Abstract

Wearable plantar pressure measurement systems have become important in biomechanics, rehabilitation, and gait analysis. By capturing foot-ground interaction data, these systems enable researchers and
clinicians to evaluate balance, diagnose pathological gait, design personalised footwear, and monitor patient recovery. Existing research in this domain has predominantly focused on extracting insights about
the human body, with applications centred on musculoskeletal health, motor control, and performance
assessment. However, despite the rich information embedded in plantar pressure signals, their potential
to characterise properties of the external environment, particularly the surfaces on which humans walk,
has received little to no attention.
This thesis addresses this unexplored dimension by investigating whether plantar pressure data can
be leveraged to infer surface characteristics, explicitly focusing on hardness. Unlike conventional approaches to surface analysis, which rely on specialised equipment or destructive testing, wearable plantar pressure systems provide a low-cost, portable, and unobtrusive means of measurement. The central
research question guiding this work is whether pressure patterns generated by the foot during locomotion can be systematically mapped to the mechanical properties of the underlying surface.
To explore this question, we developed a low-cost in-shoe sensor array capable of capturing plantar
pressure distributions in real time. Experiments were conducted by collecting walking trials across
surfaces of varying hardness, ranging from soft foam and natural ground to rigid concrete and tiled
flooring. The recorded plantar pressure profiles were then analysed to identify distinctive signatures
associated with different surface types. Statistical and machine learning techniques were employed to
classify surfaces based on the extracted features, enabling quantitative assessment of surface hardness
from wearable data.
The results demonstrate that surface hardness exerts a measurable influence on plantar pressure distribution, and that these differences can be reliably captured through the proposed in-shoe system. Our
findings establish, for the first time, that wearable plantar pressure measurements can be repurposed for
surface characterisation. This represents a conceptual shift in how wearable sensing technologies are
perceived, broadening their utility from human biomechanics to context-aware environmental interaction.
The implications of this work extend across several domains. In rehabilitation and sports science,
knowledge of surface properties can inform safer training and recovery protocols. In robotics, surface
characterisation through human-robot shared sensing could improve terrain adaptation strategies. From an urban perspective, large-scale deployment of such systems could support real-time monitoring of
ground conditions in public spaces. Overall, this thesis contributes a methodological innovation and a
new application space for wearable plantar pressure systems, bridging the gap between human-centered
sensing and surface characteristics.

Abstract

Domain specific narrative texts are seen in education, governance, healthcare, legal and multiple
other domains. These texts encompass different kinds of general and domain-specific knowledge. To
make these niche narrative texts understandable and usable both by humans and machines, they need to
be processed through Natural Language Processing (NLP) mechanisms and made conducive for use in
desired domain specific applications. A specific kind of knowledge is spatial knowledge. It is useful for
enabling the fundamental cognitive skill of spatial understanding that underpins essential human interactions with the environment. Spatial knowledge enables one to perform basic navigation and orientation,
such as planning routes, understanding and explaining locations of objects and people, and describing
objects by their spatial features - shape, size, and appearance. Pre-trained Language Models (PLMs)
have fundamentally transformed NLP by serving as powerful, generalized language processors trained
on massive datasets. Their primary importance lies in achieving state-of-the-art performance across
diverse NLP tasks, often without the need for extensive task-specific retraining.
The work done as part of the thesis combines the above three aspects and comprises of automatic
extraction, representation and analysis of spatial knowledge in domain specific narratives with the help
of PLMs. Further, the work includes demonstration of the utility of the extracted knowledge in niche
use-cases. More specifically, the work is divided into two major aspects - PLMs as reasoners of spatial
knowledge and PLMs as containers of spatial knowledge. As part of the first aspect, we consider
an existing knowledge representation, the Message Sequence Chart (MSC) and enrich it with spatial
knowledge through a novel task of motion reasoning. It is through this task of motion reasoning, we
check how PLMs function as reasoner of dynamic spatial knowledge. As part of the second aspect, we
gauge PLMs for their familiarity of a specific kind of spatial knowledge - facts in world geography. We
construct a moderately sized dataset of facts related to 24 different geographical relations and check
multiple encoder and decoder PLMs through a mask filling exercise. Additionally, we enrich these
PLMs through fine-tuning to enhance their understanding of this particular kind of spatial knowledge.
The major research contributions of the thesis include (i) proposition of the MSC as a narrative
representation formalism, (ii) enrichment of the MSC with spatial knowledge for enabling it as a spatiotemporal knowledge representation, (iii) proposition of the novel task of motion reasoning under the
sub-area of spatial reasoning, (iv) the masked sentences dataset for the effort on gauging geography
knowledge in PLMs and its enrichment, and (v) proposition of novel applications namely crime report
gap finding, enhancing spatial fidelity in the filmstrip representation, niche tourism search and wikidata
augmentation. For several of these contributions, the developed datasets and code have been made available publicly for reproducibility as well as encouraging further research. Furthermore, a subset of
work in the thesis was carried out as part of a collaboration project between my organization - TCS
Research and IIIT - Hyderabad. The code/data artifacts and learning obtained as part of the PhD have
been employed in several Proof-of-Concepts (PoCs) developed and showcased for clients and internal
teams at TCS.

Abstract

Automatic Speech Recognition (ASR) has witnessed rapid progress with the advent of self-supervised
learning and large-scale multilingual models. However, ASR for morphologically complex and lowresource languages remains challenging due to limited labeled data, orthographic inconsistencies, tokenization difficulties, and the computational demands of large-scale systems. Perso-Arabic languages
such as Persian, Arabic, and Urdu exemplify these difficulties, where script variations, rich morphology, complex word formation, and inconsistent text normalization complicate both preprocessing and
subword modeling for ASR systems.
Current state-of-the-art approaches largely rely on scaling model size and data volume, often overlooking the importance of linguistic structure, script-aware tokenization, and domain relevance. For
low-resource settings, scale-driven strategies are computationally prohibitive and frequently suboptimal.
This thesis investigates whether efficient and competitive ASR performance can instead be achieved
through linguistically informed preprocessing, morphologically-aware tokenization, and targeted crosslingual continual pretraining, without dependence on excessively large models.
We first introduce a unified script-aware preprocessing pipeline tailored to Perso-Arabic languages,
addressing normalization, tokenization, phonetic parsing, and text cleaning to reduce orthographic noise
and improve data consistency. Building upon this foundation, we construct a scalable multilingual unlabeled speech corpus of approximately 3,000 hours across Persian, Arabic, and Urdu. Using this corpus,
we systematically evaluate continual pretraining strategies under different initialization conditions and
integrate SentencePiece-based subword modeling within a CTC-based Wav2Vec framework, carefully
adapting token segmentation to handle word boundaries and repeated character collapse. All downstream fine-tuning and evaluation are conducted in a monolingual setting to isolate transfer effects.
Our results demonstrate that a 300M parameter model, when adapted through targeted continual
pretraining and morphologically-aware tokenization, achieves performance competitive with systems
over five times larger. Notably, the proposed model outperforms Whisper Large v3 on Persian and yields
strong results on Arabic and Urdu despite using substantially fewer parameters and labeled resources.
These findings challenge the prevailing assumption that ASR quality scales primarily with model
size. Instead, we show that data relevance, adaptation strategy, and linguistically informed tokenization
play a more critical role in low-resource scenarios. This thesis provides a practical and resource-efficient
pathway toward competitive ASR for Perso-Arabic languages and offers a reproducible experimental
template that may inform future multilingual extensions.

Abstract

Modeling realistic human-object interaction in three-dimensional environments is a fundamental
challenge in computer vision, with broad implications for robotics, virtual reality, and embodied artificial intelligence. Despite significant progress in human motion generation and 3D scene understanding,
existing approaches often treat motion, interaction, and environment as separate problems, limiting their
ability to generate physically plausible and context-aware behavior.
This thesis addresses this challenge through a unified perspective on scene-aware human-object interaction, grounded in both real-world systems and data-driven modeling. We begin by developing a
system for real-time human pose understanding and interaction analysis based on joint-level reasoning,
formalized in our published patent application. This system demonstrates how structured representations of human motion can enable robust, interpretable, and deployable interaction-aware applications.
Building on this foundation, we introduce a large-scale dataset and benchmark for full-body human
motion with object interaction in realistic 3D scenes. This dataset enables the study of scene-aware
grasping and exposes key limitations of existing methods in handling interaction under environmental
constraints. We then propose a learning-based framework for generating physically plausible, sceneaware human-object interactions that explicitly models the interplay between body motion, object manipulation, and scene geometry.
To enable scalable deployment of such systems, we further develop a fast and robust geometry processing framework based on feature-aware mesh simplification, which preserves interaction-relevant
geometric properties while significantly reducing computational complexity.
We also explore the extension of interaction-aware modeling to conditional settings, including multimodal human motion generation. In particular, we highlight the role of object interaction in ensuring
consistency between generated motion and environmental constraints, demonstrating that naive integration of motion and interaction leads to physically implausible results.
Through extensive experiments and evaluations, we demonstrate that the proposed approaches significantly improve the realism, physical plausibility, and scalability of human-object interaction in 3D
environments. By bridging real-world systems, data-driven modeling, and geometry processing, this
thesis contributes towards the development of integrated frameworks for human-centric 3D understanding and interaction.

Abstract

Classical motion planning for robotic manipulation includes a set of general algorithms that aim to
minimize a scene-specific cost of executing a given plan. This approach offers remarkable adaptability,
as they can be directly used off-the-shelf for any new scene without needing specific training datasets.
However, without a prior understanding of what diverse valid trajectories are and without specially
designed cost functions for a given scene, the overall solutions tend to have low success rates within a
certain time limit. Conversely, deep-learning-based planners achieve high accuracy within their training
domains but are notoriously brittle when faced with out-of-distribution environments. To bridge this
gap, we first introduce Ensemble-of-costs-guided Diffusion for Motion Planning (EDMP) [2]. EDMP
leverages a diffusion model to learn a prior over kinematically valid trajectories while incorporating
scene-specific "collision-cost" guidance directly at the time of inference. By employing an ensemble of
multiple cost functions to capture diverse scene nuances, EDMP significantly improves success rates and
generalizes to novel tasks, such as manipulating arbitrary held objects, without retraining.
Despite its robustness, EDMP, and similar waypoint-based diffusion planners, faces a fundamental
bottleneck: the requirement for a substantial number of denoising steps, which leads to slow inference
speeds. To address this, we propose Guided Polynomial Diffusion for Motion Planning (GPD)[1],
where we introduce diffusion in the parametric space of trajectories, where the parameters are
represented as Bernstein coefficients. We show that this representation greatly improves the effectiveness
of the cost-function guidance and the inference speed. We also introduce a novel stitching algorithm
that leverages the diversity in diffusion-generated trajectories to produce collision-free trajectories
with just a single cost function-guided model. We demonstrate that our approaches outperform SOTA
diffusion-based motion planners for manipulators and provide an ablation study on key components.

Abstract

Document question answering has witnessed substantial progress with the development of large visionlanguage models. Despite these advances, most existing systems remain limited in their interpretability,
as they typically generate answers without explicitly and precisely identifying the visual evidence on
which those answers are based. This deficiency is particularly pronounced in real-world documents,
where supporting evidence may appear in curved text, be fragmented across multiple spatially separated
regions, be embedded within tables or charts, or exist at different semantic levels such as phrases, lines,
and blocks. Under such conditions, conventional grounding approaches based on coarse bounding boxes
are often insufficient, as rectangular regions fail to capture the irregular and fine-grained structure of
document evidence.
This thesis addresses these limitations through M3Grounder, a novel mask-based framework for
grounded document question answering. In contrast to prior approaches that represent evidence using
rectangular coordinates, the proposed framework formulates grounding as a segmentation problem
and predicts pixel-level evidence masks that are tightly aligned with the answer generation process.
The model autoregressively generates answer text interleaved with special grounding tokens, enabling
explicit correspondence between individual answer spans and their associated supporting regions. This
formulation naturally supports multi-span grounding, in which a single answer may depend on multiple
disjoint regions, and multi-granular grounding, in which evidence may be localized at the phrase, line,
or block level depending on the semantic scope required by the question. This design enables answer
generation interleaved with grounding tokens and hierarchical evidence prediction.
To realize this formulation, the thesis introduces a set of architectural and training mechanisms
designed to improve the precision and faithfulness of document grounding. In particular, M3Grounder
integrates document understanding with segmentation-based evidence prediction, employs a bleedsuppression objective to reduce spillover into irrelevant neighboring regions, and incorporates a hierarchical enclosure mechanism that encourages spatial consistency across phrase-, line-, and block-level masks.
By enforcing containment relationships across these granularities, the framework more faithfully reflects
the hierarchical structure of document layouts and supports more interpretable evidence attribution. Collectively, these components enable finer and more reliable localization than prior box-based grounding
methods, especially in documents containing irregular text geometry or densely structured layouts.
Beyond model design, this thesis also introduces GroundingDocQA, a large-scale data generation
pipeline for grounded document question answering, and GroundingDocQA-Bench, a benchmark for
the joint evaluation of answer correctness and grounding fidelity. The dataset construction pipeline is designed to support a broad range of document types, including text-rich pages, curved-text documents,
and graphics-heavy documents such as charts and infographics. This enables supervision for a wide
variety of grounded reasoning scenarios that extend beyond conventional extractive settings. The
benchmark complements standard answer-centric evaluation by explicitly assessing whether predicted
evidence is spatially accurate, semantically appropriate, and faithful to the generated answer. In this way,
the thesis promotes a more comprehensive evaluation paradigm for document question answering, one
that treats evidence quality as a first-class criterion alongside answer accuracy.
Extensive empirical results demonstrate that mask-based grounding constitutes a more expressive
and effective alternative to coarse region attribution for document question answering. The proposed
framework yields improvements in evidence faithfulness, better captures irregular and spatially distributed
support, and provides stronger grounding performance across challenging document layouts. More
broadly, this thesis argues that the next generation of document question answering systems should not be
evaluated solely by whether they answer correctly, but also by whether they can provide precise, verifiable,
and semantically appropriate evidence for their predictions. In this respect, the thesis advances document
question answering toward more interpretable, trustworthy, and practically deployable system

Abstract

Recent advances in text-to-image generation have enabled generation of highly realistic images. Text
alone, however, can only provide coarse control over the generation process. As a result, generative
models fail to serve applications which require precise control over various scene elements. One important scenario is multi-object scene control, where the aim is to generate a scene with precise control
over 3D object arrangements.
Most of the existing works for 3D control in generation fail to account for multi-object scenes,
primarily focusing on single object scenes only. Even methods that do allow some form of multi-object
control fail to generate inter-object occlusions, resulting in unrealistic images.
This thesis contributes two research works along this direction. Compass Control is the first work
which enables multi-object 3D orientation control in text-to-image generative models. This is achieved
by introducing special orientation tokens directly in the text prompt, one token per object in the scene.
The orientation token is output by a learnable MLP, which takes desired orientation for that object as
input. Further, a binding mechanism is introduced to bind each compass token to its corresponding
object, thus enabling disentangled multi-object control.
Compass Control, along with other works on 3D scene control, have a consistent limitation - they
fail to generate inter-object occlusions. We find that this limitation stems from lack of occlusion awareness in the input scene representations, which fail to model 'hidden objects' in the scene. Building on
these insights, we propose SeeThrough3D, a model for 3D layout conditioned generation that explicitly
models occlusions. We introduce an occlusion-aware 3D scene representation (OSCR), where objects
are depicted as translucent 3D boxes placed within a virtual environment and rendered from desired
camera viewpoint. The transparency encodes hidden object regions, enabling the model to reason about
occlusions, while the rendered viewpoint provides explicit camera control during generation. We condition a pretrained flow based text-to-image image generation model by introducing a set of visual tokens
derived from our rendered 3D representation. Furthermore, we apply masked self-attention to accurately
bind each object bounding box to its corresponding textual description, enabling accurate generation of
multiple objects without object attribute mixing.
The code and datasets for both these works have been made publicly available to encourage future
research in this direction.

Abstract

Intermittent water supply, the predominant mode of piped water distribution across urban India,
forces buildings to rely on Overhead Tanks (OHTs) filled by electric pump motors. In institutional
buildings such as campus hostels, the electricity consumed by these motors represents a significant
recurring operational cost. The recent mandated adoption of Time-of-Day (ToD) electricity tariffs,
which levy a 20% surcharge during morning and evening peak hours and offer a 20% discount during
off-peak hours, presents a concrete opportunity to reduce this cost by shifting motor operation to cheaper
tariff windows. Conventional OHT controllers use fixed volume thresholds that are entirely agnostic to
the prevailing tariff, activating the pump motor at the same lower bound irrespective of whether the
moment falls in an expensive peak window or a cheap off-peak window. This tariff-agnostic behaviour
leads to unnecessary expenditure that can be eliminated without any hardware modification.
This thesis presents an Internet of Things (IoT)-based framework that closes this gap. A retrofit
camera-based sensing system was deployed at a 500-resident campus hostel at International Institute of
Information Technology, Hyderabad (IIIT-H), Hyderabad, collecting approximately 105,000 cumulative
water meter readings at two-minute intervals over four months. A Hamming distance-based correction
algorithm removed digit misclassification errors from the raw reading sequence, after which 154 clean
motor ON/OFF event pairs were extracted. A minimum safe motor ON trigger height of hˆ
trig = 0.50 m
(volumetric equivalent 18.75 kL) was derived from Torricelli's theorem and the observed peak consumption rate, establishing a physically grounded hydraulic lower bound on the motor activation threshold.
A proportional ToD-aware threshold modulation policy was formulated with analytically derived scale
factors of 0.50, 0.80, and 1.00 for peak, normal, and off-peak tariff hours respectively, requiring no
hardware changes.
A discrete-time consumption-driven simulation evaluated the policy against the real four-month consumption trace. Total pumping cost fell from Rs 53,390 to Rs 48,150, a saving of 9.8%, driven by a
91.9% reduction in peak-hour motor runtime from 676 to 54 hours. Total motor runtime remained unchanged at 2,048 hours in both configurations, confirming that the saving arises entirely from the timing
of motor operation. Water availability was maintained throughout, with the simulated tank volume never
dropping below the hydraulic safety minimum at any point over the four-month horizon.

Abstract

The global rise in pediatric myopia has intensified the need for objective biomarkers to understand
the functional retinal changes associated with refractive error. The pupillary light reflex (PLR) serves as
a non-invasive window into the health of rods, cones, and intrinsically photosensitive retinal ganglion
cells (ipRGCs). However, traditional pupillometry is often constrained by stationary hardware, narrow
spectral stimuli, and the challenges of testing pediatric populations. This thesis explores the use of a
Virtual Reality Head-Mounted Display (VR-HMD) with integrated infrared eye tracking as a portable,
high-precision platform for broad-spectrum chromatic pupillometry.
A cohort of 117 children (ages 6-14) was recruited and categorized into emmetropic, low myopic,
and high myopic groups. Participants were exposed to an eight-color paradigm (Red, Green, Blue,
Yellow, White, Purple, Indigo, and Black) generated on an OLED display, followed by a custom computational pipeline (PuReST algorithm) to extract quantitative metrics. The VR-HMD platform was
first validated by reproducing canonical PLR waveforms, demonstrating that it effectively disengages
the vergence-accommodation conflict through full-field stimulation.
Statistical analysis revealed that myopic children exhibit a significantly greater Maximum Constriction Amplitude (MCA) in response to high-illuminance stimuli (Yellow and White light) compared to
emmetropes, suggesting an altered dopaminergic modulation or increased photosensitivity to luminance.
Furthermore, myopes demonstrated a significantly faster Maximum Constriction Velocity (MCV) in response to long-wavelength Red light, supporting the hypothesis of an upregulated L-cone pathway adaptation to myopic defocus. A striking "anomalous" response was observed for Indigo light, where participants displayed three distinct phenotypes: constriction, dilation, or no change. A targeted follow-up
study (N=43 adults) deconstructed this effect, revealing that the anomaly was driven by a negative luminance contrast relative to the background, creating a perceptual conflict between chromatic constriction
and luminance-driven dilation. Finally, exploratory case studies of participants with strabismus, high
hyperopia, and amblyopia further demonstrated the system's sensitivity to diverse neuro-visual pathologies.
This research establishes the VR-HMD as a valid scientific instrument for quantitative pupillometry.
The findings provide new physiological evidence for functional neuroretinal differences in the myopic
eye and position VR-based chromatic pupillometry as a scalable solution for tele-ophthalmology and
the identification of predictive biomarkers for myopia progression.
Keywords: Chromatic Pupillometry, Virtual Reality, Myopia, Pediatric Vision, Retinal Photoreceptors, Pupillary Light Reflex.

Abstract

The rapid evolution of machine learning from task-specific systems to general-purpose decisionmakers has amplified concerns around transparency, reliability, and trust. Despite strong performance,
modern models remain opaque, often exhibiting unpredictable behavior and generating outputs that are
fluent yet unfaithful to the input or task constraints. These challenges necessitate principled oversight
mechanisms that can both explain model behavior and detect failures in a scalable and reliable manner.
This thesis addresses these challenges through the lens of behavioral faithfulness, focusing on two
complementary problems: (1) generating faithful explanations of model behavior, and (2) detecting unfaithful model outputs. We introduce SLED (Sample Learning to Explain Divergence), a framework
for producing global natural language explanations that characterize where two machine learning models converge and diverge in their predictions. SLED synthesizes representative inputs via gradient-based
optimization to identify regions of agreement and disagreement, and leverages large language models
to generate interpretable explanations grounded in these behavioral patterns. Unlike prior approaches
based on local feature attributions or training data access, SLED provides model-level insights with
minimal data requirements.
Experiments across 13 datasets and 11 model configurations show that SLED consistently outperforms baselines such as MaNtLE, LIME, and Anchors, achieving improvements of 18-24% in faithfulness and 10-22% in simulatability. Ablation studies demonstrate that the framework is sample-efficient,
robust to the choice of explainer models, and benefits from regularization that ensures realistic synthetic
samples. Human evaluations further validate its utility, with users achieving a simulatability score of
63.5% and reporting improved understanding of model behavior.
In parallel, this thesis formalizes hallucinations as failures of faithfulness, enabling systematic analysis across tasks. We propose a computationally efficient, NLI-based approach for detecting hallucinations in natural language generation tasks such as definition modeling, machine translation, and paraphrase generation. This framework models hallucination detection as a function of both model inputs
and training distributions, allowing it to generalize across domains.
Overall, this work establishes faithfulness as a unifying principle for trustworthy AI. By grounding
explanations in observable behavior and reframing hallucinations as measurable failures of that behavior, it bridges the gap between model decision processes and human understanding, contributing toward
more transparent and reliable machine learning systems.