Abstract

The field of autonomous driving has seen many advances in recent times. In this thesis, we try to look at some important challenges in autonomous driving from the perspective of a single (monocular) camera: Real-Time Scene (Road/Lanes) Layout and multi-object simultaneous localisation and mapping in the outdoor scene. To tackle the challenges faced by monocular setups/systems (such as scale ambiguity in monocular reconstruction, dynamic object localisation, and uncertainty in feature representation), we transpose the above problems into orthographic (bird’s-eye view) space. Thus, we perform road/lane layout reconstruction, localisation and mapping of dynamic participants and the ego camera vehicle in the scene with an orthographic view as the configuration space. By assuming only the height of the egocamera above the ground, we leverage single-view metrology cues to accurately localise the ego-vehicle and all other traffic participants in bird’s-eye view. We demonstrate that our system outperforms prior work that uses strictly greater information, highlighting each design decision’s relevance via an ablation analysis

Abstract

Barak valley is an area in the north-eastern part of India where traditionally the practice of shifting cultivation has been more prevalent than sedentary agriculture. In the recent past, population growth coupled with the geographic isolation of the area has further aggravated the decline of forest land cover in the area. The region that encompasses the Barak valley, shows varied land covers and has diverse population densities. While some land use and land cover changes are gradual and slow, some can be rapid and fast. Also, many of these changes are dependent on the biophysical and socio-economic drivers of change, including policy inputs. So, there is a need to understand these interactions and how these changes will be affected by the different factors in the near future. This thesis focuses on modelling land use changes in the Kathakal basin of Barak valley, primarily changes in the shifting cultivation patterns, over a period of 1988 to 2005 using an agent-based model after understanding changes in the recent past. Each district is modelled as an agent of change in the agent-based model, 1. to capture the interactions between the various drivers like population changes, infrastructure development and land use practices and 2. to decide on the land resource allocation across the district at the land parcel level including allotment of shifting cultivation regions. The model accounts for both socio-economic as well as geographic factors, like access to infrastructure while making the land use decision on a year-on-year basis. The model considers the need for shifting cultivation land area for both a staple crop – paddy (rice) and a non-staple crop based on the demand against the supply estimated from irrigated and rain-fed cropping regions. While the model has been fine-tuned based on the data till 1997, the simulated model outcome of 2005 is validated against an existing remote sensing derived land use map of the region. The model shows an accuracy of 98% for the test period till 1997 and achieves an overall accuracy of about 94% for 2005 predicted land use pattern. While the aggregated results across the region show good concurrence, the randomness inherent in the choice of the shifting cultivation land limits the ability to predict the precise locations. Further, the thesis will discuss the challenges of different land use conditions prevailing in the two districts and how the model is able to adapt to those in providing a good estimate of the land use changes in a region as diverse as Barak Valley. Based on the model and its applicability to the Barak valley region, the research further explores the impact of various policies on the choice of land use over a period of 30 years, i.e., the land use patterns between 2005 and 2035. Four different scenarios, including a business-as-usual (BAU) scenario were modelled, to highlight how either ecological or economic considerations affects the choice of land use in the two diverse existing land use conditions of the districts of Hailakandi and Mamit. The results show that depending on whether the district is largely rural or urban, has sparse or dense population, and has access to irrigation infrastructure, the land use responses are different. The BAU scenario shows that without intervention of any kind, it is difficult to stop the spread of shifting cultivation due to the land pressures from population growth and the need for increased economic returns from the land. The model shows that infrastructure development and incentivising settled forms of agriculture can have a positive impact on reducing shifting cultivation, though the rate of this change is dependent on both socio-economic and bio-physical conditions of the region.

Abstract

In the current digital era, about 80% of the digital data which is being generated is unstructured and unlabeled natural language text. Research efforts are going on for improving text mining techniques to automatically organize, analyze, and extract useful information from the voluminous text data. In the development cycle of information retrieval and text mining applications, text representation is the most fundamental and critical step as its effectiveness directly impacts the performance of the application. Three important properties of the text representation models, which make them suitable for practical applications, are representational power, interpretability of features, and unsupervised learnability. Words and word sequences (such as sentences and documents) are the natural units of the text which are subjected to vector representation to handle the given text data. With the advent of deep learning, a family of neural language models emerged to represent words as low dimensional, distributed, and dense vectors. These word representations are popularized as word embeddings. Word embeddings are learned from huge corpora, so they encode a lot of information and have high expressive power. In the case of word sequences, traditional methods provide interpretability and support unsupervised learning, but they suffer from the issues of low representational power and scalability. The deep learning based word embedding methods are successful in producing vectors with high representational power. However, most of the deep learning based methods are notorious for human uninterpretabilty and work only in supervised learning environments. In this thesis, we propose improved word sequence representation models by exploiting the frequency distributional and spatial distributional properties of the word embeddings. Firstly, we propose an alternative word sequence representation framework in the context of longer texts such as documents. The existing vector averaging based models represent the document as a position vector in the same word embedding space. As a result, they are unable to capture the multiple aspects as well as the broad context in the document. Also, due to their low representational power, the existing approaches perform poorly at document classification. Furthermore, the document vectors obtained using such methods have uninterpretable features. In this work, we propose an improved document representation framework that captures multiple aspects of the document with interpretable features. In this framework, instead of representing a document in word embedding space, it is represented in a distinct feature space where each dimension is associated with a potential feature word that has relatively high discriminatory power. A given document is modeled as the distances between the feature words and the document. We have proposed two criteria for the selection of potential feature words and a distance function to measure the distance between the feature word and the document. Experimental results on multiple datasets show that the proposed model consistently performs better at document classification over the baseline methods. Secondly, we propose a weighted averaging based word sequence embedding method in the context of shorter texts such as sentences. The proposed weighting scheme captures the contextual diversity of words based on their geometry in the embedding space. It is observed that the simple unweighted vector averaging ignores the discriminative power of words and treats all the words as equal in the given sentence. Assigning the weights to words based on their discriminative power helps in building better word sequence representations. Recent literature introduced word weighting schemes based on the word frequency distribution into the simple averaging model. The frequency-based weighted averaging models augmented with the denoising steps are shown to outperform many complex deep learning models. However, these frequency-based weighting schemes derive the word weights solely based on their raw counts and ignore the diversity of contexts in which these words occur. The proposed weighting algorithm is simple, unsupervised, and non-parametric. Experimental results on semantic textual similarity tasks show that the proposed weighting method outperforms all the baseline models with significant margins and performs competitively to the current frequency-based state-of-the-art weighting approaches. Furthermore, as the frequency distribution-based approaches and the proposed word embeddings geometry-based weighting approach capture two different properties of the words, we define hybrid weighting schemes to combine both varieties. We also empirically demonstrate that the hybrid weighting methods perform consistently better than the corresponding individual weighting schemes. Overall, we have proposed a new word sequence representation framework and wei

Abstract

In the healthcare domain, medical and patient interactions are crucial for diagnosis. Initially, AI models developed for healthcare centered only on monolingual data. However, such models do not cater to the multilingual regions, where most conversations are Code-Mixed. Our thesis concentrates on developing the Code-Mixed Medical Task-Oriented Dialog System. This task-oriented dialog system aims to help the user consult a medical specialist based on their symptoms. We train the dialog system on the code-mixed task-oriented dialog dataset ‘Su-Vaid’. The dataset contains 3005 Telugu-English Code-Mixed dialogues between patients and doctors with 29k utterances covering ten specializations with an average code-mixing index (CMI) of 33.3%. The major components of our dialog system are Natural Language Understanding (NLU), Dialog Manager (DM), and Natural Language Generation (NLG) modules. We manually annotated the conversational dataset with intent and slot labels. We also present baselines to establish benchmarks on the dataset using existing state-of-the-art Natural Language Understanding (NLU) models. We improved the existing baselines using contextual ground truth intent labels and processing the slots as chunks. We have used Rule-based Dialog Manager and template-based NLG module in our Dialog System. While interacting with the system, the user may fail to mention a few symptoms. In such a case, the dialog system must remind the user by suggesting relevant symptoms. We have incorporated a suggestion system into our dialog system. Apart from interacting with the user to recommend the medical specialist, the dialog system needs to be more human-like (empathetic). Also, human-human conversations in the healthcare domain are mostly empathetic. To achieve human-like behavior, we have included empathy via language accommodation in our dialog system. Also, interactions between patients and medical practitioners can sometimes be challenging. Health care workers are at greater risk of workplace abuse than most other workers, with nurses and family physicians rated as most at risk of abusive encounters with patients. Types of abusive encounters range in severity, from verbal threats to more extreme encounters, such as stalking and physical assault. Hate speech and physical abuse must be strongly condemned for a better function of health care. Such conversations must be identified by the system so that the system can take necessary action on such users. So we have included hate speech detection in our dialog system. Finally, we have evaluated all the major components of our dialog system using state-of-the-art baselines and classification metrics and we chose the top performing models to integrated them into the final version of the system.

Abstract

Autism Spectrum Disorder (ASD) is a lifelong heterogeneous developmental disorder that is characterized by abnormal development of the brain. Deficits in social skills, incapability to articulate language, abnormal sensory-motor movements, and stereotyped behaviors are mainly observed in children with ASD. The underlying biological markers for the diagnosis and treatment of ASD are not known yet. The current behavior-based diagnosis of ASD is arduous and requires expertise. It has been demonstrated that the non-invasive method such as resting-state functional magnetic resonance imaging (rs-fMRI) provides a means to examine functional connectivity patterns in the brain, which can help diagnose various neurodegenerative and psychiatric disorders, including ASD. Most previous studies associated ASD with atypical functional connectivity (FC) between different pairs of regions. However, brain connectivity is dynamic and varies extensively among brain states. In this thesis, we explored both static and dynamic functional connectivity based features to understand the fundamental group differences between ASD patients and typically developing (TD) subjects. Firstly, we proposed a Multilayer Perceptron (MLP) based classification model with autoencoder pretraining for classifying ASD from TD based on static functional connectivity (sFNC) extracted from rs-fMRI scans of the ABIDE-1 (a publicly available dataset from Autism Brain Imaging Data Exchange consortium). Our model achieves new state-of-the-art performance on the ABIDE-1 dataset with a 10- fold cross-validation accuracy of 74.82%. Further, we use the Integrated Gradients (IG) and DeepLIFT techniques to identify the correlations between brain regions that contribute most to the classification task. Our analysis identifies the following regions associated with ASD: Left Lingual Gyrus, Right Insula, Right Cuneus, Right Middle Frontal Gyrus, and Left Superior Temporal Gyrus. Interestingly, these regions in the brain are primarily responsible for social cognition, language, attention, decisionmaking, and visual processing, which are known to be altered in ASD. Secondly, we investigated the dynamic functional connectivity (dFNC) between 53 independent components among 188 ASD and 195 TD subjects sampled from the ABIDE-I consortium. We estimated dFNC using sliding window-based approaches and identified four distinct dynamic states through hard-clustering analysis. Hyper-connectivity within the cognitive control domain, between cognitive control and default mode network have been identified among ASD subjects. Hyper-connectivity within the default mode network has been found among TD individuals. Further, we estimated the dynamic temporal properties such as fractional time spent, mean dwell time per state and observed significant differences between ASD and TD groups. ASD subjects are found to have significantly longer dwell time in one of the states (4) when compared to TD individuals. We also found a significantly increased occurrence of the same state (4) in ASD subjects, whereas other states (1 and 3) are more frequent in TD subjects. Overall, both static and dynamic-based methods have been explored to find out the ASD biomarkers. While there is broad consensus in the brain network profiles between sFNC and dFNC, the temporal profile of brain state dynamics is additionally available with dFNC analysis and may potentially contribute to disease biomarkers. In addition, we have done extensive replication studies of both sFNC and dFNC based models for ASD classification or characterization in the literature. We observe great discrepancy between what is reported and what could be replicated or reproduced. Based on these analyses, we propose a set of recommendations for future studies that encompass factors such as dataset selection criteria, preprocessing pipepline, proper reporting of selected samples, atlas selection, and hyperparameter choices and reporting for the proposed models.

Abstract

In recent years, due to the advent of technology and the Internet, the amount of opinionated data targeting products has increased exponentially. With this increase, there has emerged a need to understand the opinions and their associated sentiment to enhance the feedback loop for organizations manufacturing the products and users looking for opinions about the usability of the same to base their future purchasing decisions on. The major bottleneck in achieving the above is the constraint on resources available for certain languages. We explore ways to work through these constraints and attempt to achieve good results for the tasks of Sentiment Analysis and it’s variants. In this thesis, We first attempt to extract language invariant features for downstream tasks like sentiment analysis to be able to retain decent performance in a low resources setting. We find that the ability to do so is task-dependent and hypothesize patterns in tasks where the approach can and cannot work effectively. We then take a closer look at the reasons for the poor performance of models in Aspect Term Extraction and Aspect Term Polarity Classification for Hindi, which is a variant of Sentiment Analysis based on Opinion Mining. After a detailed analysis of the same, we conclude that there is a gap in the state of the existing gold standard dataset for Hindi. We then go ahead and describe our methodology for developing a high-quality dataset parallel to the Gold English dataset for these tasks and establish that the new dataset adequately represents the task. To further improve the state of Aspect Term Extraction (ATE) and Aspect Term Polarity Classification (ATPC), we develop a novel architecture that achieves new state-of-the-art results for Hindi and near state-of-the-art results for English. We also show the fullness of our method in solving the task in a multilingual setting and achieving near-state-of-art results and hence establishing that for tasks where we cannot extract language invariant features, we can develop models that can learn features crucial for the task in a manner which can be leveraged to give high performance reliably.

Abstract

The rapid advancement of autonomous driving systems is transforming the future of transportation and urban mobility. However, these systems face significant challenges when deployed in complex and unstructured traffic environments, such as those found in many cities in south-east Asian countries like India. This thesis aims to address the challenges related to data collection, management, and generation for autonomous driving systems operating in such scenarios. The primary contributions of this work include the development of a data collection toolkit, the creation of a comprehensive driving dataset for unstructured environments (IDD-3D), and the proposal of a synthetic data generation framework (TRoVE) based on real-world data. The data collection toolkit presented in this thesis enables sensor fusion for driving scenarios by treating sensor APIs as separate entities from the data collection interface. This framework allows for the use of any sensor configuration and demonstrates the smooth creation of driving datasets using our framework. This toolkit is adaptable to different environments and can be easily scaled, making it a crucial step towards creating a large-scale dataset for Indian road scenarios. The IDD-3D dataset provides a valuable resource for studying unstructured driving scenarios with complex road situations. We present a thorough statistical and experimental analysis of the dataset, which includes high-quality annotations for 3D object bounding boxes and instance IDs for tracking. We highlight the diverse object types, categories, and complex trajectories found in Indian road scenes, enabling the development of robust autonomous driving systems that can generalize across different geographical locations. In addition, we provide benchmarks for 3D object detection and tracking using state-of-the-art approaches. The TRoVE synthetic data toolkit offers a framework for the automatic generation of synthetic data for visual perception, leveraging existing real-world data. By combining synthetic data with real data, we show the potential for improved performance in various computer vision tasks. The data generation process can be extended to different locations and scenarios, avoiding the limitations of bounded data volumes and variety found in manually designed virtual environments. This thesis contributes to the development of systems in complex environments by addressing the challenges of data acquisition, management, and generation. By bridging the gap between the current state-of-the-art and the needs of unstructured traffic scenarios, this work paves the way for more robust and versatile intelligent transportation systems that can operate safely and efficiently in a wide range of situations

Abstract

The internet serves as a vast repository of information covering a diverse array of topics, ranging from blogs and articles to websites. However, it is important to note that not all of this information is valuable or relevant. Navigating through the plethora of content in order to gain a comprehensive understanding of a particular topic can be a daunting and time-consuming task. Furthermore, it is all too common to invest time in reading content that ultimately proves to be unimportant or irrelevant. Given the inherent limitations of the human cognitive capacity to process large quantities of information, concise and relevant summaries are highly sought after in order to efficiently and effectively comprehend complex subjects. Summarization is a computational task that condenses textual information into a concise version by including only the most essential and relevant information. There are two main approaches to summarization: extractive and abstractive. Extractive summarization involves selecting sentences based on their importance, while abstractive summarization involves introducing new words or phrases in the summary. Document summarization has been studied for over three decades by the NLP community. However, progress in Indian language summarization has been limited due to the lack of high-quality datasets and benchmark models, which has motivated us to work towards developing resources and benchmarks for Indian languages. In this thesis, we have developed text summarization resources for Indian languages in three different settings: mono-lingual, cross-lingual, and multi-lingual. The initial focus of this thesis is on mono-lingual summarization, specifically creating a high-quality dataset for the popular south Indian language, Telugu. We propose a pipeline that crowd-sourced summarization data and then aggressively filtered the content via: automatic and partial expert evaluation. Using the pipeline, we create a high-quality Telugu abstractive summarization dataset (TeSum). The dataset consists of 20,329 document-summary pairs, which were created by 347 annotators and evaluated by 3 raters. We carefully designed annotation guidelines that consider the parameters of Relevance, Readability, and Creativity. Additionally, we compared our dataset with existing Telugu summarization datasets. By training a summarization system on multiple languages, the system can learn to represent concepts in a shared space, regardless of the language in which they are expressed. This shared representation learning can be useful for transfer learning, as it enables the model to apply knowledge gained from one language to another language. To achieve this, we perform the multi-lingual and cross-lingual summarization for Indian languages. For multi-lingual summarization, we utilized the Indian Language Summarization (ILSUM) dataset to create baselines, which includes Hindi, Gujarati, and Indian English. We test the proposed filters on ILSUM data to perform the quality assessment. We conducted experiments with different pre-trained sequence-to-sequence models to identify the best-performing model for each language. Our work also involved an in-depth analysis of the impact of k-fold cross-validation when dealing with limited data. Additionally, we performed experiments using a combination of the original and filtered versions of the data to assess the effectiveness of the pre-trained models. We present the PMIndiaSum, a new cross-lingual and highly parallel summarization dataset for languages in India. The dataset covers 4 language families, 14 languages, and 196 language pairs. We detail the approaches taken to derive this dataset, including data acquisition, cleaning, quality assurance, and inspection. In addition, we publish benchmarks for various methodologies, such as fine-tuning pre-trained language models and summarization-and-translation. Experimental results suggest that the provision of multilingual data enhances cross-lingual summarization between Indian languages. Furthermore, this thesis also delves into multi-perspective scientific document summarization. Our objective is to develop a model that can generate a generic summary encompassing various aspects covered by multiple reference summaries of a scientific document. We describe the different pre-trained models used in this task, as well as the challenges encountered during the process.

Abstract

Synthesis, capture and analysis of a highly complex 3D terrain structure are essential for critical applications such as river/flood modelling, disaster mitigation planning, landslide modelling and flight simulation. On the other hand, synthesis of natural-looking 3D terrains finds its applications in the entertainment industry such as computer gaming and VFX. This thesis explores novel learning-based techniques for the generation of immersive and realistic 3D virtual environments, catering to the needs of the aforementioned applications. The generation of virtual worlds involves multiple components, including terrain, vegetation, and other objects. We primarily focus on three key aspects for virtual world generation: 1) develop novel AI-enabled 3D terrain authoring solutions based on real-world satellite and aerial data using a learning-based framework, 2) L-systems grammar-based 3D tree generation and 3) rendering techniques that are used to create high-quality visualizations of the generated world. Terrain generation is a critical component of 3D virtual world generation, as it provides the foundational structure for the environment. Traditional techniques for 3D terrain generation involve procedural generation, which relies on mathematical algorithms to generate landscapes. However, deep learning techniques have shown promise in generating more realistic terrain, as they can learn from real-world data to produce new, varied, and realistic landscapes. In this thesis, we explore the use of deep learning techniques for 3D terrain generation, which can produce realistic and varied terrains with high visual fidelity. Specifically, we propose two learning-based novel frameworks for Interactive 3D Terrain Authoring & Manipulation and Adaptive Multi-Resolution Infinite Terrain Generation. In addition to terrain generation, vegetation is another important component of virtual world generation. Trees and other plants provide visual interest and can help create a more immersive environment. L-systems are a popular technique for generating realistic vegetation, as they are capable of generating complex structures that resemble real-world plants. In this thesis, we propose a variant of the L-systems for 3D tree generation and compare the results to traditional procedural generation techniques. Finally, rendering is a critical component of 3D virtual world generation, as it is responsible for creating the final visual output that users will see. In addition to terrain and tree generation, this thesis also covers rendering techniques used to visualize the generated virtual world. We explore the use of real-time rendering techniques in conjunction with terrain generation to achieve high-quality visual results while maintaining performance. Overall, the research presented in this thesis aims to advance the state-of-the-art in virtual world generation and contribute to the development of more realistic and immersive virtual environments. We performed extensive empirical evaluation on publicly available datasets to report details qualitative and quantitative results demonstrating the superiority of the proposed methods over existing solutions in the literature.

Abstract

Accessibility to text for machines i.e. for downstream tasks, has been improved greatly using mainstream NLP tasks such as Machine Translation and Summarization. In this dissertation, however, we delve into two lesser-explored research areas: Text Simplification (reducing the linguistic complexity of text) and Conversation Disentanglement (extracting individual conversation threads from multi-party dialogues). We pursue text simplification by analyzing the claim of utilizing control tokens in input text to produce simplified versions with precise variations along key text attributes [1]. The effect of control tokens on simplifications was isolated individually and in particular combinations through dedicated models. We show that though control tokens can help mold the outputs closer to the target simplifications, they often cause undesired interactions between control tokens and text attributes as well as within control tokens. This sometimes results in outputs being trivial source text variants that adhere to the control token constraints but do not furnish true simplifications. We offer methods to curb some of these trivial outputs e.g. masking named-entities to shield them from unwanted replacements and to curb the extent of data sparsity using pre-trained input embeddings. Our modifications produce fewer trivial outputs and show better resilience to unseen contexts. As part of our work in conversation disentanglement, we explore the most widely used expertannotated resource in the field: Ubuntu-IRC (IRC) dataset [2]. We question IRC’s utility for training generalizable models for other domains which currently lack similar large-scale, high-quality datasets, e.g., meeting transcripts & movie dialogues. We address this concern by analyzing IRC for potential sources of platform-based specificity. We find that using direct mentions (explicit references to a message’s recipient) in IRC is one such instance of platform specificity that occurs abundantly in the dataset. To measure its impact on model performance, we create Ubuntu-IRC-Hard – a variant of IRC without direct mentions. We show that the performance of past approaches (and proposed baselines) degrades significantly without direct mentions. An in-depth analysis of models is conducted using qualitative metrics based on dialogue properties which reveal brittleness and variations among models across properties. We introduce two methods that leverage these insights to alleviate this brittleness: (i) A weighted-loss formulation to better represent less frequent sub-ranges of dialogue properties in the data, and (ii) An informed-ensembling technique that determines a subset of models best suited for each property sub-range. Both methods surpass SOTA [3] performance and score more consistently across dialogue properties.