Abstract

Terrain, representing features of an earth surface, plays a crucial role in many applications such as simulations, hazard prevention and mitigation planning, route planning, analysis of surface dynamics, computer graphics-based games, entertainment, films, to name a few. With recent advancements in digital technology, these applications demand the presence of high-resolution details in the terrain. However, currently available public datasets, providing terrain scans in the form of Digital Elevation Models (DEMs) have low resolution compared with the terrain information available in other modalities like aerial images. Publicly available DEM datasets for most parts of the world have a resolution of 30 m whereas the aerial images or satellite images are available at a resolution of 50 cm. The cost involved in capturing of such high-resolution DEMs (HRDEMs) turns out to be a major hurdle for making such high-resolution available in the public domain. This motivates us to provide a software solution for generating high-resolution DEM from the existing low-resolution DEMs (LRDEMs). In natural image domain, super-resolution has set up higher benchmarks by incorporating deep learning based solutions. Despite such tremendous success in image super-resolution task using deep learning solutions, there are very few works that have used these powerful systems on DEMs to generate HRDEMs. A few of them used additional modalities as aerial images or satellite images, temporal sequence of DEMs etc., to generate high-resolution terrains. However, the applicability of these methods is highly subject to the available input formats. In this research effort, we explore a new direction in DEM super-resolution by using feedback neural networks. Availing the capability of feedback neural networks to redefine the features learned by shallow layers of the network, we design DSRFB, a DEM super-resolution architecture that generates high-resolution DEM with a super-resolution factor of 8X with minimal input. Our experiments on Pyrenees and Tyrol mountain range datasets show that DSRFB can perform near to the state-of-the-art without using information from any additional modalities like aerial images. Further, by understanding the limitations of DSRFB, which primarily occur in case of highly degraded low-resolution input. In such cases, the major structures are entirely lost and the reconstruction becomes challenging. In such cases, to avail the elevation cues from alternate sources of information becomes necessary. To utilize such information from other modalities, we inherit the attention mechanism from natural language processing (NLP) domain. We integrate the attention mechanism into the feedback network to present Attentional Feedback Module (AFM). Our proposed network, Attentional Feedback vivii Network (AFN) with AFM as a backbone, outperforms the state-of-the-art methods with the best margin of 7.2%. We also emphasize on the reconstruction of the structures across patch boundaries. While generating HRDEM by splitting large DEM tiles into patches, we propose to use overlapped tiles and generate an aggregated response to dilute the artefacts due to structural discontinuities. To summarize, in this research, we propose two methods DSRFB and AFN to generate a high- resolution DEM from existing low-resolution DEM. While DSRFB achieves near to the state-of-the-art performance, coupling DSRFB with attentional mechanism (i.e., AFN) outperforms state-of-the-art methods.

Abstract

This work is an attempt to study polarization on social media data, in the Indian context. We focuson multiple controversial and talked about events and topics in the context of India, namely 1) theSabarimala Temple (located in Kerala, India) incident which became a nationwide controversy whentwo women under the age of 50 secretly entered the temple breaking a long standing temple rule thatdisallowed women of menstruating age (10-50) to enter the temple and 2) the Indian government’s moveto demonetise all existing 500 and 1000 denomination banknotes, comprising of 86% of the currencyin circulation, in November 2016 3) The Pulwama-Balakot incidents and its effect on popular sentimenttowards India’s Prime Minister Narendra Modi 4) Social media activity on events surrounding multiplelynching incidents from 2017 to 2019 and 5) the Indian Economy from 2013 to 2019 and popularresponse towards it on social media. We gather tweets from news media, and from common people onTwitter around these events in various time periods. Then, we pre-process and annotate them with theirsentiment polarity and emotional category, and analyse trends to help us understand changing polarityover time around controversial events. While this study tries to find trends and patterns to decode theaspect of polarization in India’s social fabric, it’s also a case study on all these events and how Indiareacted to each one of them. The tweets collected are in English, Hindi and code-mixed Hindi-English.

Abstract

Over the last decade, a tremendous emphasis has been laid on collection and digitization of a vast number of books leading to the creation of so-called ‘Digital Libraries’. Projects like Google Book and Project Gutenberg have made significant progress in digitizing over millions of books and making it available to the public. Efforts have also been made from the perspective of Indic languages where the task to identify and recognize books from several Indian languages has been undertaken by the National Digital Library of India. Advantages of digital libraries can be manifold. Digitization of ancient manuscripts ensures the preservation of knowledge and promotes research. Books in digital libraries are indexed which facilitates easy search and retrieval. They are easy to store and do not take as much effort in maintenance as their physical counterparts. One of the most important steps in the digitization effort is the recognition and conversion of physical pages into editable text using an OCR. There are commercial OCRs available like Tesseract and Abby fine reader, however, the ability of an OCR to recognize text without committing too many errors depends very much on the print quality of the pages as well as font style of the type-written text. A pre-trained OCR will invariably make errors across pages whose distribution is different in terms of fonts and print quality from the pages on which it was trained. If the domain gap is too large then the number of error words will be too high which will result in investing significant effort in the correction. Since the books need to be indexed, one cannot afford to have too many word errors in the OCR recognized pages. Thus, a major effort must be spent on correcting the error words, misclassified by the OCR. Manually correcting each isolated error word will incur a huge cost and is infeasible. In this thesis, we look at methods to improve OCR accuracy with minimum human involvement. To this effect, we propose two approaches. In the first approach, we strive to improve the OCR performance via an efficient post- processing technique where we aim to group similar erroneous words and correct them simultaneously. We argue that since a book has a common underlying theme, it will contain many word repetitions. These word co-occurrences can be taken advantage of by grouping similar error words and correcting them in batches. We propose a novel clustering scheme which combines features from both images as well as its text transcription to group error word predictions. The grouped error predictions can then be corrected either automatically or with the help of a human annotator. We show via experimental verification that automatic correction of error word batches might not be the most efficient way to correct the error words and employing a human annotator to verify the error word clusters will be a more systematic way to address the issue. Next, we look at the problem of adapting an OCR to a new dataset without requiring too many annotated pages. Traditional norm dictates finetuning the existing OCR on a portion of target data. However, even annotating a portion of data to create image-label pairs can be a costly affair. For this, we employ a self-training approach where the OCR is finetuned on its own predictions from the target dataset. To curtail the effects of noise present in the predictions, we include only those samples in the training set on which the model is sufficiently confident. We also show that by employing various regularization strategies we can outperform the traditional finetuning method without the need for any additional labelled data. We further show that by combining self-training with finetuning we can achieve a maximum gain in terms of OCR accuracy across all the datasets. We furnish thorough empirical evidence to support all our claims.

Abstract

Image quality is a fundamental problem in computer vision. For variety of applications, for instance scanning documents, QR codes, bar codes or algorithms like object detection, recognition, tracking, scene understanding etc. images with good contrast, high illumination and sharpness are desired. Similarly in computer graphics for information visualisation, animations, presentations etc. aesthetically pleasing design and good colorization of the images are desired. Therefore the definition of image quality depend on the context and application of the image. In this thesis we attempt to address various challenges pertaining to image quality, (1) for natural imaging, we explore a novel approach for predicting the capture quality of the images taken in the wild. (2) For Graphics designs, we explore the aesthetic quality of images by suggesting multiple aesthetically pleasing colorization of graphics designs. Due to increasing advancements and portability of smartphone cameras, it has become a default choice for capturing images in the wild. However, there are quality issues with camera captured images due to reasons like lack of stability during capture process. This hinders the automatic workflows which takes camera captured images as input e.g. Optical Character Recognition (OCR) for documents image, face detection/recognition from human image etc. Part of this thesis is focused on Image Quality Assessment (IQA), the aim is to quantify the degradation like out-of-focus blur and motion artefacts in a given image. One of the major challenge in IQA for images captured in the wild is that, we do not have ground truth to measure the capture quality. Therefore various previous attempts of IQA require human in loop for creating the ground truths for capture quality of images. Large user studies are conducted and mean human opinion scores are then used as measure for quality. In this work we use a signal processing based technique to generate the IQA ground truth, and propose a comprehensive IQA dataset which is a good representative of the real degradation during the process of capture. Further, we propose deep learning based approach to predict image quality for captures in the wild. Such IQA algorithm can be helpful in the cause by either giving online quality suggestion during capture or rating the quality post capture. Another dimension to the image quality is aesthetic quality. Increasing usage of internet, social media and advancement in the mobile camera, photography has become a very popular hobby and interest to a large section. Even for a well captured image, people use varieties of filters and effects post capture to enhance the appearance of the image e.g. adjusting color temperature, contrast or even blurring part of vi vii the image (bokeh effect) etc. Therefore the capture quality is not enough to define the aesthetic quality of the image. However, one of the major factors defining the aesthetic quality of image is colorization. Particularly in computer graphics domain, where artificially generated images already have well defined structures, shape and components. Therefore sharpness or capture quality are not relevant, but on the other hand, color quality plays a very important role in visualization and appearance of the image. In natural images, largely colors are associated with semantics e.g. sky is always blue or grass are green etc. whereas in animations and computer graphics where objects are loosely associated with semantics, this lead to more choices of colors. Hence the problem of colorization becomes more challenging in graphics domain, where overall appearance of the images are more than its naturalness. Therefore, this work also covers aesthetic quality of graphics images, here instead of measuring the color quality, we propose algorithm to produce better coloring suggestions for the given graphics images.

Abstract

Dysarthria is a disorder resulting from weaknesses of neuromuscular execution in motor speech production due to brain tumors, brain injury, stroke, cerebral palsy, and facial paralysis. The abnormalities in the resonance, articulation, respiration, phonation, and prosody of speech are associated with dysarthria leads to poor speech intelligibility. The symptoms of poor speech quality and reduced intelligibility can be used to identify the dysarthria. The degree to which the listener understands the dysarthric individual’s speech is referred to as speech intelligibility of that speaker. Dysarthric speech detection and intelligibility assessment are very important steps in the clinical diagnosis of dysarthria. The subjective intelligibility assessment methods can be influenced by the listener’s familiarity with patients, the contextual, suprasegmental factors, and semantic/syntactic features. Moreover, the subjective intelligibility assessment methods are costly and time-consuming. On the other hand, objective intelligibility assessment methods are economical, repeatable, reliable, and can help remote patient rehabilitation monitoring. The growing evidence suggesting that clinicians are becoming more receptive to objective intelligibility assessment systems in which the trained acoustic model can assess the speech intelligibility. An objective assessment method like acoustic to articulatory representation of dysarthric speech can effectively uncover the dysarthria specific features that are useful for the assessment. Additionally, the detection of articulatory changes from the speech is useful for the assessment of dysarthria. Hence, the extraction of features from speech signals is considered to be one of the most important steps in developing clinical tools for the automatic detection and assessment of dysarthria. In literature, few attempts have been made to detect dysarthria, assess the intelligibility of dysarthric speech, and identify the type of dysarthria. The main objectives of this thesis are dysarthric speech detection (binary classification task) and dysarthric speech intelligibility assessment (four class classification task). In this thesis, all the studies are done using the UA-Speech database. In the first study, the effect of tongue tip movement in subjects with dysarthria was investigated by analyzing the duration of rhotic approximant sounds. The duration of rhotic approximant was measured using the third formant trajectory estimated from speech. The formant trajectory is estimated from the spectrogram computed using quasi-closed-phase (QCP) analysis that is an accurate method to estimate formants. This study showed that the duration of rhotic approximant is a good indicator of the severity level of dysarthria. This method requires sophisticated landmark detection techniques to measure the rhotic approximant duration for the automatic detection and assessment of dysarthria. Moreover, the duration feature showed variations due to gender, speaker, and phonetic context. Therefore, subsequent studies of the thesis focused on other state-of-art signal processing methods for the robust detection and assessment of dysarthria. The previous study anticipated that subtle variations in vocal tract system dynamics need to be captured by the feature representation for an accurate assessment of dysarthria. In this regard, narrow bandpass filtering based instantaneous spectral representation was investigated in this thesis to capture these variations. A new feature representation called perceptually enhanced narrow bandpass filtering based cepstral coefficient (PE-NBFCC) was proposed to detect and assess dysarthria. This study showed that the proposed features performed better than conventional features in terms of classification accuracy in both dysarthria detection and assessment tasks. Further to improve the performance of detection and assessment systems, phase information in the speech signals was explored. In this regard, the importance of analytic phase information was examined by proposing narrow bandpass filter bank-based instantaneous frequency cepstral coefficients (NBFB-IFCC). The proposed features showed better performance compared to standard features in terms of accuracy and F1-Score. The score-level fusion of proposed features with the magnitude spectral features improved the classification accuracy, which indicates the complementary information present in the analytic phase feature. Finally, this thesis explored the excitation features extracted using epoch-based speech processing. This study proposed a method for epoch extraction, namely zero-phase pitch selective bandpass filtering (ZP-PSBF). This study showed that the importance of the excitation source features for the detection and intelligibility assessment. The issues addressed in this thesis are summarized as follows: • Acoustic analysis of rhotic approximants was made, and the relation betwee

Abstract

In this thesis, we address the novel and highly challenging problem of estimating the layout of a complex urban driving scenario. Given a single color image captured from a driving platform, we aim to predict the bird’s eye view layout of the road, lanes, sidewalks and other traffic participants. The estimated layout should reason beyond what is visible in the image, and compensate for the loss of 3D information due to the perspective projection. We dub this problem amodal scene layout estimation, which involves hallucinating scene layout for even parts of the world that are occluded in the image. Firstly, we present MonoLayout, a deep neural network for real-time amodal scene layout estimation from a single image.MonoLayout maps a color image of a scene into a multi-channel occupancy grid in bird’s eye view, where each channel represents occupancy probabilities of various scene components. We represent scene layout as a multi-channel semantic occupancy grid, and leverage adversarial feature learning to “hallucinate” plausible completions for occluded image parts. We extend several state-of- the-art approaches for road-layout estimation and vehicle occupancy estimation in bird’s eye view to the amodal setup and thoroughly evaluate against them. By leveraging temporal sensor fusion to generate training labels, we significantly outperform current art over a number of datasets. Next, we aim to provide a comprehensive benchmark for the task of amodal layout estimation on KITTI and Argoverse datasets and also shift our attention towards estimating more fine-grained atr- ributes such as lanes, crosswalks, vehicles, etc. . To this end, we introduce AutoLay, a new dataset for amodal layout estimation in bird’s eye view. AutoLay includes precise annotations for (amodal) layouts for 44 sequences from the KITTI dataset. In addition to fine-grained attributes such as lanes, sidewalks, and vehicles, we also provide detailed semantic annotations for 3D pointclouds. To foster reproducibil- ity and further research in this nascent area, we open-source implementations for several baselines and current art. Further, we propose VideoLayout, a real-time neural net architecture that leverages tempo- ral information from monocular video, to produce more accurate and consistent layouts. VideoLayout achieves state-of-the-art performance on AutoLay benchmark, while running in real-time. We show the potency of the aformentioned methods using several abltaion studies and also show applications of amodal layout estimation on downstream tasks.

Abstract

In this thesis, I investigate the effects of certain psychoactive substances on the brain and propose a novel method to deal with addiction. The first section deals with the effect of marijuana on the different hippocampus subfields. The hippocampus has the highest concentration of cannabinoid receptors, the primary binding site for THC, the primary pscyhoactive constituent of marijuana. But this concentration varies across the subfields. We also check for the effects of varied consumption in this study by splitting the cannabis consuming group into two subgroups: heavy consumers and light consumers. The next section deals with a method to administer transcranial magnetic stimulation in patients with addiction and depression disorders amongst others. We focus on using a personalised locus rather than a population driven locus. The main issue faced is the highly variable nature of the dorsolateral prefrontal cortex, the target site for the administration. Subsequently, we highlight the importance of preprocessing and quality control in the processing of magnetic resonance images in both humans and non-human primates. Existing approaches in this area fail to deal with multiple artifacts often encountered while working on these scans. We then use quality control to develop a novel pipeline to preprocess macaque structural images. We also use quality control in the development of a state of the art pipeline for preprocessing and processing human diffusion weighted tensor images.

Abstract

The human body is a complex structure consisting of multiple organs, muscles and bones. However, in regard to modeling human motion, the information about fixed set of joint locations (with fixed bone length constraint) is sufficient to express the temporal evolution of poses. Thus, one can represent a human motion/activity as temporally evolving skeleton sequence. In this thesis, we primarily explore two key problems related to modeling of human motion, the first is efficient indexing & retrieval of human motion sequences and the second one is automated generation/synthesis of novel human motion sequences given input class prior. 3D Human Motion Indexing & Retrieval is an interesting problem due to the rise of several data-driven applications aimed at analyzing and/or re-utilizing 3D human skeletal data, such as data-driven animation, analysis of sports biomechanics, human surveillance, etc. Spatio-temporal articulations of humans, noisy/missing data, different speeds of the same motion, etc. make it challenging and several of the existing states of the art methods use hand-craft features along with optimization-based or histogrambased comparison in order to perform retrieval. Further, they demonstrate it only for very small datasets and few classes. We make a case for using a learned representation that should recognize the motion as well as enforce a discriminative ranking. To that end, we propose, a 3D human motion descriptor learned using a deep network. Our learned embedding is generalizable and applicable to real-world data - addressing the aforementioned challenges and further enables sub-motion searching in its embedding space using another network. Our model exploits the inter-class similarity using trajectory cues and performs far superior in a self-supervised setting. State of the art results on all these fronts is shown on two large scale 3D human motion datasets - NTU RGB+D and HDM05. In regard to the second research problem of Human Motion Generation/Synthesis, we aimed at long-term human motion synthesis that can aid to human-centric video generation [9] with potential applications in Augmented Reality, 3D character animations, pedestrian trajectory prediction, etc. Longterm human motion synthesis is a challenging task due to multiple factors like long-term temporal dependencies among poses, cyclic repetition across poses, bi-directional and multi-scale dependencies among poses, variable speed of actions, and a large as well as partially overlapping space of temporal pose variations across multiple class/types of human activities. This paper aims to address these challenges to synthesize a long-term (> 6000 ms) human motion trajectory across a large variety of human activity 1Video for Human Motion Synthesis. 2Video for Human Motion Indexing and Retrieval. classes (> 50). We propose a two-stage motion synthesis method to achieve this goal, where the first stage deals with learning the long-term global pose dependencies in activity sequences by learning to synthesize a sparse motion trajectory while the second stage addresses the synthesis of dense motion trajectories taking the output of the first stage. We demonstrate the superiority of the proposed method over SOTA methods using various quantitative evaluation metrics on publicly available datasets. In summary, this thesis successfully addressed two key research problems, namely, efficient human motion indexing & retrieval and long-term human motion synthesis. In doing so, we explore different machine learning techniques that analyze the human motion in a sequence of poses (or called as frames), generate human motion, detect human pose from an RGB image, and index human motion.

Abstract

Recently, a lot of the standard algorithms in machine learning and computer vision need to be redesigned for handling large scale problems. In this thesis, we explore one such fundamental algorithm, which is inherent in almost every multi-view reconstruction system pipeline in computer vision. Recently, bundle adjustment has been the key component in Structure-from-Motion (SfM) problem. Bundle adjustment (BA) jointly optimizes the camera parameters and feature points parameters according to an objective function of reprojection errors. This joint optimization of camera parameters and feature points paramters for multi-view reconstruction formulate the BA problem as a non-linear least squares problem which is solved by some variant of the traditional Levenberg-Marquardt (LM) algorithm [44]. Most of the computation in LM goes into repeatedly solving the normal equations that arise as a result of linearizing the objective function. Some widely used methods like Cholesky Decomposition and Preconditioned Conjugate Gradients [25], to solve these normal equations still face some challenges in robustness, accuracy, and efficiency. In this thesis, we propose a deflated algebraic two-grid method has been used as a preconditioner for Generalized Minimal Residual Method (GMRES) [58], for solving these normal equations arising in the Bundle Adjustment process. We show that the proposed method is several times faster than iterative schur and jacobi schur [2] which are considered as state-of-the-art methods to solve the Bundle Adjustment problem. We verify the competence of our algorithm by benchmarking it against the stateof-the-art methods on publicly available Bundle Adjustment in the Large (BAL) dataset [3].

Abstract

As demands for high volume data exchanges increase in many memory applications, significant efforts have been devoted to improving I/O interface performance to achieve high bandwidth. However, the impedance mismatch between I/O driver and a transmitting channel causes signal reflection, interfering with the incoming data in terms of overshoot, undershoot, or ringing. I/O interface requires a minimum variation of output I/O driver impedance over process, temperature, and voltage variations to ensure high-quality signal integrity. The traditional approach of using an external resistor component to generate a constant reference for impedance calibration is not an effective solution considering the constraints due to shrinking package form factor and low-cost limitations of memory applications. This thesis proposes a novel I/O driver impedance calibration architecture and its circuit design using internally generated constant current reference. Also, designed a current mode Bandgap Reference circuit, which generates a constant current across Process and Temperatures variations. A voltage detect circuit is proposed to capture Supply Voltage variations and to compensate the current reference accordingly. To minimize the error in calibration codes, a low offset comparator is designed. IO driver with a combination of MOS device and a series poly resistor is used for better linearity in impedance. This thesis proposes and implements an independent calibration of Pullup and Pulldown IO drivers. The pullup driver calibration codes are not reused for pulldown driver calibration. It reduces the mismatch in impedance of Pullup and Pulldown drivers by eliminating propagation of any quantization error in Pullup calibration codes to Pulldown driver calibration. The proposed calibration scheme and the whole circuit was designed and implemented in 28nm standard CMOS technology using 1.8V supply voltage. In spice simulations of the proposed calibration design/scheme, the impedance variation of both pullup and pulldown driver is approximately +/-7% across process, temperature, and voltage variations. This reduced IO driver impedance variation is achieved using the proposed internally generated constant current source without using any external component resistor.