Abstract

Human brain is undoubtedly the most magnificent yet delicate arrangement of tissues. This serves as the seat of such a wide span of cognitive functions and behaviors. The neuronal activities within every neuron, collectively observed over network(s) of these interconnected neurons, manifest themselves into patterns at multiple scales of observations. Many brain imaging techniques such as fMRI, EEG, MEG etc. measure these patterns as electro-magnetic responses. These patterns supposedly play the role of unique neuronal signatures of the vast repertoire of cognitive functions. Experimentally, it is observed that different neuronal populations participate coherently to generate a signature for a cognitive function. These signatures could be investigated at the micro-scale corresponding to responses of individual neurons to external-current stimuli, at the meso-scale related to populations of neurons that show similar metabolic activities and in turn these populations, also known as regions of interest (ROIs), communicate via complex arrangement of anatomical fiber pathways leading to signatures at the macro-scale. The holy grail of neuroscience is thus to computationally decipher the interplay of this complex anatomical network and the complex functional patterns corresponding to the cognitive behaviors at various scales/levels. Each scale of observation, depending on the instruments of measurement, has its own rich spatiotemporal dynamics that interacts with higher and lower levels in complex ways. Large-scale anatomical fiber pathways are represented in a matrix that accounts for inter-population fiber strength known as structural connectivity (SC) matrix. One of the popular modalities to capture large-scale functional dynamics is resting-state fMRI, and statistical dependence between these inter-population BOLD signals is captured in functional connectivity (FC) matrix. There are many models that provide computational accounts for the relationship between these two matrices as deciphering this relationship will provide the mechanism by which cognitive functions arise over the structure. On one hand, there are many non-linear dynamical models that describe the biological phenomenon well but are expensive and intractable. On the other hand there are linear models that compromise on the biological richness but are analytically feasible. This thesis is concerned with the analysis of the temporal dynamics of observed resting-state fMRI signals over the large-scale human cortex. We provide a model that has a bio-physical explanation as well as an analytical expression for FC given SC. Reaction-diffusion systems provide a computational framework for the emergence of excitatory-inhibitory activities at the populations as reactions and their interactions as diffusion over space and time. The spatio-temporal dynamics of the BOLD signal governed by this framework is constrained with respect to the anatomical connections thereby separating the spatial and temporal dynamics. Covariancematrix of this signal is estimated thus getting an estimate of the functional connectivity matrix. The covariance matrix or the BOLD signal in general is expressed in terms of the graph-diffusion-kernels thus forming an analytically elegant expression. Most importantly, the model for FC abstracts out biological details and works in the realm of spectral graph theoretic constructs providing the necessary ease for computational analysis. As this model learns the combination parameters of multiple diffusion kernels and kernels themselves, it is called Multiple Kernel Learning (MKL) model. Apart from superior quantitative performance, the model parameters may act as biomarkers for various cognitive studies. Albeit, the model parameters are learned for a cohort, the model preserves subject-specificity. These parameters can be used as a measure for inter-group differences and dissimilarity identification as has been employed for age-group identification as an example in this thesis. Essentially MKL model partitions FC into two constituents: influence of the underlying anatomical structure into diffusion kernels and the cognitive theme of temporal structure into the model parameters, thus predicting FCs specific to subjects within the cognitive conditions of the cohort. Even though MKL is a cohort based model, it maintains sensitivity towards anatomy. Performance of the model drastically drops down with alterations in SC and model parameters, but does not overfit to the cohort. Resting state fMRI BOLD signals have been observed to show non-stationary dynamics. Such multiple spatio-temporal patterns, represented as dynamic FC matrices, are observed to be cyclically repeating in time motivating use of a generic clustering scheme to identify latent states of dynamics. We propose a novel solution that learns parameters specific to the dynamic states using a graph-theoretic model (temporal-Multiple Kernel Learning, tMKL) and finally p

Abstract

Synthetic or artificial speech has been developed steadily during the last decades. Especially, the intelligibility has reached an adequate level for most applications, especially for communication impaired people. This thesis is concerned with the development of deployable text to speech (TTS) systems for Indian languages. The research described here was conducted with two principle aims: To build stable, scalable and unrestricted synthesis systems for Indian languages; and to design such systems by (a) making observations about the structure of the data and (b) keeping the human in loop so that we end up with systems both acceptable and efficient. In order to do that, I first present a baseline synthesis system for Indian languages built employing unit selection and concatenation as the algorithm and syllables as the basic unit with reduced vowel epenthesis based backoff to handle the missing units. During synthesis, the required utterance specified as a string of syllables help in the selection of appropriate units from the stored database. The system uses word to phone mapping to handle the words from a different language (English) and the selected units are concatenated using a variant of Waveform Similarity Overlap Addition (WSOLA) based method. Two extensions to the system are discussed. • Inspired by the smoothness of the speech obtained after the backend concatenation algorithm, a method is presented to exploit the signal correlation as a join cost in the viterbi algorithm itself with an aim to further improve the naturalness of the system. Two variants of signal similarity are examined - cross correlation based and average magnitude difference function based. • Modeling the prosodic characteristics is an important trait for a text to speech system. Deep generative models which are the standard today benefit from a low dimensional continuous input rather than an array of 1-hot k encoding for the phonemes as is used today. A method has been discussed to obtain continuous representation for the phonemes using a matrix factorization approach and show that the representation can be used to model prosody, by predicting the phoneme durations. The system can be retrained in 6 hours on the same languages and should take approximately the same amount of time for any new language.

Abstract

Abstract International Relations deals with the interaction of political entities in the international arena. Iden- tifying and analyzing these interactions in near real-time is important to predict upcoming political alignments and fathom upcoming conflicts. This effort is widespread as part of governmental diplo- macy. Our approach presents a computational notion of this same phenomenon. This requires analysis and data which was scarce and highly secured. Technology and computation, was limited and expen- sive. Classified documents regarding the interactions weren’t public. With increased digitization and penetration of internet across domains, this research has renewed impetus. One of the major tasks of analyzing actor interactions is to identify events involving actors from the news. Event is one of the most important temporal phenomena. It is that which has a beginning and an end. Automatic coding or classification of events happening in international relations is an important part of social science data ecosystem. The last few decades have witnessed significant work in detecting political events in the international arena. Most of the current work involves classification of news based on pattern matching from a large set of verb patterns, political actors, compound nouns, compound verb phrases, reference to pronouns and deep parsing of sentences in news articles. We introduce a methodology for extracting international relations events using news media data. This method involves a graph based unsupervised learning method for extracting topics in the news articles and identifying events based on various entities: actors and other spatial and temporal features. Since the same event might be reported by multiple news media across the world, multiple events corresponding to the same real-world event would be detected. This is referred as Event Coreference in NLP. We work on the problem of Event Coreference resolution which consists of finding clusters of event mentions that refer to the same event. Since events are complex, they co-refer fully as well as partially. We extract full co-reference, partial co-reference (sub-event and member-of relations) for the events extracted based on rule-based approach and obtain substantially good results. International relations events require an ontology for the classification of events to analyze the in- teractions among actors. We propose a novel mediation ontology for the categorization of international relations events and also introduce a novel method of classifying events through the mediation ontol- ogy using Beth Levin Verbs Classification, word2vec and Universal dependencies. The selected feature space is a result of mapping language entities to ontological entities. Hereafter, we explore various applications of international relations events. We demonstrate the relevance of our work in IREvent2Story. Event detection is a key aspect of story development which is itself composed of multiple narrative layers. Most of the narratives are template-based and follow a narration theory. We demonstrate a narrative from events detected in the international relations which presents interactions of international actors over various topics and other interactive visualizations. Another application we demonstrate is IREvent2Perspectives. Media is a classic example of the freedom to expression which may incorporate polar sentiments on the reportage. We create a system for readers with various perspectives on the same international relations event from various news sources across the world. Our study opens newer avenues for journalists and policy makers to exploit the opportunities for data driven diplomacy. It is hoped, that this research finds base especially with national and international organizations that the inexact and amorphous nature of world diplomacy is handled on hard facts and real analysis.

Abstract

Last few years have seen a tremendous increase in interest for autonomous driving. As advances have been made in this field, autonomous driving has faced various challenges. In this thesis, we present vision based solutions to curb and intersection detection problems faced by self-driving cars. In the first part of the thesis, we present a discriminative approach to the problem of curb detection under diverse road conditions. Curb detection is a critical component of driver assistance and autonomous driving systems. We define curbs as the intersection of drivable and non-drivable area which are classified using dense Conditional random fields(CRF). In our method, we fuse output of a neural network used for pixel-wise semantic segmentation with depth and color information from stereo cameras. CRF fuses the output of a deep model and height information available in stereo data and provides improved segmentation. Further, we introduce temporal smoothness using a weighted average of Segnet output and output from a probabilistic voxel grid as our unary potential. Finally, we show improvements over the current state of the art neural networks. Our proposed method shows accurate results over a large range of variations in curb curvature and appearance, without the need of retraining the model for the specific dataset. While it is important to detect road boundaries, another interesting problem is to know how the boundaries fork and lead to an intersection. In the second part of the thesis, we focus on the problem of road intersection detection. Long-short term memory networks(LSTM) models have shown considerable performance on a variety of problems dealing with sequential data. In this thesis, we propose a variant of Long-Term Recurrent Convolutional Network(LRCN) to detect road intersection. We call this network as IntersectNet. We pose road intersection detection as binary classification task over a sequence of frames. The model combines deep hierarchical visual feature extractor with recurrent sequence model. The model is end to end trainable with a capability of capturing the temporal dynamics of the system. We exploit this capability to identify road intersection in a sequence of temporally consistent images. The model has been rigorously trained and tested on various different datasets. We think that our findings could be useful to model behavior of autonomous agent in the real world.

Abstract

The thesis brings out technology impact on the two key political processes viz. news reporting and public protests. In the first part theoretical models are developed for the attention seeking ability of the news headlines online, based on the theories of human curiosity termed as the information seeking behaviour without utilitarian motive. A classifier is built to identify the clickbaity headlines indicating high degree of curiosity they generated in reader’s mind. The methodology is also used to study the headline dataset scraped and collated from the digital newspaper start-ups in India for their ability to attract reader views and attention. News headline gets spotted by the reader for the novelty of the topic, it creates surprise in readers’ mind by the choice of word arrangements used. Headline stoking optimal reader anticipation attracts a click pointing to its autonomous significance independent of the narrative. Finally headline’s structural, syntactic and semantic features create information gap in reader’s mind, by elevating the information reference point and the headline which is ”self-important”, salient, and promises to close the information gap piques reader curiosity and triggers clicking behaviour. The work also computationally validates a century old postulate about the relationship between curiosity and stimulus levels by computing novelty and surprise from Facebook reader response to news headlines. We have plotted the curious responses of reactions, shares and comments against the stimuli causing novelty and surprise to check their impact on the reader curiosity. The result of inverted ”U” curve, is as per the psychological modelling done during late 19th century by Wilhelm Wundt hypothesizing intermediate arousal potential. The second part of the thesis brings out the biography of public outrage online by computationally analyzing two recent incidents viz. Jyoti Singh incident(2012) at Delhi and murder of noted journalist Gauri Lankesh(2017). This study has tried to prise-open the manner in which online outrage builds up and evolves into politically meaningful public conversations. It suggests that outrage by itself has a short life, is intense in its expression but dies quickly. However, it appears that where it builds on a certain robust mapping of emotions and incidents to each other, it has a good chance to develop an afterlife as a larger public conversation, and thus impacts politics.

Abstract

Inverse Kinematics (IK) problem deals with finding an appropriate actuator configuration that makes the end effector Cartesian coordinates and pose match the given coordinates and pose. In most of the cases, it is enough if we can solve IK for the given coordinates whatever may be the pose. Real time calculation of inverse kinematics with dynamically stable configuration is of high necessity in humanoid robots as they are highly susceptible to lose balance. General Inverse Kinematic solvers may not guarantee real-time control of the end-effectors in external coordinates along with maintaining stability. This work proposes a methodology to generate joint-space trajectories of stable configurations for solving inverse kinematics using Deep Reinforcement Learning (RL). Our approach is based on the idea of exploring the entire configuration space of the robot and learning the best possible solutions using an actor-critic based policy learning in continuous action spaces, Deep Deterministic Policy Gradient (DDPG). The proposed strategy was evaluated on the highly articulated upper body of a humanoid model with 27 degree of freedom (DoF). The trained model was able to solve inverse kinematics for the end effectors with 90% accuracy while maintaining the balance in double support phase. Following the success of learning a general IK solver for goal reachability tasks of a single hand of humanoid, a more challenging problem of solving IK for both hands of the humanoid simultaneously is taken up. For addressing this problem, the proposed methodology was extended with significant changes in the state vector and reward function modelling. The extended strategy was then evaluated on the highly articulated upper body of the given humanoid for learning multi-goal reachability tasks of both hands along with maintaining stability in double support phase. Results show that the trained model was able to solve inverse kinematics for both the hands, where the articulated torso contributed to both the tasks. However it was observed that DDPG was unstable and in some cases even though it was stable initially, the cumulative reward started degrading later. Consequently, this works moves on to address these instability issues by using multi-task reinforcement learning. However, most reinforcement learning algorithms are inefficient for learning multiple tasks in complex robotic systems, where actions between different tasks are not fully separable and share a set of actions. In case of tasks where actions are fully separable, each task policy can be learnt independently using different policy networks. However when the actions are not fully separable, the policies cannot be learnt independently. In such environments a compound policy may be learnt with shared neural network parameters, which performs multiple tasks concurrently. However such compound policy may get biased towards a task or the gradients from different tasks negate each other, making the learning unstable and sometimes less data efficient. In this work, we propose a new approach for simultaneous learning of multiple tasks sharing a set of common actions in continuous action spaces, which we call as DiGrad (Differential Policy Gradient). The proposed framework is based on differential policy gradients and can accommodate multi-task learning in a single actor-critic network. We also propose a simple heuristic in the differential policy gradient update in case of partially separable actions to further improve learning. In order to show the efficiency of framework, the proposed architecture was tested on the given humanoid for learning multi-goal reachability tasks (each end effector has a different goal) and the results were compared with that of DDPG. Training results show that DiGrad converges faster than DDPG and removes the instability in training. Finally a comparative analysis with different settings in DiGrad along with DDPG was shown.

Abstract

The cellular users’ demand has increased drastically with the emergence of mobile internet technologies. In order to satisfy the demand, cellular service providers have improved each generation of cellular networks over its predecessors. However, the fifth generation (5G) cellular technologies would revolutionize the network architecture and would go beyond the trend of previous generations. Device-to-Device (D2D) communication is one such revolutionary technology in which the mobile users communicate with each other directly without traversing the base station. D2D communication is actively studied with special focus on public safety in 3GPP. D2D communication with network control has many other use-cases. Also, the 5G cellular technology includes millimeter wave (mmWave) for communication. Along with massive bandwidth availability at mmWave frequencies, the advantage of extensive frequency reuse and directive propagation is gained. The integration of device-to-device (D2D) and mmWave technology would improve the system performance and spectral efficiency to a great extent. Moreover, with the emerging trends in Internet-of-Things (IoT), massive device connectivity is required with high connection speed. Hence, D2D communication technology plays a vital role in maintaining Quality of Service (QoS) in IoT applications. The thesis studies D2D communication in inband mode where the resources of the licensed spectrum are used for communication. The work emphasizes on the system performance improvement because of the integration of mmWave technology and D2D communication. Underlay mode D2D communication, where the devices reuse the resources of a cellular user, is analyzed. The problems of resource allocation and interference avoidance are addressed. Then, the importance of fairness in resource allocation and the power allocation are studied. An approach for resource allocation and power allocation is proposed which is adaptive to traffic demand. Results show that the proposed scheme can significantly enhance the system performance in terms of fairness and spectral efficiency. Inband D2D communication with selective operation in underlay and overlay modes aiming to maximize QoS is explored. Spectrum underutilization is the key challenge in overlay mode which is addressed in this work. In areas with high call traffic density, D2D communication can operate in overlay mode because of its low complexity as resources are dedicated and the absence of interference with cellular users. In order to maximize spectrum utility, resources are dynamically assigned to overlay mode based on traffic demand. System performance in overlay mode is analyzed. A dynamic mode selection scenario is considered while addressing resource allocation, power control and interference avoidance. Thus, formulated problem is solved using a single heuristic algorithm for resource management in both the modes. Performance of the proposed algorithm is evaluated through simulations. The proposed scheme has optimal performance for a dense 5G network in controlled underlay and overlay operations and overcomes the problem of spectrum wastage in overlay mode by its selective operation.

Abstract

Abstract LiDAR has been emerging as one of the most sought after technologies with it’s novelty to perceive 3D environments accurately. One of the popular application of the lidar data has been object segmen- tation of different objects from a given scene. Lidar as a data format can record the 3D structure of the buildings in urban areas with precise object information. Such platforms to record lidar are mostly clas- sified into three types that is terrestrial, aerial and mobile based lidar systems. The data being recorded from respective platforms varies a lot in terms of range, coverage and point density. Our work presents new approaches to segment buildings from a given lidar scene with flexibilty to work with lidar data from multiple different platforms. The challenge of using LiDAR datasets is the very high resolution of datasets, with high computa- tional requirements in visualizing and processing the datasets. The current methods of object segmen- tation and extraction and classification of aerial LiDAR data are manual and tedious. Our work tries at modelling this problem into a generic pipeline which can be used to segment different varieties of lidar datasets. The pipeline constructed is a result of multiple geometric approaches taken to segment the buildings. The massiveness of data to process, the accuracy of groundline estimation, detection of connected roofplanes and occlusion and overlap of multiple objects in a given lidar scene poses a range of challenges while processing and segmenting such data. Our work tries to overcome some of the challenges and shortcomings of the existing works by proposing two new methods of segmenting objects. Objects in this thesis refer to only buildings. The justification of segmenting only buildings and not focusing on any other type of objects is primarily due to the uniformity in shape of the building and its adherance to geometric properties. In case of other objects, such advantages are not present making their segmentation more challenging and difficult to formulate. The scope of the thesis is limited to only aerial LiDAR because of it’s larger coverage and more processing friendly density compared to terrestrial scans. Terrestrial LiDAR can not capture the roof planes of the buildings while in case of aerial LiDAR we can capture the roof planes of the buildings, which are later used to segment buildings from the given scene. Existing methods of building segmentation from a LiDAR scene mostly involves the usage of image processing based techniques like morphological operations and fitting done in a plane through RANSAC algorithm. These methods often are highly dependent on the resolution of the input data being fed into it. Some methods also take the help of other data to aid in segmentation. Our work tries to tackle the shortcomings of the above works and present a new and novel method to segment buildings from a given scene. We propose two different approaches in segmenting buildings from a given scene. The commonality of the different approaches is that both of them focus on using the facade planes of the buildings to segment them. The algorithm has been tested in multiple datasets ranging from synthetic and real-world datasets. A bottom-up geometric rule-based approach was used initially to devise a way to segment buildings out of the LiDAR datasets. Multiple lidar datasets were used to check the accuracy and performance of the algorithm. Preliminary results show successful segmentation of the buildings objects from a given scene in case of synthetic datasets and promissory results in case of real-world data with an accuracy of 68%. Our work tries to solve a part of the puzzle by segmenting only the buildings in a given lidar scene automatically with limited tuning, in an intuitive manner. The further advantage of the proposed work is its non-dependence on any other form of data required except LiDAR. It is an unsupervised method of building segmentation, thus requires no model training as seen in supervised techniques. It focuses on extracting the walls of the buildings to construct the footprint, rather than focusing on the roof. The focus on extracting the wall to reconstruct the buildings

Abstract

The thesis presents immensely pipelined parallel FPGA based architectures for Image segmentation by a graph-based partitioning approach and by state-of-the-art CNN’s approach. Efficient and real-time segmentation of color images has a variety of importance in many fields of computer vision such as image compression, medical imaging, mapping and autonomous navigation. Being one of the most computationally expensive operations, it is usually done through software implementation using high-performance processors. In robotic systems, however, with the constrained platform dimensions and the need for portability, low power consumption and simultaneously the need for real-time image segmentation, we envision hardware parallelism as the way forward to achieve higher acceleration. Field-programmable gate arrays (FPGAs) are attractive alternatives for this task because of their reconfigurability, high per-watt performance in a small physical area. They exceed the computing speed of various software-based implementations by breaking the paradigm of sequential execution and accomplishing more per clock cycle operations by enabling hardware level parallelization at an architectural level. The first contribution proposes three novel architectures for a well known Efficient Graph-based Image Segmentation algorithm. These proposed implementations optimize time and power consumption when compared to software implementations. The hybrid design proposed, has notable furtherance of acceleration capabilities delivering at least 2X speed gain over other implementations, which henceforth allows real-time image segmentation that can be deployed on Mobile Robotic systems. The second contribution focuses on FPGA architectures designed for state-of-the-art approach CNN’s for image segmentation and classification. Convolutional Neural Networks (CNN’s) are rapidly gaining popularity in varied fields such as computer vision, information retrieval, and mobile robotics. Due to their increasingly deep structures, modern CNN’s are continuously becoming computationally and memory intensive. Although this enhances accuracy, it becomes difficult to deploy them on energy-constrained mobile devices such as drones. As a consequence, hardware accelerators such as FPGAs with their inherent hardware parallelism, have come up with an attractive alternative. The major bottleneck while implementing huge networks on FPGA is meeting high memory throughput requirement of CNN’s with limited on-chip memory. Hence, this work proposes a high-performance FPGA based architecture - Depth Concatenation and Inter-Layer Fusion based ConvNet accelerator-DeCoILFNet which exploits the intra-layer parallelism of CNN’s by flattening across the depth and combining it with the inter-layer fusion. DeCoILFNet perfectly pipelines the data flow across convolution layers which significantly reduces off-chip memory accesses and maximizes the throughput by using multiple line buffers. To validate our approach, we demonstrate results for first five convolution layers of the VGG-16 network implemented on Xilinx Virtex7 VC709 FPGA board in Verilog .

Abstract

Deep neural networks have yielded immense success in speech recognition, computer vision and natural language processing. However, the exploration of deep neural networks for news recommendation systems has received a relatively less amount of attention. Also, different recommendation scenarios have their own issues which creates the need for different approaches for recommendation. With news stories coming from a variety of sources, it is crucial for news aggregators to present interesting articles to the user to maximize their engagement. This creates the need to have a recommendation system which accounts for both the content of the articles and the user preferences. Methods such as Collaborative Filtering, which are well known for general recommendations, are not suitable for news because of the short life span of articles and because of the large amount of articles published each day. Typically, it is desirable that a news recommendation system be able to discriminate between and select articles from a pool to recommend to a user as soon as they are published. Apart from this, such methods do not harness the information present in the sequence in which the articles are read by the user and hence are unable to account for the various interests of the user which may keep changing with time. Alternatively, the other class of models based on Content Filtering, can handle cold start problems but in the long run tend to suffer from the problem of over specialization. In this thesis, we address these issues in a step-by-step manner. We start off with a problem that is commonly associated with deep learning based methods i.e lack of sufficient data. In order to tackle this issue we come up with an item-based collaborative filtering approach which utilizes Markov Decision Process (MDP). We also come up with a novel semantic similarity measure which we incorporate as a reward for the MDP. This helps us gain insights about the various interests that users may have by only having prior knowledge of a few articles read by them in the past. We then move on to the exploration of various deep learning based methods to tackle our problem. We give importance to utilizing the content of the articles, and taking into account the historical reading data of a user. We attempt to solve the cold-start problem, and make effective recommendations for users who have had little to no interaction with items. We design the model to keep learning parameters based on implicit feedback from the users. A description of how we go about it is described as follows. We first come up with a user profiling based approach and use it in combination with a Deep Semantic Structured Model (DSSM). We then later expand the model and utilize a recurrent neural network with an attention mechanism. Such a mechanism helps us discriminate between the various interests of the user. We then come up with the Recurrent Attentive Recommendation Engine (RARE). RARE consistsof two components and utilizes the distributed representations of news articles. The first component is used to model the user’s sequential behaviour of news reading in order to understand her general interests i.e to get a summary of her interests. The second component utilizes an article level attention mechanism to understand her particular preferences. We feed the information obtained from both the components to a Siamese Network in order to make predictions which pertain to the user’s generic as well as specific interests. We carry out extensive experiments to establish the effectiveness of our methods. We also perform experiments to prove the efficacy of our model on solving the item cold-start cases as well as making effective recommendations for users who have had very little interaction with articles. Finally, we experiment with a novel 3D Convolutional Neural Network based model in an attempt to solve similar problems as the ones we tried to address earlier. Applying 3D convolution helped us identify both spatial (features of a particular article) as well as temporal information (features present in the sequence of articles read by the user) which are pertinent to a user’s interest. The initial results of this experimentation are also presented in this thesis.