Abstract

The thesis is a novel attempt to propose a methodology which could help create and profile any semi-open organisation using publicly available data about its members and how they communicate with each other. The thesis is divided into two parts. In the first part, we lay down 5 methods through which we can create a virtual image of any organisation. The second part gives out a practical application of the proposed methodology where we find insights into an organisation and the movement of its members. We use the data available about an organisation in the public domain, from their websites, newspapers and other open source data. To illustrate our methodology, we use the instance of the Sangh Parivar with its affiliates along with the parent body, the Rashtriya Swayamsevak Sangh to perform a comprehensive analysis. We performed Social Network Analysis on some of the sub-fronts of the network using the method of co-occurrence to deduce the relationship between different organs of the Sangh Parivar. The results show a significant change in the way many sub-organisations are placed in the network with varying influence. We used another step of our methodology to show a Path Dependency in the Bharatiya Janata Party by studying the trail formed by the elected Members of the Parliament in the last 30 years. Using the connections these elected MPs make with the members from outside the political fraternity along with the Path Dependency and their relationship with the other fronts of the Sangh Parivar, we attempt to give out a complete picture of the members of the people associated with the Sangh Parivar using basic computational tools and publicly available data.

Abstract

Deep learning models have evolved remarkably, and are pushing the state-of-the-art in various problems across domains. At the same time, the complexity and the amount of resources these DNNs consume has greatly increased. Today’s DNNs are computationally intensive to train and run, especially Convolutional Neural Networks (CNNs) used for vision applications. They also occupy a large amount of memory and consume a large amount of power during training. This poses a major roadblock to the deployment of such networks, especially in real-time applications or on resource-limited devices. Two methods have shown promise in compressing CNNs: (i) Binarization and (ii) Pruning. We explore these two methods in this thesis. The first method to achieve improvements in computational/spatial efficiency is to binarize (1-bit quantize) the weights and activations in a network. However, naive binarization results in accuracy drops for most tasks. In this work, we present a Distribution-Aware approach to Binarizing Networks (DABN) that allows us to retain the advantages of a binarized network, while improving accuracy over binary networks. We also develop efficient implementations of DABN across different architectures. We present a theoretical analysis of DABN to show the effective representational power of the resulting layers, and explore the forms of data they model best. Experiments on popular sketch datasets show that DABN offers better accuracies than naive binarization. We further investigate the question of where to binarize inputs at layer-level granularity and show that selectively binarizing the inputs to specific layers in the network could lead to significant improvements in accuracy while preserving most of the advantages of binarization. We analyze the binarization tradeoff using a metric that jointly models the input binarization error and computational cost. We introduce an efficient algorithm to select layers whose inputs are to be binarized. We discuss practical guidelines based on insights obtained from applying the algorithm to a variety of models. Experiments on the Imagenet dataset using AlexNet and ResNet-18 models show 3-4% improvement in accuracy over fully binarized networks with minimal impact on compression. The improvements are even more substantial on sketch datasets like TU-Berlin, where we match state-of-the-art accuracy, getting more than 8% increase in accuracies over binary networks. We further show that our approach can be applied in tandem with other forms of compression that deal with individual layers or overall model compression (e.g., SqueezeNets). In contrast to previous binarization approaches, we are able to binarize the weights in the last layers of a network, which enables us to compress a large fraction of additional parameters. The second method explored is pruning. We investigate pruning neural networks from a graph-theoretic perspective. Efficient CNN designs like ResNets and DenseNet were proposed to improve accuracy vs efficiency trade-offs. They essentially increased the connectivity, allowing efficient information flow across layers. Inspired by these techniques, we propose to model connections between filters of a CNN using graphs which are simultaneously sparse and well-connected. Sparsity results in efficiency while well-connectedness can preserve the expressive power of the CNNs. We use a well-studied class of graphs from theoretical computer science that satisfies these properties known as Expander graphs. Expander graphs are used to model connections between filters in CNNs to design networks called XNets. We present two guarantees on the connectivity of X-Nets: (i) Each node of a layer influences every node in a layer O(logn) steps away, where n is the number of layers between the two layers (ii) The number of paths between two sets of nodes is proportional to the product of their sizes. We also propose efficient training and inference algorithms, making it possible to train deeper and wider X-Nets effectively. Expander based models give a 4% improvement in accuracy on MobileNet over grouped convolutions, a popular technique which has the same sparsity but worse connectivity. X-Nets give better performance trade-offs than the original ResNet and DenseNet-BC architectures. We achieve model sizes comparable to state-of-the-art pruning techniques using our simple architecture design, without any pruning.

Abstract

With the advent of internet of things (IoT) technology, millions of smart devices will be connected to each other as well as to the internet. These connected devices will generate a huge amount of data and collaboratively dispense meaningful insights about the surrounding physical phenomenon by sensing, processing and actuating other devices to cater to various applications. In IoT networks, the sensor nodes need to continuously measure and transmit sensor data to the sink node or to the cloud directly. The communication cost in this transaction is usually high as compared to other operations of a sensor node. Keeping in mind that these battery-operated devices have a limited lifetime, efforts have been made in the wireless sensor networks (WSN) and IoT literature to improve the lifetime of the nodes. Of the many possible ways to increase the lifetime, reducing the sensor data communication to the sink node or other nodes is one of the most efficient ways. These data-transmission reduction techniques are often referred to as data reduction schemes. This thesis focuses on the popular data reduction schemes based on the classical least mean square (LMS) algorithm and the Shewhart test-based scheme. The algorithms have been implemented on a custom-designed low-cost sensor node using easily accessible hardware. The results of the proposed implementations have been evaluated upon real-time deployment of the nodes in our lab. Various results and measurements such as percentage reduction in data transmission, the current consumption of the node and estimation of the battery lifetime have been quantified based on the observations. Comparison of the LMS and Shewhart test-based data reduction schemes on different datasets encourages us to improve the accuracy of the estimated time series data at the sink node. To achieve this, a piggybacking and interpolation based scheme has been proposed along with the Shewhart test-based scheme to improve the mean square error (MSE) of the estimated data, without increasing the number of data transmissions to the sink. The proposed algorithm has also been implemented on the custom-designed sensor node. The current consumption and battery lifetime have been analyzed. It is seen that there is an inherent trade-off between the decrease in the MSE and the reduction in the battery-lifetime for the number of piggybacks chosen. However, the decrease in the battery lifetime is negligible as compared to the improvement in the accuracy. Later, the Shewhart test-based data reduction scheme is implemented on NodeMCU and integrated with oneM2M standard to show the proof-of-concept for using these algorithms for industrial and real-world IoT applications.

Abstract

Community-driven Question Answering (CQA) platforms such as Stack Overflow, Quora, Yammer, AnswerHub, etc., have gained immense popularity in recent years among the Internet users to seek, learn and share information. These platforms not only provide users with the means to discuss their queries with the community and fulfill their information needs, but also serve the purpose of knowledge-base creation. Each CQA platform encourages rich content by recognizing contributions of its users in terms of different metrics of reputation. Thus, peers on these platforms dissuade other users to post low- quality content by closing, disliking or not answering their questions. This is expected to keep the platform focused; however, the aggressive attitude of the community towards marking the posts as low- quality leads to negative impact on users’ experience of the platform (especially for new users) leading to reduced interest in the platform. Thus, to improve the user experience and maintain the content quality on the CQA platforms includes issues such as the increasing count of low-quality questions, aggressive communities and the editing process for the low-quality questions on the platforms. In this thesis, we address two issues related to the content quality management at the CQA platforms: (1) We note that editing closed questions can be time consuming, especially for new users: Often, the edited versions of the closed questions may still be found to be unsuitable by moderators and remain closed for a long time. This motivates the need of automated mechanisms that can potentially help users with appropriate feedback on their closed, disliked or unanswered questions, so that they can obtain rel-evant answers in a timely fashion without getting unnecessarily stuck waiting for moderators’ feedback.(2) We also note that another reason behind questions getting closed is duplicity. The community marks them as duplicate considering their semantic similarity with the related questions already available on the platform. Although users can search for related questions using the search & retrieval systems of these platforms before posting their questions. But most of these systems capture and return questions primarily based on syntactic similarity which may not meet the exact need of the user. To address the above-mentioned issues: First, we present a framework to assist the users in the re-opening process of closed questions on the platform. We build a predictive modeling framework that suggests to a user as to whether the edited version of his/her closed question will get successfully re-opened or not. This can assist users at large by retracting them from entering the review process with improper edits. To learn these models effectively, we analyze the closed questions of established & non-established users (determined by their reputation score). We note that the established users have higher odds of getting their closed questions reopened than non-established users. Thus, we present an approach to leverage the better editing skills of established users to train a binary classifier that determines whether the edited closed questions will get reopened or not. In order to train such a classi- fier, we identify a rich set of relevant and effective features based on exploratory analysis of a popular CQA platform, namely Stack Overflow. Through empirical evaluation of the proposed approach using publicly available data from Stack Overflow CQA platform, (i) we demonstrate that the proposed pre- dictive modeling framework can be used to estimate whether an edited version of a closed question will get reopened or not, (ii) we show that by leveraging such a framework, the users can potentially save substantial amount of time that currently goes waste while waiting for a response from the moderators. Second, to reduce the number of duplicate questions we present a mechanism to generate a set of factual questions from the accepted answers of the top-rated questions on the Stack Overflow platform. The mechanism is based on the SRL (Semantic Role Labels) parse of the sentences in the accepted answers of these questions. We demonstrate that the generated set of factual questions can potentially increase the syntactic coverage of the questions on the platform, thus yielding better results while using the search & retrieval system of the platform. Eventually, this can lead to lesser number of duplicate closed questions and improved content quality on the Stack Overflow platform.

Abstract

In this thesis we mainly focus on three major aspects of computational humour recognition. We start with categorizing humour based on the classical theories of humour along with features like theme, emotions and topics. We then look at the problem of recognizing humour in conversations and broadcasted speeches which are more complex and large than short jokes. Finally, we try to differentiate between different types of off-color humour and try to detect insulting remarks from off-colour humour in which dark humour is often misclassified as insulting humour. Most scholarly works in the field of computational detection of humour derive their inspiration from the incongruity theory. Incongruity is an indispensable facet in drawing a line between humorous and non-humorous occurrences but is immensely inadequate in shedding light on what actually made the particular occurrence a funny one. Classical theories like Script based Semantic Theory of Humour(SSTH) and General Verbal Theory of Humour(GVTH) try and achieve this feat to an adequate extent. We adhere to a more holistic approach towards classification of humour based on these classical theories with a few improvements and revisions. Through experiments based on our linear approach and performed on large data-sets of short jokes, we are able to demonstrate the adaptability and show componentizability of our model, and that a host of classification techniques can be used to overcome the challenging problem of distinguishing between various categories and sub-categories of jokes. Almost all the studies done in the field ofcomputational humour recognition has been done on datasets consisting of short jokes, tweets and puns. We try to detect humour in conversations and broadcasted speeches as they are complex and contains more contextual information when compared to short jokes. For the purpose of automatic humour detection in monologues we built a corpus containing humorous utterances of TED talks and for dialogues we analysed data from a popular TV-sitcom Friends whose canned laughter gives an indication of when the audience would react. We classified dialogues/monologues into humorous and non-humorous by using multiple deep learning methods. Our experiments on the data show that such deep learning methods outperform the baseline by 21 accuracy points respectively on the TED Talk dataset. Off colour humour is a category of humour which is considered by many to be in poor taste or overly vulgar. Most commonly, off-colour humour contains remarks on particular ethnic group or gender, violence, domestic abuse, acts concerned with sex, excessive swearing or profanity. Blue humour, black humour and insult humour are types of off-colour humour. Blue and black humour unlike insult humour are not outrightly insulting in nature but are often misclassified because of the presence of insults and harmful speech. We then provide an original data-set consisting of nearly 15,000 instances and a novel approach towards resolving the problem of separating black and blue humour from offensive humour which is essential so that free speech on the internet is not curtailed. Our experiments show that deep learning methods outperforms other n-grams based approaches like SVMs, Naive Bayes and Logistic Regression by a large margin.

Abstract

Autonomously navigating to a goal pose is central to several robotic applications such as inspection, surgery, object manipulation, transportation and more. Often navigation frameworks require the goal pose to be specified in Cartesian space and a localization module that estimates the current pose of the robot. However, in the absence of a metric-scale map and an accurate position estimation module, it is difficult to attain the correct goal pose. Visual servoing approaches, therefore represent the goal pose in the form of an image, which is more intuitive for humans and easier to obtain as compared to the 6-DoF coordinates. The image feedback from a robot's camera is further employed by visual servoing approaches for localizing the robot in image space and navigating it to the goal pose. Existing visual servoing approaches control the robot such that the overlap between the current image and the goal image is iteratively maximized. Therefore, existing approaches do not generalize while servoing to different objects or even instances from the same category. We introduce a novel problem of instance invariant visual servoing, where the robot is required to servo a given instance in the desired pose represented by a goal image of a different instance from the same category. This problem is more relevant in practical scenarios since the same robot can handle multiple object instances without changing the goal image. We initially tackle the problem of instance invariant visual servoing using a geometric approach. We propose a novel visual servoing control law based on a linear combination of 3D models representing the given object category. Our approach is capable of accommodating the deformation in shapes among object instances and the resulting controller is able to smoothly navigate the robot to the desired pose even though the instances are different. To tackle the large variation in appearance across object instances, we further propose part-based visual features that uniquely capture the locations of an object's parts in images. Advantages of using such part-aware semantics are two-fold. (i) It conceals the illumination and textural variations from the visual servoing algorithm. (ii) Semantic keypoints enable us to accurately match descriptors across instances. To circumvent the requirement of 3D models by the above-mentioned geometric solution, we propose a discriminative learning based framework for visual servoing across object instances. This approach is able to learn the desired pose from previously seen examples of goal images and is able to navigate the robot despite appearance and shape variations. Specifically, we learn a binary classifier that discriminates the goal image from images captured from other viewpoints for that object category. The classification error is then used to navigate the robot towards the desired pose. Furthermore, we design controllers for linear, kernel and exemplar Support Vector Machine (SVM) and empirically discuss their performance in the visual servoing context. To address large intra-category variation in appearance, we introduce Principal Oriented Glyph (POG) features that are easier to obtain as compared to part-based features. We next consider the problem of navigating a robot in unstructured environments using deep learning techniques. Motivated by recent breakthroughs in performance of data driven methods on recognition and detection tasks, we aim to learn visual feature representations suitable for servoing tasks in unstructured and unknown environments. In contrast to existing visual servoing approaches that require the knowledge of a scene geometry especially depth and camera parameters, we present an end-to-end learning based approach for visual servoing in diverse scenes that is agnostic to the knowledge of a scene's geometry and camera parameters. This is achieved by training a Convolutional Neural Network (CNN) over color images with synchronized camera poses. After assessing the capability of CNNs in classical visual servoing tasks. We next apply them to our instance invariant visual servoing task. However, catering to shape, appearance and viewpoints variations all together is a difficult task for a CNN. Thus, instead of directly regressing to the desired pose, we use CNNs to predict locations of part based keypoints in images so that appearance variations are decoupled. To tackle geometrical and viewpoint variations, we further present a pose induction strategy based on the part-based keypoints predicted by our network. Specifically, we reconstruct the locations of parts of the given instance in 3D, from a sequence of 2D keypoints estimated by the network and align this reconstruction with a template instance. This aligned reconstruction is then used to localize the robot and navigate it the desired pose. Finally, we employ our framework for the autonomous inspection of vehicles using Micro Aerial Vehicles (MAVs), which is vital

Abstract

Homo sapiens have evolved over centuries owing to their curiosity. We have travelled millions of miles and spent countless years in quest for answers. Fast forward to today, the invention of the internet has democratised access to enormous amounts of information at the click of a button. For the past two decades, search engines like Google have been the primary way of finding information online. However, search engines break down when a user presents complex information expressed as natural language questions. Additionally, as more people access the web from mobile devices on the move, the need for software that can interpret and answer questions becomes more important. Recently, voice assistants aim to provide us with natural and declarative access to information. The proliferation of the Internet in newer demographics has led to many new phenomena. One such phenomenon called “code-mixing” has been frequently observed in online social media and has attracted a lot of research and interest from sociolinguists. Code-mixing (CM) is the mixing of linguistic units, such as phrases, words or morphemes, from multiple languages. It is prevalent in countries where people speak multiple languages with native proficiency such as India. It helps in speeding-up communication and allows a wider variety of expression due to which it has become an accessible mode of expression on the web. Hence, there is a need for the development of software to allow these communities to communicate naturally and ask questions in code-mixed. In this thesis, we study the design of an effective Question Answering (QA) framework for QA over Knowledge Bases (KB). A given question may have multiple candidate answers amongst which we have to pick the correct one. We adopt a two-step approach: candidate (answer) generation and candidate (answer) re-ranking to answer questions. We propose a Triplet-Siamese-Hybrid CNN (TSHCNN) to re-rank candidate answers. We show experiments on the SimpleQuestions dataset which uses Freebase as the underlying KB to answer questions. We demonstrate the effectiveness of our approach on monolingual question answering task for English, beating previous benchmarks. We also study how existing QA resources for English and bilingual embeddings can be effectively used in a resource-constrained setting such as for monolingual Hindi and CM QA. We discuss the creation of our QA dataset for CM questions, study the effectiveness of our QA framework and do an in-depth analysis of approaches that work for CM questions. We show that our system that works with CM questions in its original form fares better compared to translating them and using a QA system trained on monolingual English to obtain answers.

Abstract

Scene understanding is a very crucial task in computer vision. With the recent advances in research of autonomous driving, road scene segmentation has become a very important problem to solve. Few exclusive datasets were also proposed to address this research problem. Advances in other methods of deep learning gave rise to deeper and computationally heavy models and were subsequently adapted to solve the problem of semantic segmentation. In general, larger datasets and heavier models are a trademark of segmentation problems. This thesis is a direction to propose methods to reduce annotation effort of datasets and computational efforts of models in semantic segmentation. In the first part of the thesis, we introduce the general sub-domain of semantic segmentation and explain the challenges of dataset annotation effort. We also explain a sub-field of machine learning called active learning which relies on a smart selection of data to learn from, thereby performing better with lesser training. Since obtaining labeled data for semantic segmentation is extremely expensive, active learning can play a huge role in reducing the effort of obtaining that labeled data. We also discuss the computational complexity of the training methods and give an introduction to a method called knowledge distillation which relies on transferring the knowledge from complex objective functions of deeper and heavier models to improve the performance of a simpler model. Thus, we will be able to deploy better performing simpler models on computationally restrained systems such as self-driving cars or mobile phones. The second part is a detailed work on active learning in which we propose a Region-Based Active Learning method to efficiently obtain labelled annotations for semantic segmentation. The advantages of this method include: (a) using lesser labeled data to train a semantic segmentation model (b) proposing efficient models to obtaining the labeled data thereby reducing the annotation effort. (c) transferring model trained on one dataset to another unlabeled dataset with minimal annotation cost which significantly improves the accuracy. Studies on Cityscapes and Mapillary datasets show that this method is effective to reduce the dataset annotation cost. The third part is the work on knowledge distillation and we propose a method to improve the performance of faster segmentation models, which usually suffer from low accuracies due to model compression. This method is based on distilling the knowledge during the training phase from models with high performance to models with low performance through a teacher-student type of architecture. The knowledge distillation allows the student to mimic the performance of the teacher network, thereby improving its performance during inference. We quantitatively and qualitatively validate the results on three different networks for Cityscapes dataset. Overall, in this thesis, we propose methods to reduce the bottlenecks of semantic segmentation tasks, specifically related to road scene understanding: the high cost of obtaining labeled data and the high computational requirement for better performance. We show that smart annotation of partial regions in data can reduce the effort of obtaining labels. We also show that extra supervision from a better performing model can improve the accuracies for computationally faster models which are useful for real-time deployment.

Abstract

The problem of estimating scene-depth from a single image has seen great progress recently. It is one of the foundational problems in computer vision and hence been studied from various angles. Since the advent of deep learning, most of the approaches are data driven. These methods train high-capacity models with large amount of data in an end-to-end fashion. They rely on ground-truth depth, which is hard to capture and process. Recently self-supervised methods have been proposed which rely on view supervision as an alternative. These methods minimize photometric reconstruction error in order to learn depth. In this work,we propose a geometry-aware generative adversarial network to generate multiple novel views from a single image. Novel views are generated by learning depth as an intermediate step. The synthesized views are discerned from real images using discriminative learning. We show the gains of using the ad- versarial framework over previous methods. Furthermore, we present a structured adversarial training routine to train the network, going from easy examples to difficult ones. The combination of adversarial framework, multi-view learning, and structured training produces state-of-the-art performance on unsupervised depth estimation for monocular images. We also compare our method with human depth perception by conducting a series of experiments. We investigate the existence of monocular depth cues like relative size, occlusion and height in the visual field in artificial vision systems. With quantitative and qualitative experiments, we highlight the shortcomings of artificial depth perception and propose future avenues for research.

Abstract

Sentiment is an important feature of any text. Feelings and opinions are purely subjective, unlike facts. Analyzing these opinions accurately is a challenging task. However, using the sentiment, we are looking to understand the attitude of a writer with regard to a specific topic in a piece of text. Sentiment analysis has been an important topic of research since a very long time. Simply put, the task involves a system that would be predicting (classifying) the sentiment of a given input sentence as either positive, neutral or negative. Sometimes we have more fine-grained classes and sometimes we even evict the neutral class based on what we need to use the sentiment for. The capability to automatically compute the sentiment for things such as user reviews, critique on movies, e-commerce etc is of tremendous importance. It helps organizations and businesses take multiple large scale important decisions which are based strongly on user satisfaction. Movies are one of the most prominent means of entertainment. The favourite pastime of many people would be watching movies. They provide a break from the hectic schedule a person goes throughout the day. People often prefer to express their views online in English as compared to other local languages because of many factors such as non availability of applications which makes typing in local languages easy whereas typing in English is very easy. Even if one manages to write a review in a local language there are very less number of platforms which accept such reviews. This leaves us with a very little amount of data in languages apart from English to work on. The widespread use of the Internet in recent times has led to large volumes of data related to movies being generated and shared online. People watch movies, write reviews and give ratings online. This method of broadcasting opinions has gained a lot of popularity ever since. However, this led to a decrease in the quality of opinions that were shared. Due to this, people find it challenging and difficult to browse through all the opinions. This issue of bogus and random opinions is witnessed in a lot of cases where the user can provide feedback in a quick manner such as multiple choice options, checkboxes etc. Movie ratings and genres fall in this category. Movie ratings and genres play an important role in tasks such as user movie recommendations, verifying the relationship between user-submitted reviews and ratings etc. The ability to predict the correct rating/genre of a movie would be useful considering these aspects. In this thesis, we attempt to solve problems we face in real life with the help of sentiment. One problem is encountering invalid information(wrong ratings) by using the sentiment of the movie reviews. We propose methods to predict the movie rating based on its summary. We then set out to use priors that are generally available with movie summaries in order to improve the accuracy. In order to achieve this, we consider the associated movie reviews as well while predicting the rating and provide insights on why this helps our models perform better. We use sentiment of reviews along with the summary in order to predict the rating more accurately since the sentiment captures a lot of essential information that can aid rating prediction. The majority of methods used to study NLP problems employed shallow machine learning models and time-consuming, hand-crafted features for a very long time. Many problems were encountered due to this. One of the problem is the curse of dimensionality as linguistic information was represented with sparse representations. However, with the recent popularity and success of word embeddings, sentence embeddings, neural network based models have achieved better results on various language-related tasks as compared to traditional machine learning models like SVM or logistic regression. Deep learning methods are starting to out-compete the statistical methods on some challenging natural language processing problems with singular and simpler models. We propose deep learning models to solve our problem at hand and compare the results with traditional methods. The other problem is preserving sentiment during translation. An important aspect of translation is ensuring that the complete meaning (including the sentiment/opinion) of the source text is translated appropriately. Consider product reviews, retaining the sentiment is an important aspect of translating these reviews. We cannot afford to have phrases such as “not at all good” get translated to “not good” since there is a significant difference in the sentiment of the two phrases. Most of the metrics that evaluate machine translators only consider n-grams, number of overlaps, etc in order to produce translation scores. This does not account for the preservation of sentiment. In order to solve this other problem, we propose a new metric that considers the sentiment of a sentence along with the existing me