Abstract

Homo sapiens have evolved over centuries owing to their curiosity. We have travelled millions of miles and spent countless years in quest for answers. Fast forward to today, the invention of the internet has democratised access to enormous amounts of information at the click of a button. For the past two decades, search engines like Google have been the primary way of finding information online. However, search engines break down when a user presents complex information expressed as natural language questions. Additionally, as more people access the web from mobile devices on the move, the need for software that can interpret and answer questions becomes more important. Recently, voice assistants aim to provide us with natural and declarative access to information. The proliferation of the Internet in newer demographics has led to many new phenomena. One such phenomenon called “code-mixing” has been frequently observed in online social media and has attracted a lot of research and interest from sociolinguists. Code-mixing (CM) is the mixing of linguistic units, such as phrases, words or morphemes, from multiple languages. It is prevalent in countries where people speak multiple languages with native proficiency such as India. It helps in speeding-up communication and allows a wider variety of expression due to which it has become an accessible mode of expression on the web. Hence, there is a need for the development of software to allow these communities to communicate naturally and ask questions in code-mixed. In this thesis, we study the design of an effective Question Answering (QA) framework for QA over Knowledge Bases (KB). A given question may have multiple candidate answers amongst which we have to pick the correct one. We adopt a two-step approach: candidate (answer) generation and candidate (answer) re-ranking to answer questions. We propose a Triplet-Siamese-Hybrid CNN (TSHCNN) to re-rank candidate answers. We show experiments on the SimpleQuestions dataset which uses Freebase as the underlying KB to answer questions. We demonstrate the effectiveness of our approach on monolingual question answering task for English, beating previous benchmarks. We also study how existing QA resources for English and bilingual embeddings can be effectively used in a resource-constrained setting such as for monolingual Hindi and CM QA. We discuss the creation of our QA dataset for CM questions, study the effectiveness of our QA framework and do an in-depth analysis of approaches that work for CM questions. We show that our system that works with CM questions in its original form fares better compared to translating them and using a QA system trained on monolingual English to obtain answers.

Abstract

Scene understanding is a very crucial task in computer vision. With the recent advances in research of autonomous driving, road scene segmentation has become a very important problem to solve. Few exclusive datasets were also proposed to address this research problem. Advances in other methods of deep learning gave rise to deeper and computationally heavy models and were subsequently adapted to solve the problem of semantic segmentation. In general, larger datasets and heavier models are a trademark of segmentation problems. This thesis is a direction to propose methods to reduce annotation effort of datasets and computational efforts of models in semantic segmentation. In the first part of the thesis, we introduce the general sub-domain of semantic segmentation and explain the challenges of dataset annotation effort. We also explain a sub-field of machine learning called active learning which relies on a smart selection of data to learn from, thereby performing better with lesser training. Since obtaining labeled data for semantic segmentation is extremely expensive, active learning can play a huge role in reducing the effort of obtaining that labeled data. We also discuss the computational complexity of the training methods and give an introduction to a method called knowledge distillation which relies on transferring the knowledge from complex objective functions of deeper and heavier models to improve the performance of a simpler model. Thus, we will be able to deploy better performing simpler models on computationally restrained systems such as self-driving cars or mobile phones. The second part is a detailed work on active learning in which we propose a Region-Based Active Learning method to efficiently obtain labelled annotations for semantic segmentation. The advantages of this method include: (a) using lesser labeled data to train a semantic segmentation model (b) proposing efficient models to obtaining the labeled data thereby reducing the annotation effort. (c) transferring model trained on one dataset to another unlabeled dataset with minimal annotation cost which significantly improves the accuracy. Studies on Cityscapes and Mapillary datasets show that this method is effective to reduce the dataset annotation cost. The third part is the work on knowledge distillation and we propose a method to improve the performance of faster segmentation models, which usually suffer from low accuracies due to model compression. This method is based on distilling the knowledge during the training phase from models with high performance to models with low performance through a teacher-student type of architecture. The knowledge distillation allows the student to mimic the performance of the teacher network, thereby improving its performance during inference. We quantitatively and qualitatively validate the results on three different networks for Cityscapes dataset. Overall, in this thesis, we propose methods to reduce the bottlenecks of semantic segmentation tasks, specifically related to road scene understanding: the high cost of obtaining labeled data and the high computational requirement for better performance. We show that smart annotation of partial regions in data can reduce the effort of obtaining labels. We also show that extra supervision from a better performing model can improve the accuracies for computationally faster models which are useful for real-time deployment.

Abstract

The problem of estimating scene-depth from a single image has seen great progress recently. It is one of the foundational problems in computer vision and hence been studied from various angles. Since the advent of deep learning, most of the approaches are data driven. These methods train high-capacity models with large amount of data in an end-to-end fashion. They rely on ground-truth depth, which is hard to capture and process. Recently self-supervised methods have been proposed which rely on view supervision as an alternative. These methods minimize photometric reconstruction error in order to learn depth. In this work,we propose a geometry-aware generative adversarial network to generate multiple novel views from a single image. Novel views are generated by learning depth as an intermediate step. The synthesized views are discerned from real images using discriminative learning. We show the gains of using the ad- versarial framework over previous methods. Furthermore, we present a structured adversarial training routine to train the network, going from easy examples to difficult ones. The combination of adversarial framework, multi-view learning, and structured training produces state-of-the-art performance on unsupervised depth estimation for monocular images. We also compare our method with human depth perception by conducting a series of experiments. We investigate the existence of monocular depth cues like relative size, occlusion and height in the visual field in artificial vision systems. With quantitative and qualitative experiments, we highlight the shortcomings of artificial depth perception and propose future avenues for research.

Abstract

Sentiment is an important feature of any text. Feelings and opinions are purely subjective, unlike facts. Analyzing these opinions accurately is a challenging task. However, using the sentiment, we are looking to understand the attitude of a writer with regard to a specific topic in a piece of text. Sentiment analysis has been an important topic of research since a very long time. Simply put, the task involves a system that would be predicting (classifying) the sentiment of a given input sentence as either positive, neutral or negative. Sometimes we have more fine-grained classes and sometimes we even evict the neutral class based on what we need to use the sentiment for. The capability to automatically compute the sentiment for things such as user reviews, critique on movies, e-commerce etc is of tremendous importance. It helps organizations and businesses take multiple large scale important decisions which are based strongly on user satisfaction. Movies are one of the most prominent means of entertainment. The favourite pastime of many people would be watching movies. They provide a break from the hectic schedule a person goes throughout the day. People often prefer to express their views online in English as compared to other local languages because of many factors such as non availability of applications which makes typing in local languages easy whereas typing in English is very easy. Even if one manages to write a review in a local language there are very less number of platforms which accept such reviews. This leaves us with a very little amount of data in languages apart from English to work on. The widespread use of the Internet in recent times has led to large volumes of data related to movies being generated and shared online. People watch movies, write reviews and give ratings online. This method of broadcasting opinions has gained a lot of popularity ever since. However, this led to a decrease in the quality of opinions that were shared. Due to this, people find it challenging and difficult to browse through all the opinions. This issue of bogus and random opinions is witnessed in a lot of cases where the user can provide feedback in a quick manner such as multiple choice options, checkboxes etc. Movie ratings and genres fall in this category. Movie ratings and genres play an important role in tasks such as user movie recommendations, verifying the relationship between user-submitted reviews and ratings etc. The ability to predict the correct rating/genre of a movie would be useful considering these aspects. In this thesis, we attempt to solve problems we face in real life with the help of sentiment. One problem is encountering invalid information(wrong ratings) by using the sentiment of the movie reviews. We propose methods to predict the movie rating based on its summary. We then set out to use priors that are generally available with movie summaries in order to improve the accuracy. In order to achieve this, we consider the associated movie reviews as well while predicting the rating and provide insights on why this helps our models perform better. We use sentiment of reviews along with the summary in order to predict the rating more accurately since the sentiment captures a lot of essential information that can aid rating prediction. The majority of methods used to study NLP problems employed shallow machine learning models and time-consuming, hand-crafted features for a very long time. Many problems were encountered due to this. One of the problem is the curse of dimensionality as linguistic information was represented with sparse representations. However, with the recent popularity and success of word embeddings, sentence embeddings, neural network based models have achieved better results on various language-related tasks as compared to traditional machine learning models like SVM or logistic regression. Deep learning methods are starting to out-compete the statistical methods on some challenging natural language processing problems with singular and simpler models. We propose deep learning models to solve our problem at hand and compare the results with traditional methods. The other problem is preserving sentiment during translation. An important aspect of translation is ensuring that the complete meaning (including the sentiment/opinion) of the source text is translated appropriately. Consider product reviews, retaining the sentiment is an important aspect of translating these reviews. We cannot afford to have phrases such as “not at all good” get translated to “not good” since there is a significant difference in the sentiment of the two phrases. Most of the metrics that evaluate machine translators only consider n-grams, number of overlaps, etc in order to produce translation scores. This does not account for the preservation of sentiment. In order to solve this other problem, we propose a new metric that considers the sentiment of a sentence along with the existing me

Abstract

The Discrete Fourier transform (DFT) is one of the most prominent algorithms of signal processing. The algorithm for computing the DFT effectively is known as the Fast Fourier Transform (FFT) algorithm. FFT core is the key component in many of the digital signal processing, Audio, and video signal and telecommunication systems applications and most of the advanced communication systems need an FFT core to process the Orthogonal Frequency Division Multiplexing (OFDM) function. Cooley-Tukey radix-2 FFT algorithm is one of the most popular FFT algorithms which tries to reduce the computational complexity of DFT. In recent times, the use of FFT processors is even more significant and it is rated among the top 10 algorithms with greatest influence on the practice and development of engineering. Since an FFT core is the basic module in all of these applications, the performance and accuracy of the FFT core are of prime importance. Hence, we use the IEEE 754 half precision format to represent the floating point numbers. The increasing demands for speed and performance in some real-time DSP applications make sequential systems inadequate and led to the design of special-purpose parallel FFT processors that include several arithmetic units operating in parallel. In this work, we try to implement the new Modified Radix-2 decimation-in-time (DIT) FFT algorithm with reduced arithmetic complexity which is in the form of Tangent FFT algorithm. As implementation of the Tangent FFT forms is relatively unexplored even though these FFTs give a lot of advantage by reducing the number of computations. This algorithm is implemented using a design which is similar to that of a perfect shuffle architecture as they possess high computation rates by parallel processing and low latencies. Additionally, we proposed a new bit slicing scheme for the IEEE 754 half precision floating representation format to represent the twiddle factors in the proposed architecture to obtain a better implementation of the Modified Radix-2 DIT FFT algorithm. Further, we tried to implement the FFT algorithm using optimal Canonical Signed Digit (CSD) based complex constant multipliers in an attempt to reduce the area occupied by the complex multipliers and increase the precision/SNR of the output. The entire proposed system has been designed using Verilog Hardware description language. All the simulations are performed and verified using Xilinx Vivado tools and synthesis are done on Cadence Encounter RTL compiler with a UMC CMOS 180nm technology library

Abstract

Multiple access schemes play a crucial role in determining the performance of wireless networks. Already established access schemes which were part of the first generation (1G) to fourth generation (4G) cellular networks are time division multiple access (TDMA), frequency division multiple access (FDMA), code division multiple access (CDMA), and orthogonal frequency division multiple access (OFDMA). Though these techniques provide good service with simple receivers, they lack the ability to tackle emerging challenges arising due to monumental rise in the number of devices accessing wireless channel. In this background, non-orthogonal multiple access (NOMA) is seen as a suitable candidate not only to handle the connectivity challenges but also to improve spectral efficiency significantly. NOMA has been in the research literature for long, but it has garnered a lot of interest recently and extensive research is underway. Major hurdles in making NOMA a reality is the complexity involved in realizing successive interference cancellation (SIC) and imperfections in the channel state information (CSI). Hence to make NOMA a reality, effective algorithms that addresses these issues are needed. In this thesis, we focus on system design with optimal resource allocation and relay precoder design in a cooperative relay network, which employs NOMA. The proposed optimal design aims at maximizing the weighted sum-rate. The design problems turn out to be a non-convex optimization problem. We employ minorization-maximization (MM) technique to tackle the non-convexity of the optimization problems. Then, we address the problem of maximization of weighted sum-rate (WSR) while making the system performance robust to errors in the CSI. To solve this problem, we establish a relationship between WSR and weighted mean square error (WMSE) and solve the latter problem which is simpler to obtain a solution for the former. Cognitive radio (CR) is another technology that can help in solving the spectrum scarcity problem. NOMA and CR in combination is expected to improve spectral efficiency significantly. But very few works have explored the two technologies together. So, we consider the design of secondary system working on principles of NOMA in a cognitive radio network by ensuring interference to a primary node is below the permissible threshold. Then, we modify the algorithm to take into account the errors in the CSI and thus ensuring the system performance is robust. Apart from the theoretical analyses, extensive numerical simulations have been performed in order to verify the effectiveness of our proposed designs in terms of sum-rate, weighted mean square error, convergence, etc.

Abstract

The construction in hilly regions is generally constrained by local topography resulting in vertical as well as horizontal irregularities in the buildings. These irregularities cause irregular distributions of mass and stiffness of the structures. As a result, the dynamic properties such as natural period and mode shape of such buildings vary significantly when compared to those constructed on the flat terrain. During past earthquakes like 2005 Kashmir earthquake, 2011 Sikkim earthquake, 2015 Nepal earthquake, severe damages were observed in such constructions on hill slopes. The damage attributed to these buildings was observed to be disproportionately higher when compared to those on a flat land, despite smaller intensity of shaking. This damage is attributed to several reasons viz.,a. Landslide issues, Natural period, Shear distribution in columns and Torsional effects. Therefore, there is an urgent need to understand the seismic behaviour of such buildings standing on hill slopes and include corresponding changes in codal provisions. In the current study, two important seismic parameters, Natural Period and Base Shear of the structure have been studied. Fundamental Natural Period of a building is one such important dynamic property which has its significant use especially during the design phase of the building i.e., while calculation of base shear. Hence, to understand the same the natural period study has been divided into two parts i.e., in first part, natural period of buildings constructed on hill slopes have been obtained and in the second part, parametric study was done to obtain the best fit curve. In the first part, Natural Period of nine buildings constructed on the hill slopes of Mussoorie were obtained using Ambient Vibration testing. From the recorded data, it has been observed that the existing code provisions for estimating the approximate natural period of a building are not efficient enough in the case of such buildings with a standard deviation of errors of 0.1. Since the available data is too less to derive any expression, in the second part, the data has been extrapolated by modelling 270 similar buildings in SAP 2000 by varying different parameters like angle of inclination (5o to 45o), base dimension (3x3 to 5x5) and height (3 to 7 stories). Regression Analysis has been performed on the above data to derive an Empirical Relation to estimate Fundamental Natural Period of buildings constructed on slopes. It has been observed that the standard deviation of errors of the proposed expression with respect to the recorded data has reduced to 0.05. Another major issue with the step back buildings constructed on hill slopes is the distribution of the total Base Shear acting on the structure among the respective columns. The irregular configurations of a step back building causes stiffness differences in the structure. This results in short column/uphill column attracting the maximum base shear causing severe damages to the corresponding short columns and hence the whole structure. For example, in a 3x3 bays building standing on 30o slope, it has been observed that all the short columns were attracting 118% of the total base shear acting on the structure whereas the largest ones were attracting only 1.5%. To understand and reduce these stiffness differences, different building configurations using strut elements were studied. And the configuration of struts which supports optimum distribution of stiffness differences has been identified. The short column of proposed model attracts only 38% of total base shear acting on the structure and hence reducing the design cost as well as damage vulnerability of the structure.

Abstract

Energy consumed in the building sector accounts for 20.1% of the total delivered energy consumed worldwide. In 2016, the residential and commercial sectors in the US accounted for around 40% of the total US energy consumption, which largely contributes to green house gas (GHG) emissions. Immediate action in the building sector is essential to avoid hazardous climate change and also to incorporate energy efficiency in the building design. The early design stage offers more opportunities for use of energy simulation in achieving energy efficiency because there are fewer constraints from other design decisions. Although there are a large number of building simulation tools available, these tools are mostly limited to the design development stage and not the early design stage. In the early design stage, there are a large number of variables involved, and it is difficult for design teams to consider all the solutions and take decisions because this might lead to missing some parameter combinations. In addition, in the early design stage, there are very few users of building energy simulation tools because: • Performing energy simulation requires sufficient knowledge of building physics and expertise • Simulating a large number of combinations takes significant time and computation power • After simulating a large number of combinations, extracting relevant information from output data can be tedious and difficult to understand To fill this gap, an early design methodology has been developed for energy efficient building design. The methodology includes: 1. Getting inputs from the user in a simple user interface 2. Generating input files for the simulations 3. Performing energy simulations efficiently by using parallel simulations and/or using machine learning techniques to speedup simulations 4. Identifying clusters in output data to communicate the strategies to users in the form of humanly describable rules. The contributions of this thesis are as follows: (i) Machine learning regression techniques to speed up the parametric simulations. There were two case studies performed for two different cities in which the use of two algorithms (kNN and SVR) were shown to increase speed of parametric energy simulations by approximately 40 times. (ii) Axis Aligned Hyper Rectangle (AAHR) clustering method for building energy simulation data was used to produce humanly describable or easy to understand rules for design strategies. An energy model was developed and simulated for five cities in different climates and findings from these are discussed in the thesis. (iii) Development of a tool called eDOT to demonstrate the working of early design methodology. eDOT considers five variable parameters, which are orientation, aspect ratio, window to wall ratio (WWR), overhang profile angle, and glass type. Once the user enters these parameters, backend simulations are performed in parallel on the cloud. The user is provided with interactive visualisation of output data. An algorithm is applied to find strategies in output data in humanly understandable rules. To use this tool, the user does not need not to be an expert at using any energy simulation tools. (iv) Development of the Surrogate City Finder (SCF) tool to find similar weather data based on location, latitude range, altitude range, temperature range and distance range. (v) An approach was developed using EnergyPlus simulations to calculate the equivalent SHGC for various fixed/dynamic shading configurations. For the fenestration, the prescriptive method provides maximum allowable U-value and Solar Heat Gain Coefficient (SHGC). Some modifications are allowed to SHGC if fixed horizontal overhangs and/or vertical fins are installed on the fenestration. The modified SHGC with shades is called the equivalent SHGC for code compliance requirements. In this approach, equivalent SHGC is calculated using energy simulations for a reference case (with fixed/dynamic shades) and other parametric cases with varying SHGC (without shades).

Abstract

Cameras and mobile phones have become integral part of our everyday lives as they become portable, powerful and cheaper. We capture and share hundreds of pictures and videos with our friends, family and social connections. Similarly, large volume of such visual content is generated in surveillance, entertainment, and biometrics applications. Without any doubt, people are the most important objects that dominate in these visual content. For instance, photos taken in a family event or movie videos focus around humans. It is utmost important to automatically detect, identify and analyze people appearing in images to obtain a better understanding of these content and make decisions around them. In this thesis, we consider the problem of person detection and recognition in images. This is a well explored topic in vision community with a vast literature focused on these problems. The current state-of-the-art recognition systems are able to identify people with high degree of accuracy in scenarios where images have high resolution, contain visible and near frontal faces, and recognition systems have access to sufficiently large training gallery. However, these systems need significant improvement in challenging real-world applications such as surveillance or entertainment videos where one needs to handle several practical issues such as non-visibility of faces, limitation of training samples, domain mismatch, etc in addition to other instance variations such as pose, illumination, and resolution. While there are plenty of challenges pertaining to person recognition, we are interested in some of the open challenges that are relevant from the deployment perspective in diverse recognition scenarios. We first consider people detection that is a pre-requisite for a recognition system. We detect people in images by detecting their faces through an exemplar based detector. Exemplar approach detects faces through hough voting using an exemplar training database indexed with bag-of-words method. We propose two key ideas referred as “Visual phrases” and “Contextual weighting” into the exemplar approach that improves its performance significantly. We show that visual phrases which encode dependencies between visual features are discriminative and propose a strategy to incorporate them into exemplar voting. We also introduce the notion of spatial consistency for a visual feature which weights each occurrence of a feature based on its global context. Our evaluation of popular in-the-wild face detection benchmarks demonstrate significant improvement obtained using these proposed ideas. We then focus on person recognition and consider several issues encountered in practical recognition systems. We initially address the common and important issue of “unavailability of sufficient training samples” during recognition. We propose a solution based on semi-supervised learning that can efficiently learn from a small amount of labeled data and a large amount of unlabeled data. We demonstrate how the similarities between labeled and unlabeled samples can be effectively exploited to improve the performance. We then consider the problem of “domain mismatch” between training gallery (source) and probe instances (target). We consider a recognition setup in which the objective is to identify people in a collection of probe images using a training gallery collected from different domain. We propose a novel two-stage solution that generates the labels of a few confident seed images from the target domain and propagate their labels to remaining images using a graph based framework. We evaluate our approach in several practical recognition scenarios such as movie videos and photo-albums. We then consider a different recognition scenario in which faces are not completely visible due to occlusion or people facing away from the camera. To deal with such occluded and partially or completely non-visible faces, we exploit information from other body regions such as head, upper body and body to improve the recognition. When considering different body regions, pose of different body regions pose a serious challenge. To handle the issues of unreliable facial region and pose variation, we propose a technique that learns multiple pose-specific representations from different body regions. Our approach involves training a separate deep convolutional network for each pose and then combining their predictions using adaptive weights determined by the pose of the person. Person recognition approaches based on multiple body regions however require training multiple deep convolution networks for different body regions resulting in large number of parameters with slower training and testing procedures. To overcome these, we develop an end-to-end person recognition approach based on pooling and aggregation of discriminative features from multiple body regions. Our end-to-end convolutional network pools features from several pre-determ

Abstract

Representation of words as dense vectors has recently gained traction following the introduction of word2vec model. Traditional word representation models are sparse, long and each dimension represents a word in vocabulary. Dense vectors in contrast are dense meaning most values are non zero, short meaning the dimension of vectors is very less compared to size of the vocabulary and also implicitly encode semantics within the dimensions as the dimensions no longer represent a word from the vocabulary. These advantages have made them the model of choice for various machine learning models. In this work we introduce a method to improve word embeddings and show the utility of word embeddings in tasks of controversy detection and item recommendation. In the first part we develop a method to improve word embeddings of a language using resources of some other language. Word embeddings learned from text corpus can be improved by injecting knowledge from external resources, while at the same time also specializing them for similarity or relatedness. These knowledge resources (like WordNet, Paraphrase Database) may not exist for all languages. We introduce a method to inject word embeddings of a language with knowledge resource of another language by leveraging bilingual embeddings. First we improve word embeddings of German, Italian, French and Spanish using resources of English and test them on variety of word similarity tasks. Then we demonstrate the utility of our method by creating improved embeddings for Urdu and Telugu languages using Hindi WordNet, beating the previously established baseline for Urdu. In second part we demonstrate a method to detect controversy on news issues. Detecting controversial news topics on web is a relevant problem today. It helps to identify the issues upon which people have divided opinion and is specially useful on topics such as a presidential election, government reforms, climate change etc. First we use word embeddings to find out similar news articles about a given topic. Then we perform an analysis of peoples reaction on social media to these news articles. We use sentiment analysis and word matching to accomplish this task. We show the application of our method for detecting controversial topics during the US Presidential elections 2016. The crucial step in controversy detection is to eliminate bias due to people from a particular political spectrum, so we carry out our analysis on a group of articles reporting an incident, rather than a single article. We use word embeddings to find articles reporting on a topic from different sources like left, right or mainstream. The quality of embeddings used has a direct effect on the performance of the system. In third part, we show another application of the word embedding literature where we borrow the concepts of word embeddings and retrofitting to recommend items in an aggregated marketplace.