Abstract

The evolution of dating technologies from matchmaking only services to dating technology ‘mediums’ has brought tremendous shifts to both dating paradigms as well as dating cultures over the years. Personal ads encouraged the shift in the agency to choose partners from family to individuals and computer dating in the 60s and 70s played a hand in challenging traditional norms of limiting oneself to one partner and only from within one’s social class. These dating technologies, however, only served as matchmaking services leaving all the actual preliminary ‘dating’ or ‘gauging’ to in-person, FtF (face to face) meetings. The launch of online dating through Match.com transformed dating experiences by bringing both matchmaking and preliminary dating under one umbrella, enabling finding and gauging matches as potential partners through chat on the same medium. From newspaper ads to VHS tapes, Match.com, and now Tinder, the aim of dating technologies has remained the same - to gauge potential dates that could spark a good real-life date. In the meanwhile, fully immersive VR has evolved as a powerful, embedded, and immersive technology medium to transform dating experiences once again. While existing chat-based dating mediums like Tinder claimed the benefits of internet technologies going mobile, fully immersive VR displaces mobility in favour of bodily immersion. Each dating technology provides different mechanisms to ‘gauge’ matches depending on its affordances with online dating web and mobile apps already thoroughly explored by existing literature. However, there is no rigorous research examining ‘dating’ or romantic interaction in VR, despite social interaction being a serious topic of exploration in the HCI community. This thesis aims to push the discourse on social interaction a step further by analyzing the dynamics of romantic reciprocality in a fully immersive VR application. Through a qualitative study, the first few chapters of the thesis examine a customizable social VR application ‘RecRoom’ as a dating technology medium to analyze how dimensions of interaction - including but not limited to voice, haptics, and spatiality - influence the dynamics of dating experiences. Tinder is employed as a contrasting chat-based medium to situate and deepen our learnings about dating in VR. The principal finding from our research is fully immersive VR allows for better, efficient, and effective ‘gauging’ of matches resulting in well-informed decisions to meet (or not) virtual partners in real life than existing chat-based mediums like Tinder. We base this conclusion on a combination of factors such as environmental alleviation of pressure, spontaneity, real-time interaction, better avenues of attraction leading to improved gauging of matches, and experience of first dates. However, the very aspects of fully immersive VR that make it a better medium to gauge matches open the floodgates to ethically mixed experiences - with online harassment taking a new form and inflicting genuine harm. The penultimate chapter endeavors to close the thesis by outlining the negative aspects of fully immersive VR, particularly harassment and raises the crucial need for the revision of ethical norms governing human- VR interactions. The goal of this chapter is to highlight the ethical implications of a medium capable of affording full bodily immersion, encouraging informed and responsible development of future VR dating apps.

Abstract

Learning computational models for visual attention (saliency estimation) is an effort to inch machines/robots closer to human visual cognitive abilities. Data-driven efforts have dominated the landscape since the introduction of deep neural network architectures. In deep learning research, the choices in architecture design are often empirical and frequently lead to more complex models than necessary. The complexity, in turn, hinders the application requirements. In this work, we identify four key components of saliency models, i.e., input features, multi-level integration, readout architecture, and loss functions. We review the existing state of the art models on these four components and propose novel and simpler alternatives. As a result, we propose two novel end-to-end architectures called SimpleNet and MDNSal, which are neater, minimal, more interpretable and achieve state of the art performance on public saliency benchmarks. SimpleNet is an optimized encoder-decoder architecture and brings notable performance gains on the SALICON dataset (the largest saliency benchmark). MDNSal is a parametric model that directly predicts parameters of a GMM distribution and is aimed to bring more interpretability to the prediction maps. The proposed saliency models can be inferred at 25fps, making them suitable for real-time applications. We also explore the possibility of improving saliency prediction in videos by using the image saliency models and existing work.

Abstract

Music signal processing is a sub branch in signal processing which is a promising area these days. Music analysis based on signal processing techniques paved a new way of generation and analysis of music. Music analysis and recognition of various swaras and ragas is inherent to human understanding. Music signal processing include various areas of research such as Synthesis of Music, Transcription, Classification, Music Information Retrieval, Raga Classification, Tala Classification, Instrument/ Voice Identification, Audio Matching, Source Separation, Tonic Identification, Intonation/melodic/rhythmic analysis, Music emotion recognition etc. The concept of Raga and Tala is integral part of Indian Classical music. Raga is the melodic component while Tala is the rhythmic component in the music. Hence, classification and identification of Raga and tala is a paramount problem in the area of Music Information Retrieval (MIR) systems. Although there are seven basic Talas in Carnatic Music, a further subdivision of them gives a total of 175 ragas. There are 72 melakartha ragas and more than thousand janya ragas. Statistical and machine learning approaches are proposed in Literature Survey to classify Ragas and Talas. However, they use complete musical recording for training and testing. As part of this thesis, a novel approach is proposed for the first time in Carnatic music to classify Carnatic music recordings using repetitive structure called Thumbnails. We proposed a parallel CNN-RNN models to classify Ragas and Talas in Carnatic music using ’Thumbnails’.

Abstract

In the era of autonomous navigation, visual servoing plays a pivotal role in tasks that involve vision, control, machine learning and embedded systems. Deep learning has evolved hand-in-hand with the advancements in various technical fields and has become a part and parcel in the current technological inventions. This thesis aims at improving the current state of research in Image Based visual servoing using deep learning architectures. The proposed pipelines make an attempt to carry out visual servoing in unforeseen environments thus making them scene agnostic. We also focus on developing an unsupervised control mechanism that forecasts into a fixed time-horizon and helps in achieving faster target driven visual navigation. We mainly put forward 2 architectures. In the first architecture(chapter 3), we propose a two-fold solution: (i) We consider optical flow as our visual features, which are predicted using a deep neural network. (ii) These flow features are then systematically integrated with depth estimates provided by another neural network using interaction matrix. We further present an extensive benchmark in a photo-realistic 3D simulation across diverse scenes to study the convergence and generalisation of visual servoing approaches. We show convergence for over 3m and 40 degrees while maintaining precise positioning of under 2cm and 1 degree on our challenging benchmark where the existing approaches that are unable to converge for majority of scenarios for over 1.5m and 20 degrees. Furthermore, we also evaluate our approach for a real scenario on an aerial robot. Our approach generalizes to novel scenarios producing precise and robust servoing performance for 6 degrees of freedom positioning tasks with even large camera transformations without any retraining or fine-tuning. In the second architecture(chapter 4), we present a Deep model predictive visual servoing framework (Deep MPC) that can achieve precise alignment with optimal trajectories and can generalize to novel environments. Our framework consists of a deep network for optical flow predictions, which are used along with a predictive model to forecast future optical flow. For generating an optimal set of velocities we present a control network that can be trained on-the-fly without any supervision. This work bridges the gap between classical and learning-based control for 6-DoF image-based visual servoing (IBVS) in novel environments with continuous action and state space. We select optical flow to represent our states instead of directly working with images. The dense optical flow is predicted using deep neural networks. Subsequently, we reformulate a predictive model for forecasting the evolution of states given a sequence of actions (velocities). We then learn a recurrent control network on-the-fly in an unsupervised fashion for computing an optimal set of velocity for a given goal state. We show superior performance vis-a-vis Deep Visual Servoing methods due to a receding horizon controller even as the framework generalizes to new environments without needing to retrain or finetune. The controller regresses to a continuous space of outputs over 6-DoF. Again, through extensive simulations on photo-realistic indoor settings of the popular Habitat framework, we show significant performance gain due to the proposed formulation vis-a-vis recent state of the art methods. Specifically, we show a faster convergence and an improved performance in trajectory length over recent approaches.

Abstract

The advent of social media has immensely increased the number of opinions and arguments voiced on the internet. Social media platforms comprise a significant part of an individual’s social interaction. These interactions also generate many opinions on issues where there is a significant division—these virtual interactions, which often result in debates, manifest cases of aggression. Various online platforms like forums, blogs, and so on help users post comments and reply to other users’ comments. Some of these comments can be aggressive, hate speech, lovable, offensive languages etc. With the growing population on social media, interactions over the web have increased and have become aggressive, and related activities like cyberbullying, trolling, hate speech, etc. have also increased manifold across the globe. Thus, aggressive online behaviour incidents have become a significant source of social conflict, potentially resulting in an activity of a criminal nature. Thus, a fundamental challenge for identifying aggression on social media is to classify it from offensive or vitriolic languages. For the task of Aggression Detection, we used a Hindi-English code-mixed dataset provided for the shared task in the 1st Workshop on Trolling, Aggression and Cyberbullying (TRAC-1). Keeping these ideas in mind, we developed a system to discriminate between Overtly Aggressive, Covertly Aggressive and Non-aggressive content in texts. While research has been focused mostly on analyzing aggression, stance, and other dimensions of speech in isolation from each other, this work also attempts to gain an extensive and fine-grained understanding of aggression and figurative language use patterns when voicing an opinion. However, this task is daunting since natural language is fraught with ambiguities, and language in social media is boisterous. So, specialized techniques are required to handle issues related to these data streams’ unstructured and dynamic nature — it can be further used in various contexts to analyze and gain insights from social behaviours. Since the users on these social media platforms tend to write in an informal tone in real-time, it is relatively natural to mix languages as they ease communication. This factor could be attributed to these users being informal, being multi-lingual, or non-native language speakers. However, it adds another layer of complexity on top of the dynamic layer of social media data. This thesis explores and develops techniques that can further help us to gain in-depth insights from such data. We also present a code-mixed dataset in English-Hindi, of opinion on a politico-social issue. We annotate it across multiple dimensions: aggression, hate speech, emotion arousal, and figurative language usage (such as sarcasm/irony, metaphors/similes, puns/word-play) across varied modalities. Like vi vii the one presented, such in-depth datasets are required to analyze the not so apparent forms of verbal aggression displayed on social media and analyze the social dynamics of opinion. The thesis also hopes to understand linguistic patterns better when voicing an opinion and showing aggression. Furthermore, such datasets also facilitate classification models that leverage corpora annotated for auxiliary tasks through transfer learning, joint modelling, and semi-supervised label propagation methods.

Abstract

The flexible capacitive pressure sensors are one of the most essential and famous devices with vast applications in automobile, aerospace, marine, healthcare, wearables, consumer, and portable electronics. The fabrication of pressure sensors in a cleanroom is expensive and time-consuming; however, the sensitivity, linearity, and other performance factors of those pressure sensors are exceptional. Moreover, sometimes we require sensors that are not expensive and can be fabricated rapidly where the other performance factors don’t need to be highly remarkable. In this modern era, household materials and DIY (Do-it-yourself) techniques are quite helpful, highly utilized. They are recommended to fabricate low-cost sensors and healthcare devices for personalized medicine and low-cost consumer electronics. Different flexible capacitive pressure sensors are presented and experimentally characterized for acoustic and air-pressure monitoring in this thesis. The design criteria of a cantilever-based capacitive pressure sensor are discussed. The three different designs are analysed with aspect ratios of 1.5, 1.0, and 0.67. The sensor with an aspect ratio of 0.67 shows maximum sensitivity (mechanical and electrical), better response time, and the 1st and 2nd mode of resonant frequencies is comparatively less than the other two. The cantilever designs are susceptible to slight pressure; therefore, the diaphragm-based normal mode capacitive pressure sensor is introduced in the second chapter, which defines the design criteria of diaphragm shapes. The five different diaphragms analysed are circular, elliptical, pentagon, square, and rectangular shapes. The circular capacitive pressure sensor shows maximum sensitivity, however, maximum non-linear response. The rectangular capacitive pressure sensor has a maximum linear response; however, the sensitivity is the lowest. However, to design the pressure sensor for bit higher pressure monitoring and increase the linear output regime, we analyse the single and double touch mode capacitive pressure sensors. The single-touch mode capacitive pressure sensor shows linear response for 8 – 40 kPa however double touch mode capacitive pressure sensor shows the linear response for 14.24 – 54.9 kPa. The presented sensor offers high sensitivity and an accurate response; therefore, we can utilize these sensors for various applications like tire-pressure monitoring, hear-rate, asthma monitoring, etc. The capacitive pressure sensor is chosen because of its advantages like low-temperature drift, high sensitivity, and ease of design.

Abstract

Semiconductor industry has long struggled to develop systems that operate at higher speeds, thereby reaching giga-hertz or even tera-hertz in the recent times. However, newly emerging applications like biomedical, environmental monitoring, surveillance etc., place contrast requirements to these high speed systems. These applications do not require higher speeds, however they do not have access to wall power (in most cases) and must resort to energy harvesting, miniaturized batteries, thereby placing power consumption as the bottleneck. Additionally, systems powered through energy harvesting must be able to work with lower supply voltages, while systems powered by miniaturized batteries must be able to work for wide supply voltages precluding the need of voltage regulator. For biomedical applications, these systems should also occupy lower form factor, so that they could lead to less invasive surgeries. Temperature is one of most important sensing modality for any IoT system, and hence temperature sensor must be designed keeping the above constraints in mind. Consequently, we propose 47nW temperature sensor in TSMC 180nm technology node and 419pW temperature sensor in UMC 55nm technology node. Both the sensors are based on completely different architectures. In TSMC 180nm technology, temperature is transduced to delay using temperature characteristics of resistance and capacitance. The architecture is digital friendly, since delay can be easily converted to digital code. Since temperature to delay conversion depends mainly on passive elements, the architecture is highly resilient to supply variations. To further push the power consumption from nano-watt to sub nano-watt regime and reduce the area further, we exploit the temperature dependency of gate-leakage in lower technology nodes. A process and supply-invariant proportional-to-absolute-temperature voltage (VP T AT ) is generated by using BJTs and gate leakage characteristics in UMC 55nm technology. The obtained VP T AT is highly resilient to process and supply variations and occupies lower form factor. Biasing circuits being common to any analog circuits, must also be designed for the above constraints. Consequently, we propose 50nA current reference in TSMC 180nm technology node and 60pA current reference, 600mV voltage reference in UMC 55nm technology node. Both the current references are based on completely different architectures. In TSMC 180nm technology, we design the 50nA current reference by cancelling the temperature coefficient of voltage and temperature characteristics of resistance. A voltage bias circuit is designed in a way that the obtained current reference temperature coefficient is independent of process variations, necessitating only single point trim. However, this architecture cannot be used for designing sub-nW currents, as it requires impracticably high resistances. To push the power consumption of current reference for sub-nW regime without compromising on area, we exploit the temperature characteristics of gate leakage in lower technology nodes. The proposed 60pA current reference is the first sub-nW current reference that solves the problem of exponential power consumption with temperature and process. With a slight modification to this architecture, we develop a 600mV voltage reference for sub-nW regime. Both these sub-nW voltage and current references require only single-point trim. All the above mentioned voltage and current references work for wide temperature range and show high supply resilience over wide supply ranges. Sleep mode timers are essential to any duty cycled IoT nodes for synchronizing data transmission and data reception. Although active power consumption is reduced by duty cycling, sleep mode power consumption of timers becomes a bottleneck in scaling the overall system power consumption. These timers are also expected to be robust to supply and temperature variations, as error in frequency necessitates additional guard band time, thereby increasing the overall system power consumption. RC oscillators are preferred due to their excellent temperature and supply stability, However scaling power consumption to sub-10nW, requires impracticably high resistances in the order of hundreds of mega ohms/giga ohms. Although gate leakage in thin oxide transistors serve as effective replacements for large resistance in lower technology nodes, TSMC 180nm does not have gate leakage and requires large physical resistances. Consequently, we propose a resistance amplifier, a CMOS circuit that realizes the characteristics of large resistance using small resistance, thereby saving area and hence cost. This resistance amplifier is substituted in the conventional offset compensated RC oscillator architecture, and the overall functionality of the system is evaluated.

Abstract

In urban driving scenarios, forecasting future trajectories of surrounding vehicles is of paramount importance. It is an essential component in the autonomous driving pipeline as the forecasted trajectories are used as inputs in the planning and control modules of the autonomous vehicles. While several approaches for the problem have been proposed, the best-performing ones tend to require extremely detailed input representations (e.g., image sequences). As a result, such methods do not generalize to datasets they have not been trained on. On the other end of the representation spectrum, some approaches represent the vehicle of interest in a scene in terms of geometric properties such as 3D location w.r.t ego vehicle, rotational parameters, and velocity. These approaches suffer from a lack of semantics / contextual information and cannot produce the most accurate forecasts. In this thesis, we propose intermediate representations that are particularly well-suited for trajectory forecasting. We show that these intermediate representations, generated using off-the-shelf computer vision methods for object detection and semantic segmentation, provide the right balance between competing features of expressiveness and generalizability. As opposed to using texture (color) information from images, we condition on semantics and train an autoregressive model to accurately predict future trajectories of traffic participants (vehicles). We demonstrate that semantics provide a significant boost over techniques that operate over raw pixel intensities/disparities. Uncharacteristic of state-of-the-art approaches, our representations and models generalize across different sensing modalities (stereo imagery, LiDAR, a combination of both), and also across completely different datasets, collected across several cities, and also across countries where people drive on opposite sides of the road (left-handed vs. right-handed driving). We also conduct detailed ablation studies to understand which semantics generalize across datasets and demonstrate their importance in the generalizability of the model. Additionally, we demonstrate an application of our approach in multi-object tracking (data association).

Abstract

Few resource-rich languages like English and French have been extensively analyzed for Natural Language Processing (NLP) tasks. Domain expertise is essential requirement for studying the properties of a language to build linguistic resources, prepare annotated training data, select appropriate features, configure parameters and set exceptions in the systems for building models for solving NLP tasks of a language. And to build data driven models like, deep learning models, huge amount of annotated or untagged training data is required. So, handling each of the remaining 6500+ “resource-poor” languages would require the same amount of intensive effort, expertise, expenses, time and large training datasets. Hence, it is required that we build domain independent and language independent data driven systems which can work reasonably and effectively with less amount of training data without requiring domain expertise. For this purpose, in this thesis, we propose generic concepts and data driven methods which can be used to build systems for solving the NLP tasks of resource-poor languages. We propose a generic associative classification approach called associative context classification which we have developed using our proposed context based list concept that groups items of some specific context and other proposed parameters and concepts. In our research, we have demonstrated the application of this proposed approach in developing solutions to a few representative Natural Language Processing tasks. We have focused on developing semi-supervised methods using small sized annotated data. Our methods perform well even with less amount of training data without using domain knowledge explicitly and hence, are especially suitable for resource-poor languages which lack domain resources. Our proposed approach is based on associative classification and on the one sense per collocation hypothesis which states that the sense of a word in a document is effectively determined by its context. Hence, our proposed approach can be applied for NLP tasks which depend on collocation property. We have validated the utility of our proposed approach for NLP tasks of resource-poor languages by successfully applying it for developing generic methods for Part-of-Speech tagging and Word Sense Disambiguation tasks. Part-of-speech (POS) tagging is a NLP classification task that assigns a POS tag or other lexical class marker to an item or to each item in the sentence. Here, we use the term “item” to represent all the words and tokens of a language. All the available POS taggers including the state-of-the-art taggers require training data and linguistic resources like dictionaries in large quantities. These taggers do not perform well for resource-poor languages. So, there is a need to develop generic semi-supervised tagging methods which use untagged corpus and require less or no lexical resources. Most of the existing vi vii semi-supervised techniques require large untagged corpus, while for many resource-poor languages, even obtaining a small untagged corpus is hard. Word sense disambiguation (WSD) is a classification task which involves determining the correct meaning of each word in a sentence or phrase based on the neighboring context items. Automated WSD methods use knowledge structures like, WordNet and dictionaries and hand crafted features and rules crafted by domain and linguistic experts from the training data. This is a costly and time taking process and requires extensive amount of domain resources and linguistic expertise. These requirements make it difficult to design a WSD algorithm for resource-poor languages and hence domain independent methods are needed to be developed for this task also. For our experiments, we have cleaned and prepared resource-rich English, resource-moderate Hindi and resource-poor Bengali, Marathi, Tamil, Telugu and Urdu language datasets for POS tagging experiments and English, Hindi and Marathi language datasets for WSD experiments. As part of our research work, we have developed: • Two semi-supervised POS tagging methods using proposed associative context classification approach. • Various ensemble POS tagging methods using proposed associative context classification approach, Support Vector Machine, Conditional Random Field, Decision Tree and Semi-supervised Condensed Nearest Neighbor method. • One semi-supervised associative WSD model using proposed associative context classification approach. • One ensemble WSD model using proposed associative context classification approach and Support Vector Machine. • One negation rule finding algorithm for the POS tagged data to find annotation errors from the tagged data.

Abstract

Analytic reasoning differences, as gauged from intelligence metrics, in students engaged in streams requiring a predominantly divergent (arts) or convergent thinking (science and engineering) is a topic of interest. In this thesis an attempt is made to examine the differential application of solution analysis in standard non-verbal intelligence tests by fine arts and engineering students, two cohorts with distinct academic coursework, skill training and attributed differences in their thinking. As understood from previous research, intelligence being a multi-dimensional entity, can be broadly categorized into two divisions (Cattel,1963) - a) Fluid Intelligence (Gf) b) Crystallized intelligence (Gc). The former refers to the ability of humans to solve novel problems and is measured using tests requiring induction, deduction and reasoning. The latter defines the skill of humans in applying existing knowledge and experience, mostly evaluated by verbal comprehension and working memory tests. There are debates about fluid intelligence being an innate ability or genetic factor but measures of Gf have been observed to be affected by various factors such as socio-economic differences, gender, skill training etc. Keeping in context the larger debate on the role of each factor and the correlations, the aim of my study is to find whether skill training creates a difference in performance in a Gf test. Ravens Progressive Matrices (RPM) - a standard measure of fluid intelligence was used to examine not just the score-wise performance but the analytical processes applied and problem solving technique. The premise for considering fine arts students for a fluid intelligence test with shapes, patterns and constrast, is their artistic talent to create identified visual patterns, manipulate geometrical shapes and sharper contrast elimination. Other reasoning applied for the selection of this participant cohort for comparison is that they differ in their thinking style based on Hudson’s theory (1966), that this variation in thinking style is reflected in choice of profession/education. That is, this theory proposed that convergent thinkers display a rule-based thinking and engage in science, mathematics while divergent thinkers are defined as engaging in freethinking and excel in arts and/or humanities. Various arguments have been made on the accuracy of this distinction, fine arts such as music also have certain set of rules (mathematical) while fields such as engineering encourage divergent thinking in design and applications. Hence, in my work I extend the research on this theory by exploring the approach towards analytical problem solving and make inferences on the thinking styles as per the above-proposed theory, with two distinct cohorts – computer science and fine art students. To examine further the role of academic skill training, the engineering students were grouped into art-trained and non art-trained sets A difference in performance just based on numerical data such as scores would be insufficient. Insights of the approach to solution can be extracted from the differences in visual strategies applied with eye-tracking technique as valid method. This technique has been established in literature as efficient for analyzing attention patterns, which is said to reveal the underlying cognitive processes applied. Hence in this study, visual search strategy (scan-path) was inspected to extract the approaches towards a problem solution for each item of the RPM test. Additional measurements included analysis of fixation duration and area of interest in the matrix elements and options spaces. The selection of RPM for the current study is based on established concepts : RPM being a patternbased test, studies have indicated that visual information processing or visual strategies might be crucial to understand and solve these questions. To further identify the underlying differential visual strategies applied, the Navon stimuli and Embedded Figures Task (EFT) were also administered. Navon stimuli allows for insights to the variations in visual information processing technique, principally global vs. local processing and the Embedded Figures Test is utilized to analyze field independence and its association to thinking style. The results show that engineering students performed better than art students in terms of score (T-test: p <0.0001) while no significant difference was seen among art-trained and non art-trained engineering students (p >0.05). Analysis of eye-tracking data showed considerable differences in the various performance metrics such as time and fixations. Scan-paths generated from fixation mapping were compared within-subject and for each question. First, three major scan-path directions that indicated pattern identification variations were observed - Horizontal, Vertical and Defocused [elements of matrix scanned in no particular direction]. On an average, artists pref