Abstract

This thesis is a culmination of the work done on event detection, annotation and analysis. We present here the development of the detection of events on a large scale for low resource languages in two ways from a computational and a linguistic perspective. From the linguistic perspective, we discuss the creation of a language specific event annotation and representation task for Kannada, a morphologically rich resource poor Dravidian language and Hindi, a popular Indo-Aryan language. From a computational perspective, we look into leveraging information from resource rich languages and use transfer learning in order to detect events in a resource poor environment as well. We analyze events in Hindi briefly and Kannada, in depth. As most information retrieval and extraction tasks are resource intensive, very little work has been done on Kannada NLP, with almost no efforts in discourse analysis and dataset creation for representing events or other semantic annotations in the text. In this thesis, we linguistically analyze what constitutes an event in this language, the challenges faced with discourse level annotation and representation due to the rich derivational morphology of the language that allows free word order, numerous multi-word expressions, adverbial participle constructions and constraints on subject-verb relations. Therefore, this is one of the first attempts at a large scale discourse level annotation for Kannada, which can be used for semantic annotation and corpus development for other tasks in the language. On the other hand, from a processing viewpoint, detection of TimeML events in text have traditionally been done on corpora such as TimeBanks. Traditional architectures revolve around highly feature engineered, language specific statistical models. In this thesis, we also present a Language Invariant Neural Event Detection (ALINED) architecture. ALINED uses an aggregation of both sub-word level features as well as lexical and structural information. This is achieved by combining convolution over character embeddings, with recurrent layers over contextual word embeddings. We find that our model extracts relevant features for event span identification without relying on language specific features. We compare the performance of our language invariant model to the current state-of-the-art in English, Spanish, Italian and French. We outperform the F1-score of the state of the art in English by 1.65 points. We achieve F1-scores of 84.96, 80.87 and 74.81 on Spanish, Italian and French respectively which is comparable to the current states of the art for these languages. We also introduce the automatic annotation of events in Hindi and Kannada with an F1-Score of 77.13 and 67.30 respectively.

Abstract

This work presents the idea of using a space-filling fractal curve as the heating element in micro heaters. Since the heating element of a micro-heater is based on metal coils, that will be the costliest component of a micro-heater. However, if we try to increase the spacing between the metal coil, that leads to hotspots and non-uniform distribution. Hence, a cost-effective heater should have an excellent uniform temperature distribution while having minimum metal consumption. Further, non-uniform temperature distribution will give rise to the non-uniform temperature gradient between the environment and body. The geometry of the Heater plays a crucial role in its average temperature (Tµ) and uniformity. This work talks about multiple types and orders of space filling fractal curves as the geometry for heaters, modelled and simulated all of them as heaters and showed that they are on par with the typical double-spiral heat-pad in terms of several criteria like average temperature and temperature uniformity while being cost-effective due to optimal distribution of metal. This work presents modelling of modified Hilbert, Moore and Peano fractal curves as micro-heaters. In order to make a fair comparison among the heaters, a constant power of 1 W was supplied to all the heaters, also the heaters were confined to a fixed area of 30 mm x 30 mm. Finite Element Analysis (FEA) results showed that heating elements based on the geometries of Peano order 3 curve, Hilbert & Moore order 4 curves outperformed double-spiral heating element in terms of temperature uniformity with less metal consumption (cost effective), also all the geometries are modelled with same material, thickness and width. Heaters based on Peano order 3, Hilbert & Moore order 4 curves, showed a non-uniform temperature distribution (𝜎/𝜇) of 2.2%, 4.4% and 4.3% respectively whereas double spiral had a non-uniformity of 5.8%. However, the average temperature (Tµ) was lower by 10 K for Hilbert and Moore order 4 and 50 K for Peano order 3. Further, heaters one fractal order lower than these, i.e., Peano order 2, Hilbert & Moore order 3, have given the average temperature (Tµ) close to double spiral heater while consuming only half the metal that a double spiral takes, thus promising significant reduction in metal consumption. Peano order 2, Hilbert and Moore order 3, showed a a non-uniform temperature distribution of 7.24%, 11.7% and 11.6% respectively, which makes them slightly more non-uniform than the double-spiral Heater. With many fractal patterns to choose from, space-filling fractal curves can provide very good performance while being cost effective as the heating elements in a micro heater.

Abstract

The Cooperative Target Observation (CTO) problem is an optimization problem in which a set of observer agents attempt to maximize the collective time during which each target from among a set of targets, is observed by at least one observer agent within an area of interest. The CTO problem has been well pursued in the multi-agents and robotics literature due to the problem being at the core of a number of applications such as surveillance, inspection, search-and-rescue among others. However, prior works are limited to handling homogeneous target types and the sole criterion of the set of observers is to maximize the observation of targets. In this work, we expand the CTO problem to include heterogeneous target types and constrain observers to incorporate other criteria during decision making while maximizing target observation. To this purpose, we pose Wildlife Monitoring domain as a CTO problem and propose a Multi Criteria Decision Analysis (MCDA) based algorithm named MCDA-CTO, to maximize the observation of different animal species and to effectively handle multiple target types (animal species) and the multiple criteria that arise due to the targets and environmental factors. To demonstrate the working of our algorithm, we present the performance results on two settings: (a) UAV setting i.e., Unmanned Air Vehicles (UAV) are modeled as observers and (b) UGV setting i.e., Unmanned Ground Vehicles (UGV) are modeled as observers. UAVs have uncertainty in observation of targets which makes it challenging to develop a high-quality monitoring strategy while UGVs are affected by the terrain they operate in i.e., UGVs have a different maximum speed in different terrains, thus needing to make suitable trade-offs. We therefore develop monitoring techniques that explicitly take actions to improve belief about the true type of targets being observed for UAV or optimize the ability to observe targets based on terrains for UGV. Furthermore, it is often reasonable to assume that the observers may themselves be a subject of observation by unknown adversaries (poachers). Randomizing the observer’s actions can therefore help to make the target observation strategy less predictable. We provide experimental validation which shows that the techniques we develop provide a higher performance along with better randomization than state of the art approaches.

Abstract

Code-mixing (CM) refers to the mixing of various linguistic units (morphemes, words, modifiers, phrases, clauses and sentences) primarily from two or more participating grammatical systems within a sentence [10]. The development of CM NLP systems has significantly gained importance in recent times due to an upsurge in the usage of CM data by multilingual speakers. However, this proves to be a challenging task due to the complexities created by the presence of multiple languages together. The complexities get further compounded by the inconsistencies present in the raw data on social media and other platforms. In this thesis, we explore methods to efficiently parse the immensely popular Bengali-English CM which is widely spoken in India and Bangladesh. We present a neural stack-based dependency parser for CM data of Bengali-English by utilizing pre-existing resources for closely related Hindi-English CM as well as monolingual treebanks for Bengali, Hindi, and English. To address the issue of scarcity of annotated resources for the Bengali-English CM pair, we present a rule-based system to computationally generate synthetic CM dataset from parallel treebanks of Bengali and English. The generated synthetic CM data does not require an overhead of preprocessing for language identification and normalization as we get orthographic norms from standard corpora. Next, we project automatic annotations from Bengali and Hindi-English treebanks to Bengali-English using simple heuristics creating a synthetic CM treebank for Bengali-English (Syn-BE). Incorporating Syn-BE into our neural stacking parser further improves its performance. Our final model achieves an accuracy of 89.63% for POS tagging and 76.24% UAS and 61.41% LAS points for dependency parsing. This model improves parsing results by 1.37% LAS points when compared with a stacking model utilizing pre-existing Hindi-English and other monolingual resources. We also present a dataset of 500 Bengali-English tweets annotated under Universal Dependencies scheme which can be utilized for evaluation of the system as well as providing seed training data.

Abstract

Neural Machine Translation (NMT), together with multilingual formulations have arisen as the de-facto standard in translating a sentence from a source language to a target language. However, unlike many western languages, the available resources like training data of parallel sentences ortrained models which can be used to build and demonstrate applications in other domains are limited for the languagesin the Indian subcontinent. This work takes a major step towards closing this gap.In this work, we describe the development of state-of-the art translation solutions for 10 Indian lan-guages and English. We do this in four parts described below:1.Considering the Hindi-English language pair we successfully develop an NMT solution for anarrow-domain, demonstrating its application in translating cricket commentary.2.Through heavy data augmentation, we extend the above to the general domain and build a state-of-the art MT system for Hindi-English language pair. Further, We extend to five more languagesby taking advantage of multiway formulations.3.WedemonstratetheapplicationoftheNMTincontributingmoreresourcestothealreadyresource-scarce field, expanding to 11 langauges and its application in a multimodal task of translating a talking face to a target language with lip synchronization.4.Next, we improve both data-situation and performance for machine translation in 11 Indian Lan-guages iteratively to place our models in a standardized, comparable set of metrics setting up forfuture advances in the space to comprehensively evaluate and compare against.

Abstract

The widespread access to the Internet has led to a meteoric rise in audio-visual content consumption. Our content consumption habits have changed from listening to podcasts and radio broadcasts to watching videos on YouTube. We are now increasingly preferring the highly engaging nature of video calls over plain voice calls. Given this considerable shift in desire for audio-visual content, there has also been a surge in video content creation to cater to these consumption needs. In this fabric of video content creation, especially those containing people talking, lies the problem of making these videos accessible across language barriers. If we want to translate a deep learning lecture video in English to Hindi, it is not only that the speech should be translated but also the visual stream, specifically, the lip movements. Learning to lip-sync arbitrary videos to any desired target speech is a problem with several applications ranging from video translation, to readily creating new content that would otherwise require humongous efforts. However, speaker-independent lip synthesis for any voice, and language is a very challenging task. In this thesis, we tackle the problem of lip-syncing videos in the wild to any given target speech. We propose two new models in this space: one that significantly improves the generation quality and the other significantly improving on lip-sync accuracy. In the first model, LipGAN, we identify key issues that plague the current approaches for speakerindependent lip synthesis that prevent them from reaching the generation quality of speaker-specific models. Specifically, ours is the first model to generate face images that can be pasted back into the video frame. This feature is crucial for all the real-world applications where the face is just a small part of the entire content being displayed. We show that our improvements in quality lead to multiple real-world applications that have not been demonstrated in any of the previous lip-sync works. In the second model, Wav2Lip, we investigate why current models are inaccurate while lip-syncing arbitrary talking face videos. We hypothesize that the reason is weak penalization. This finding allows us to create a lip-sync model that can generate lip-synced videos for any identity and voice with remarkable accuracy and quality. We re-think the current evaluation framework for this task and propose multiple new benchmarks, two new metrics, and a Real world lip Sync Evaluation Dataset (ReSyncED). Also, using our model, we show applications on lip-syncing dubbed movies and animating real CGI movie clips to new speech. We also demonstrate a futuristic video call application that is useful for poor network connections. Finally, we present two major appli cations that our model can impact the most social media content creation and personalization and video translation. We hope that our advances in lip synthesis open up new avenues for research in the space of talking face generation from speech.

Abstract

The recent surge of data and trends in machine learning and deep learning helped to increase our understanding of language and thereby resulting in many applications. But the same is not true for many under resourced languages. Telugu is one such low resource Indian language. This work aims to shed light on the research of Telugu language. We present corpora and their analysis in the areas of Conversational Dialogue Systems, Named Entity Recognition and Emotion Prediction respectively. We present a Telugu conversational corpus, the first ever corpus to the best of our knowledge. We have built an end-to-end dialogue system using the corpus and performed various experiments with sequence to sequence encoder and attention decoder model involving word order, translation, vocabulary size, transliteration and word representations. The second is a Telugu-English code-mixed social media corpus for Named Entity Recognition(NER), the first ever corpus to the best of our knowledge. We have experimented with traditional machine learning methods such as Conditional Random Fields(CRFs), an undirected probabilistic graphical model, Decision Trees and also using a deep learning method, Long Short term Memory Networks(LSTMs). We have proposed feature functions for Named Entity Recognition which were used in the CRF. We reported an F1-score of 0.96, 0.94 and 0.95 with CRFs, Decision Trees and Bidirectional LSTMS respectively. The third is a Telugu-English code-mixed social media corpus for Emotion prediction, the first ever corpus to the best of our knowledge. We have proposed feature functions for Emotion Prediction which were used in an experiment with Support Vector Machines(SVM). We have also experimented with deep learning methods such as Long Short term Memory Networks(LSTMs) and Bidirectional LSTMs. SVM, LSTMs and Bidirectional LSTMs reported an accuracy of 58%, 60.92% and 70.74% respectively

Abstract

The aim of this research is to cultivate, nurture and establish the formal foundation of arts and their relations in association with time and space. Primarily we begin with the question that, why there exist so many forms of art? What constitutes the existence of their plural beings? How are they formally distinct from each other? And what are the formal features which connect them as well. We start with the concepts of forms and language in arts and gradually move towards conceiving the following four chapters in the thesis. Chapter 1. Origin and Foundations of Arts Chapter 2. Idea of Form, Mind, Body, and Truth Chapter 3. Blending of Sound and Vision: Study of Rāgamālā Paintings in Association with the North Indian Classical Music Chapter 4. Blending of Site and Architecture: Installation as a Form of Art The idea is to examine a creative mind by re-visiting the rational space of each art form in terms of their formal purity and origin. We use the term ‘purity’ or ‘a pure being’in the sense of exactness, something which is apt, just that and nothing else. It refers to the qualitative existence of a being and doesn’t imply any moral ground. Any work of art necessarily requires actions to move from one space to another in the imaginary horizon. It’s a kind of force for any constructive creation. Is there any creation without construction? We claim that there is no true creation without having a construction. And here comes the question of rationality. To be more specific, the rational being of every art form. Form itself is saṅskār-rahit or devoid of saṅskār-s. But any formal action is necessarily saṅskār- hit or born out of saṅskāra-s. There is an idea of ‘muse’ in the Greek mythology which refers to the spirit of arts. ‘Muse’-s and spirits are plural beings. But how and where does one get the spirit-s/ muse-s from? We instigate that the spirit- s/muse-s are born out of saṅskāra-s. Saṅskāras are sheltered in the cultivated memories. The cultivated memories are born out of repetitive act-s/ habit-s/ ādab-s etc. It gives birth of skills.The skills for imagination, judgement, re-creation and transformation in the process of creating a work of art to investigate the nature of reality beyond its physical existence. The nature of art construes the growth of a creative mind. And the nature of its construal is completely a mental phenomenon until it gets transformed into the material existence. Art can be purely a mental phenomenon. But any work of art necessarily requires a material body. And that’s how it gets visibility in the real world. So the act of giving a material body to the ideas of any creative mind needs to be examined to validate its formal identity and the nature of its transformal quality at the same time. Work of art is a re-creation or anukṛti of imagination and memory. The act of its mechanism involves a rational structure irrespective of conscious or sub-conscious state of any creative mind. We study and examine the fundamentals of arts in connection with their origins and foundations in the first chapter of our thesis. It’ll falsify the notion that the act of an artistic creation is irrational

Abstract

The Indian IT industry is being continuously disrupted by rapidly improving automation technologiesthat tend towards replacing repetitive IT support work and back-office tasks previously driven by humanactors. This increase in efficient automation-based technologies leverages software to carry out back-office tasks almost independently of human- where foreign organizations originally invested in. Thetrend of automating manual work practices in the industry has generated extreme debates over theirprojected influence on hiring patterns, employment strategies, job profiles, work practices, and the overalleconomy. Using a socio-technical framework, the thesis aims at investigating the manifold consequencesarising out of introducing automation techniques in the software Quality Testing (QT) segment of theIndian IT industry. Ethnographic methods were employed for the collection of qualitative researchdata, to identify key socio-economic factors influencing the decision to automate a testing environment.The resulting discourse has been viewed through an Information and Communications Technology forDevelopment (ICTD) lens, by primarily analyzing first-person responses of Quality Testers injectingvalue into the testing job profile and colligating these discourses with existing QT practices. Subsequently,the significance of designing improved QT work practices in combination with relevant skill-trainingstrategies has been detailed, to enable IT professionals to exploit their implicit knowledge, businessprocess knowledge, critical thinking, and computing skills while collaborating with automated QTprocedures.The second part of the thesis focuses on the reskilling and up-skilling required to take the edge off jobthreats in the IT industry owing to work automation. India is home to the largest under-25 demographicprofile in the world, requiring a widespread, skill-oriented educational model, equipping youth to thrivein highly dynamic job markets. As a response to the huge demand for technical education, a large privateskill tutoring ecosystem has sprung up in Ameerpet, Hyderabad, India, whose pedagogical model can beadopted by Massive Open Online Courses (MOOCs) to improve their underlying foundation of teachingtechniques. The thesis offers reasons as to why learners are inclined to enroll in the traditional physicalmodels of Ameerpet classrooms over online training courses. This further draws attention towards designrecommendations that can be incorporated in online learning platforms to entice learners who often getmarginalized in the more formal and competitive education system and opt for Ameerpet-like skillinghubs. The principal offering in this context would be to propose a shift in the perspective of MOOCsto include job readiness by escorting suggestions in course curriculum, course delivery, and marketingstrategies

Abstract

Livestreaming platforms enable content producers or streamers to broadcast creative content to a potentially large viewer base. Chatrooms form an integral part of such platforms, enabling viewers to interact both with streamer and amongst themselves. Streams with high engagement (many viewers and high active chatters) are typically considered engaging and often promoted to end users by means of recommendation algorithms, and exposed to better monetization opportunities via revenue share from platform advertising, viewer donations and third-party sponsorships. Given such incentives, some streamers make use of fraudulent means to increase perceived engagement by simulating chatter via fake “chatbots” which can be purchased from online marketplaces. This inorganic engagement can negatively influence recommendations, hurt streamer and viewer trust in the platform, and harm monetization for honest streamers. In this study, we tackle the novel problem of automating detection of chatbots on livestreaming platforms. To this end, we first formalize the livestreaming chatbot detection problem and characterize differences between botted and genuine chatter behaviour observed from a real-world livestreaming chatter dataset collected from Twitch.tv. We then propose SHERLOCK and BOTHUNT methods, which posits a two-stage approach of detecting chatbotted streams, and subsequently detecting constituent chatbots. Finally, we demonstrate effectiveness on both real and synthetic data: to this end, we propose a novel strategy for collecting labeled, synthetic chatter dataset (typically unavailable) from such platforms, enabling evaluation of proposed detection approaches against chatbot bahaviors with varying signatures. The SHERLOCK approach achieves 97% precision/recall on the real world dataset and +80% F1 score across most simulated attack settings and BOTHUNT achieves 86% accuracy for real world dataset and 93% accuracy across all attack settings. This thesis is a timely contribution to the area of computer science specially combating astroturfing, needed to mitigate the spread of fraudulent bot users on Live streaming Platforms. The results from this thesis can be used to build real world solutions to mitigate the spread of untrustworthy or botted streams, fake users, etc. on live streaming platforms in the future.