Abstract

Identifying language information from speech utterance is referred to as spoken language identification. Language Identification (LID) is essential in multilingual speech systems. There are various conditions under which the performance of LID systems are sub-optimal, such as short duration, background noise, channel variation, and so on. There have been efforts to improve performance under these conditions, but the impact of speaker emotion variation on the performance of LID systems has not been studied. Hence in contrast to the previous studies, for the first time in the literature, the present work investigated the impact of emotional speech on language identification. In this work, different emotional speech databases have been pooled to create the experimental setup. A dataset of this kind was not available for LID, and is a contribution of this thesis. Additionally, state-of-art i-vectors, time-delay neural networks (TDNN), long short term memory (LSTM), and deep neural network (DNN) x-vector systems have been considered to build the LID systems. Performance of the LID system has been evaluated for speech utterances of different emotions in terms of equal error rate and Cavg. The results of the study indicate that the speech utterances of anger and happy emotions degrades performance of LID systems more compared to the neutral and sad emotions. To that effect, we investigated adaptation approaches for improving the performance of LID systems by incorporating emotional utterances in the form of adaptation dataset. Hence, we studied a prosody modification technique called Flexible Analysis Synthesis Tool (FAST) to vary the emotional characteristics of an utterance in order to improve the performance, but the results were inconsistent and not satisfactory. Therefore, we propose a combination of Recurrent Convolutional Neural Network (RCNN) based architecture with multi stage training methodology, which outperformed state-of-art LID systems. Keywords: TDNN, LSTM, DNN x-vector, Flexible Analysis Synthesis Tool, Recurrent Convolutional Neutral Network, multistage training

Abstract

Last decade has witnessed a growing role played by online platforms in enabling socio-political movements and shaping narratives. Social networking sites have evolved into being active sites of political culture and begin to function as a “public sphere”. This has also enabled users on these platforms to conduct campaigns and coordinate acts; engaging in collective user practice of “trolling” or shutting down voices. We observed a recurring form of online collective action that played out on Indian online platforms. Users organise over Twitter to initiate a collective uninstalling/down voting of an app endorsed by a celebrity they have ideological disagreements with. The logic is simple: it is a call to boycott the product the associated celebrity is endorsing in hopes of causing economic or reputational consequences (poor review, low rating, uninstalls) to the company, forcing them to disassociate with the said celebrity. This would potentially scare public figures from taking stances on controversial issues which challenges their narrative. We pick the case of an online boycott of an e-commerce application Snapdeal, initiated by the Indian Right on Twitter for our study. This boycott campaign involved calls to collectively uninstall, give poor ratings, and leave bad app reviews on App store, over their brand ambassador Aamir Khan issuing a controversial statement on increasing intolerance in India. As this statement didn’t sit well with the Indian Right, they sought to leverage reputational harm as a way to force Aamir Khan to ”apologise” for it, and also to create precedence to scare public figures on challenging the right-wing narrative in the country. We conduct a mixed methods investigation to unpack co-ordinated user practice and behaviour in such online campaigns. We investigate how platform features get appropriated to serve the goals of the campaign, and how tweet framing strategies were effective in mobilising and calling to action. Through this thesis we make two contributions, 1) provide empirical results on user behaviour and tactics on mobilising for online collective action 2) theorise and present an argument how these campaigns are able to achieve very similar user goals as platforms designed for collective action do. We conclude with implications for design on building better online civic systems that rely on engaging large number of people and, how designers need to be cognisant of appropriation of features lest they become a moderation challenge.

Abstract

The ability to successfully imitate natural processes is vital for understanding the dynamic behaviour of atoms and molecules. These include reaction rates, mechanisms, energy configurations and macroscopic thermodynamic properties. For any such simulation, it is vital that a potential is constructed which can replicate the dynamics as closely as possible. Widely used classical potentials are constructed by fitting certain parameters with experimental results and then these potentials are used to run simulations under varied conditions. A drawback of this approach is its limited transferability, so the potential might not be suitable to simulate a system under conditions that differ vastly from the parameterization conditions, for example different solvents, temperatures, pressures etc. The accuracy of an empirical force field also depends on the quality of the parameterization that was done based on various approximations and the different sources of data used. The computationally expensive nature of ab initio molecular dynamics simulations severely limits its ability to simulate large system sizes and long time scales, both of which are necessary to imitate experimental conditions. In this work, we explore an approach to make use of the data obtained using the quantum mechanical density functional theory (DFT) on small systems and use deep learning to subsequently simulate large systems by taking liquid argon as a test case. A suitable vector representation was chosen to represent the surrounding environment of each Ar atom, and a ∆-NetFF machine learning model where, the neural network was trained to predict the difference in resultant forces obtained by DFT and classical force fields was introduced. Molecular dynamics simulations were then performed using forces from the neural network for various system sizes and time scales depending on the properties we calculated. A comparison of properties obtained from the classical force field and the neural network model was presented alongside available experimental data to validate the proposed method.

Abstract

Reading and writing documents is one among the primary skills with which we gather and communicate information. With the emergence of artificial intelligence (AI), researchers are in constant pursuit to build intelligent algorithms that can bring our physical and digital worlds close to each other. One such important domain is document image analysis, where we delve into the problem of understanding content from scanned document image collections. Considering “words” as the basic unit in understanding a document, in this thesis, we address the problem of finding the best possible representation for word images. Representation learning has been a key investigation for an AI problem. The primary goal of this thesis is to learn efficient representations for word images that encode its content. An ideal representation should be invariant to multiple fonts, handwritten styles and less sensitive to noise and distortions. In the past, representations have been handcrafted, specific to modalities (printed, handwritten), and sensitive to the complexities in handwriting in multi-writer scenarios. In this work, we choose the paradigm of learning from data using deep neural networks. We take our inspiration from the fact that given large amounts of annotated data, modern deep neural networks can inherently learn better representations. In this thesis, we also relax the need for large annotated datasets by heavily capitalizing on synthetically generated images. We also introduce a novel problem of learning semantic representation for word images which encodes the semantics of the word and reduces the vocabulary gap that exists between the query and the retrieved results. The first contribution of this thesis is a simple technique to generate large amounts of synthetic data, useful for pre-training deep neural networks. This led to the creation of IIIT-HWS dataset which is now widely used in the document community. The other major contributions of this thesis are: (a) the design of a deep convolutional architecture (named as HWNet) for learning an efficient holistic representation for word images, (b) a joint embedding scheme to project words and textual strings onto a common subspace, and (c) a novel form of word image representation which respects the word form along with its semantic meaning. The learned representations are evaluated under the tasks of word spotting and word recognition. We report state-of-the-art performance on popular datasets under both modern/historical and handwritten/printed document images while keeping the representation size compact in nature. Finally, in order to validate the proposed representations of this thesis, we present some interesting use cases such as (i) finding similarity between a pair of handwritten documents images, (ii) searching for keywords from online lecture videos, and (iii) building word retrieval system for Indic scripts.

Abstract

Neural Machine Translation (NMT) is a rapidly advancing MT paradigm and has shown promising results for many language pairs, especially in large training data scenarios. But Indian to English language Machine Translation is a challenging problem, owing to multiple factors including the structural and morphological difference, in addition to the lack of sufficient parallel training data, thereby demanding efficient strategies to improve the translation quality of the NMT systems. Although NMT is a promising approach, it still lacks the ability of modeling deeper semantic and syntactic aspects of the language. In machine translation with a low-resource setting, resolving data sparseness and semantic ambiguity problems can help improve its performance. In this thesis, we investigate utilizing extra syntactic and semantic linguistic factors in the context of the NMT framework for a low resource language pair i.e. Hindi-English. We propose a new architecture to incorporate explicit linguistic input features into the state-of-the-art Transformer network and demonstrate considerable performance improvements. Despite the massive success brought by neural machine translation, it has been noticed that the vanilla NMT often lags behind conventional machine translation systems, such as statistical phrase-based translation systems, for low-resource language pairs. In the past few years, various approaches have been proposed to address this issue but not much work has been done for Indian languages in this context. In this thesis, we also present our efforts towards building efficient Neural Machine Translation systems between Indian languages (specifically Indo-Aryan languages) and English via exploring the effectiveness of Multilingual Learning and Transfer Learning. We describe techniques to leverage the language relatedness among Indo-Aryan languages to improve the translation quality for individual language pairs. We also present our new approach Multilingual Transfer Learning which outperforms the aforementioned techniques. Neural MT models are generally trained using a Maximum Likelihood Estimation (MLE) objective and are tested with sequence level evaluation metrics such as BLEU. To address this inconsistency issue, Reinforcement Learning (RL) methods have been adopted to directly optimize sequence-level objectives. In this thesis, we also present an approach for training Neural Machine Translation systems using Advantage Actor-Critic method from Reinforcement Learning. Our approach directly optimizes the model parameters with respect to the task specific scores, unlike conventional maximum likelihood estimation and is fit for problems with low resource settings, large action space & delayed rewards. We also demonstrate experiments to leverage our approach to further boost the performance of NMT systems using source & target monolingual data for a low resource language pair like Hindi-English.

Abstract

The performance of speech recognition (ASR) system degrades when there is a mismatch between training and operating environments. The presence of expressive (emotional) speech is one among the mismatches in operating environments as majority of ASR systems are trained using neutral speech. The emotional state of the speaker induces changes in the speech characteristics and effects the ASR system in practical scenarios. The goal of this thesis is to improve the performance of ASR systems in these emotional conditions. The key challenge in addressing this research problem is the lack of resources, where the existing emotional databases are limited in the number of speakers and their size. The main focus of this thesis is to create the required infrastructure to study this challenging problem for low resource Telugu language and present different exploratory studies to evaluate the accuracy of Telugu ASR systems. This thesis investigates several different techniques at various stages of the recognition process that are suitable for building an emotionally robust ASR system. In the first study, prosody modification is employed at the pre-processing level of the speech recognizer. Model-based and feature-space adaptation approaches are also analyzed towards the improvement of ASR systems. These emotion adaptation strategies were studied using various deep neural network (DNNs) architectures and shown to be effective in comparison with baseline Gaussian mixture models (GMMs). The experiments are conducted using IIT Kharagpur simulated emotion speech corpus (IITKGP-SESC) and IIIT-Hyderabad Telugu naturalistic emotional speech corpus (IIIT-H TNESC). Some studies and the major conclusions are: • Prosody parameters such as pitch, duration and energy play a vital role in the analysis of emotional speech. In the first study, prosody modification is used at the pre-processing level to convert the emotional speech to neutral speech. This prosody modified emotional speech has shown an improvement in the performance of the ASR system at the preprocessing level. • In the second study, prosody modification is used to create the training data from the given neutral to emotional speech. Different ASR models were built for the generated emotional speech along with the existing neutral speech. An emotion recognition system built using differenced prosody features is used to route speech to the corresponding ASR model. • From the above studies, we observed that the emotion data used for training resulted in better performance than pre-processing emotional speech. Hence the feasibility of extending the emotion adaptation algorithms of GMM-HMM acoustic models to DNN based models is explored using model-based and feature-space adaptation approaches in the third study. The best performance is observed for TDNN based acoustic models which use utterance level decisions as their objective function instead of a standard frame level decision. • Feature space adaptation strategies have performed better than model space adaptation techniques. The auxiliary features appended to the conventional MFCCs contain emotion specific information, which helps in better handling of emotional speech by the ASR systems. Model-based adaptation could have performed well, when sufficient emotional data is provided for adaptation. fMLLR based adaptation are effective in handling the emotion-specific information in comparison with MAP adaptation for building the ASR systems.

Abstract

The choice of platform to launch one’s product/service on in order to maximize the adoption the product/service gets with minimal marketing costs is a matter of high-stakes for product makers. This choice is especially tougher for startup founders or very early stage product makers as they often have highly limited resources and hence cannot afford to either launch on multiple platforms to maximise the opportunity for adoption or to launch on a single platform and get the decision wrong. Yet, it has been very tough and complicated for product makers to make this choice due to the lack of any framework or theoretical foundation enabling this decision. In this thesis, we attempt to create a scale that product makers can use to evaluate various platforms available to them and make the logical choice of platform in a pre-launch stage with minimal effort and costs. Through this thesis, we have evaluated seven factors namely Platform readiness index, Product platform fit, Perceived security, Perceived price, Contextual accessibility, Contextual modality compatibility and Degree of plasticity in an attempt to create a scale that product makers can use to make a decision regarding which platform would be ideal for them to launch their product/service on. These seven factors were arrived at through the learnings from the journey of a real startup, Bucker. The learnings from Bucker were further conceptually strengthened through a literature review. Two confirmatory studies were then conducted further to evaluate the validity of these seven factors to belong to the Pros Scale. The first study expanded the existing model Decomposed Theory of Planned behaviour and utilized the statistical tool structural equation modelling to answer whether or not the seven factors have an impact on the potential adoption a product/service would get when launched through a particular platform. The second study was in an attempt to answer if three of the factors, namely Degree of plasticity, Perceived security and perceived price, are differentiating between various platforms to ensure the parsimony of the PROS scale and utilized the statistical tool ANOVA for this. Through these studies, we were able to find substantial evidence for five factors namely Platform readiness index, Product platform fit, Perceived price, Contextual modality compatibility and Degree of plasticity to belong on the Pros scale . In addition to providing a practical scale early-stage product makers can use to evaluate the platform choices they have to decide which platform to launch their product/service on, we have also successfully expanded the existing D-TPB model to now include platform factors as significant influencers of the adoption and thus advancing the theoretical models in the information systems adoption research field.

Abstract

MLWare applications is an upcoming term used to refer to Software applications that use MachineLearning approaches/algorithms (in part) to address the application’s objective. The development andutilization of such applications is growing rapidly in every major sector with the increase in speedand volume of data collection/analysis. These applications tend to be complex and hence maintainingthem can be a difficult task. The complexity of the applications may be due to the inherent complexityof algorithms used or may be accidental due to the structural and dynamic relationships between thevarious sub-systems of the application. Maintaining such complex systems can a difficult task as thereare challenges such as comprehension of complex code base, lack of documentation, lack of supporttools, etc. Furthermore, such applications tend to be algorithm centric. As a result, there is a lack ofsoftware engineering rigour associated with building such applications.Researchers have started to work on integrating software engineering practices while buildingMLware applications. Applying software engineering practices, specifically pattern based approachesfor the development of MLware applications can potentially improve reliability, robustness,extensibility, scalability and other such quality attributes. Another problem with MLware applicationsis the need for an explanation associated with the decisions arrived at during various stages. This isdifficult as some Machine Learning models are black-box models and hence explainability is a difficultcharacteristic to achieve.In this thesis, developing maintainable, scalable, and explainable MLware applications usingsoftware engineering practices is described and a proof of concept implementation for the specific caseof credit risk evaluation is detailed. Microservices based architecture is used to develop the applicationwith the use of Strategy design pattern.This helps with scalability and maintainability of theapplications. A simple approach to understand the decision made by the machine learning model isalso provided. We detail the set of practices using a credit risk evaluation MLware application thathelps a loan granting officer in deciding whether to grant the loan or not. The implementation is doneusing Django framework and is deployed on Heroku server

Abstract

Social media has provided new opportunities to users for creating, accessing, and sharing information in a location independent and timeless fashion. Recent years have witnessed exponential rise in the number of individuals and organizations using social media for sharing healthcare information which is publicly accessible. Medical social media is a subset of social media, in which the interests of the users are specifically devoted to medicine and health related issues. Medical social media encompasses healthcare related texts in generic social media platforms such as WordPress, Twitter, Facebook, Quora, Instagram, YouTube as well as medical forums such as patientslikeme.com, https://patient.info/forums, doctorslounge.com and kevinmd.com/blogs. People use medical social media for a variety of reasons such as seeking answers to specific questions, giving expert advice about a particular drug or a treatment, spreading awareness, sharing experiences, reporting discoveries and findings, voicing opinions and forming communities. Apart from individual users, a lot of medical organizations such as hospitals, clinics, and insurance companies are also actively contributing to medical social media. Medical social media data plays a crucial role in several applications such as studying the unintended effects of a drug (pharmacovigilance), hiring potential participants for a clinical trial, promoting a drug, monitor public health and healthcare delivery. However, extracting information from medical social media is challenging due to various reasons. First, medical social media posts are highly noisy; they are plagued with incorrect spellings, incorrect grammar, non-standard abbreviations and slang words. Owing to this, the current medical information extraction tools fail to extract concept mentions such as imence pain in ma leg which are very frequent in medical social media. Secondly, several applications rely on human labeled examples for training supervised machine learning models. Manually creating such datasets is effort-intensive and expensive. Lastly, several applications require the persona associated with a medical social media post. Medical social media contributors belong to various persona such as patient, consultant, journalist, caretaker, researcher and pharmacist. Identifying the medical persona from the content of a medical social media post is a challenging task. In this thesis, we propose solutions for three important problems related to information extraction from medical social media. In our first contribution, we address the problem of medical persona classification which refers to computationally identifying the medical persona associated with a particular medical social media post. We formulate this as a supervised multi-class text classification task and propose a neural model for it. In order to minimize the human labeling effort, we propose a distant supervision based approach to heuristically obtain labeled examples which can be used for training the model. In our second contribution, we address the task of medical concept normalization, which aims to map concept mentions such as not able to sleep to their corresponding medical concepts such as Insomnia. We propose neural models which are capable of mapping any concept mention to its corresponding medical concept in standard medical vocabularies such as SNOMED CT1 . There are several challenges associated with existing methods for normalizing medical concept mentions. First, creating training data is effort intensive, as it requires manually mapping medical concept mentions to entries in a target lexicon such as SNOMED CT. Secondly, existing models fail to map a mention to target concepts which were not encountered during the training phase. Thirdly, current models have to be retrained from scratch whenever new concepts are added to the target lexicon, which is computationally expensive. We propose a neural model which overcomes these limitations. Our model scales to millions of target concepts and trivially accommodates a growing target lexicon size without incurring a significant computational cost. While our approach reduces the need for human-labeled examples, it does not completely eliminate it. In order to overcome this grave practical challenge we propose a distant supervision based approach to train our model. We extract informal medical phrases and medical concepts from patient discussion forums using a synthetically trained classifier and an off-the-shelf medical entity linker respectively. We use pretrained sentence encoding models to find the k-nearest phrases corresponding to each medical concept. The resultant mappings are used to train our model, which shows significant performance improvements over previous methods while avoiding manual labeling. In our third contribution, we focus on the problem of automatic simplification of medical text. Patients and caregivers are increasingly usi

Abstract

Soil act as supporting medium for buildings as loads from the buildings transfer to soil. Ability of soil to resist these loads depends on type of building and on type of soil. Existing soil tests and geotechnical investigations in practice according to code show the behavior of soil based on only type of soil and discuss results before construction of building. Behavior of soil under the action of these building loads requires attention. This study initiated to considered magnitude of load based on height of buildings and type of soil based on IS 1893 (Part-1)-2016. From earlier studies this study categorized geotechnical investigations based on each low, medium and highrise structures founded on hard, medium and soft soils respectively. Combined behavior of soil and sub-structure of building plays an important role in building safety. Soil-structure interaction (SSI) is important phenomenon which consider soil and substructure behavior together. In past case studies SSI in many of structures studied only under seismic events where behavior of soil becomes complex. It is seen that respective seismic design of these building according to IS code is not sufficient for building safety as type of soil is not considered to required extent in building design. Not only in seismic design even in general design of buildings type of soil is ignored and base of building is assumed to be fixed. This study compare fixed base and flexible base by considering low, medium and high-rise buildings founded on hard, medium and soft soils designed for IS 1893 (Part-1)-2016, fixed base and flexible base respectively. This study considered low (5-storey), medium (10-storey) and high-rise (20-storey) buildings founded on different type of soils according to IS code as hard, medium and soft soils. These 9 buildings designed for base shear obtained from IS 1893 (Part-1)-2016, fixed base and flexible base respectively show that buildings are over designed when compare with flexible base. In hard soil, building designed for IS Code is 11%, 21%, 33% overdesigned than flexible base building, in medium soil it is 9%, 21%, 33% overdesigned and in soft soil it is 25%, 45%, 59% overdesigned in all low, medium and high-rise buildings respectively. Buildings with overdesign are assumed to be safe, to check the safety of these overdesigned buildings linear dynamic analysis and pushover analysis were performed, from these analysis it is proved that flexible base design is safe and economic than IS code and fixed base design except for high-rise buildings on soft soil. For these high-rise buildings soil improvement techniques are suggested as mentioned in Appendix B.