Abstract

Code-switching is a phenomenon of mixing grammatical structures of two or more languages under varied social constraints. The code-switching data differ so radically from the benchmark corpora used in NLP community that the application of standard technologies to these data degrades their performance sharply. Unlike standard corpora, these data often need to go through additional processes such as language identification, normalization and/or back-transliteration for their efficient processing. In this thesis, we investigate these indispensable processes and other problems associated with syntactic parsing of code-switching data and propose methods to mitigate their effects. In particular, we study dependency parsing of code-switching data of Hindi and English multilingual speakers from Twitter. We present a treebank of Hindi-English code-switching tweets under Universal Dependencies scheme and propose domain adaptation techniques to efficiently leverage monolingual syntactic annotations and the annotations from the Hindi-English code-switching treebank. Firstly, we propose modifications to the parsing models which are trained only on the Hindi and English monolingual treebanks. We have shown that code-switching texts can be efficiently parsed by the monolingual parsing models if they are intelligently manipulated. Against an informed monolingual baseline, our parsing strategies are at-least 10 LAS points better. Secondly, we propose a neural stacking model for parsing that efficiently leverages part-of-speech tag and syntactic tree annotations in the code-switching treebank and the monolingual Hindi and English treebanks. We also present normalization and back-transliteration models with a decoding process tailored for code-switching data. Our neural stacking models achieve an accuracy of 90.53% for POS tagging and 80.23% UAS and 71.03% LAS for dependency parsing. Results show that our neural stacking parser is 1.5% LAS points better than the augmented parsing model and our decoding process improves results by 3.8% LAS points over the first-best normalization and/or back-transliteration.

Abstract

We demonstrate the effectiveness of an end-to-end trainable hybrid CNN - RNN architecture in recog- nizing Urdu text from printed documents, typically known as Urdu OCR , and from Arabic text embedded in videos and natural scenes. When dealing with low-resource languages like Arabic and Urdu, a major adversary in developing a robust recognizer is the lack of large quantity of annotated data. We overcome this problem by synthesizing millions of images from a large vocabulary of words and phrases scraped from Wikipedia’s Arabic and Urdu versions, using a wide variety of fonts downloaded from various online resources. Building robust recognizers for Arabic and Urdu text has always been a challenging task. Though a lot of research has been done in the field of text recognition, the focus of the vision community has been primarily on English. While, Arabic script has started to receive some spotlight as far as text recognition is concerned, works on other languages which use the Nabatean family of scripts, like Urdu and Persian, are very limited. Moreover, the quality of the works presented in this field generally lack a standardized structure making it hard to reproduce and verify the claims or results. This is quite surprising considering the fact that Arabic is the fifth most spoken language in the world after Chinese, English, Spanish and Hindi catering to 4.7% of the world’s population, while Urdu has over a 100 million speakers and is spoken widely in Pakistan, where it is the national language, and India where it is recognized as one of the 22 official languages. In this thesis, we introduce the problems related with text recognition of low-resource languages, namely Arabic and Urdu, in various scenarios. We propose a language independent Hybrid CNN - RNN architecture which can be trained in an end-to-end fashion and prove it’s dominance over simple RNN based methods. Moreover, we dive deeper into the working of its convolutional layers and verify the robustness of convolutional-features through layer visualizations. We also propose a method to synthe- size artificial text images to do away with the need of annotating large amounts of training data. We outperform previous state-of-the-art methods on existing benchmarks by quite some margin and release two new benchmark datasets for Arabic Scene Text and Urdu Printed Text Recognition to instill interest among fellow researchers of the field.

Abstract

Large scale emergencies are part of life that can require mass evacuations. Planning for such emergencies is an important component of traffic management to minimize any losses arising due to them. As human cognitive behaviour plays an important role during response to emergencies, understanding such cognitive behavior will help us plan better. The cognitive behavior of each individual driver agent may include greed (hurry), mob mentality and other unexpected biases which may slowdown the clearing up of traffic. To provide an efficient solution to this problem, we study the effect of human cognitive behaviors on traffic patterns during an emergency situation. We make the assumption that during the emergency, there are traffic police or other personnel trying to guide the vehicles so that the traffic clearance is sped up (to minimize the human or property losses), as they have a global view of the emergency and have accurate real-time updates. Another assumption is that the traffic police use the Ford-Fulkerson Algorithm, which is a standard algorithm to perform an efficient evacuation using flow model, to maximize the flow in the (traffic) network and then perform an extensive agent based analysis to study the traffic patterns that arise due to a variety of human factors and biases. While human biases can cause disruption to plans generated by the Ford Fulkerson algorithm, there are situations where we may want encourage behaviors such as greed. For example, we may want an ambulance to reach its destination at the earliest possible time hence would like it to display a greedy behavior. This implies that different vehicles may need to have different priorities. Current evacuation models, including Ford-Fulkerson and other known algorithms, overlook such a scenario, and treat all the vehicles as same. To overcome this drawback, we also focus on the problem of prioritized routing of vehicles without affecting the overall evacuation time. Prioritized routing may happen even during normal times, but it becomes even more important during emergencies, as emergency vehicles, police vehicles and large vehicles (such as buses) that carry a lot of people need to have a higher priority in terms of evacuation. To summarize, we study the following two problems in this thesis: (a) How human biases effect plans made for emergency evacuation (b) Incorporate priority of vehicles when generating plan for evacuation. Through a series of experiments performed using the well-known traffic simulator SUMO, we make the following contributions: (a) We validate that Ford-Fulkerson Algorithm is indeed efficient for clearance of vehicles in case of non-prioritized evacuation (b) Driver agents that do not follow the specified directions due to their preferences and biases lose out on an average (c) In line with our expectations, we find that having a prior knowledge and/or real-time updates about the environment can indeed offset the effect of human biases to a good extent. (d) We map the prioritized routing problem to the minimum-cost maximum-flow problem, a standard problem formulation in network flow theory. (e) We then develop the Prioritized Routing Assistant for Flow of Traffic (PRAFT) that casts the prioritized routing problem which includes a notion of priority of vehicles and priority of routes into the minimum cost maximum flow problem. (f) Through a series of experiments performed using the well-known traffic simulator SUMO, we could establish that PRAFT indeed maps higher priority vehicles to better quality routes and is monotonic in the sense that decreasing priority order of vehicles maps to a decreasing route quality.

Abstract

Consequent to the discovery of their catalytic and regulatory functions, RNA has been found to be associated with numerous biological processes including regulation of gene expression and protein synthesis. To participate in these cellular processes, RNA molecules are required to be folded into functionally competent structures. Unlike double stranded DNA, the single stranded RNA folds onto itself and thereby displays a diverse repertoire of noncanonical base pairs. Unique geometry, stability and physicochemical properties of these base pairs define the structural landscape of RNA molecules. In this regard, modification of nucleobases constitute an important feature. On one hand, they in- crease the diversity in RNA’s structural alphabets and, on the other, the subsequent changes in their respective electronic properties give rise to a new class of noncanonical base pairs. Interestingly, some nucleobase modifications remain ‘invisible’ even in the high resolution X-ray crystal structures (e.g., protonations, tautomerization, etc.), and can only be inferred through circumstantial evidences. In this thesis quantum mechanics (QM) based methods in conjunction with structural bioinformatics tech- niques have been implemented to study the putative roles of four different nucleobase modifications (e.g. metal ion binding, protonation, tautomerization and synthetic modification) in determining the geometry, stability and function of different noncanonical base pairs observed in functional RNAs. It has been illustrated here that Mg2+ binding not only can stabilize the intrinsically unstable ge- ometry of numerous important RNA base pairs, but also may fine tune their geometries and interaction energies. Among these base pairs, some are also stabilized by Class II nucleobase protonation (i.e., when the loaded proton is not sequestered between two bases). By evaluating the optimized geometries and interaction energies of such base pairs, in the presence and absence of protonation, a proof of con- cept has been established for a strategy to detect hitherto undetected occurrence of N7 and N3 proto- nated guanine base pairs. This also opens up the possibility of the discovery of undetected protonation in other base pairing systems. Consequences of nucleobase protonation has also been studied for base triples, which reveal that nucleobase protonation often leads to cooperative binding of three distantly placed nucleotides. In the context of protein synthesis, it is known that selection of wrong aminoacyl tRNA due to mismatch primarily at the first two positions of the codon-anticodon (C-AC) helix leads to missense errors. Recent crystal structures of complete 70S ribosome revealed that GU mismatches may occur at the first position of C-AC helix, where they mimic the GC like canonical geometry. It has been argued here that nucleobase tautomerization (enolic forms of uracil/guanine) is necessary for such spatial mimicry and is sufficient to maintain the overall interaction of the first C-AC pair with the monitoring residue (A1493) of 16S rRNA. In addition to the aforementioned findings, modified nu- cleobases have been studied, in the light of their important application in biomimetics and RNA based nanotechnology. In this work it has been evaluated why substitution of thymine by 6-Ethynylpyridone (a synthetic thymine analogue) increases the thermostability of double helical stretches. Overall, this thesis highlights that geometry and stability of RNA base pairs are sensitive towards different types of nucleobase modifications. Characterization of such sensitivity enriches the understanding about the sequence-structure-function relationship in RNA.

Abstract

The non invasive measurement of chromophores in blood is in great need for diagnosis of major health hazards. Frequent monitoring of blood glucose levels, along with blood urea will allow early detection of diabetes, liver dysfunction and renal diseases. The non invasive instrument used for accomplishing these tasks must meet several criteria like, accuracy and precision of the measurement. Many efforts have been made by many research groups all over the world in making the procedure as much less invasive as possible by trying to reduce the blood drawn for test to micro liters or by reducing the cross sectional area of the suction needle. The device should be portable, low-cost and reliable overtime. There are many modes of diagnosis that satisfy these criteria but most of them are still in a developmental stage. Although many methods have been proposed and are available in literature, near infrared (NIR) spectroscopy has the advantage of meeting these rigorous demands.Initially, based on the intrinsic property of the analyte molecule, the NIR spectroscopy has been carried out and the wavelength of peak absorption has been identified. Using the identified wavelength a finger clip comprising of sensor and detector has been developed for collecting the in vivo PPG (photoplethsymograph) signals. These non invasive signals are used for estimating the concentration of specific blood analyte. Various statistical techniques have been used for carrying out the calibration of the in vivo signals obtained from the photoplethsymograph based sensing circuit. Secondly, these signals are calibrated using artificial neural networks to predict the analyte concentration. As typical blood analytes, blood glucose and urea are used to establish the proposed methodology to noninvasively estimate the blood parameters. Altogether 282 patients were available for measurement of glucose out of which for 100 samples the reference values are measured using Accucheck glucometer and for remaining 182 samples the reference values are measured using traditional lab based method. For the first 100 samples the testing has been carried out after taking informed consent from patients out of which 50 patient data have been used for training the neural networks and testing has been carried out on remaining 50 patients. The training of neural networks has been done offline using MATLAB and then implemented on FPGA to minimize the design circuitry. For estimating the concentration of urea in blood, samples are collected from total of 70 patients. For this invasive urea measurement is carried out using lab based method. For estimating the blood glucose concentration, the data has been calibrated using two types of neural network architectures conventional neuron model and the inverse delayed (ID) neuron model and the results are compared. It is found that the mean square error in estimating the glucose concentration was found to be 1.89mg/dL using ID function model of neuron as against 5.48mg/dL using conventional neuron model with 10 neurons in hidden layer for each model. In a similar way the blood urea concentration has been estimated and the mean square error is found to be 2.23mg/dL. For measuring the blood urea concentration, the artificial neural network based regression analysis has been carried out. However, different components in blood, although having different absorption spectrum, there is some overlap between absorption spectra of different components. Hence the photoplethsymography output is subjected to different statistical techniques like artificial neural networks(ANN), principal component analysis(PCA). Finally, an attempt has been made to identify the optimal orientation of optode pair with respect to the skin surface using Rayleigh Rice Vector Pertubation Surface Scattering theory, surface roughness models for skin surface roughness have been created and optimal orientation of the optode pair with the skin surface has been theoritically estimated. The results of the theoritical estimation has also been suitably verified with the help of experiments both in the case of reflective and transmissive photoplethsymography.

Abstract

Advertisements (ads) are often filled with strong affective content covering a gamut of emotions intended to capture viewer attention and attempt to convey an effective message. However, most approaches to computationally analyze the emotion present in ads are based on the text modality and only a limited amount of work has been done on affective understanding of advertisements videos from the content and user-centric perspectives. This work attempts to bring together recent advances in deep learning (especially in the domain of visual recognition) and affective computing, and use them to perform affect estimation of advertisements. We first create a dataset of 100 ads which are annotated by 5 experts and are evenly distributed over the valence-arousal plane. We then perform content-based affect recognition via a transfer learning based approach to estimate the affective content in this dataset using prior affective knowledge gained from a large annotated movie dataset. We employ both visual features from video frames and audio features from spectrograms to train our deep neural networks. This approach vastly outperforms the existing benchmark. It is also very interesting to see how human physiological signals, such as that captured by Electroencephalography (EEG) data are able to provide useful affective insights into the content from a user-centric perspective. Using this time series data of the electrical activity of the brain, we train models which are able to classify the emotional dimensions of this data. This also enables us to benchmark this user-centric performance and compare it to the content-centric deep learning based models, and we find that the user-centric models outperform the content-centric models, and set the state-of-the-art in ad affect recognition. We also combine the two kinds of modalities (audiovisual and EEG) using decision fusion and find that the fusion performance is greater than either single modality, which shows that human physiological signals and the audiovisual content contain complementary affective information. We also use multi task learning (MTL) on top of the features of each kind to exploit the intrinsic relatedness of the data and boost the performance. Lastly, we validate the hypothesis of better affect estimation being able to enhance a real world application by supplying the affective values computed by our methods to a computational advertising framework to get a video program sequence with ads inserted at emotionally relevant points, determined to be appropriate based on the affective relevance between the the program content and the ads. Multiple user studies find that our methods significantly outperform the existing algorithms and are very close (and sometimes better than) human level performance. We are able to achieve much more emotionally relevant and non disruptive advertisement insertion into a program video stream. In summary, this work (1) compiles an affective ad dataset capable of evoking coherent emotions across users; (2) explores the efficacy of content-centric convolutional neural network (CNN) features for affect recognition (AR), and find that CNN features outperform low level audio-visual descriptors; (3) study user-centric ad affect recognition from Electroencephalogram (EEG) responses (with conventional classifiers as well as a novel CNN architecture for EEG) acquired while viewing the content that outperform content descriptors; (4) Examine a multi-task learning framework based on CNN and EEG features which provides state of the art AR from ads; (5) Demonstrates how better affect predictions facilitates more effective computational advertising in a real world application.

Abstract

In recent years, Internet connected devices have increased signicantly. Technologies such as Internet of Things and Cloud Computing are enabling more and more devices to connect to the Internet. With rise in number of Internet connected devices, new computational paradigms such as Edge computing and Fog Computing are emerging. People are even voluntarily providing their Internet connected devices for computation and storage. These connected devices are not only communicating with the central cloud resource, but also with other connected devices using variety of protocols. All these changes are making the network infrastructure very complex, dense and heterogeneous. In this dynamically altering and growing scenario, existing traditional network infrastructures are inadequate. To fulfill the growing data requirements, the network service providers need to update the infrastructure hardware and software parameters dynamically. They need to manage co-operation, co-ordination and co-existence among diverse network types. For this, novel self configuring resource management techniques are required. Networks built on foundations of software defined paradigm provide the solution for above mentioned challenges. In this direction, we have presented novel methods for allocating resources at different levels of network infrastructure. In the rst part, computational resources optimization for IoT devices has been done. IoT device density is increasing and current philosophy of processing requests at cloud is not appropriate for emerging IoT domains such as health care and real time control. We have considered to use variety of devices available at the network access layer. This includes the devices voluntarily given by users, dedicated edge servers and cloud infrastructure. The proposed system learns the optimal operating parameters during initial runs. Using the knowledge acquired in the learning phase, an integer linear programming problem is formulated to minimize the mean time to complete the request for all the IoT nodes. The solution of the formulated problem provides fair resource allocation for all the IoT nodes. Later, considering the unreliable nature of the voluntary devices, the learning and formulation has been extended to incorporate probability of failure of these devices. A multi-objective optimization problem has been formulated and solved using Genetic algorithm. Second part covers the economic way to configure the physical infrastructure of a software defined wireless network (SDWN). In a SDWN, the radio units can be configured dynamically. This feature gives the flexibility to change the operating parameters on the go. Resources can be allocated dynamically as per operating conditions. Utilizing these features, cost effective way to configure access layer of modern network is presented. An integer linear programming problem, with objective of cost minimization and indirect quality of service constraints, has been formulated and solved. In the third part, we have looked into the problems related to time optimization in spectrum sensing. As it is expected to have dense populated wireless IoT devices, spectrum must be utilized effectively. In a SDWN, the signal processing happens in software. It gives an opportunity to perform spectrum sensing in software using innovative ways. In our approach, the historical occupancy records of the channels are considered for sensing time allocation. The problem is formulated and solved using integer linear programming with special practical constraints. The problem is also formulated using quadratic programming method and interesting observations have been presented.

Abstract

Text-to-speech (TTS) synthesis is typically carried out in two ways: (1) By concatenating waveform segments of units (often dubbed unit selection synthesis (USS)) and (2) By predicting speech parameters from text using statistical models (also called statistical parametric speech synthesis systems (SPSS)). Most commercial TTS systems use USS approach as it produces highly natural speech. However, the USS approach requires the recorded waveforms to be stored which demands memory, but the statistical approach alleviates this by modeling the speech compactly in a parametric form. Also, using the waveform directly offers little scope to alter the characteristics to produce different varieties like speakers, genders, voice-qualities, languages, etc. On the other hand, the parameters of a statistical model can be suitably transformed to produce the desired variations. The above advantages (compactness and flexibility) come at the cost of the speech sounding slightly robotic than the unit-selection counter-part. A typical SPSS system has several components namely text feature extraction, speech parameter extraction, aligning text and speech features, a text feature-to-speech parameter regression model and a duration prediction model. Each of these components are independently hand-engineered making the SPSS system susceptible to errors in any one of them. The loss in naturalness of SPSS output has been majorly attributed to the limitations of the regression model (also dubbed acoustic model) to capture the complexity of mapping from text features to speech parameters and the representations used for text and speech data. In addition, the use of separate alignment model leads to erroneous averaging in acoustic modeling. In this thesis, we address the issues of acoustic modeling, textual representation, acoustic representation, multilingual multispeaker synthesis and end-to-end synthesis with implicit text feature extraction and alignment model within the SPSS framework. Techniques developed to address these issues include the use of recurrent neural networks for acoustic modeling, contextual representation using hidden layeroutput, unit-level acoustic representation using recurrent auto-encoder, language and speaker codes, and windowing based attention modeling for end-to-end synthesis using sequence-to-sequence (seq2seq) neural networks. The major conclusions of this thesis work are: • Recurrent neural networks improve the acoustic modeling by capturing required amount of context and generating smooth parameter trajectories than the deep neural network based SPSS. • Speaker and language codes can be used to build multilingual and multispeaker synthesizer using single recurrent neural network. Building a single model for multiple languages and speakers offers a compact model which is also capable of polyglot synthesis. • A unit-level acoustic representation can be derived using a seq2seq auto-encoder. This representation is also useful in building a unit-level SPSS. • An end-to-end TTS system can be trained from characters to speech waveform directly using seq2seq with attention model. As a result, much of the hand-engineered components of SPSS system can be substituted by a single seq2seq neural network thereby reducing the effect of independent errors.

Abstract

Advances in medical field and imaging systems have resulted in a series of devices that sense, record, transform and process digital data. In the case of human eyes this digital data is fundus images, which are images of the back part of our retina. Automatic analysis of these images is required to process large amount of data and help doctors make the final diagnosis. Retina images has 3 major visible landmarks: Optic disk(OD), macula and blood vessels. In retina images, OD appears as a bright elliptical structure, macula appear as a small dark region and blood vessel appears as dark tree branch like structure. In this thesis, we have proposed methods for detection of retina landmarks. Accurate detection of OD and macula is important as computer assisted diagnosis systems uses location of these landmarks for understanding the retina image and using clinical facts about retina for improving diagnosis. Retina landmark detection also aids in assessing the severity of diseases based on the locations of abnormalities relative to these landmarks. We first used retina atlas for OD and macula detection. The idea of retina atlas is inspired by brain 3D atlas [34]. We create 2 retina atlases: intensity atlas and probability atlas, by annotating public datasets locally. We use probabilistic atlas for OD and macula detection but detection rates and accuracy of the system is low. To achieve better detection, we than used Generalized motion patterns(GMP) [14][23] for OD and macula detection. The GMP is derived by inducing motion to an image, which serves to smooth out unwanted information while highlighting the structures of interest. Our GMP based detection is fully unsupervised and its results outperformed all other unsupervised methods. The results are comparable to that of supervised methods. The proposed GMP based system is completely parallelizable and handles illumination differences efficiently. Blood vessels are another important retina landmark and we find that the current research uses evaluation measure like sensitivity, specifity, accuracy, area under curve and matthew correlation coefficient for evaluating vessel segmentation performance. We find several gaps in current evaluation measures and propose local accuracy, which is an extension of [39]. We show that local accuracy is especially useful in settings, where segmentation of weak vessels and accurate estimation of vessel width is required.

Abstract

After the emergence of internet, online advertising has evolved as a major form of advertising in this century. The most important types of online advertising are sponsored search, contextual advertising and banner advertising. Banner advertising or display advertising is one of the predominant modes of online advertising. In this thesis, we present the approaches to improve the performance of display advertising. The three important stakeholders in display advertising are advertiser, publisher and user. The publisher manages the ad space of the website(s) and several advertisers demand ad slots for advertisements. The advertiser aims to reach/spread the ad to the maximum number of targeted users of the website whereas the publisher aims to maximize the revenue by satisfying the demands of increased number of advertisers. The users visit web pages and the corresponding ads. The major issues involved in display advertising scenario are developing efficient approaches (i) for the publisher to meet the demands of the maximum number of advertisers, (ii) to schedule the ads to ad slots of web pages (iii) for auctioning and charging of ad slots (iv) for targeting ads and (v) for managing guaranteed and non-guaranteed contracts. In this thesis, we have made efforts to develop efficient ad allocation approaches for the publisher and a scalable management framework for display ad space management. Firstly, we propose an improved approach for ad slot allocation by exploiting the notion of coverage patterns. In the literature, an approach is proposed to extract the knowledge of coverage patterns from the transactional databases. In the display advertising scenario, we propose an efficient ad slot allocation approach by exploiting the knowledge of coverage patterns extracted from the clickstream transactions. We explore the factors in adapting the knowledge of coverage patterns to benefit display advertising. Using the knowledge of coverage patterns extracted from a website clickstream transactional database, we identify supply of expected user visits to available ad slots of the website. Coverage patterns extracted from website clickstream transactional database contains a set of distinct web pages with corresponding coverage or percentage of users. With this information, we develop an efficient ad slots allocation approach which helps the publisher in selecting appropriate sets of web pages that satisfy coverage demands of advertisers and at the same time reduces repeated display of ads to users of the website. The proposed allocation framework, in addition to the step of extraction of coverage patterns, contains mapping, ranking and allocation steps. The experimental results on both synthetic and real world clickstream datasets show that the proposed approach could meet the demands of increased number of advertisers and reduces the boredom faced by user by reducing the repeated display of advertisements. Secondly, we propose a framework to harvest the pages views of the web. It can be observed that online advertising provides an opportunity for product sellers and service providers to reach customers and has become a key factor in the growth of economy. It is a major source of revenue most importantly for the search engine and social networking sites. Currently, the number of websites registered comes to a billion. Each day, a typical website receives the number of visitors ranging from hundreds to millions. In a few years, the relative association of users with internet as well as the number of websites will grow exponentially. In this context, it is possible for a product seller or service provider to reach every potential customer through display advertising. We propose that rather than managing a single website, the publisher manages the aggregated advertising space of a collection of websites. In this framework, websites can be divided into clusters based on the topic (or any other interesting measure) and the interested publishers can manage the aggregated advertisement space of the cluster. The proposed framework can provide wider opportunity for the publishers to play an active role in connecting the sellers with potential buyers. As a result, the advertisement space could be expanded significantly and it will provide the opportunity for increased number of publishers to market the aggregated advertisement space of millions of websites in an organized way. It will also help in balancing the management of banner advertising market in a transparent and controlled way. Overall, in this thesis we have shown that the knowledge of coverage patterns extracted through clickstream transactions of the websites helps in efficient allocation of available display ad space to advertisers. In addition, we have proposed a scalable plan to exploit the banner advertisement space of the entire web. We hope that the proposed approaches enthuse the ad management companies to adopt and enable sellers to connect with