Abstract

Past decade witnessed an explosive growth in the amount of unstructured data, especially in the public domain, mainly due to Web 2.0 and social media. This has created a need for applications that extract structured information from such noisy data. Automatic extraction of structured information from unstructured data is called information extraction. Structured information thus extracted is stored in a Knowledge Base. The knowledge base stores facts about entities like name, type and other attributes. On the one hand, the information extraction task utilizes the facts of the entity stored in knowledge base to refine the extraction process, while on the other, the facts extracted refines the knowledge base facts further. Thus knowledge base provides structure and guidance to the extraction task, and gets enhanced by the results of the extraction task. Here we see that the tasks of entity extraction and knowledge base enhancement are mutually dependent and mutually beneficial. Hence this thesis proposes methods to enhance both the tasks, in an effort to build a strong and sound named entity extraction system. Documents typically talk about multiple named entities. All the named entities mentioned in the document are not equally important to the content of the document. Named entities that are important to the document are called salient named entities. In this thesis we propose a method to identify salient named entity of the document. Importance of a named entity can also be judged by understanding how the named entity is semantically related to other named entities mentioned in the document. We propose a method to identify the presence of such semantic relations within named entities. For example, semantic relation like attribute or category in a product title. Understanding the salience of a named entity and its semantic relations help the Named Entity Extraction task in extracting the important information from text, while filtering out unimportant information. Performance of Named Entity Extraction methods depend on the size and structure of the context of the named entity mention in the text. While bigger size and better structure of context results in improved performance of the Named Entity Extraction, lower size and poor structure of context results in reduced performance. We propose three different Named Entity Extraction approaches tailored to the varying size and structure of the context in this thesis. The proposed methods perform on par with state-of-the-art methods with improved latency. Named Entity Extraction approaches that work on lesser context and poorer structure, increasingly depend on non-textual signal like global coherence of entities in the knowledge base. Conventional EL performs well for popular entities but performs poorly for less popular (a.k.a tail) entities, because conventionally EL methods depend on richness of entity neighborhood in the Knowledge Graph (KG). In this context ‘KB’ refers to Knowledge Base like Wikipedia and ‘KG’ refers to Knowledge Graph like Wikipedia Hyperlink Graph. Tail entities have sparse entity neighborhood and hence EL methods perform poorly on them. In this thesis we propose ELDEN, an EL system that overcomes the degree sparsity problem of tail entities. ELDEN enriches entity’s neighborhood in a KG by extracting high quality mentions of entity from a web corpus using Pointwise Mutual Information (PMI) measure. ELDEN outperforms state-of-the-art EL systems while achieving significant improvement in linking tail entities, achieving best results on CoNLL and TAC datasets. We follow up the discussion on ELDEN with a discussion on information retrieval task that is improved by use of KB entities. We propose a novel method for enhancing classification performance of research papers into ACM computer science categories using KB entities, both Wikipedia and Freebase entities. All through this thesis we present five methods of improving Named Entity Extraction using Knowledgebases. We conclude the thesis by looking at how Knowledgebases can be improved with Named Entity Extraction. We review and analyze the main approaches of New Entity Identification (NEI) in Named Entity Extraction systems. We analyze the features and share insights from reproducing state-of-the-art results, suggesting future improvements.

Abstract

The Cooperative Target Observation (CTO) problem has been of great interest in the multi-agents and robotics literature due to the problem being at the core of a number of applications for example surveillance, inspection and search-and-rescue. In CTO problem, the observer agents attempt to maximize the collective time during which each moving target agent is being observed by at least one observer agent in the area of interest. The area of interest can include airport, border patrolling or public events. Micro-drones are typically used for such operations as with large gatherings and protests, or even when monitoring individuals, it is often difficult to maintain an overview. They can be pretty useful for surveillance allowing for a (top) view of the happenings on ground. The CTO problem in most of the prior works is solved by finding the best suited strategy for observer agents, considering the target agent’s movement to be Randomized. Given our focus on surveillance domain, we modify this assumption to make the targets strategic and present two target strategies namely Straight-line strategy and Controlled Randomization strategy. We then modify the observer strategy proposed in the literature based on the K-means algorithm to introduce five variants over the K-means algorithm and provide experimental validation across the strategies. In surveillance domain, it is often reasonable to assume that the observers may themselves be a subject of observation for a variety of purposes by unknown adversaries whose model may not be known. Randomizing the observer’s action can help to make their target observation strategy less predictable, so that the unknown adversary cannot harm the observer agent. As the fifth variant, we therefore introduce Adjustable Randomization into the best performing observer strategy among the four variants over K-means algorithm, where the observer can adjust the expected loss in reward due to randomization depending on the situation.

Abstract

Understanding the precise 3D structure of an environment is one of the fundamental goals of computer vision and is challenging due to a variety of factors such as appearance variation, illumination, pose, noise, occlusion and scene clutter. A generic solution to the problem is ill-posed due to the loss of depth information during imaging. In this paper, we consider a specific but common situation, where the scene contains known objects. Given 3D models of a set of known objects and a cluttered scene image, we try to detect these objects in the image, and align 3D models to their images to find their exact pose. We develop an approach that poses this as a 3D-to-2D alignment problem. We also deal with pose estimation of 3D articulated objects in images. We evaluate our proposed method on BigBird dataset and our own tabletop dataset, and present experimental comparisons with state-of-the-art methods. In order to find the pose of an object, we come up with a hierarchical approach whereby we first an initial estimate of the pose and thereby refine it using a robust algorithm. Obtaining the initial estimate is crucial as the refinement is entirely dependant on it. Estimating the object proposals or region proposals from an image is a well-known but difficult task, as the complexity of the problem intensifies due to the presence of object-object interaction and background clutter. We tackle the problem by coming up with a robust Convolutional Neural Network based method which learns object proposals in a supervised manner. As we need region proposals at object level, we solve the problem of instance-level semantic segmentation, where each pixel in the image is classified into one of the known classes. Moreover, two pixels are labelled differently if they belong to two different instances of the same class. We show quantitative and qualitative comparison of our proposed network models with previous approaches, and show our results on the challenging PASCAL VOC dataset.

Abstract

A given piece of textual information produced by any user or agent is said to be inappropriate if the expressed intent may cause anger, annoyance to certain users or exhibits lack of respect, rudeness, discourteousness towards certain individuals/communities or may be capable of inflicting harm to oneself or others. A search engine should regulate its query completion suggestions by detecting and filtering suchqueries as it may hurt the user sentiments or may lead to legal issues thereby tarnishing the brand image. Hence, automatic detection and pruning of such inappropriate queries from completions and related search suggestions is an important problem for most commercial search engines. The problem is rendered difficult due to unique challenges posed by search queries such as lack of sufficient context, natural language ambiguity and presence of spelling mistakes and variations. In this Thesis, we propose a novel deep learning based technique for automatically identifying inap- propriate query suggestions. We propose a novel deep learning architecture called “Convolutional Bi-Directional LSTM (C-BiLSTM)” which combines the strengths of both Convolution Neural Networks (CNN) and Bi-directional LSTMs (BLSTM). Given a query, C-BiLSTM uses a convolutional layer for extracting feature representations for each query word which is then fed as input to the BLSTM layer which captures the various sequential patterns in the entire query and outputs a richer representation encoding them. The query representation thus learnt passes through a deep fully connected network which predicts the target class. C-BiLSTM doesn’t rely on hand-crafted features, is trained end-end as a single model, and effectively captures both local features as well as their global semantics. Evaluating C-BiLSTM on real-world search queries from a commercial search engine reveals that it significantly outperforms both pattern based and other hand-crafted feature based baselines. Moreover, C-BiLSTM also performs better than individual CNN, LSTM and BLSTM models trained for the same task. Rapid growth of chatbots and Interactive Gaming Systems expect large, real time, clean (non-inappropriate) human conversation data to train their models. High demand of social networking sites provide a platform for users to interact and share opinions publicly. Inappropriate conversations and posts in these platforms may also lead to loss of business and damaging company’s image. So we extended our idea to identify inappropriate conversations in chat data. We have applied various techniques to solve this problem and observed that BLSTM performs better than LSTM and Boosted Decision Tree approaches.

Abstract

Understanding the meaning of words is essential for most natural language processing tasks. Word representations are means to mathematically represent the meaning of a word in a way that computers can understand. These representations are often in the form of vectors in which words are represented in a continuous vector-space of fixed dimensionality also referred to as word embeddings. In this thesis, we focus on generating better and reliable word representations for low resource languages. Many languages, though widely spoken, are largely under-represented in this area of research. One of the main reasons for this is the lack of reliable evaluation metrics to compare between different approaches of building these embeddings. Word similarity task is a widely used, computationally efficient method to directly evaluate the quality of word vectors. It relies on finding correlation between human assigned similarities between words, and those between corresponding word vectors. We release word similarity datasets for six low resource languages – Urdu, Telugu, Marathi, Punjabi, Tamil and Gujarati. For the construction of these datasets, our approach relies on translation and re-annotation of word similarity datasets of English. We also present baseline scores for word representation models using state-of-the-art techniques for Urdu, Telugu and Marathi by evaluating them on newly created word similarity datasets. For linguistically similar languages, we show that it is possible to use the better trained word representations of the more resourceful language for the other language in the pair, using a projection learning approach which relies on a mapping between words having similar meaning from the two languages. This cross-lingual vector space transformation results in state of the art results on word similarity test sets of French and German - an increase of 13% in case of French and 19% for German, using English as the source language. We also go on to demonstrate that this approach is better suited for linguistically similar language pairs like Hindi-Urdu (where 60% words are simply transliterations of each other) than English-German or English-French. We go on to see how we modelled prefix-suffix based morphoology using a similar technique.

Abstract

The field of data mining has emerged to extract knowledge hidden in large databases for better decision making. The process of frequent pattern (a set of items represents a pattern (or an itemset)) mining finds interesting information about the association among the items in a transactional database. The notion of support is employed to extract the frequent patterns. A pattern is called a frequent pattern if it satisfies the user-defined threshold on minimum support. An important criterion to assess the interestingness of a frequent pattern is its temporal occurrences in a database. That is, whether a frequent pattern is occurring periodically, irregularly, or mostly at specific time intervals in a database. The class of frequent patterns that are occurring periodically within a database are known as periodic-frequent patterns. Finding these patterns is a significant task with many real-world applications like improving the performance of recommender systems, intrusion detection in computer networks, discovering events in Twitter. Current periodic-frequent pattern models cannot handle datasets in which multiple transactions share a common timestamp or when transactions occur at irregular time intervals. This issue limits the applicability of the model as in many real-world databases like e-Commerce, Twitter, etc., transactions share a common timestamp and uneven time gaps exist in between the consecutive transactions. Most previous models on periodic-frequent pattern mining have focused on finding all patterns in a transactional database that satisfy the user-specified minimum support (minSup) and maximum periodicity (maxP er) constraints. The minSup constraint controls the minimum number of transactions that a pattern must cover in a database. The maxP er constraint controls the maximum duration between the two transactions below which a pattern should reoccur in a database. The usage of a single minSup and maxP er for an entire database leads to the rare item problem, because real-world databases have a non-uniform item distribution, which considers that items have different support and periodicity values. Also, current periodic-frequent pattern models have focused on discovering full periodic-frequent patterns, i.e., finding all patterns that have exhibited complete cyclic repetitions throughout the entire database. These models evaluate the periodic interestingness of a frequent pattern by determining whether all of its inter-arrival times are within the user-specified maxP er threshold. Therefore, the model cannot assess the partial periodic behavior of a frequent pattern in a database. However, partial periodic-frequent patterns are more common due to the imperfect nature of real-world databases. So, to address the above issues, in this thesis, we are proposing two improved approaches that discover periodic-correlated patterns and partial periodic-frequent patterns in non-uniform temporal databases, respectively. In the first approach, we tackle rare item problem by proposing a improved model that discovers periodic-correlated patterns in a non-uniform temporal database. In this thesis, we consider temporal database as a collection of transactions, ordered by their timestamps. Further, a temporal database facilitates multiple transactions to share a common timestamp and allows time-gaps in between consecutive transactions. A temporal database is said to be non-uniform if it contains items with dissimilar support and periodicity. To tackle rare item problem in non-uniform temporal databases, the proposed model considers a pattern as interesting if its support and periodicity are close to that of its individual items. The existing all-confidence measure is used to determine how close is the support of a pattern with respect to the support of its individual items. A new interestingness measure, called periodic-all-confidence, is being proposed to determine how close is the periodicty of a pattern with respect to the periodicity of its individual items. A pattern-growth algorithm has also been discussed to find periodic-correlated patterns. Experimental results show that the proposed model is efficient and tackles rare item problem. We discuss the usefulness of periodic-correlated patterns with a real-world case study on FAA-Accidents database and show that the proposed model may be utilized to discover interesting periodic-correlated patterns involving both frequent and rare items effectively. In the second approach, we have introduced a improved model to discover partial periodic-frequent patterns in non-uniform temporal databases. The proposed model lets the user specify a different maximum inter-arrival time (M IAT ) for each item. An inter-arrival time of a pattern is considered periodic (or cyclic) if it is no more than period. Thus, different patterns may satisfy different period depending on their items’ M IAT values. This solves the rare item problem in partial periodic-frequent pat

Abstract

This research aims to investigate the crossmodal correspondences between pitch and space. Crossmodal correspondences refer to the associations that exist between different physical stimulus attributes or properties in different modalities. In the context of this thesis, we see the association between pitch as one physical attribute and space as another. Research refers to this as pitch-space mapping and has been frequently investigated through various methods. In recent times, however, pitch-space mapping has been investigated by employing the stimulus-response compatibility (SRC) paradigm. SRC refers to the fact that performance is better when stimuli and responses are paired with each other in certain ways i.e. stimulus and response share common features than when they do not. Perception-action coupling is facilitated due to such correspondences resulting in higher accuracy and faster response times (Rts). Previous research shows that performance is better when a high pitch is responded with up or right responses and a low pitch is responded with down or left responses, called the spatial-pitch association of response codes (SPARC) effect. In this thesis, we have investigated the SPARC effect by introducing certain manipulations which are now briefly explained. Our first manipulation stems from the need to employ feedback in an SRC based task, more specifically SPARC effect in this case. Despite the intuitive coupling of perception-action, studies investigating the SPARC effect have, however, used feedback to manipulate the stimulus-response mapping. Feedback contradicts the purpose of intuitive stimulus-response mapping by enabling short-term learning. Our first section primarily investigates the role of feedback on SPARC effect. The second study of the thesis explores the role of loudness in pitch-space mapping. Even though all the recent studies more or less establish the pitch-space mapping, little has been questioned about the dimension of loudness that can interact with the spatial representation of pitch. It has been shown that loudness and pitch interact in the perception of a multidimensional stimulus such as auditory tone. They form an inseparable perceptual couple and therefore variation in one can cause a subjective variation in the other. Hence, pitch perception is confounded with loudness. Moreover, loudness too has its own spatial representation which can interact with the spatial representation of pitch. Our second study, therefore, investigates the role of loudness equalization on the SPARC effect. Finally, the third question addresses the role of blindness in pitch-space mapping. Our motivation for this study comes from the fact that even though visually impaired have an absent modality of vision, they have functional spatial representations of the world around derived through other modalities. They also exhibit similar if not better performance in pitch perception tasks as well. Hence, it was intriguing to see whether a lack of vision has any role on the SPARC effect. We found consistent vertical SPARC effect and horizontal SPARC effect in the first two manipulations. Moreover, both feedback and loudness equalization influenced the horizontal SPARC effect. The third study showed no SPARC effect in visually impaired participants. However, the results from third study are not conclusive due to the limitations and challenges described in the corresponding chapter.

Abstract

Code-mixing is a frequently encountered phenomenon in day-to-day natural language communication in the multilingual and bilingual communities. The phenomenon is so widespread that is often considered a different, emerging variety of the language. Computational modeling of the phenomenon of code-mixing and code-switching assumes particular relevance with the advancement of social media. However, computational studies for both textual and speech processing of code-mixing suffer from a sincere disadvantage: lack of data. In this thesis, we present the development of a Phonetically Balanced read speech corpus of Code-Mixed Hindi-English (the PBCM corpus). A Large Code-Mixed (LCM corpus) has been extracted from selected sections of two widely read Hindi newspapers. These sections (namely: Sports, Technology and Lifestyle) contain frequent English insertions embeddded within the matrix of Hindi sentences. Phonetic balance in the corpus has been established by selecting sentences that contain triphones lower in frequency than a predefined threshold. The assumption with the compulsory inclusion of such rare units is that the high frequency triphones will inevitably be included [66]. Using this metric, the Pearson’s correlation coefficient of the phonetically balanced corpus with a large code-mixed reference corpus was recorded to be 0.996. Statistics on the phone and triphone distribution have been presented, to graphically display the phonetic likeness between the large reference corpus (LCM corpus) and the corpus sampled through our method. A pilot PBCM corpus (PBCM-Phase I) has been recorded by 4 volunteer Hindi speakers. As a second contribution of this thesis, we present a baseline automatic speech recognition (ASR) system for code-mixed read speech in Hindi-English, developed upon the extrapolation of monolingual training resources. Two distinct monolingual acoustic models (Hindi and English) have been implemented to train a neural network and subspace GMM based speech recognition framework. The testing has been performed on PBCM-Phase I corpus. We examine the combination of a code-mixed trigram language model with each of the acoustic models described above. We contrast the combination of the code-mixed language model with a monolingual language model, and see drastic improvements in accuracy. The Hindi monolingual acoustic model, in combination with the code-mixed language model is our best performing experimental setup. (word error rate: 41.22%).

Abstract

Morphology is the branch of linguistics that deals with words, their internal structure, how they are formed, and their relationship to other words in the same language. It involves analyzing the structure of words and parts of words, such as stems, root words, prefixes, and suffixes. It also looks at parts of speech, intonation and stress, and the ways context can change a word’s pronunciation and meaning. In most languages, if not all, many words can be related to other words by rules that collectively describe the grammar for that language. For example, English speakers recognize that the words dog and dogs are closely related, differentiated only by the plurality morpheme “-s”, only found bound to nouns. With recent advancements in computational linguistics, we can now learn one-hot vector represen- tation for each word, also called word representations, from monolingual corpus of a language (training corpus). Word representations have been shown to contain syntactic as well as semantic (morphologi- cal) regularities. These word representations are being widely used to solve problems of various areas of natural language processing. These include but are not limited to dependency parsing, named entity recognition and parsing. One major requirement for learning good word representations (word embeddings) is large enough corpus to train. Size of training corpus directly affects the corresponding quality of word representations our model learns. Many languages, even though widely spoken, suffer from being computationally resource poor, which results in relatively poorer trained word embeddings. On top of it, morphologically rich languages suffer from morphologically induced data sparsity, since there are cases, where one morphological form of a word is common but other is rare in the same training corpus. Hence to learn better word representations for low resourced languages and to better exploit morpho- logical regularities present in distributional word representations, we present a language independent, unsupervised method for building word embeddings using morphological expansion of text by exploit- ing morphological regularities present in distributed word representations. Our model handles the prob- lem of data sparsity and yields improved word embeddings by relying on training word embeddings on artificially generated sentences. We evaluate our method using small sized training sets on eleven test sets for the word similarity task across seven languages. Further, for English, we evaluated the impacts of our approach using a large training set on three standard test sets. Our method improved results across all languages. We also present an unsupervised, language agnostic approach for exploiting morphological regular- ities present in high dimensional vector spaces. We propose a novel method for generating embeddings of words from their morphological variants using morphological transformation operators. We evaluate this approach on MSR word analogy test set with an accuracy of 85% which is 12% higher than the previous best known system.

Abstract

Cultural objects are highly interpretive and complex in nature. Though technological advancements have made it possible to digitize and preserve knowledge of cultural value, capturing details of such objects in-situ has rarely been explored. In this thesis, we develop a platform called Curarium for indepth exploration of cultural collections, within and outside a museum setting. A focus group of 17 researchers gathered cultural collections on a trip to Odisha. The collections contain information pertaining to Pattachitra paintings, Gotipua performances, sustainable crafts, museum objects and processions. The data-sets are unique in nature and content, as they provide for an interplay between different kinds of objects. In this thesis, we examine the cultural value in objects unexplored before - like processions, and what they comprise. To accommodate the different kinds of data, we develop a meta-data structure that would encapsulate all the relevant information. We import the curated data collection into Curarium and utilize the various in-house tools to discover patterns, create annotations, collaborate and visualize the data. Curarium provides a novel approach in delving deep into a cultural object. However, we recognize a gap in cross-referencing objects within its natural environment. In lieu of this, the thesis proposes developing narratives about cultural objects as an approach to capturing information. We develop a platform to generate narratives based on the digital surrogates. We conduct a user study with the focus group to develop narratives on the aforementioned data-sets on a platform called Narratavium. On further research, we recognize these narratives to be of three types - within-object, within-collection, and user generated thematic narratives. Upon recognizing plots within art objects and identifying themes, we account for the human experience in interacting with these cultural objects in their natural environment. This narrows the removed appreciation of the culture by the audience. In this thesis, we utilize sequential art methods to study Pattachitra paintings, Aristotlean narrative theories to study processions and performances, and qualitative analysis to generate themes. By visualizing the narrative information, we gain insights into nuances of the objects and the collections as a whole. We use charts and clusters to extract popular themes in a collection. We develop methods to identify conflict points in a narrative and understand transitions in a story plot. We also study information retention in the process of re-narration, and relevant objects in a collection through frequently told narratives.