Abstract

This research aims to investigate the crossmodal correspondences between pitch and space. Crossmodal correspondences refer to the associations that exist between different physical stimulus attributes or properties in different modalities. In the context of this thesis, we see the association between pitch as one physical attribute and space as another. Research refers to this as pitch-space mapping and has been frequently investigated through various methods. In recent times, however, pitch-space mapping has been investigated by employing the stimulus-response compatibility (SRC) paradigm. SRC refers to the fact that performance is better when stimuli and responses are paired with each other in certain ways i.e. stimulus and response share common features than when they do not. Perception-action coupling is facilitated due to such correspondences resulting in higher accuracy and faster response times (Rts). Previous research shows that performance is better when a high pitch is responded with up or right responses and a low pitch is responded with down or left responses, called the spatial-pitch association of response codes (SPARC) effect. In this thesis, we have investigated the SPARC effect by introducing certain manipulations which are now briefly explained. Our first manipulation stems from the need to employ feedback in an SRC based task, more specifically SPARC effect in this case. Despite the intuitive coupling of perception-action, studies investigating the SPARC effect have, however, used feedback to manipulate the stimulus-response mapping. Feedback contradicts the purpose of intuitive stimulus-response mapping by enabling short-term learning. Our first section primarily investigates the role of feedback on SPARC effect. The second study of the thesis explores the role of loudness in pitch-space mapping. Even though all the recent studies more or less establish the pitch-space mapping, little has been questioned about the dimension of loudness that can interact with the spatial representation of pitch. It has been shown that loudness and pitch interact in the perception of a multidimensional stimulus such as auditory tone. They form an inseparable perceptual couple and therefore variation in one can cause a subjective variation in the other. Hence, pitch perception is confounded with loudness. Moreover, loudness too has its own spatial representation which can interact with the spatial representation of pitch. Our second study, therefore, investigates the role of loudness equalization on the SPARC effect. Finally, the third question addresses the role of blindness in pitch-space mapping. Our motivation for this study comes from the fact that even though visually impaired have an absent modality of vision, they have functional spatial representations of the world around derived through other modalities. They also exhibit similar if not better performance in pitch perception tasks as well. Hence, it was intriguing to see whether a lack of vision has any role on the SPARC effect. We found consistent vertical SPARC effect and horizontal SPARC effect in the first two manipulations. Moreover, both feedback and loudness equalization influenced the horizontal SPARC effect. The third study showed no SPARC effect in visually impaired participants. However, the results from third study are not conclusive due to the limitations and challenges described in the corresponding chapter.

Abstract

Code-mixing is a frequently encountered phenomenon in day-to-day natural language communication in the multilingual and bilingual communities. The phenomenon is so widespread that is often considered a different, emerging variety of the language. Computational modeling of the phenomenon of code-mixing and code-switching assumes particular relevance with the advancement of social media. However, computational studies for both textual and speech processing of code-mixing suffer from a sincere disadvantage: lack of data. In this thesis, we present the development of a Phonetically Balanced read speech corpus of Code-Mixed Hindi-English (the PBCM corpus). A Large Code-Mixed (LCM corpus) has been extracted from selected sections of two widely read Hindi newspapers. These sections (namely: Sports, Technology and Lifestyle) contain frequent English insertions embeddded within the matrix of Hindi sentences. Phonetic balance in the corpus has been established by selecting sentences that contain triphones lower in frequency than a predefined threshold. The assumption with the compulsory inclusion of such rare units is that the high frequency triphones will inevitably be included [66]. Using this metric, the Pearson’s correlation coefficient of the phonetically balanced corpus with a large code-mixed reference corpus was recorded to be 0.996. Statistics on the phone and triphone distribution have been presented, to graphically display the phonetic likeness between the large reference corpus (LCM corpus) and the corpus sampled through our method. A pilot PBCM corpus (PBCM-Phase I) has been recorded by 4 volunteer Hindi speakers. As a second contribution of this thesis, we present a baseline automatic speech recognition (ASR) system for code-mixed read speech in Hindi-English, developed upon the extrapolation of monolingual training resources. Two distinct monolingual acoustic models (Hindi and English) have been implemented to train a neural network and subspace GMM based speech recognition framework. The testing has been performed on PBCM-Phase I corpus. We examine the combination of a code-mixed trigram language model with each of the acoustic models described above. We contrast the combination of the code-mixed language model with a monolingual language model, and see drastic improvements in accuracy. The Hindi monolingual acoustic model, in combination with the code-mixed language model is our best performing experimental setup. (word error rate: 41.22%).

Abstract

Morphology is the branch of linguistics that deals with words, their internal structure, how they are formed, and their relationship to other words in the same language. It involves analyzing the structure of words and parts of words, such as stems, root words, prefixes, and suffixes. It also looks at parts of speech, intonation and stress, and the ways context can change a word’s pronunciation and meaning. In most languages, if not all, many words can be related to other words by rules that collectively describe the grammar for that language. For example, English speakers recognize that the words dog and dogs are closely related, differentiated only by the plurality morpheme “-s”, only found bound to nouns. With recent advancements in computational linguistics, we can now learn one-hot vector represen- tation for each word, also called word representations, from monolingual corpus of a language (training corpus). Word representations have been shown to contain syntactic as well as semantic (morphologi- cal) regularities. These word representations are being widely used to solve problems of various areas of natural language processing. These include but are not limited to dependency parsing, named entity recognition and parsing. One major requirement for learning good word representations (word embeddings) is large enough corpus to train. Size of training corpus directly affects the corresponding quality of word representations our model learns. Many languages, even though widely spoken, suffer from being computationally resource poor, which results in relatively poorer trained word embeddings. On top of it, morphologically rich languages suffer from morphologically induced data sparsity, since there are cases, where one morphological form of a word is common but other is rare in the same training corpus. Hence to learn better word representations for low resourced languages and to better exploit morpho- logical regularities present in distributional word representations, we present a language independent, unsupervised method for building word embeddings using morphological expansion of text by exploit- ing morphological regularities present in distributed word representations. Our model handles the prob- lem of data sparsity and yields improved word embeddings by relying on training word embeddings on artificially generated sentences. We evaluate our method using small sized training sets on eleven test sets for the word similarity task across seven languages. Further, for English, we evaluated the impacts of our approach using a large training set on three standard test sets. Our method improved results across all languages. We also present an unsupervised, language agnostic approach for exploiting morphological regular- ities present in high dimensional vector spaces. We propose a novel method for generating embeddings of words from their morphological variants using morphological transformation operators. We evaluate this approach on MSR word analogy test set with an accuracy of 85% which is 12% higher than the previous best known system.

Abstract

Cultural objects are highly interpretive and complex in nature. Though technological advancements have made it possible to digitize and preserve knowledge of cultural value, capturing details of such objects in-situ has rarely been explored. In this thesis, we develop a platform called Curarium for indepth exploration of cultural collections, within and outside a museum setting. A focus group of 17 researchers gathered cultural collections on a trip to Odisha. The collections contain information pertaining to Pattachitra paintings, Gotipua performances, sustainable crafts, museum objects and processions. The data-sets are unique in nature and content, as they provide for an interplay between different kinds of objects. In this thesis, we examine the cultural value in objects unexplored before - like processions, and what they comprise. To accommodate the different kinds of data, we develop a meta-data structure that would encapsulate all the relevant information. We import the curated data collection into Curarium and utilize the various in-house tools to discover patterns, create annotations, collaborate and visualize the data. Curarium provides a novel approach in delving deep into a cultural object. However, we recognize a gap in cross-referencing objects within its natural environment. In lieu of this, the thesis proposes developing narratives about cultural objects as an approach to capturing information. We develop a platform to generate narratives based on the digital surrogates. We conduct a user study with the focus group to develop narratives on the aforementioned data-sets on a platform called Narratavium. On further research, we recognize these narratives to be of three types - within-object, within-collection, and user generated thematic narratives. Upon recognizing plots within art objects and identifying themes, we account for the human experience in interacting with these cultural objects in their natural environment. This narrows the removed appreciation of the culture by the audience. In this thesis, we utilize sequential art methods to study Pattachitra paintings, Aristotlean narrative theories to study processions and performances, and qualitative analysis to generate themes. By visualizing the narrative information, we gain insights into nuances of the objects and the collections as a whole. We use charts and clusters to extract popular themes in a collection. We develop methods to identify conflict points in a narrative and understand transitions in a story plot. We also study information retention in the process of re-narration, and relevant objects in a collection through frequently told narratives.

Abstract

Dopamine plays a major role in learning and reward. Dopaminergic neurons in the ventral tegmental area (VTA) exhibit the discrepancy between expected and actual rewards known as reward prediction error (RPE). But how these signals are computed in the brain remains unknown. One of the earliest attempts of investigation of animal learning involved pairing an unconditioned stimulus (US) with a cue or conditioned stimulus (CS) and observing that animal starts responding to the CS after some point in time. This is the basis of Pavlovian learning or classical conditioning, a fundamental learning mechanism in animals. The model described here focuses on the mechanism of reward prediction error within Pavlovian learning and does not deal with other conditioning phenomena. To acquire a deeper understanding of the roles of the cerebral structures involved in RPE computation, we need a neuro-computational model implicating them more faithfully in Pavlovian learning. The approach described here is a model that proposes a circuit involving a dissociation between magnitude expectation and timing expectation and explaining more precisely how the computation of the reward prediction error happens inside the VTA. The model described here proposes that the two dimensions involved in computation of reward prediction errors i.e., magnitude and time could be computed separately and later combined unlike the way it is done in traditional reinforcement learning models. The model is built on biological evidences and is able to reproduce various aspects of classical conditioning, namely, the progressive cancellation of the predicted reward, the predictive firing from conditioned stimuli, the twin peaks of dopamine firing and delineation of early rewards by showing more firing for sooner early rewards and less for early rewards that occur with a longer latency in accordance with biological data. Key predictions for experiments are made regarding how neural substrates execute this computation which would validate the model. Potential implications for the reinforcement learning theory are also discussed.

Abstract

The structure of built environment is defined by the collection of fundamental units contained in it. Two of the most basic units in an urban setting are Roads and Buildings. The buildings form most of the origin and destination pairs for intra-city movements and the road-networks aid these movements. The basic aim of this thesis is to identify patterns of arrangements of these elementary units in any urban agglomeration along with deriving a syntactic meaning out of the arrangement geometry and secondly, how the movements patterns and dynamic occupancy of spaces are often affected by the synergy of roads and buildings. We try to identify combinations of arrangement of these two that increase the odds in favor of increasing attractor values of spaces. In past, several studies were carried out to extract patterns in urban environment which were often driven by considering networks only. A few studies paid attention to buildings and their characteristics such as their volumetric capacities and their locations along the network. The approach we have chosen to define symmetries and regularities in structures is hybrid in nature as it considers both buildings and road-networks as well as their intrinsic interactions such that the buildings and roads in closer vicinity inherit the properties of each other. The spatial indices we used for analyzing built environments are based on navigational properties of any location which suggest accessibility of the location depending on its connectivity with neighborhoods. These indices are motivated by a combination of Space Syntax configurational theory that uses topological (Angular & Segment) distances along network and Urban Network Analysis which has metric distances as basis for accessibility computations. The accessibility based on topological as well as metric computations is defined as function of how often a spatial unit lies on the shortest paths while commuting all origin-destination pairs in its neighborhood, how many of surrounding places can be reached from this place, how compactly packed is the neighborhood for a given set of fixed radii, and do the shortest paths between origin-destination pairs resemble straightest distances. These characteristics can be discussed for both buildings and roads. We use spatial autocorrelation for our chosen accessibility indices to analyze significance of occurrence of specific patterns of neighborhoods for city of Hyderabad, India, and thus consider not only spatial units but areas as whole. We discuss the inconsistencies in methodologies used, and propose a weight metrics for urban road network, weights are driven by dimensions of buildings. This enables us to reduce computational complexity by a significant margin using road networks as the only analysis layer for making configurational computations. We verify that it is intrinsic property of spatial organization that more similar areas tend to occur in closer vicinity of each other by using K Means clustering. In the process, we also propose that the clusters formed with varying radii of analysis are suggestive of spatial properties of urban environment. Lastly, we present a data driven methodology to classify regions in terms of amount vehicular traffic. In this technique, Average Annual Daily Counts (AADT) of vehicular traffic are used to represent roads into three categories of high, medium and low which are then superimposed on buildings. The chosen spatial indices form feature vectors for training the model while the vehicular traffic categories superimposed on buildings act as prediction target labels. To get the best accuracy model, we compare performances of Logistic Regression Classification model, Decision Tree Classification model and ensemble based Gradient Boosted Classification model. The ensemble based model using gradient boosting algorithm outperforms all other models in this classification task. While discussing all methodologies to denominate areas in terms of their symmetry and constrained by relative carriage of vehicular traffic, we go from a local measure of trip distances towards global trip distances that implies we go from inspection of nearby regions in close proximity of area under observation towards larger distances. This process enables us to draw comparisons between areas corresponding to their obvious choices of being preferred for various trip distances. For example, an area being preferred for short trip distances might not be a good fit for longer trips and vice versa. The approaches presented in this can draw conclusive reports from the perspective of architectural and infrastructural planning for new city sites and modifying the existing ones.

Abstract

The amount of videos recorded and shared on the internet has grown massively in the past decade. The most of it is due to the cheap availability of mobile camera phones and easy access to social media websites and their mobile applications. Applications such as Instagram, Vine, Snapchat allows users to record and share their content in matter of seconds. These three are not the only such media sharing platform available but the number of active monthly users are 600 , 200, and 150 million respectively indicate the interest people have in recording, sharing and viewing their content [1,3]. The number of photos and videos collectively shared on instagram alone crosses 40 billion [1]. Vine contain approximately 40 million videos created by the users [3] and on a daily basis played 1 billions times. This cheaply available mode of data can empower many learning tasks which require huge amount of curated data. Also the videos contain novel viewpoints, and reflect real world dynamics. Different from the content available on older established websites such as Youtube, the content shared here is smaller in length (typically few seconds), contains description and associated hash-tags. Hash-tags can be thought of as keywords assigned by the user to highlight the contextual aspect of the shared media. However, unlike english words these don’t have a definite meaning associated to them as the description is heavily reliant on the content, along which the hash-tags are used. To clearly decipher the meaning of the hash-tag one requires the associated media. Hence, Hash-tags are more ambiguous and difficult to categorise than English words. In this thesis, we attempt to shed some light on applicability and utility of videos shared on a popular social media website vine.co. The videos shared here are called vines and are typically 6 seconds long. They contain with them description composed of a mixture of english words and hash-tags. We try recognising actions and recommend hash-tags to an unseen vine by utilising the visual and the semantic content and the hash-tags provided by the vines respectively. By this we try to show how this untapped resource of popular social media format can prove beneficial for resource intensive tasks which require huge amount of curated data. Action recognition deals with categorising the action being performed in a video to one of the seen categories. With the recent developments, considerable precision is achieved on established datasets. However, in an open world scenario, these approaches fail as the conditions are unpredictable. We show how vines are a much difficult categories of videos with respect to the videos currently in circulation for such tasks. To avoid manual annotations for vines we develop a semi-supervised bootstrapping approach. If one is to manually annotate vines this would defeat the purpose of easily available vines. We iteratively build an efficient classifier which leverages the existing dataset for 7 action categories and also the visual, semantic information present in the vines. The existing dataset forms the source domain and the vines compose the target domain. We utilise semantic word2vec space as a common subspace to embed video features from both, labeled source domain and unlabeled target domain. Our method incrementally augments the labeled source with target samples and iteratively modifies the embedding function to bring the source and target distributions together. Additionally, we utilise a multi-modal representation that incorporates noisy semantic information available in form of hash-tags. Hash-tags form an integral part of vines. Adding more and relevant hash-tags can expand the categories for which a vine can be selected. This enhances the utility of vines by providing missing tags and expanding the scope for the vines. We design a Hash-tag recommendation system to assign tags for an unseen vine from 29 categories. This system uses a vines’ visual content only after accumulating knowledge gathered in an unsupervised fashion. We build a Tag2Vec space from millions of hash-tags using skip-grams using a corpus of 10 million hash-tags. We then train an embedding function to map video features to the low-dimensional Tag2vec space. We learn this embedding for 29 categories of short video clips with hash-tags. A query video without any tag-information can then be directly mapped to the vector space of tags using the learned embedding and relevant tags can be found by performing a simple nearest-neighbor retrieval in the Tag2Vec space. We validate the relevance of the tags suggested by our system qualitatively and quantitatively with a user study.

Abstract

Quantum information theory has emerged as one of the new frontiers of science and technology over the last few decades. Unlike classical information theory that deals with the classical bit, quantum information theory is centred around the concept of a qubit. In contrast to the classical bit that can either be a 0 or 1, the qubit exists in a superposition of both states at the same time. The importance of quantum resources is evident from the fact that are certain information processing tasks like teleportation which cannot be accomplished with only classical resources and even though there are certain tasks like secret sharing which have classical counterparts, their outputs are greatly enhance when the classical resources are combined with quantum resources. While quantum superposition and entanglement are wellestablished quantum resources, recently, significant research has been carried out to explore the possibility of quantum coherence as a resource for information processing applications. This thesis is divided into four chapters. While the first two chapters deal with the fundamentals of quantum information theory and quantum coherence, the last two chapters primarily deal with the two research problems on quantum coherence. The first chapter serves as a revision of the important introductory concepts of quantum information theory, starting with the significance of the famous Stern-Gerlach experiment and the rise of randomness to mathematical formalism and postulates of quantum mechanics. Then, we explore the connection between randomness and lack of information and further explore how classical information is different from quantum information alongwith a brief overview of its corresponding measures. Finally, the chapter ends with a layout of widelyused quantum resources along with its utility in some of the famous information processing tasks. The second chapter mainly deals with a brief synopsis on quantum coherence. At first, it discusses the formulation of resource theory of quantum coherence, which is followed by an introduction to the widely accepted framework of quantifying coherence and some of the popularly used coherence quantifiers. Finally, the chapter concludes with the application of coherence in the fields of quantum thermodynamics, quantum algorithms, interference phenomena and phase discrimination. The third chapter addresses the first research topic of the thesis : broadcasting of quantum coherence. As quantum coherence has recently emerged as a key candidate for use as a resource in various quantum information processing tasks, hence, it is of utmost importance to explore the possibility of creating a greater number of coherent states from an existing coherent pair. This process is known as broadcasting of coherence. This chapter starts with a brief discussion on the famous no-cloning theorem, followed by an overview of approximate cloning transformations. After that, the motivation behind broadcasting of coherence is explored along with an introduction to the definitions of optimal and non-optimal broadcasting. In both these definitions, the most general two-qubit state is taken as the input to the cloner, while the most incoherent states are used as blank states of the cloner. This chapter leads to two important results. Firstly, it has been proved that while optimal broadcasting is not possible (for both local and non-local cloners), nonoptimal broadcasting cannot be ruled out. Secondly, in case of non-optimal broadcasting, the coherence introduced in the output states of the cloner will always be lesser than the initial coherence of the input state. Finally, three classes of mixed states, namely, the statistical mixture of the most coherent state (MCS) and the most incoherent state (MIS), the Wernerlike states (WLS) and the Bell-diagonal states (BDS), are taken up for obtaining their respective ranges of non-optimal broadcasting in terms of their corresponding input state parameters. The fourth and final chapter deals with the second research topic of the thesis : some peculiar properties of robustness of coherence (ROC). Here, it has been shown that robustness of coherence, in contrast to many popular quantitative measures of quantum coherence derived from the resource theoretic framework of coherence, may be sub-additive for a specific class of multi-partite quantum states. Furthermore, this chapter also highlights how the subadditivity is affected by admixture with other classes of states for which ROC is superadditive. Moreover, it has been shown that pairs of quantum states may have different orderings with respect to relative entropy of coherence, l1-norm of coherence and ROC and the difference in ordering for coherence measures chosen pairwise has been numerically studied.

Abstract

In the thesis, an ontology and event grammar of tennis is proposed in order to enhance the methods of analysis in tennis. This work specifically focuses on the analysis required for improving the game from the coach’s point of view. Our ontological study has a vocabulary involving about 760 reusable terms related to tennis which are broadly divided into 12 top classes. The scope of this thesis deals primarily with three classes ‘play’, ‘score’ and ‘court’ and related terms out of the twelve. These terms and their relationships are defined with the help of event trees, score graphs and labelled images which are based on the rules legislated by International Tennis Federation. Any game of tennis involves a particular sequence of events. With the help of event trees, a generative event grammar of any play has been designed. This grammar generates all possible legal sequences of events that can take place in a game of tennis. However, a particular game is a unique sequence of such possible events. A game of tennis is represented as a finite state automaton, at any given point it can be in exactly one defined state. Annotated data is crucial in any form of analysis. We have created a real time annotator for tennis on the android platform based on the proposed event grammar. This integration of grammar into the application reduces the error rate and also makes the process efficient. The annotated file is then processed for generating play statistics at two levels. First, basic features such as success rates of shots/serves, number of forehands/backhands hit/miss etc. are analyzed. These features are very similar to those presented during the broadcast tennis videos in major tournaments although our annotation has more data like sub-events. Secondly, we infer sub-events between shots. Such detailed annotation points towards deeper inquiry as to why a shot was not executed properly, if it failed. An ergonomic and easy to replicate video documentation eight camera standard for the game of tennis is proposed. For deeper analysis, we create a data set of (singles) tennis games in accordance with the reduced version of the standard. Computationally two main features were extracted from this video data set: A. Player detection by a generalized feature based motion tracking algorithm, and B. Ball tracking by an improvised object-and-trajectory based algorithm. Every shot played is defined as a 18-feature tuple in the ontology. Selected features from the definition were computed by extracting ball trajectory and player position in real world situations. The features thus extracted are then classified and validated using SVM and KNN techniques. The features extracted along with their accuracies are: spin of the ball (83.85%), trajectory height (95.56%), position (94.65%) and number of bounces (98.93%). Additional application of shot prediction (88.75%), which predicts the next likely shot to be hit based on the player and ball bounce positions, was also implemented. An adaptive probabilistic model was developed to analyze a player’s game by predicting the conditional probabilities of an event based on a previously occurred event. A graph based on event trees is constructed to store the occurrences of a particular event and features. The data changes dynamically as the game proceeds and at any given point will give an experimental probability of the next likely events based on the current and past events. Proposed analysis reduces search space for coach to identify areas of improper execution and work towards improving player’s performance.

Abstract

Satellite imagery in the recent past has gone through numerous advancement. Different satellite sensors have been developed with the capability to capture the data at different scales from coarse to very fine. The finer resolution data provides detailed information which helps in improving the object detection for more number of land cover classes. Satellite imagery forms the backbone of remote sensing. Remote sensing has a wide range of applications in Earth sciences like geology, military, economic and humanitarian purposes. Urban remote sensing is useful for analysing and planning. Building detection is one of the most important aspects of urban remote sensing. Building footprint detection is very useful in wide range of applications like disaster management, urban planning, cartography, environment modeling, all require knowledge of man made structures. Even now, mapping of buildings is done using remote sensing based on visual interpretation or manual digitizing of digital images. These methods are tedious, time consuming, requires qualified people, therefore it has virtually no scope for real-time applications. This work presents an approach to automatic building detection from very high resolution (VHR) satellite images. The work includes discussion of various image processing techniques used and challenges faced with detection of different building models. Monocular satellite data is used for this work. Satellite images typically contain elements like development areas, vegetation, water bodies, barren land etc. Development areas include buildings, roads, pavements and other such man-made entities. The first task of building detection process is to remove the natural entities and focus on development regions. This task is realized using Independent Component Analysis (ICA). Three input planes derived from the LAB and LUV colorspace are used as input to the ICA which in turn converts them into three independent output planes. One of these output planes primarily contains man-made structure pixels and other two primarily contain vegetation and other natural entities. Development areas are detected using pixels obtained from the plane and appropriate thresholding. The only drawback of this technique is buildings are obtained in the form of individual pixels and as a result, object-level performance of the detection is not as good as its pixel-level performance. The second phase of this work builds on the learning of ICA based building detection. The candidate building pixels obtained through ICA are now used to extract maximally stable extremal regions (MSER) which are then filtered using geometric properties to obtain final candidate buildings. The technique is aimed at improving object-level performance. The achieved Precision of 80.64% and Recall of 83.65 % in case of object-level evaluation. The work offers an unsupervised building detection technique which is robust towards size, shape, color, types of rooftops and shadows. A wide test image set consisting of 15 localities is used to evaluate the performance of the complete detection process. This technique achieved higher object-level performance compared to the only ICA-based technique. The ICA and MSER combined approach have potential to detect buildings under different scenarios, not compromising with accuracy.