Abstract

In recent times, there have been an increase in Optical Character Recognition (OCR) solutions for recognizing the text from scanned document images and scene-texts taken with the mobile devices. Many of these solutions works very good for individual script or language. But in multilingual environment such as in India, where a document image or scene-images may contain more than one language, these individual OCRs fail significantly. Hence, in order to recognize texts in the multilingual document image or scene-image, we need to, manually, specify the script or language for each text blocks. Then, the corresponding script/language OCR is applied to recognize the inherent tasks. This is a step which is preventing us to move forward in the direction of fully-automated multi-lingual OCRs. This thesis presents, two effective solutions to identify the scripts and language of document images and scene-texts, automatically. Even though, recognition problems for scene texts has been highly researched, the script identification problem in this area is relatively new. Hence, we present an approach which represents the scene-text images using mid-level strokes based features which are pooled from the densely computed local features. These features are then classified into languages by using an offthe-shelf classifier. This approach is efficient and require very less labeled data for script identification. The approach has been evaluated on recently introduced video script dataset (CVSI). We also introduce and benchmark a more challenging Indian Language Scene Text (ILST) dataset for evaluating the performance of our method. For script and language identification in document we investigate the utility of Recurrent Neural Network (RNN). These problems have been attempted in the past with representations computed from the distribution of connected components or characters (e.g. texture, n-gram) from a larger segment (a paragraph or a page). We argue that, one can predict the script or language with minimal evidence (e.g. given only a word or a line) very accurately with the help of a pre-trained RNN. We propose a simple and generic solution for the task of script and language identification without any special tuning.This approach has been verified on a large corpus of more that 15.03M words across 55K documents comprising 15 scripts and languages. The thesis aims to provide a better recognition solutions in document and scene-texts space by providing two simple, but effective solutions for script and language identification. The proposed algorithms can be used in multilingual settings, where the identification module will first identify the inherent script or language of incoming document or scene-texts before sending them to corresponding script/language recognition module.

Abstract

Understanding human appearance in images and videos is one of the most fundamental and explored area in the field. Describing human appearance can be interpreted as a concoction of smaller and more fundamental related aspects like posture, gesture, outlook etc. By doing so we try to grasp holistic sense from semantically lower level information. The utility behind understanding human appearance and related aspects is the industrial demand for applications that involve analyzing humans and their interaction with surroundings. This thesis work tackles two of such related aspects: i. more fundamental problem, human pose estimation and ii. deeper understanding from cloth parsing based on pose estimation. Determining the human body joint locations and configuration is quizzed as human pose estimation problem. In this work we address the problem of human pose estimation for video sequence data type. We exploit the availability of redundant information from redundant data type. The proposed iteratively functioning methodology has the first iteration involving parsing data from a quintessential generic base model. For the following iterations, we run a 3 step pipeline: grabbing confidently positive detections from previous iteration using our novel selection criteria, fine-tuning external-to-base parameters to local distribution by synthesizing exemplars from picked ones, and enforcing the learned information using an updated amalgamation model. The resulting pipeline propagates correctness in temporal neighborhoods of a video sequence. Previous methods that use the same base models have relied more on tracking strategies. From the unbiased experiments conducted, our approach has proven to be much more robust and overall better performing. In the second half, we indulge in determining a more deeper understanding of human aspects i.e. cloth parsing. This involves predicting the cloth types worn by humans and their segmented regions in images. Our work focuses on incorporating robustness to a previously formulated method. Conceivably, determining hot regions for each cloth type is dependent on the underlying pose skeleton of the human, eg. hat will be worn on the head. Hence, availability of pose information is key to cloth parsing problem, but incorrect body part estimations can also simply lead to false cloth detections and segmentations. The previous method uses pose information from pictorial structure based model, whereas we update the formulation to incorporate information from more robust part detectors. The changed model has shown to be performing better at more wild outdoor settings. However, the performance from available and proposed methods is below par and appears non-viable for application purposes. To answer this, we report a set of experiments in which we take different lookouts and report observations.

Abstract

With the ever-increasing influence of social media in our lives, it is important to automatically make sense out of the enormous amounts of social media data generated each day and leverage it for different applications. A politician trying to gauge her chances of winning the election, the government reaching out to understand the grievances of the citizens regarding various administrative issues, a company attempting to promote its products and perceive its market reach - all of them have one thing in common, their destination, i.e. the social media. Even a few years ago, a restaurant interested in customer feedback would have to run manual surveys, requesting customers to fill out lengthy feedback forms. Not only did this approach have limited reach, it also had the additional problem of customers not being responsive. Now, with the advent of social media, this process had become much less cumbersome, with people reporting their experiences openly and unguardedly, thereby minimizing the cost incurred to the companies many folds. However it is challenging to mine information from the huge amount of user generated unstructured social media data. Understanding this data entails primarily identifying, understanding and analysing the topics discussed in social media along with the opinions expressed towards those topics. In this thesis, we focus on this problem, by first treating the two tasks of topic and sentiment identification independently, before eventually presenting a joint model. Topic identification in social media posts have been widely studied in the recent past. In contrast, we direct our attention towards capturing the topics in the conversations that arise out of interactions between users on social media. Specifically, given a conversation on Twitter, we aim to automatically recommend a relevant tweet, treating the conversation as a tree. Existing solutions to this problem exploit similarity of a candidate tweet to a single tweet, or to past tweets in a user-pair conversation. In this work, we generalize the problem setting to recommend a tweet considering the context from an entire conversation tree which often includes tweets from multiple users. While this setting is more natural, it brings in additional challenges: (1) how to choose an anchor tweet node from the conversation tree for which a new tweet can be recommended as a reply? (2) how to choose the tweet to recommend? We learn regression models with novel features to address both the challenges, and use them to perform extractive response recommendation. The first regression model predicts the time required by a tweet node to get its first child node, while the second predicts the number of retweets received by a tweet, as a measure of its popularity and acceptability, and hence quality. Experiments with millions of tweets show that the proposed recommendation method is more accurate compared to the state-of-the-art approaches,with respect to ground-truth labels. Due to lack of manually annotated data, we have used proxy signals to infer labels and used them as ground truth. Sentiment Analysis is another important task that complements topic identification in order to capture actionable information from social media. While sentiment analysis has been studied widely, it is not a completely solved problem when it comes to the noisy nature of the social media data. In this work, we introduce some novel features and learn supervised models to improve performance in the area of sentiment analysis in Twitter. With careful feature ablation experiments, we show which of the novel features contribute to the performance improvement of our system and to what extent. It is a simple model that performs competitively with respect to state-of-the-art systems and can be quickly and easily prototyped as an end-to-end system from scratch. But, one drawback of this work is that we have not taken the topics into consideration. Given a piece of text, we assign a sentiment label to it, irrespective of the topics being discussed in that text. This is especially problematic, when multiple topics are at play and the sentiment expressed throughout the text is not uniform. In order to alleviate this shortcoming, we further aim to learn topics and sentiment together. However, it is still a pipeline model instead of a joint one. In this model, we want to overcome the drawbacks of topic-agnostic sentiment analysis and tackle the problem which is formally known as Aspect based Sentiment Analysis, in which the goal is to identify the sentiment associated with each of the aspects being discussed in a post. First the aspects or topics being discussed in the text are identified and we treat this sub-task of aspect category detection as a multi-class multi-label classification problem. Instead of assigning a single generic sentiment label to a piece of text, our goal here is to associate a sentiment label to each of the aspects or

Abstract

Consider a scenario wherein a bank with a large database of customer information intends to identify the set of customers to whom a credit line could be extended. For this, the bank might need various other information, apart from its own database, such as tax information from the income tax department, criminal history of the customers from the law enforcement authorities etc. Although all the other parties might be willing to share their respective information (databases) in order for the banking organization to compute the result, they might not be willing to compromise on the privacy of their databases. To preserve the privacy of the individual data, in an idealistic world, all parties could send their data to a Trusted Third Party (TTP) which would then apply any of the data mining techniques to compute the intended result and share it with the banking organization. The bank would only get to know the final set of customers to whom a line of credit could be extended and any local information with other parties would not be revealed other than what could be computed from the final result itself. However, in the real world, in many instances it would be infeasible to find such a Trusted third party with whom all the other sources of data would be willing to share their information. To overcome such problems, Privacy Preserving Data Mining techniques have been developed which allow the computation of a function on various databases distributed across a set of players, with out either party learning anything about the data of the other parties other than that can be derived from the final result itself. In other words, these techniques simulate the functionality of a TTP. In this thesis, we consider the problem of Privacy Preserving outlier Detection(PPOD) over vertically partitioned data. The computational and communication complexities of the proposed PPOD algorithm are sub-quadratic in the size of the dataset as opposed to the quadratic complexities of the previously known results. In order to achieve efficient algorithms in the private setting, we first develop an approximation algorithm for outlier detection in the centralized setting and then extend it to the case of privacy preserving outlier detection.

Abstract

Identification of static and/or dynamic roles of different non-canonical base pairs is essential for a comprehensive understanding of the sequence-structure-function space of RNA. In the past combined structural bioinformatic and quantum mechanical (QM) approach provided insights into occurrence context and conservation and covariation pattern of non-canonical A:G W:W Cis base pair in RNAs. In this work we have investigated structural and functional occurrence context of reverse Watson-Crick purine-purine base pairs (A:A, G:G and A:G W:W Trans) which constitute an interesting class of non-canonical base pairs in RNA due to their characteristic C10-C10 distance (highest among all base pairing geometries) and parallel local strand orientation. In contrast to canonical base pairs, intrinsic interaction energies and frequencies of occurrence in a nonredundant set of RNA crystal structures of these base pairs do not correlate with each other. Therefore, it is expected that, occurrence of these base pairs will be associated with specific structural folds or functional roles. We have identified the occurrences of these base pairs in a non-redundant set of RNA crystal structures and performed structural alignments of the corresponding homologous sites followed by a detailed context analysis. Our studies reveal that occurrences of purine-purine W:W Trans pairs exhibit a bimodal distribution with respect to the corresponding local backbone geometry. One mode is populated by both A:A and G:G W:W Trans pairs and characterized by a special fold in backbone. We call it a ‘Sharp-turn’ motif. We have observed ‘Sharp-turn’ to be of significant biological importance as it acts as binding site for proteins, metabolites and ions in RNAs. It remains conserved across species with some variation in its composition and often mediates structural folds crucial for the function as in the case of HIV-1 dimerization initiation site. The other mode in distribution is explicitly associated to A:A W:W Trans pairs involved in mediating higher order interactions. A:A W:W Trans mediates in these higher order interactions by participating in specific kinds of triple/quartet formation. We also have analyzed structural context of previously undetected A:G W:W Trans base pair. It has been found to be involved in maintaining translational fidelity at A site in the ribosome via type II A-minor and G-minor interactions with codon:anticodon bases. We rationalize our observations on the basis of quantum chemical calculations performed at M05-2X/6-3++(2d,2p) level of theory. The general trends obtained from analyzing X-ray crystal structures are also consistent with trends observed in available solution NMR structures.

Abstract

n the past recent years, there has been a growing need to understand the semantics in sports games. Use of technology in analyzing player movements and understanding the action on a sports field has been growing in the past few years. Most of the systems today make use of certain tracking devices worn by players or markers with sensors placed around the play area. These trackers or markers are electronic devices that communicate with the cameras or cameramen. Other technologies such as the goal line technology popularly used in soccer helps game referees to make accurate decisions that are often misjudged by mere human perception. The primary challenges in these techniques is to make it cost effective and ease of installation and use. It is not convenient to setup markers and sensors around the playing field or to force players to wear certain recording or communication devices without affecting their natural style of playing. Placing a sensor in the game ball also poses a tricky problem of not altering the physical properties of the ball. Sports recorders and broadcasters are now looking for simple and yet effective solutions to get semantic information from a sports game. The big question here is - Can we get sufficient important data only from a video capture just as a human would without relying on external aids of markers and sensors? With advances in various computer vision algorithms and techniques the goal for the future is to analyze everything from captured video. This kind of solution is obviously more attractive to broadcasting and game recording companies as they dont need to setup extra equipment, or influence the authorities to change the match ball or players outfits. We propose a set of algorithms that does the task of automatic analysis for broadcast videos for the sports of Cricket and Soccer.. Using our approach we can automatically detect salient events in Soccer, Temporally align Cricket video with corresponding text commentaries, Localize/Register a soccer image and others. We also compare our algorithms with other state of the art approaches extensively on different datasets for a variety of tasks.

Abstract

With the scaling of CMOS feature size into sub-100 nm technology nodes, data rates in the order of giga hertz per second are possible. Input / Output (I/O) interfaces and communication media are the mains bottlenecks to achieve high data rates of the communication systems. The main limitation of the communication media is its limited bandwidth, due to which a high frequency signal passing through it experiences intersymbol interference (ISI) thereby distorting the transmitted data. The limitations imposed by the communication media are overcome by the use of additional circuitry and better communication schemes in the interface devices. Additional circuitry such as equalizers are used in the transceivers to nullify the ISI and distortions due to the communication media. Also, better signaling schemes are also employed in the interfaces to reduce the effects of communication media on the data transmitted through it. Therefore, these techniques allow us to achieve high data rates with the existing communication media. In this work, circuits to nullify the ISI and distortions of the communication media (low loss and high loss) and the use a different signaling scheme to increase the throughput through communication media are discussed. Initially, an analog front-end transceiver for enhanced data rate MIL-STD-1553B, called E-MIL-STD-1553 suitable for avionics and military applications is proposed. This work proposes an analog front-end transceiver supporting 100-Mbps data rate in a multi-drop bus environment, over a cable length of 100 meters for a target bit error rate (BER) of 10 −12 , while retaining all other specifications of MIL-STD-1553B. Complete E-MIL-STD-1553 transceiver is designed and implemented in a 3.3 V, 65 nm CMOS technology. E-MIL-STD-1553 analog transceiver is implemented in Verilog-A and Verilog-HDL respectively. Performance results shows a power consumption of 12.85 mW at 100-Mbps data rate for a 100 percent transmitter duty cycle, occupying an area of 199017 μm 2 , for a target BER of 10 −12 . Secondly, a new current mode continuous time linear equalizer (CTLE) with charge mode pole-zero adaptation scheme is proposed. The proposed equalizer is realized with common gate (CG) topology using switched capacitor based pole-zero adaptation to suit varying channel characteristics. The input impedance of the CG-CTLE is made equal to the characteristic impedance of the off-chip link, hence eliminating the need for a separate resistive termination. Since the proposed CG-CTLE acts as the first stage of the current mode receiver, there is no need for a separate trans-impedance amplifier or a pre-amplifier. The proposed CG-CTLE is implemented in a 1.1 V, 65 nm CMOS technology. The performance results show that it achieves a data rate of 15-Gbps while equalizing the loss of a 7.5 inch FR4 PCB trace. It also offers an input impedance of 44.6 ohm, SNR of 23 dB and a bandwidth of 7.575 GHz and consumes a power of 13.9 mW. Lastly, a new hybrid circuit topology for simultaneous bidirectional signaling across global short (3-mm) and long (10-mm) on-chip interconnects in a multi-channel signaling environment is proposed. The proposed hybrid circuit transmits the outbound signal and simultaneously receives the inbound signal from the combined signal, which is a superposition of the outbound and inbound signals. This work is implemented in 1.1 V, 65-nm CMOS technology. Post-layout results show that the proposed hybrid circuit operates at a data rate of 16-Gbps/ch over a 3-mm and 10-mm on-chip interconnects in a eight parallel channel environment and it achieves a transmitted signal swing of 520 mV and a received signal swing of 108 mV for a power consumption of 35.65 mW. In summary, interface circuits such as equalizers are proposed to nullify the adverse effects of high loss communications channels and hybrid circuit is proposed to increase the overall throughput of the system.

Abstract

A hand-drawn sketch is a convenient way to search for an image or a video from a database where examples are unavailable or textual queries are too difficult to articulate. In this thesis, we have tried to propose solutions for some problems in sketch-based multimedia retrieval. In case of image search, the queries could be approximate binary outlines of the actual objects. In case of videos, we consider the case where the user can specify the motion trajectory using a sketch, which is provided as a query. However there are multiple problems associated with this paradigm. Firstly, different users sketch the same query differently according to their own perception of reality. Secondly, sketches are sparse and abstract representations of images and the two modalities can not be compared directly. Thirdly, compared to images, datasets of sketches are rare. It is very difficult, if not impossible to train a system with sketches of every possible category. The features should be robust enough to retrieve classes that were not a part of training. In this thesis, the work can be broadly divided into three parts. First, we develop a motion-trajectory based video retrieval strategy and propose a representation for sketches that aims to reduce the perceptual variability among different users. We also propose a novel retrieval strategy, which combines multiple feature representations for a final result using a cumulative scoring mechanism. In order to tackle the problem of multiple modalities, we propose a sketch-based image retrieval strategy by mapping the two modalities into a lower dimensional sub-space where they are maximally correlated. We use Cluster Canonical Correlation Analysis (c-CCA), a modified version of standard CCA, for the mapping. Finally, we investigate the use of semantic features derived from a Convolutional Neural Network, and extend the idea of sketch-based image retrieval to the task of zero-shot learning or unknown class retrieval. We define an objective function for the network such that, while training, a close miss is penalized less than a distant miss. Our training encodes semantic similarity among the different classes. We perform experiments to evaluate our algorithms on well known datasets and our results show that our features perform reasonably well in challenging scenarios.

Abstract

Software Quality has been an important economic factor for every successful Software Product. It is quite challenging for large-scale software products, especially integrated software products to maintain its quality after every version release. Integrated Software Product is a software that combines two or more sub-products to achieve a similar business goal. These sub-products are interconnected to each other and share common packages of programs to form one application. Unlike non-integrated software products, integrated software products are complex in design. They require detailed exploration on the spread of a defect across entire integrated product suite along with causal analysis so as to improve overall product quality. Either due to security constraints or due to complicated workflow, quality engineers may not be able to access and analyze the source code of all sub-products together in a large scale integrated software product. Thus investigating defect dependency in large scale integrated software product is difficult. In this thesis, we present our efforts towards exploring various ways to estimate defect widespread or defect dependency of a defect in a large scale integrated software product. Current approaches are mostly non-generic and are developed for a specific product design with domain constraints and with predefined conditions. Very few approaches were found to be adaptable for large software products. However, they were neither evaluated nor validated using a real-time defect dataset of a large scale integrated software product. This motivated us to present a few approaches to estimate the defect widespread or defect dependency of a defect in a large scale integrated software product. Our approaches are heuristic by design, this is to achieve a generalized way to study the defect dependency in a software product of any domain and size. These approaches help product owners of an integrated software product to prioritize defects based on its widespread across the large software product and also help quality teams to evaluate the defect dependency so as to address dependent defects based on artifacts accessible to them. As part of our initial research, we examined Defect Dependency calculation using the concept of Generalized Dependency Degree method which is measure a of studying the dependency of an element over another. This method was primarily inspired by dependency degree approach from Rough Set Theory. We formulated this method into an approach by representing all recorded defects per sub product in a large scale integrated software product as onevset. We calculated Generalized Dependency Degree between defect sets of each sub-product so as to estimate the defect dependency over another. We implemented this approach on a real-time industrial defect dataset of a large scale integrated Human vi vii Resource Management product. Also captured significant results across various version releases. This is a simple and a generic approach to implement on a large software products. However, the level of abstraction on defect dependency calculation for a large software was limited to defects recorded at sub-product only. This prompted us to explore new ways to study defect dependency at a deeper level of abstraction in a large software product. Correlation analysis in statistics is one of the efficient way to understand the independence or dependency of elements in a given relation. We explored various ways on implementing correlation analysis on a defect dataset and formulated an approach based on correlation to study defect dependency between defects recorded in a large software product. For a better level of generalization, we have grouped defects recorded per feature in a large software product and performed defect correlation analysis to understand the defect dependency among defect sets of available software features. We evaluated this approach against Open Source (NetBeans IDE) defect dataset and captured noteworthy observations. We were able to record list of most defective features which are unstable when deployed together in a large software product.These results help product managers to prioritize defects recorded in defective features and focus towards building a stable software. However, product managers still may not be able to isolate the exact cause-effect of a single defect over entire large software product i.e. with the above proposed methods to study defect dependency, it is difficult to estimate the dependency of a single defect over entire large software product. This stimulated us towards exploring defect dependency at the lowest level of abstraction in a large software product. We composed another approach with a model framework as basis to evaluate defect dependency so as to fill this gap. We have implemented our approach using an i* modelling framework on a few real time software projects and have recorded notable observations. The approaches composed as part of th

Abstract

With the rapid growth of camera-based mobile devices, applications that answer questions such as, “What does this sign say?" are becoming increasingly popular. This is related to the problem of optical character recognition (OCR) where the task is to recognize text occurring in images. The OCR problem has a long history in the computer vision community. However, the success of OCR systems is largely restricted to text from scanned documents. Scene text, such as text occurring in images captured with a mobile device, exhibits a large variability in appearance. Recognizing scene text has been challenging, even for the state-of-the-art OCR methods. Many scene understanding methods recognize objects and regions like roads, trees, sky in the image successfully, but tend to ignore the text on the sign board. Towards filling this gap, we devise robust techniques for scene text recognition and retrieval in this thesis. This thesis presents three approaches to address scene text recognition problems. First, we propose a robust text segmentation (binarization) technique, and use it to improve the recognition performance. We pose the binarization problem as a pixel labeling problem and define a corresponding novel energy function which is minimized to obtain a binary segmentation image. This method makes it possible to use standard OCR systems for recognizing scene text. Second, we present an energy minimization framework that exploits both bottom-up and top-down cues for recognizing words extracted from street images. The bottom-up cues are derived from detections of individual text characters in an image. We build a conditional random field model on these detections to jointly model the strength of the detections and the interactions between them. These interactions are top-down cues obtained from a lexicon-based prior, i.e., language statistics. The optimal word represented by the text image is obtained by minimizing the energy function corresponding to the random field model. The proposed method significantly improves the scene text recognition performance. Thirdly, we present a holistic word recognition framework, which leverages scene text image and synthetic images generated from lexicon words. We then recognize the text in an image by matching the scene and synthetic image featureswith our novel weighted dynamic time warping approach. This approach does not require any language statistics or language specific character-level annotations. Finally, we address the problem of image retrieval using textual cues, and demonstrate large-scale text-to-image retrieval. Given the recent developments in understanding text in images, an appealing approach to address this problem is to localize and recognize the text, and then query the database, as in a text retrieval problem. We show that this approach, despite being based on state-of-the art methods, is insufficient, and propose an approach without relaying on an exact localization and recognition pipeline. We take a query-driven search approach, where we find approximate locations of characters in the text query, and then impose spatial constraints to generate a ranked list of images in the database. We evaluate our proposed methods extensively on a number of scene text benchmark datasets, namely, street view text, ICDAR 2003, 2011 and 2013, and a new dataset IIIT 5K-word, we introduced, and show better performance than all the comparable methods. The retrieval performance is evaluated on public scene text datasets as well as three large datasets, namely, IIIT scene text retrieval, Sports-10K and TV series-1M, we introduced.