Abstract

Displays have seen many improvements over the years. With advance- ments in pixel resolution, color gamut, vertical refresh rates and power con- sumption, displays today have become more personal and accessible devices for sharing and feeding information. Advancements in computer human interaction have provided novel interactions with current displays. Touch panels are now common and provide better interaction with the cyberworld shown on a device. Displays today have many shortcomings still. These include a rectangular shape, low color gamut, low dynamic range, lack of simultaneous focus and context in a scene, lack of 3D viewing, etc. Eorts are being made to create better and more natural displays than 2D at rect- angular screens we see today. Much research has gone into designing better displays including 3D displays, focus and context displays, HDR displays, etc. Such displays enhance a display directly using device level technologies such as physical, chemical and metallurgical means. This approach has been taken in the past but proves expensive and is hard to scale to better standards that may be needed in color, refresh rate, spatial and intensity resolution in the coming future. New paradigms must be explored to generate better displays that may provide better user experiences and are easy to view and interact with. The conventional direct approach requires new physical properties to be constantly discovered in order to push the envelop in display design. Such technologies are hard to come by and may take years to prove their merit, for them to be acceptable in the display design pipeline. An alternate is to use computation to enhance displays. Today compu- tation is easily available and cheap. CPUs follow the Moores law for their ever expanding compute capabilities. Multi-core CPU architectures are now common - even in hand held devices. GPUs consisting of 1500 cores deliver 1:5Tera FLOPS for $500 today. This abundance of compute capability has been applied to various systems with much success. Many areas including computational photography, computational biology and human computer in- teractions take advantage of computation to enhance a base system. Displays too have been shown to benet from such an approach. Fish tank virtual reality is an example of this, 3D viewing can be enabled for a head tracked viewer using standard displays in conjunction with computer graphics. Com- putation can thus be used to enhance displays or remove their shortcomings. Many displays are underway that combine computation with other means to provide better and natural user experiences. These may combine computa- tion with optics, display arrangement, metallurgy, sensors, etc. to bring out more than what is available on existing displays. Such displays can collec- tively fall under the term Computational Displays. There are two aspects to enhance displays using computation: using hard- ware modications and/or using algorithmic design. Computational displays, as a subject, covers both these grounds. While hardware modications can range from a simple arrangement of display elements to designing a com- pletely new hardware unit for a display system, the cost factor must be considered along with display applicability. Algorithmic design outshines hardware modication in this regard as it allows the use of any hardware and holds no bound on the design. Due to easy availability of computational devices much work can be ported to computational elements with limited dependence on hardware modication. We follow this line of thought in this thesis. We propose computational displays which employ computation to economically alleviate some of the shortcomings of todays displays us- ing algorithmic modication with limited use of hardware modication. We demonstrate the idea using four prototypes, which collectively demonstrate how computation can be used to enhance displays. Specically we propose solutions to the 3D viewing, the focus+context and the limited color resolu- tion problems in this thesis. We create four systems to this end, two dealing with rendering view-dependent 3D scenes to piecewise planar and non-planar surfaces, another giving a framework to render massive environments inter- actively to a tiled display wall, achieving focus and context for the scene, and the last providing better intensity resolution using three mixing meth- ods. The systems work on top of existing display methodologies and are independent of any specic display technology. This makes our systems scal- able to any displaying method and enhances the same using computation. In conclusion, we argue that computation/algorithms should be built into the displays themselves to enhance the visual experience.

Abstract

The rapid evolution of information technology (IT) has prompted a massive growth in digitizing books. Accessing these huge digital collections require solutions, which will enable the archived materials to be searchable. These solutions can only be acquired through research in document image understanding. In the last three decades, many signicant developments have been made in the recognition of Latin-based scripts. The recognition systems for Indian languages are very far behind the European language recognizers like English. The diversity of archived printed document poses an additional challenge to document analysis and understanding. In this work, we explore the recognition of printed text in Telugu, a south Indian language. We begin our work by building the Telugu script model for recognition and adopting an existing optical character recognition system for the same. A comprehensive study on all the modules of the optical recognizer is done, with the focus mainly on the recognition module. We then evaluate the recognition module by testing it on the synthetic and real datasets. We achieved an accuracy of 98% on synthetic dataset, but the accuracy drops to 91% on 200 pages from the scanned books (real dataset). To analyze the drop in accuracy and the modules propagating errors, we create datasets with different qualities namely laser print dataset, good real dataset and challenging real dataset. Analysis of these experiments revealed the major problems in the character recognition module. We observed that the recognizer is not robust enough to tackle the multifont problem. The classier's component accuracy varied signicantly on pages from different books. Also, there was a huge difference in the component and word accuracies. Even with a component accuracy of 91%, the word accuracy was just 62%. This motivated us to solve the multifont problem and improve the word accuracies. Solving these problems would boost the OCR accuracy of any language. A major requirement in the design of robust OCRs is the invariance of feature extraction scheme with the popular fonts used in the print. Many statistical and structural features have been tried for character classication in the past. In this work, we get motivated by the recent successes in object category recognition literature and use a spatial extension of the histogram of oriented gradients (HOG) for character classication. We conducted the experiments on 1.46 million Telugu character samples in 359 classes and 15 fonts. On this data set, we obtain an accuracy of 96-98% with an SVM classier. Typical optical character recognizer (OCR) only uses local information about a particular character or word to recognize it. In this thesis, we also propose a document level OCR which exploits the fact that multiple occurrences of the same word image should be recognized as the same word. Whenever the OCR output differs for the same word, it must be due to recognition errors. We propose a method to identify such recognition errors and automatically correct them. First, multiple instances of the same word image are clustered using a fast clustering algorithm based on locality sensitive hashing. Three different techniques are proposed to correct the OCR errors by looking at differences in the OCR output for the words in the cluster. They are character majority voting, an alignment technique based on dynamic time warping and one based on Progressive Alignment of multiple sequences. In this work, we demonstrate the approach over hundreds of document images from English and Telugu books by correcting the output of the best performing OCRs for English and Telugu. The recognition accuracy at word level is improved from 93% to 97% for English and from 58% to 66% for Telugu. Our approach is applicable to documents in any language or script.

Abstract

Dam is one of the biggest structures built on the Earth. It is known as a life line structure, as it serves the purpose of irrigation, hydro-electric power generation, flood control, domestic and industrial water supply etc., which are important for human existence. This makes dam as a reliable structure. For this reason, dam should always be designed for highest safety, resisting worst forces of nature. India is a country with over 5,100 large dams. India is also a seismically active country with over 1,000 active faults. 1988 Bihar earthquake, 1991 Uttarkashi earthquake, 1993 Killari earthquake, 1997 Jabalpur earthquake, 1999 Chamoli earthquake, 2001 Bhuj earthquake, 2002 Andaman earthquake, 2004 Sumatra earthquake, 2005 Kashmir earthquake, 2011 Sikkim earthquake are some of the earthquakes that has hit India in the recent past. Also events like 1992 Landers, 1994 Northridge, 1995 Hyogoken-Nambu and few other events that took place around the world proved how devastating an earthquake could be, particularly if it is near-field. Near-field ground motions could cause more damaging effects on structures, as they were observed to differ dramatically from the characteristics of their far-field counterparts. The propagation of fault rupture towards a site at very high velocity causes most of the seismic energy from the rupture to arrive in a single or multiple large long period pulse of motion, which occurs at the beginning of the record. This characteristic of near-field ground motions could cause damage to a wide range of structures including dams. Several dams that were built in India, which are in highly seismic zones are prone to near-field ground motions. In this regard, behavior of a concrete gravity dam subjected to near-field ground motion should be studied. In the proposed study, a concrete gravity dam is selected from the National Importance Dams of India and is numerically modelled along with its foundation and soil strata, using Applied Element Method (AEM). The dam models are analysed for the components of near-field and far-field ground motions. The comparisons between these two were drawn to understand the effects of near-field ground motions on dams. In another study, vertical displacement is given at the bed rock level to create reverse fault effect, while changing the location of dam alongside fault. Variation of displacements and stresses at the base, crest, toe, heel, neck, and on upstream and downstream sides are evaluated. For understanding the initiation and propagation of cracks, effect at various parts of the dam body is also studied. Failure of dam is observed, when subjected to near-field ground motions caused by reverse fault and strike-slip fault. Also, when subjected to fault motion, failure in the dam body is observed to be more, when the dam is modelled on hanging wall than when it is modelled on foot wall. This study proved life-line structure like dam has failed to near-field ground motion. Thus, dams should be designed to resist severe earthquakes.

Abstract

Ever so often a need arises in clinical scenarios, for integrating information from multiple images or modalities for the purposes of diagnosis and pathology tracking. Registration, the most fundamental step in such an integration, is the task of spatially aligning a pair of images of the same scene acquired from different sources, viewpoints and time. This thesis concerns the task of registration specific to three most popular retinal imaging modalities namely Color Fundus Imaging (CFI), Red-Free Imaging (RFI) and Fluoroscein Fundus Angiography (FFA). CFI is obtained under white light which enables the experts to examine the overall condition of the retina in full color. In RFI, the illuminating light is filtered to remove red color which improves the contrast between vessel and other structures. FFA is a set of time sequence images acquired under infrared light after a fluorescent dye is injected intravenously into the blood stream. This provides high contrast vessel information revealing blood flow dynamics, leaks and blockages. Retina is a part of the central nervous system (CNS) which is composed of many different types of tissues. Given this distinctive feature, a wide variety of diseases affecting different body systems uniquely affect the retina. These Systemic diseases include Diabetes, Hypertension, Atherosclerosis , Sickle cell disease, Multiple sclerosis to name a few. Recent advancements reveal a close association of retinal vascular signs to cerebrovascular, cardiovascular and metabolic outcomes. Simply put, the health of blood vessels in the eye often indicates the condition of the blood vessels (arteries and veins) throughout the body. Registration of multimodal retinal images aids in the diagnosis of various kinds of retinal diseases like Glaucoma, Diabetic Retinopathy, Age Related Macular degeneration etc. Single modality images acquired over a period of time are used for pathology tracking. Registration is also used for constructing a mosaic image of the entire retina from several narrow field images, which aids comprehensive retinal examination. Another key application area for registration is surgery, both in the planning stage and during surgery for which only optical range information is available. Fusion of these modalities also helps increase the anatomical range of visual inspection, early detection of potentially serious pathologies and assess the relationship between blood flow and the diseases occurring on the surface of the retina. The task of registering retinal images is challenging given the wide range of pathologies captured via different modalities in different ways, geometric and photometric variation, illumination artifacts, noise and other degradations. Many successful methods have been proposed in the past for the registering retinal images. A review of these methods show good performance over healthy retinal images. However, the scope of handling a wide range of pathologies is limited for most of the approaches. Further, these methods fail to register poor quality images, especially in the multimodal case. In this work, we propose a feature based retinal image registration algorithm capable of handling such challenging image pairs. At the core of this algorithm is a novel landmark detector and descriptor scheme. A set of landmarks are detected on the topographic surface of retina using Curvature dispersion measure. The descriptor is based on local projections using radon transform which characterizes local structures in an abstract sense rendering it less sensitive to pathologies and noise. Drawing essence from the recent developments in robust estimation methods, a modified MSAC(M-estimators Sample and Consensus) is proposed for false correspondence pruning. On the whole, the minor contributions at each stage of feature based registration scheme presented here are of significance. We evaluate our method against two recent schemes on three different datasets which includes both monomodal and multimodal images. The results show that our method gives better accuracy for poor quality and pathology affected images while performing on par with the existing methods on normal images.

Abstract

In the last few years, web technologies and thus social media have shown an immense growth. Social media allow users to share their experiences with others and influence connected people. Users can post their views and opinions in the form of blogs, reviews, status updates, tweets, etc. This large volume of data calls for quick and powerful automatic analytics system in order to gain useful insights about user’s tastes. Sentiment Analysis is a study of human behavior in which we extract user opinions and emotions from plain text. Sentiment Analysis is a combination of natural language processing, computational linguistics and information extraction and is applied to extract subjective information from text. Every textual information can be categorized into two main classes: Opinions and Facts. Facts are objective sentences describing some events or people or organizations which are not deniable. A good example of factual data are news articles. News articles contains the exact and correct information about the event. Opinions on the other hand are usually subjective expressions which describe people’s sentiments, feelings or emotions. The key task in sentiment analysis is to extract the attitude of the speaker or writer towards a subject. Sentiment classification can be considered as a two class (subjective vs objective) classification at top level which can be further segmented to two more classes (Positive and Negative). In general literature, sentiment analysis is primarily done as positive vs negative classification task with little (almost zero) focus on subjective vs objective classification. Web has proved to be a good source of subjective information (opinionated text) in form of reviews, blogs, forum posts, etc. but there are some cases where we face the problem of information overload. In this research, we worked on Product and Movie Reviews, Blogs and Tweets. Each of these genres of user-generated content exhibit some unique qualities which differentiate them from each other. Reviews range from few words to few paragraphs and generally talk about the specific object. Tweets on the other hand are just 140 characters and due to this character limit they are noisy, unstructured but object specific. Blogs are document like big and generally talk about one or more objects simultaneously. We can find hundreds of websites providing opinionated user-generated content, it is difficult to go through all and then make some decision. Thus, automated sentiment detection and summarization is an important task. From product and movie reviews, we extracted N-Gram based features with and without part-of-speech information and devised a scoring function to find the subjectivity expressed in the text. We also tested the same set of features on various supervised machine learning algorithms like SVM, NB, MLP, etc and proved the close correctness of our proposed scoring function. Sentiment lexicons also prove to be a useful resource in identifying the polarity of opinion expressed in text. We use SentiWordNet as our base lexicon to identify the opinion expressed in reviews. Our methods outperform the previous benchmark accuracies by 3-5%. Tweets are 140 character status updates and suffer largely in their structure and other grammatical errors. Its hard to apply NLP tools or basic rules to extract sentiment information from tweets because of its anomalies. We present a basic method to pre-process, restructure tweets and mine opinions expressed in them. We use two different datasets and perform 5 fold cross validation of our approach. Also we tested our approach on cross dataset training and testing. Proposed methods in this research show an improvement of 2-4% in classification accuracy. For English blog analysis, we propose a novel approach to perform sentiment mining and summarization based on objects (entities) in the blog. It is difficult to perform sentence level or document level sentiment classification in blogs because of their long size (in general) and contextual flow. Using basic NLP tools available, we first identify potential objects and then associated opinion towards each of these objects. Major contribution made in thesis are • Scoring function for review classification: This function can be used as a replacement to heavy duty machine learning algorithms. • Feature segregation as NLP-based and Twitter-specific features, correct pre-processing techniques for tweets. • A novel approach towards entity centric opinion mining from blogs along with entity specific opinionated dataset for English blogs. Keywords: Sentiment Analysis, Opinion Mining, User-generated Content, Blogs, Twitter, Movie Reviews, Product Reviews

Abstract

We describe the Symbiotic Software Engineering approach to produce field deployable and maintainable software systems using diverse modules developed out of distributed effort loosely tied together. These systems are heterogeneous made of different modules written in different programming languages and at different level of software engineering. This approach has been successfully applied on Machine Translation (MT) systems for nine major Indic languages in eighteen directions developed by a consortium of eleven academic/research institutions. As a result, we have engineered and deployed Sampark MT systems at http://sampark.org.in. Further a tool called Dashboard, has been developed which is based on pipe-line blackboard architecture for integration and testing of Natural Language Processing (NLP) applications. The Dashboard helps in testing of a module in isolation, as well as integration and testing of complete integrated systems. Functional module developer can avail and configure, if they so desire, the dynamic cache facility offered by the tool. It is also equipped with a user-friendly visualization tool to build, test, and integrate a system (or a subsystem) and view its component-wise performance, and step-wise processing as well. This caching facility improves performance of modules up to 20%. We have also described an approach to make MapReduce framework applicable for computationally intensive application like MT system. This framework helps to enhance the throughput of MT system uniformly by deploying it on large cluster of physical or virtual machines. Cloud gives us on-demand computing resources, so with the availability of elastic computing resources through cloud environment the completion time of any MT job irrespective of its size can be delivered within a short time.

Abstract

Learning representations for computer vision tasks has been the holy grail for the vision community for long. Research in computer vision is dedicated towards developing machines that understand image data, which takes a variety of forms such as images, video sequences, views from multiple cameras, high dimensional data from medical scanners and so on. Good representations of the data intend to discover the hidden structure in it; better insights into the nature of the data can help choose or create better features, learn better similarity measures between data points, build better predictive and descriptive models, and ultimately drive better decisions from data. Research into this field has shown that good representations are more often than not task specific: there is no single universal set of features that solves all the problems in computer vision. Consequently, feature learning for computer vision tasks is not a problem, rather a set of problems, a full fledged field of research per se. In this thesis, we seek to learn good, semantically meaningful representations for some of the popular computer vision tasks, such as visual classification, action recognition and so on. We study and employ a variety of existing feature learning approaches, and devise novel strategies for learning representations on these tasks. Additionally we compare our methods with the traditional approaches. We discuss the design choices we make, and the effects of varying the parametric variables in our approaches. We provide empirical evidence to show our representations are better at solving the tasks at hand than the traditional ones. To solve the task of action recognition, we devise a novel PLS kernel that employs Partial Least Squares (PLS) regression to derive a scalar measure of similarity between two video sequences. We use this similarity kernel to solve the tasks of hand gesture recognition and action classification. We demonstrate that our approach significantly outperforms the state of the art approaches on two popular datasets: Cambridge hand gesture dataset and UCF sports action dataset. We use a variety of approaches to tackle the popular task of visual classification. We describe a novel hierarchical feature learning strategy that uses low level Bag of Words visual words to create “higher level” features by making use of the spatial context in images. Our model uses a novel Naive Bayes Clustering algorithm to convert a 2-D symbolic image at one level to a 2-D symbolic image at the next level with richer features. On two popular datasets, Pascal VOC 2007 and Caltech 101, we demonstrate the superiority of our representations to the traditional BoW and deep learning representations. Driven by the hypothesis that most data, such as images, lies in multiple non-linear manifolds, we propose a novel non-linear subspace clustering framework that uses K Restricted Boltzmann Machines (K-RBMS) to learn non-linear manifolds in the raw image space. We solve the coupled problem of finding the right non-linear manifolds in the input space and associating image patches with those manifolds in an iterative Expection Maximization (EM) like algorithm to minimize the overall reconstruction error. Our clustering framework is comparable to the state of the art clustering approaches on a variety of synthetic and real datasets. We further employ K-RBMs for feature learning from raw images. Extensive empirical results over several popular image classification datasets show that such a framework outperforms the traditional feature representations such as the SIFT based Bag-of-Words (BoW) and convolutional deep belief networks. This thesis is an account of our efforts to do our bit to contribute to this fascinating field. We admit that research in this field will continue for a long time, for solving computer vision is still a distant dream. We hope that we have earned the right to say one day, in retrospect, that we were on the right track.

Abstract

This thesis describes the work done by me in design and development of single-robot and multi-robot exploration systems under constrained scenarios in both - structured and unstructured environments. For single robot exploration, a system was developed for road network exploration in outdoor environments with the constraints being in terms of available hardware (only camera and 2D laser range-finder for environment perception and a single laptop computer for computation) and exploration scope (vehicle moves on and explores roads only). A major challenge in designing such a system is that the uncertainties in the outdoor environment are already very high and when such additional restrictions are added in terms of hardware, the perception of environment becomes even more difficult. Apart from the physical contribution as a new platform for road network exploration, work on this system also contributed to the development of two new road intersection detection algorithms. The work on multi-robot exploration involved development of algorithms or strategies for efficient deployment of multiple robots, under a constraint that the robots should maintain continuous connectivity with a fixed base-station. Along with the continuous base-station connectivity, it is inherently expected that the system should be able to work with minimal robot hardware primarily consisting localization and mapping sensor(s) and driving actuators. Exploration of large areas using multiple robots is made possible under this constraint by using some robots as repeaters and thus, maintaining connectivity via a network with a hop chain or tree to the base station. Two strategies, termed as Hinged Chain Method (HCM) and Hinged Tree Method (HTM) with HTM being an improved version of HCM, have evolved through our work on multi-robotic exploration using this concept. HCM involves a sequential process of extending a hopping chain i.e. a chain of robots acting as repeaters from the base-station to far areas and reconfiguring the chain in a Depth- First Traversal fashion for various directions at every node. On the other hand, HTM is a two-phase parallel method evolved from HCM, which extends a tree and then subtrees in a recursive fashion to the far areas. In extreme cases (exploration tree depths of the order of number of robots), HTM gracefully degrades to HCM due to a heuristic cascade in our implementation. The proposed methods can also be further extended for 3D exploration scenarios, such as a building with multiple floors or rubble as it uses a tree/graph network to represent the connectivity map. These methods are believed to be very useful for the scenarios where a large number of low-cost robots need to be deployed for exploration under hazardous conditions such as Search and Rescue Scenarios, Planetary Exploration and military reconnaissance. The work done in these exploration systems also uncovered some common aspects in the design of constrained exploration systems, which are also presented in the conclusion of the thesis These aspects have a good scope for further research from system design and integration perspective when considering constrained exploration systems.

Abstract

Accurate and Robust estimation of optical flow continues to be of interest due to the deep penetration of digital cameras into many areas including robot navigation and video surveillance applications. The canonical approach to the flow estimation relies on local brightness constancy which has limitations. In this thesis, we re-examine the optic flow problem and formulate an alternate hypothesis that optical flow is an apparent motion of local information across frames and propose a novel framework to robustly estimate flow parameters. Pixel-level matching approach has been implemented according to the proposed formulation in which optical flow is estimated based on local information associated with each pixel. Self information and a variety of divergence measures have been investigated for capturing the local information. Results of benchmarking with the Middlebury dataset show that the proposed formulation is comparable to the top performing methods in accurate flow computation. The distinguishing aspects however are that these results hold for small as well as large displacements and the flow estimation is robust to distortions such as noise, illumination changes, non-uniform blur etc. Thus, the local information based approach offers a promising alternative to computing optical flow. We also developed a method to remove motion blur from frames by using the information measures. The effectiveness of the proposed motion estimation approach is also demonstrated on extraction of structure from motion of synthetic microtexture patterns, cardiac ultrasound sequences and colorization of black and white videos.

Abstract

Edge detection is an important image processing technique with wide range of applications. Several edge detection algorithms have been developed in the past few decades, however no single algorithm is suitable for all types of applications. Face recognition is one such important application in which edge detection plays a key role. Edge maps help in representing faces as a single unit. Here we survey the existing edge detection techniques and their suitability for extracting edges from facial images under noisy conditions. Then we introduce the proposed algorithm which is optimized for extracting edge maps from facial images under normal and noisy conditions. We employ several improvised techniques suitable for facial images. Composite gradient variation measures help in removing spurious edge points from edge maps. Using weighted wide convolution kernels reduce noise in the image while preserving the genuine edges. Similarly, low thresholding and dual maxima detection extracts edge points along face boundaries, curvatures etc... Also the proposed method is a gradient based technique which employs partial first derivates of the image maps. Generally gradient based techniques are computationally less intensive compared to other methods. The proposed method is optimized for extracting edges from images corrupted by various types of noises. Performance of several existing edge detectors is severely degraded under noisy conditions. However, using the proposed method we are able to consistently detect good quality edge maps under various levels of noises. We demonstrate this by extracting edges from images degraded by Gaussian noise, Speckle noise and Salt and Pepper noise. For performance measurement and evaluation we use Pratt’s Figure of Merit and Signal to Noise statistics. Using these measures we have compared several conventional edge detection methods with the proposed method. Also we have run the proposed method on Berkeley’s BSDS500 image dataset and compared with Pb boundary detection algorithm. Experimental results are promising and show that we are able to extract slightly better quality edge maps both under zero noise conditions and under var- ious levels of noises.