Abstract

In this paper, we propose a framework for content level ac-cess to the scanned pages of Digital Library of India (DLI).The current Optical Character Recognition (OCR) systems are not robust and reliable enough for generating accurate text from DLI pages. We propose a search scheme which fuses noisy OCR output and holistic visual features for con-tent level access to the DLI pages. Visual content is cap-tured using Bag of Visual Words (BoVW) approach. We show that our fusion scheme improves over the individual methods in terms of mean Average Precision (mAP) and mean precision at 10 (mPrec@10). We exploit the fact that OCR has a high precision while BoVW has a high recall.We use a modified edit distance to improve the order of re-sults ranked by BoVW. Experiments are carried out on largedatasets of DLI pages in Hindi and Telugu languages. Wevalidate our method on more than 10,000 pages and 4 Mil-lion words, and report a mAP of around 0.8 and mPrec@10of more than 0.9. We show improvements over BoVW by introducing query expansion. We also demonstrate a textual query interface for the search system.

Abstract

Instance retrieval has emerged as a promising research area with buildings as the popular test subject. Given a query im-age or region, the objective is to find images in the database containing the same object or scene. There has been a recents urge in efforts in finding instances of the same building in challenging datasets such as the Oxford 5k dataset[19], Ox-ford 100k dataset and the Paris dataset[20].We ascend one level higher and pose the question:Are Buildings Only Instances?Buildings located in the same geographical region or constructed in a certain time period in history often follow a specific method of construction.These architectural styles are characterized by certain fea-tures which distinguish them from other styles of architec-ture. We explore, beyond the idea of buildings as instances,the possibility that buildings can be categorized based on the architectural style. Certain characteristic features distin-guish an architectural style from others. We perform exper-iments to evaluate how characteristic information obtained from low-level feature configurations can help in classifica-tion of buildings into architectural style categories. Encour-aged by our observations, we mine characteristic features with semantic utility for different architectural styles from our dataset of European monuments. These mined features are of various scales, and provide an insight into what makes a particular architectural style category distinct. The utility of the mined characteristics is verified from Wikipedia

Abstract

Text line segmentation is a basic step in any OCRsys-tem. Its failure deteriorates the performance of OCRen-gines. This is especially true for the Indian languages due to the nature of scripts. Many segmentation algorithms are proposed in literature. Often these algorithms fail to adapt dynamically to a given page and thus tend to yield poor seg-mentation for some specific regions or some specific pages. In this work we design a text line segmentation post processor which automatically localizes and corrects the segmentation errors. The proposed segmentation post processor, which works in a “learning by examples” framework, is not only independent to segmentation algorithms but also robust to the diversity of scanned pages.We show over 5% improvement in text line segmentation on a large dataset of scanned pages for multiple Indian lan-guages

Abstract

Constructing global image representations from local fea-ture descriptors is a common step in most visual classifica-tion tasks. Traditionally, the Bag of Features (BoF) rep-resentations involving hard vector quantization have been used ubiquitously for such tasks. Recent works have demon-strated superior performance of soft assignments over hard assignments. Fisher vector representations have been shown to outperform other global representations on most bench-mark datasets. Fisher vectors (i) use soft assignments, and(ii) reduce information loss due to quantization by captur-ing the deviations from the mean. However, the Fisher vec-tor representations are huge and the representation size in-creases linearly with the vocabulary size. Recent findings report that the classification performance of Fisher vectors is proportional to the vocabulary size. Computational and storage requirements, however, discourage the use of arbi-trarily large vocabularies. Also, Fisher vectors are not in-herently discriminative. In this paper, we devise a novel strategy to compute sparse Fisher representations. This al-lows us to increase the vocabulary size with little computa-tion and storage overhead and still attain the performance ofa larger vocabulary. Further, we describe an approach to en-code class-discriminative information in the Fisher vectors.We evaluate our method on four popular datasets. Empir-ical results show that our representations consistently out-perform the traditional Fisher Vector representations andare comparable to the state of art approaches.

Abstract

MRF models have shown state-of-the-art performance for many computer vision tasks. In this work, we propose a non-local MRF model for image completion problem. The goal of image completion is to fill user specified “target” region with patches of “source” regions in a way that is visually plausible to an observer. We represent the patches in the target region of the image as random variables in an MRF, and introduce a novel energy function on these variables. Each variable takes a label from a label set which is a collection of patches of the source region. The quality of the image completion is determined by the value of the energy function. The non-locality in the MRF is achieved through long range pairwise potentials. These long range pairwise potentials are defined to capture the inherent repeating patterns present in heritage architectural images. We minimize this energy function using Belief Propagation to obtain globally optimal image completion. We have tested our method on a wide variety of images and shown superior performance over previously published results for this task.

Abstract

Frequent Itemset Mining (FISM) attempts to find large and frequent itemsets in bag-of-items data such as retail market baskets. Such data has two properties that are not naturally addressed by FISM: (i) a market basket might contain items from more than one customer intent (mixture property) and (ii) only a subset of items related to a customer intent are present in most market baskets (projection property). We propose a simple and robust framework called LOGICAL ITEMSET MINING (LISM) that treats each market basket as a mixture-of, projections-of, latent customer intents. LISM attempts to discover logical itemsets from such bagof-items data. Each logical itemset can be interpreted as a latent customer intent in retail or semantic concept in text tagsets. While the mixture and projection properties are easy to appreciate in retail domain, they are present in almost all types of bag-of-items data. Through experiments on two large datasets, we demonstrate the quality, novelty, and actionability of logical itemsets discovered by the simple, scalable, and aggressively noise-robust LISM framework. We conclude that while FISM discovers a large number of noisy, observed, and frequent itemsets, LISM discovers a small number of high quality, latent logical itemsets.

Abstract

In this paper, we propose a recognition scheme for the Indian script of Devanagari. Recognition accuracy of Devanagari script is not yet comparable to its Roman counterparts. This is mainly due to the complexity of the script, writing style etc. Our solution uses a Recurrent Neural Network known as Bidirectional LongShort Term Memory (BLSTM). Our approach does not require word to character segmentation, which is one of the most common reason for high word error rate. We report a reduction of more than 20% in word error rate and over 9% reduction in character error rate while comparing with the best available OCR system.

Abstract

Computing similarities between data samples is a fundamental step in most Pattern Recognition (PR) tasks. Better similarity measures lead to more accurate prediction of labels. Computing similarities between video sequences has been a challenging problem for the PR community for long because videos have both spatial and temporal context which are hard to capture. We describe a novel approach that employs Partial Least Squares (PLS) regression to derive a measure of similarity between two tensors (videos). We demonstrate the use of this tensor similarity measure along with SVM classifiers to solve the tasks of hand gesture recognition and action classification. We show that our methods significantly outperform the state of the art approaches on two popular datasets: Cambridge hand gesture dataset and UCF sports action dataset. Our method requires no parameter tuning.

Abstract

. In this paper, we propose the canonical correlation kernel (CCK), that seamlessly integrates the advantages of lower dimensional representation of videos with a discriminative classifier like SVM. In the process of defining the kernel, we learn a low-dimensional (linear as well as nonlinear) representation of the video data, which is originally represented as a tensor. We densely compute features at single (or two) frame level, and avoid any explicit tracking. Tensor representation provides the holistic view of the video data, which is the starting point of computing the CCK. Our kernel is defined in terms of the principal angles between the lower dimensional representations of the tensor, and captures the similarity of two videos in an efficient manner. We test our approach on four public data sets and demonstrate consistent superior results over the state of the art methods, including those that use canonical correlations.

Abstract

. Category based image search, where the goal is to retrieve images of a specific category from a large database, is becoming increasingly popular. In such a setting, the query is often a classifier. However, the complexity of the classifiers (often SVMs) used for this purpose hinders the use of such a solution in practice. Problem becomes paramount when the database is huge and/or the dimensionality of the feature representation is also very large. In this paper, we address this issue by proposing a novel method which decomposes the query classifier into set of known eigen queries. We use their precomputed results (or scores) for computing the ranked list corresponding to novel queries. We also propose an approximate algorithm which accesses only a fraction of the data to perform fast retrieval. Experiments on various datasets show that our method reports high accuracy and efficiency. Apart from retrieval, the proposed method can also be used to discover interesting new concepts from the given dataset.