Abstract

wo-view stereo problem is a well researchedproblem in 3D computer vision. Algorithms proposedin the past have focussed on rectified stereo imageswhere the epipolar lines are parallel to the horizon-tal axis.The general problem of computing stereocorrespondences for unrectified images without anyknowledge of calibration parameters is an importantproblem but unexplored as yet. Our idea in this paperis to predict depth maps from two unrectified stereoimages using a modified Flownet architecture. Since,datasets for depth map reconstruction for unrectifiedstereo images for deep learning do not exist, we havecreated a dataset of turn table sequences of 3D modelsfrom Google 3D warehouse. Following the conceptsof Attention modelling, we implement an architecturefor combining correlations computed at multipleresolutions using a simple element-wise multiplicationof the correlations to aid the architecture to resolvecorrespondences for textureless and repeated texturedsurfaces. Our experiments show both qualitaitve andquantitative improvements of depth maps over theoriginal Flownet architecture.

Abstract

Video highlights are a selection of the most interesting parts of a video. The problem of highlight de-tection has been explored for video domains like egocentric,sports, movies, and surveillance videos. Existing methods are limited to finding visually important parts of the video but does not necessarily learn semantics. Moreover, the available benchmark datasets contain audio muted, single activity,short videos, which lack any context apart from a few keyframes that can be used to understand them. In this work,we explore highlight detection in the TV series domain, which features complex interactions with the surroundings. The existing methods would fare poorly in capturing the video semantics in such videos. To incorporate the importance of dialogues/audio, we propose using the descriptions of shots of the video as cues to learning visual importance.Note that while the audio information is used to determine visual importance during training, the highlight detection still works using only the visual information from videos.We use publicly available text ranking algorithms to rank the descriptions. The ranking scores are used to train a visual pair wise shot ranking model (VPSR) to find the highlights of the video. The results are reported on TV series videos of the Video Set dataset and a season of Buffy the Vampire SlayerTV series.

Abstract

Person recognition methods that use multiple body regions have shown significant improvements over traditional face-based recognition. One of the primary challenges in full-body person recognition is the extreme variation in pose and view point. In this work,(i) we present an approach that tackles pose variations utilizing multiple models that are trained on specific poses, and combined using pose-aware weights during testing.(ii) For learning a person representation, we propose a network that jointly optimizes a single loss over multiple body regions.(iii) Finally, we introduce new benchmarks to evaluate person recognition in diverse scenarios and show significant improvements over previously proposed approaches on all the benchmarks including the photo album setting of PIPA.

Abstract

Transgender face recognition is gaining increasing attention in the face recognition community because ofits potential in real life applications. Despite extensive progress in traditional face recognition domain, it is very challenging to recognize faces under transgender setting.The gender transformation results in significant face variations, both in shape and texture gradually over time. This introduces additional complexities to existing face recognition algorithms to achieve a reliable performance. In this paper, we present a novel framework that incorporates appearance factor and a transformation factor caused due to Hormone Replacement Therapy (HRT) for recognition.To this extent, we employ the Hidden Factor Analysis(HFA) to jointly model a face under therapy as a linear combination of appearance and transformation factors.This is based on the intuition that the appearance factor captures the features that are unaffected by the therapy and transformation factor captures the feature changes due to therapy. Extensive experiments carried out on publicly available HRT transgender face database shows the efficacy of the proposed scheme with a recognition accuracy of82.36%

Abstract

We present a practical solution for generating 360◦stereo panoramic videos using a single camera. Current approaches either use a moving camera that captures multiple images of a scene, which are then stitched together to form the final panorama, or use multiple cameras that are synchronized. A moving camera limits the solution to static scenes, while multi-camera solutions require dedicated calibrated setups. Our approach improves upon the existing solutions in two significant ways: It solves the problem using a single camera, thus minimizing the calibration problem and providing us the ability to convert any digital camera into a panoramic stereo capture device. It captures all the light rays required for stereo panoramas in a single frame using a compact custom designed mirror, thus making the design practical to manufacture and easier to use. We analyze several properties of the design as well as present panoramic stereo and depth estimation results.

Abstract

Fine-grained classification is an extremely challenging problem in computer vision, compounded by subtle differences in shape, pose, illumination and appearance. While convolutional neural networks have become the versatile jack-of-all-trades tool in modern computer vision, approaches for fine-grained recognition still rely on localization of keypoints and parts to learn discriminative features for recognition. In order to achieve this, most approaches use a localization module and subsequently learn classifiers for the inferred locations,thus necessitating large amounts of manual annotations for bounding boxes and key points. In order to tackle this problem,we aim to leverage the (taxonomic and/or semantic) relationships present among fine-grained classes. The ontology tree is a free source of labels that can be used as auxiliary tasks to train a multi-task loss. Additional tasks can act as regularizers, and increase the generalization capabilities of the network. Multiple tasks try to take the network in diverging directions, and the network has to reach a common minimum by adapting and learning features common to all tasks in its shared layers.We train a multi-task network using auxiliary tasks extracted from taxonomical or semantic hierarchies, using a novel method to update task-wise learning rates to ensure that the related tasks aid and unrelated tasks does not hamper performance on the primary task. Experiments on the popular CUB-200-2011 dataset show that employing super-classes in an end-to-end model improves performance, compared to methods employing additional expensive annotations such as key points and bounding boxes and/or using multi-stage pipelines.

Abstract

In computer vision, many active illumination techniques em-ploy Projector-Camera systems to extract useful information from the scenes. Known illumination patterns are projected onto the scene and their deformations in the captured images are then analyzed. We observe that the local frequencies in the captured pattern for the mirror-like surfaces is different from the projected pattern. This property allows us to design a custom Projector-Camera system to segment mirror-like surfaces by analyzing the local frequencies in the captured images. The system projects a sinusoidal pattern and capture the images from projector’s point of view. We present segmentation results for the scenes including multiple reflections and inter-reflections from the mirror-like sur-faces. The method can further be used in the separation of direct and global components for the mirror-like surfaces by illuminating the non-mirror-like objects separately. We show how our method is also useful for accurate estimation of shape of the non-mirror-like regions in the presence of mirror-like regions in a scene.

Abstract

Modern feature extraction pipelines, especially the on esusing deep networks, involve an increasing variety of elements. With layered approaches heaping abstraction upon abstraction, it becomes difficult to understand what it is that these features are capturing. One appealing way of solving this puzzle is feature visualization, where features are mapped back to the image domain. Our work improves the generic approach of performing gradient descent (GD) in the image space to match a given set of features to achieve a visualization. Specifically, we note that coarse features of an image like blobs, outlines etc. are useful by them-selves for classification purposes. We develop an inversion scheme based on this idea by recovering coarse features of the image before finer details. This is done by modeling the image as the composition of a Laplacian Pyramid. We show that by performing GD on the pyramid in a level-wise manner, we can recover meaningful images. Results are presented for inverting a shallow network: the densely calculated SIFT as well as a deep network: Krizehvsky et al.’sImagenet CNN (Alexnet).

Abstract

Most of the traditional sketch-based image retrieval systems compare sketches and images using morphological features. Since these features belong to two different modalities, they are compared either by reducing the image to a sparse sketch like form or by transforming the sketches to a denser image like representation. However, this cross-modal transformation leads to information loss or adds undesirable noise to the system. We propose a method, in which, instead of comparing the two modalities directly, across-modal correspondence is established between the images and sketches. Using an extended version of Canonical Correlation Analysis (CCA), the samples are projected onto a lower dimensional subspace, where the images and sketches of the same class are maximally correlated. We test the efficiency of our method on images from Caltech, PAS-CAL and sketches from TU-BERLIN dataset. Our resultsshow significant improvement in retrieval performance with the cross-modal correspondence.

Abstract

Understanding the precise 3D structure of an environ-ment is one of the fundamental goals of computer vision and is challenging due to a variety of factors such as appearance variation, illumination, pose, noise, occlusion and scene clutter. A generic solution to the problem is ill-posed due to the loss of depth information during imaging. In this paper, we consider a specific but common situation, where the scene contains known objects. Given 3D models of a set of known objects and a cluttered scene image, we try to detect these objects in the image, and align 3D models to their images to find their exact pose. We develop an approach that poses this as a 3D-to-2D alignment problem.We also deal with pose estimation of 3D articulated objects in images. We evaluate our proposed method on Big Bird dataset and our own tabletop dataset, and present experimental comparisons with state-of-the-art methods.