Abstract

This research aims to study Indian folk paintings in terms of style, icons, and their interrelations. The thesis is an attempt to do the above through presenting a formal theory of paintings, built upon, and developed from pre-existing formal theories in India and the west. We have selected Indian folk paintings as a case to demonstrate our formal theory for the following reasons: first, Indian folk paintings have not been studied in terms of their formal aspects; second, their unique compositional sense of style stability and iconic similarity have remained under-researched. The thesis begins by explaining the nature of space of the painting. It gradually develops the following five core chapters: (1) Concept of style (2) Concept of icon (3) Blending of style and icons in paintings (4) Case study of Indian folk paintings and (5) Formal analysis of folk paintings of India. The idea developed in the above core chapters is to study Indian folk paintings in terms of their artistic qualities. The styles and patterns of the Indian folk paintings are as diverse as the nation. However, Indian folk paintings show diversity in style and similarity in iconic representations. For example, one can see similar iconic representations of snake, elephant, tiger and even humans being repeated across various Indian folk painting styles. The purpose of our research is to qualify these differences in style and study the artistically and contextually informed aspects influencing an artist’s craft. We will describe and analyse folk paintings in terms of their formal aspects, which helps to develop and shape our research process. Finally, to validate our research efforts in qualifying style in Indian folk paintings, with undertake an evaluation employing existing arthistorical theories and modules.From the above study, this thesis will attempt to derive and explain formal features guiding the style of an artist and her art, in this case folk paintings from India. Our thesis leads further to the following inquiry- how can we define the concept of style in painting? What are the definitions of icons? What is the nature of space of a painting which allows blending of the style and icon? What is the role of memory and imagination in creating icon and understanding a pictorial style? “If the icon is what an artist wants to expose, the style is the medium for this exposure” (Merleau-Ponty, 1993). Painting reflects the ‘way’ (style/action) and ‘what’ (icon) in a concrete form. In painting, style/action can be understood only through strokes. “These strokes are traces of the artist's moving hand. The hand movements are guided by the icon, which is etched in the artist's mind. Strokes are nothing but the parts of that icon” (MerleauPonty, 1993). For example, if the intention is to draw a tiger, one starts by drawing its parts like head, tail, body. Here the strokes are syntactic, which reveal the semantics of the icon to a spectator. Only the strokes and patches are visible in a painting. “The primary sine qua non to create or to understand icons is imagination” (Feldman, 1967). The artist using her imagination to create artwork is understandable. However, it is indeed intriguing how the viewer comprehends the artist's mind by using her imaginative prowess. How does the viewer get access to the world of the artist's imagination? The artist makes her art accessible through her artworks. The strokes and colour patches in the painting become paths through which the viewer enters the world of the artist's imagination. This research is about the role of style and icon in creating these paths and the journey that the artist undergoes while creating and, the viewer, while seeing. Thisstudy involves an analysis of five Indian folk painting styles. They belong to five different Indian states: Telangana, Odisha, West Bengal, Bihar, and Rajasthan. We tried to trace the changes and developments within a particular style and clarify the core elements in each of the region’s (mentioned above) art school style and their compositional principles. For this, we have made an exhaustive and elaborate database of thousands of images of paintings from each style. The images belong to varying time periods and are created by several artists. We adopted qualitative methodologies to interact and interview artists to know and get a clear perception of their painting styles. We have videotaped the interviews to aid in capturing the process in the making of a painting from the perspective o the artist.

Abstract

Crowd counting is an important task in security, surveillance and monitoring. There are many competitive benchmark datasets available in this domain. The data distribution in the crowd counting datasets show a heavy-tailed and discontinuous nature. This nature of the dataset is majorly ignored while building solutions to this problem. However, the skew in datasets contradicts few assumptions made by the stages of the training pipeline. As a consequence of the skew in the dataset, unacceptably large standard deviation wrt to the customarily used performance measures (MAE, MSE) is observed. To address these issues, this thesis provides modifications that incorporate the dataset skew in training and evaluation pipelines. In the training pipeline, to enable principled and balanced minibatch sampling, a novel smoothed Bayesian binning approach is presented that stratifies the entire count range. Further, these strata are sampled to construct uniform minibatches. The optimization is upgraded with a novel strata-aware cost function that can be readily incorporated into the existing crowd counting deep networks. In the evaluation pipeline, as an alternative to the customary evaluation MAE, this thesis provides three alternative evaluation measures. Firstly, a strata-level performance in terms of mean and standard deviation gives range specific insights. Secondly, relative error perspective is brought in by using a novel Thresholded Percentage Error Ratio (TPER). Lastly, a localization included counting error metric Grid Average Mean absolute Error (GAME) is used to evaluate the different networks. In this thesis, it is shown that proposed binning-based modifications retain their superiority wrt the novel strata-level performance measure. Overall, this thesis contributes a practically useful training pipeline and detail-oriented characterization of performance for crowd counting approaches.

Abstract

This thesis proposes solutions to two major problems (highlighted) faced when developing a selfdriving car using traditional modular approaches. That is to sense, percieve, localize/map, predict and plan/control. The first one is doing real-time Simultaneous Localization and Mapping SLAM at low sampling rates and/or at high linear or angular velocities. The keynote of the work showcases SROM’s ability to maintain localization at low sampling rates or at high linear or angular velocities where most popular LiDAR-based localization approaches get degraded fast. We also demonstrate SROM to be computationally efficient and capable of handling high-speed maneuvers. It also achieves low drifts without the need for any other sensors like IMU and/or GPS. Our method has a two-layer structure wherein first, an approximate estimate of the rotation angle and translation parameters are calculated using a Phase Only Correlation (POC) method. Next, we use this estimate as an initialization for a point-to-plane ICP algorithm to obtain fine matching and registration. Another key feature of the proposed algorithm is the removal of dynamic objects before matching the scans. This improves the performance of our system as the dynamic objects can corrupt the matching scheme and derail localization. Our SLAM system can build reliable maps at the same time generating high-quality odometry. We exhaustively evaluated the proposed method in many challenging highways/country/urban sequences from the KITTI dataset and the results demonstrate better accuracy in comparisons to other state-of-the-art methods with reduced computational expense aiding in real-time realizations. We have also integrated our SROM system with our in-house autonomous vehicle and compared it with the state-of-the-art methods like LOAM and LeGO-LOAM. The second one is the problem of an agent/robot with non-holonomic kinematics avoiding dynamic and static obstacles whose state and velocity noise are non-parametric. Additionally, there may be bounds/constraints on the configurational space of the robot in the form of lane/corridor boundaries. In this work we consider the problem of an agent/robot with non-holonomic kinematics avoiding dynamic and static obstacles. Additionally there may be bounds/constraints on the configurational space of the robot in the form of lane/corridor boundaries. The robot’s state and velocity noise, the lanes, the obstacles, and the robot’s control noise are modelled as non-parametric distributions as Gaussian assumptions of noise models are violated in real-world scenarios. Under these assumptions, we formulate a robust MPC that samples robotic controls effectively in a manner that aligns the robot to the goal state while avoiding obstacles and staying within the lane bounds under the duress of such non-parametric noise. In particular, the MPC incorporates a distribution matching cost that effectively aligns the distribution of the current collision cone to a certain desired distribution whose samples are collision-free. This cost is posed as a distance function in the Hilbert Space, whose minimization typically results in the collision cone samples becoming collision-free. We show tangible performance gains compared to methods that model the collision cone distribution by linearizing the Gaussian approximations of the original nonparametric state and obstacle distributions. We also show superior performance to methods that pose a chance constraint formulation of the Gaussian approximations of non-parametric noise without subjecting such approximations to further linearizations. The performance gain is shown both in terms of trajectory length and control costs that vindicates the efficacy of the proposed method. Finally, we show the proposed method being used to navigate with a non holonomic differential drive robot in real-time in a realistic setting in Gazebo with dynamic and static obstacles. To the best of our knowledge, this is the first presentation of non-holonomic collision avoidance of stationary obstacles, moving obstacles and lane constraints in the presence of non-parametric state, velocity, actuator and lane boundary noise models.

Abstract

The last few decades have witnessed tremendous growth in the sensors industry, spurred by advancements in electronics, nanoscience, biotechnology and computer science. They are employed for a wide variety of applications and they possess the potential to play a pivotal role in biochemical applications. Among the various sensing modalities being explored, we have Radio Frequency based sensing. It studies the interaction between EM waves and matter and uses this interaction for qualitative and quantitative analysis. RF sensors offer a reduction in process complexity, speed, cost, portability and ease of integration with existing technology. In this work, Radio Frequency (RF) based sensing technique is applied for different biochemical applications. In particular, emphasis is given to combating two significant challenges of RF sensing, namely, specificity and reusability. In addition to that, the use of RF sensing in new sub-domains is explored, which would enable the development of a portable biochemical sensing platform. The sensors put into use in this work are Interdigitated Capacitor based sensor and Microstrip Patch based sensor. This sensor is reusable and can be put into use for diverse applications. Reusability of the sensor implies that it can not only be used several times, but the same sensor can detect different sets of analytes with equal efficiency. We have performed both computational and experimental studies on surfactants, nanoparticles and biomolecules using RF sensing. Surfactants are essentially surface-active agents that are amphiphilic. Above a specific concentration called the Critical Micelle Concentration (CMC), they self assemble to form micelles. The estimation of CMC is vital for all the varied applications of surfactants. The first application develops a fast, facile, efficient method for CMC detection that works well for all classes of surfactants. With their plethora of inherent advantages, Nanoparticles play vital roles in the biochemical and biomedical domains. Their study is essential to utilize them for diverse applications efficiently. There is a growing need to develop a methodology that performs preliminary characterization of nanoparticles in the absence of sophisticated laboratory facilities. In the second application, we study nanoparticles of various morphology and arrangements and characterize them. Our third application deals with developing a biomolecule detection platform that is rapid, portable, efficient and reusable. Gold nanoparticles are functionalized to improve the reusability and specificity aspects of conventional RF sensors. Further, the Biotin-Streptavidin binding is chosen to demonstrate the methodology

Abstract

The popularity of egocentric cameras and their always-on nature has lead to the abundance of daylong first-person videos. Because of the extreme shake and highly redundant nature, these videos are difficult to watch from beginning to end and often require summarization tools for their efficient consumption. However, traditional summarization techniques developed for static surveillance videos, or highly curated sports videos and movies are, either, not suitable or simply do not scale for such hours long videos in the wild. On the other hand, specialized summarization techniques developed for egocentric videos limit their focus to important objects and people. In this work, we present a novel unsupervised reinforcement learning technique to generate video summaries from day long egocentric videos. Our approach can be adapted to generate summaries of various lengths making it possible to view even one minute summaries of one’s entire day. The technique can also be adapted to various rewards, such as distinctiveness, indicativeness of the summary. When using the facial saliency-based reward, we show that our approach generates summaries focusing on social interactions, similar to the current state of the art. Quantitative comparison on the benchmark Disney dataset shows that our method achieves significant improvement in Relaxed F-Score (RFS) (32.56 vs. 19.21) and BLEU score (12.12 vs. 10.64). Finally, we show that our technique can be applied for summarizing traditional, short, hand-held videos as well, where we improve the state of the art F-score on benchmark SumMe and TVSum datasets from 41.4 to 45.6 and 57.6 to 59.1 respectively

Abstract

Random Number Generators (RNGs) are the solution for cryptographic applications to enhance hardware security. These RNGs should have three specific properties unpredictability, aperiodic, and good statistical criteria. This brief presents a True Random Number Generator (TRNG) based on Ring oscillators' jitter with MLFSR. The MLFSR is augmented with a set of prime primitive polynomials, Boolean, and nonlinear functions to attain a non-linear, unpredictable, and extended sequence period. Paper mainly focused on achieving high randomness by integrating the TRNG with a new promising crypto engine Keccak as a post-processing block, leading to more extensive security in data transfer, encryption keys, data authenticity of ICs, and IoT based applications. The TRNG design is coded in Verilog HDL and implemented on the FPGA Zed board. Hashing is performed with a throughput of 2.4Gbps at 100MHz either with RNG data or SHA data. The HRNG (Hardware Random Number Generator) is an integrated block of TRNG and hashing along with the IP cores of memory, clock Generator, and Integrated Logic Analyzer, implemented on the FPGA hardware. Evaluated the randomness of the generated non-deterministic bit streams (10Mb) using the NIST 800-22 & Diehard test suite and successfully passed.

Abstract

The launch of level 5 autonomous driving is imminent which would result in removal of controls like steering wheel, acceleration and brakes. This warrants the need for a better system to direct the self-driving agent to perform maneuvers if the passenger wishes to do so. Hence, this thesis proposes a set of solutions tackle the problem of vision and language navigation in an autonomous driving setting which has been broken down into multiple sub problems. The first problem we attempt to solve, we present a simple baseline for visual grounding for autonomous driving which outperforms the state of the art methods, while retaining minimal design choices. The framework minimizes the cross-entropy loss over the cosine distance between multiple image ROI features with a text embedding (representing the given sentence/phrase). We use pre-trained networks for obtaining the initial embeddings and learn a transformation layer on top of the text embedding.We perform experiments on the Talk2Car dataset and achieve 68.7% AP50 accuracy, improving upon the previous state of the art by 8.6%. Humans have a natural ability to effortlessly comprehend linguistic commands such as “park next to the yellow sedan” and instinctively know which region of the road the vehicle should navigate. Extending this ability to autonomous vehicles is the next step towards creating fully autonomous agents that respond and act according to human commands where knowing only the location of the object of interest in the scene is not enough. To this end, a novel task of Referring Navigable Regions (RNR), i.e., grounding regions of interest for navigation based on the linguistic command is proposed. RNR is different from Referring Image Segmentation (RIS), which focuses on grounding an object referred to by the natural language expression instead of grounding a navigable region. For example, for a command “park next to the yellow sedan,” RIS will aim to segment the referred sedan, and RNR aims to segment the suggested parking region on the road. We introduce a new dataset, Talk2Car-RegSeg, which extends the existing Talk2car dataset with segmentation masks for the regions described by the linguistic commands. We benchmark the proposed dataset using a novel transformer-based architecture. We present extensive ablations and show superior performance over baselines on multiple evaluation metrics. A downstream path planner generating trajectories based on RNR outputs confirms the efficacy of the proposed framework.

Abstract

Dialogue systems help us to access information and services more effectively. This thesis presents the work done on developing Interactive Question Answering Systems, a type of dialogue systems, for English and Telugu. The domains chosen for English was Hyderabad MMTS enquiry and for Telugu it was Hyderabad Tourism. Both the systems take input in the form of text and the output is also text. To be able to study a wider range of user behaviour both the systems are developed for a different domain and language. The MMTS Question Answering system follow the frame based approach. This model can handle spelling errors and can also deal with insufficient information in the user query. In this system, the data is stored using SQL databases. We convert the user query to a proper SQL query and extract required information from the database. We tried to port the above model into Telugu language for Tourism domain but the process failed because of a lack of proper language processing tools for Telugu language. So we had to design a new approach. The approach used for Telugu can be extended to other resource poor Indian languages as well. The dialogue model consists of two parts namely Data Management and Query Processing. Data Management deals with storing the data in a particular format which helps in easy and quick retrieval of requested information. Query Processing deals with producing a relevant system response for a user query. Our model can handle code-mixed queries which are very common in Indian languages and also handles context which is a major challenge in dialogue systems. It also handles spelling mistakes and a few grammatical errors. The model is domain and language independent. Using this model, we developed a system for Telugu language for ’Tourist places of Hyderabad’ domain. As there is no automated evaluation tool available for dialogue systems, we went for human evaluation of our system. Five people evaluated our system and the results are reported in the thesis. In this thesis, we have also made an attempt to extract adjacency pairs from conversational units with multiple utterances from each user using dialogue acts. Adjacency pairs help in automatic extraction of dialogue structure. In dealing with minimal information in utterances, this becomes useful. In the machine learning algorithms that we were using for Natural language Generation, we relied on conversational units where the speakers have uttered only one sentence in their turn as training data, This restricts the data, and a good amount of manual work goes into converting these conversations units to adjacency pairs. To ease this process, we have tried to automate the process of extracting adjacency pairs from the data using dialogue acts.

Abstract

Reinforcement Learning (RL) enables agents to learn how to perform various tasks from scratch. In domains like autonomous driving, recommendation systems, domestic service robots, and more, the need for privacy of individuals interacting with such systems has become evident. We aim to investigate the quality of solutions that claim to protect the reward functions that contain sensitive personal information, from being exploited. Optimal RL policies learned could cause a privacy breach if the policies memorize any part of the private reward. Differential Privacy (DP) - a popular technique used for computational privacy, empowers us to provide mathematically quantifiable privacy guarantees for queries made to “privatised” algorithms. Our work introduces the reward reconstruction attack that tries to reverse engineer the reward from a private policy using Inverse RL algorithms. This attack is the key part of the novel Privacy-Aware Inverse RL Analysis (PRIL) framework. We assume that the entire training process and the optimal policy is released publicly. We study the set of existing differentially-private RL policies derived from various RL algorithms including Value Iteration, Deep Q Networks (DQNs), and Vanilla Proximal Policy Optimization (PPO). We propose the new PRIL analysis framework, that performs reward reconstruction as an adversarial attack on private policies that the agents may deploy. For this, we introduce the novel reward reconstruction attack, wherein we seek to reconstruct the original reward from a privacy-preserving policy using an Inverse RL algorithm. An adversary must do poorly at reconstructing the original reward function if the agent uses a tightly private policy. Using the PRIL framework, we empirically test the effectiveness of the privacy guarantee offered by the private algorithms on multiple instances of OpenAI Gym environments of varying complexities, specifically the discrete state-space Frozen Lake domain and the continuous state-space Mountain Car domain, employing a different Linear Programming based Inverse RL method for each domain. From our experiments, we observe that there is no indication of the policy improving at reconstructing the reward with a relaxation in the budget - thus rendering all strategies ineffective at providing true reward privacy. Based on the analysis performed, we infer that there exists a gap between the current standard of privacy offered and the standard of privacy needed to protect reward functions in RL. We quantify the extent to which each private policy protects the reward function by measuring distances between the original and reconstructed rewards. We calculate the distance between the original reward and the reconstructed reward via various distance metrics. The larger the reward distance, the better the private agent’s ability to protect the sensitive reward function using which it was trained. We present the trends in test-time performance utility of private agents upon varying the level of privacy guaranteed (by altering the privacy budget during training). We also formally discover the differentially-private Bellman Update algorithm along with its proof of sensitivity. This is one of the first few works in this sub-field, and we hope that it is pursued with more rigour in the future.

Abstract

To execute meaningful real-world tasks, robots must perceive and understand the objects that surround them. Manipulation, autonomous driving, and human-robot interaction are just a few examples where robots must reason about previously unseen instances of objects. We present DRACO, a method for Dense Reconstruction And Canonicalization of Object shape from one or more RGB images. Canonical shape reconstruction—estimating 3D object shape in a coordinate space canonicalized for scale, rotation, and translation parameters—is an emerging paradigm that holds promise for a multitude of robotic applications. Prior approaches either rely on painstakingly gathered dense 3D supervision, or produce only sparse canonical representations, limiting real-world applicability. DRACO performs dense canonicalization using only weak supervision in the form of camera poses and semantic keypoints at train time. During inference, DRACO predicts dense object-centric depth maps in a canonical coordinatespace, solely using one or more RGB images of an object. Extensive experiments on canonical shape reconstruction and pose estimation show that DRACO is competitive or superior to fullysupervised methods. This thesis delves into understanding objects to capture their pose and shape from a single image. Canonicalization is the operation of mapping an input representation to an invariant representation with respect to an operation. This input representation can be a point cloud of an object. For the scope of this thesis, we fix this operation to be a 3D transformation. Lets take an example of an object, say a car. By finding a canonical frame for this car, we have its 3D reconstruction in a frame that is invariant to any 3D transformation. This is important for tasks such as registration, feature propagation and annotation. Using this invariant 3D pose we can register different poses of the car to a single consistent frame. This single consistent frame across multiple cars is called the canonical frame. Humans have an inherent understanding of this canonical frame. For instance, we know that the headlights of the car are towards its forward direction. This forward direction is with respect to our mental canonical frame and this allows us to estimate the pose of the car with respect to our view. canonicalization – the process of mapping an object instances to a category-level container – has emerged as a useful tool for category-level understanding. We can extend datasets,automatically annotate for parts, register and manipulate, if we can map every object to the canonical orientation. Our method, DRACO, estimates a canonical frame by predicting NOCS maps and reconstructs the object of interest in using a monocular image. NOCS maps are 2D images that store the location of every pixel with respect to a canonical frame that is invariant to transformation. Previous methods use pain-stakingly collected datasets to learn a dense canonical frame or weakly-supervise canonicalization for sparse keypoints. Our method uses weak-supervision in the form of semantic keypoints and camera relative motion to learn an object’s reconstruction and canonical frame which performs on par or better against fullysupervised methods. This enables estimating canonical frame for in-the-wild objects that enables canonicalizing them. Extensive experiments on our new large-scale mixed reality dataset (20K images/category) demonstrate on par shape reconstruction and canonicalization quality compared to state-of-the-art fully-supervised approaches, and weakly-supervised baselines that strictly use more information. This thesis introduces concepts of single-view depth estimation for reconstructing objects using a monocular image and delves into canonicalization in a weakly-supervised manner.