Abstract

Depression has always been an exciting topic of research due to its high prevalence and complex underlying mechanisms. Cognitive impairment, alterations in mood, emotion dysregulation, aversion to activity are just a few of the consequences of depression. Despite there being an ocean of literature present there, attempting to understand depression, we have just scratched the surface. There is still a lot to explore about the intricacies of this perplexing subject. This thesis is yet another attempt to understand and evaluate how depression changes how we understand and represent information (or concepts). In this work, our primary focus is on emotional concepts. In a nutshell, we are trying to look for answers to the question, ’Whether or not there exist any differences in the way people exhibiting moderate to severe depressive symptoms encode emotional concepts, compared to people with no or minimal depressive symptoms?’. To address this question, we first conducted a pilot study involving 48 participants. A corpus of 50 emotional words of varying hedonic and arousing content was given to the participants in this study. They were instructed to rate their affective experience of valence and arousal for each word on a 9-point self-assessment manikin (SAM) scale. This study aimed to record the Indian affective experience of emotional words and construct a comprehensive affective word database for the following main study. Fifteen words were selected from the corpus, which according to the participants’ responses, were capable of accurately capturing the required affect and arousal. These 15 words were clustered on the arousal-valence bi-dimensional model into five groups (3 belonging to each) and used as stimuli for the second study. An additional objective of the study was to compare and contrast the affective responses of the Indian population with the data collected by other studies on the western people to see if there existed any cultural differences between them. Following the pilot study, we conducted a detailed psychological study involving 107 participants and instructed them to generate properties for 15 emotional words (from the pilot study) presented to them as stimuli. Using state-of-the-art computational models, we evaluated the data collected to efficaciously shed some light on the linguistic, semantic, and sentiment differences in the properties generated by the two groups of samples (participants exhibiting no or mild symptoms and participants showing moderate to severe symptoms). These two groups were created after assessing the predisposition emotional state of the entire sample individually by using Beck Depression Inventory-II (BDI). We found an overall increased negative sentiment and decreased positive sentiment in the properties generated by the sample showing depressive symptoms. Emotion analysis of the data reflected that the sample with

Abstract

Augmented Reality and Virtual Reality(AR/VR) applications can become far more widespread if they can photo-realistically capture our surroundings and modify them in different ways. It could include editing the scene’s lighting, changing the objects’ material, or augmenting virtual objects onto the scene. There has been a significant amount of work done in this domain. However, most of these works capture the data in a controlled setting consisting of expensive setups such as light stages. These methods are impractical and cannot scale. Thus, we must design solutions that capture scenes casually from offthe-shelf devices commonly available to the public. Further, the user should be able to interact with the captured scenes and modify these scenes in exciting directions, such as editing the material or augmenting new objects into the scene. In this thesis, we study how we can produce novel views of a casually captured scene and modify them in interesting ways. First, we present a neural rendering framework for simultaneous novel view synthesis and appearance editing of a casually captured scene using off-the-shelf smartphone cameras under known illumination. Existing approaches cannot perform novel view synthesis and edit the materials of the scene objects. We propose a method to explicitly disentangle appearance from lighting while estimating radiance and learn an independent lighting estimation of the scene. This allows us to generalize arbitrary changes in the scene’s materials while performing novel view synthesis. We demonstrate our results on synthetic and real scenes. Next, we present PanoHDR-NeRF, a neural representation of an indoor scene’s high dynamic range (HDR) radiance field that can be captured casually without elaborate setups or complex capture protocols. First, a user captures a low dynamic range (LDR) omnidirectional video of the scene by freely waving an off-the-shelf camera around the scene. Then, an LDR2HDR network converts the captured LDR frames to HDR, which are subsequently used to train a modified NeRF++ model. The resulting PanoHDR-NeRF representation can synthesize full HDR panoramas from any location in the scene. We also show that the HDR images produced by PanoHDR-NeRF can synthesize correct lighting effects, enabling the augmentation of indoor scenes with synthetic objects that are lit correctly. Through these works, we demonstrate how we can casually capture scenes for AR/VR applications that the user can further edit.

Abstract

Human action recognition, with its irrefutable and varied use cases across fields of surveillance, robotics, human object interaction analysis and many more, has gained critical importance and attention in the field of compute vision. Traditionally entirely based on RGB sequences, action recognition domain has shifted focus towards using skeleton sequences due to the easy availability of skeleton data capturing apparatus and the release of large scale datasets, in recent years. Skeleton based human action recognition, having superiority in terms of privacy, robustness and computational efficiency over traditional RGB based action recognition, is the primary focus of this thesis. Ever since the release of large scale skeleton action datasets namely NTURGB+D and NTURGB+D 120, the community has solely focused on developing complex approaches, ranging from CNNs to complex GCNs and more recently transformers, to achieve the best classification accuracy for these datasets. However, in this rat race for state of the art performance, the community turned a blind eye to a major drawback at the data level which bottlenecks even the most sophisticated approaches. This drawback is where we start our explorations in this thesis. The pose tree provided in the NTURGB+D datasets contains only 25 joints, out of which only 6 joints (3 for each hand) are finger joints. This is a major drawback since only 3 finger level joints are not sufficient enough to distinguish between action categories such as ”Thumbs up” and ”Thumbs down” or ”Make ok sign” and ”Make victory sign”. To specifically address this bottleneck, we introduce two new pose based human action datasets - NTU60-X and NTU120-X. Our datasets extend the largest existing action recognition dataset, NTU-RGBD. In addition to the 25 body joints for each skeleton as in NTURGBD, NTU60-X and NTU120-X dataset include finger and facial joints, enabling a richer skeleton representation. We appropriately modify the state of the art approaches to enable training using the introduced datasets. Our results demonstrate the effectiveness of these NTU-X datasets in overcoming the aforementioned bottleneck and improving the state of the art performance, overall and on previously worst performing action categories. Pose-based action recognition is predominantly tackled by approaches that treat the input skeleton in a monolithic fashion, i.e. joints in the pose tree are processed as a whole. However, such approaches ignore the fact that action categories are often characterized by localized action dynamics involving only small subsets of part joint groups involving hands (e.g. ‘Thumbs up’) or legs (e.g. ‘Kicking’). Although part-grouping based approaches exist, each part group is not considered within the global pose frame, causing such methods to fall short. Further, conventional approaches employ independent modality streams (e.g. joint, bone, joint velocity, bone velocity) and train their network multiple times on these streams, which massively increases the number of training parameters. To address these issues, we introduce PSUMNet, a novel approach for scalable and efficient pose-based action recognition. At the representation level, we propose a global frame based part stream approach as opposed to conventional modality based streams. Within each part stream, the associated data from multiple modalities is unified and consumed by the processing pipeline. Experimentally, PSUMNet achieves the state of the art performance on the widely used NTURGB+D 60/120 dataset and dense joint skeleton dataset NTU 60-X/120-X. PSUMNet is highly efficient and outperforms competing methods which use 100%-400% more parameters. PSUMNet also generalizes to the SHREC hand gesture dataset with competitive performance. Overall, PSUMNet’s scalability, performance and efficiency make it an attractive choice for action recognition and for deployment on compute-restricted embedded and edge devices. Finally, we conclude this thesis by exploring new and more challenging frontiers under the umbrella of skeleton action recognition namely ”in the wild” skeleton action recognition and ”non-contextual” skeleton action recognition. We introduce Skeletics-152, a curated and 3D pose dataset derived from the RGB videos included in the larger Kinetics-700 dataset to explore in the wild skeleton action recognition. We further introduce, Skeleton-mimetics, a 3D pose dataset derived from recently introduced non-contextual action dataset-Mimetics. By benchmarking and analysing various approaches on these two new dataset we lay the ground for future exploration in these two challenging problems within skeleton action recognition. Overall in this thesis, we draw attention to prevailing drawbacks in the existing skeleton action datasets and introduce extensions of these datasets to counter their shortcomings. We also introduce a novel, efficient and hi

Abstract

Unmanned traffic management (UTM) has been gaining popularity these days because we are switching to autonomous navigation for drones in a real-world scenario. Unmanned Aerial Systems (UAS) Traffic Management (UTM) provides a reliable holistic solution for multi-UAV missions including air space design, motion planning, communication, control, geo-fencing, contingency management, etc. Motion planning is one of the key elements involved in UTM. Motion planning includes generating collision-free trajectories from the starting position to the goal position for the UAV in an uncertain and dynamic 3D environment. These trajectories can be planned for many kinds of UAVs, they include holonomic UAVs like a quadrotor or non-holonomic UAVs like fixed-winged aircraft (FW A). Traditionally these collision-free trajectories are planned using methods like velocity obstacles and their variants to avoid obstacles in a 3D space. These obstacles can be static or dynamic whereas the aerial vehicles can also be of various kinds like quadrotor drones or fixed-winged aircraft. So, we start with discussing an improvement on the basic velocity obstacle scheme for a single agent like a quadrotor and an FWA in an uncertain 3D environment. This improvement on the basic velocity obstacle to handle measurement and control noise (uncertainty in the environment) is done by extending it to a probabilistic version called probabilistic versions of Velocity Obstacles (PVO). We explore Probabilistic Inverse Velocity Obstacle (PIVO) as an alternative to PVO for free-flying quadrotor systems. Inverse Velocity Obstacles compute effective controls from a sequence of observations on other agents without the need to access ego state information. As a direct consequence of this, the ego state noise is not entailed in probabilistic formulations bringing in verifiable advantages in the form of reduced path lengths, less conservative maneuvers, and reduced occurrences of stopping/hovering to let others pass. These advantages are vividly tabulated in this thesis, showcasing the efficacy of PIVO as an alternative to probabilistic versions of Velocity Obstacles. In particular, we show the benefits of PIVO over PVO in sample complexity as well as overall trajectory lengths. We also show the efficacy of our probabilistic formulation in handling non-parametric and often multimodal noise distributions. Afterward, we extend our navigation and collision avoidance scheme for multiple drones of the same kind navigating in the same 3D space. Here we propose a predictive optimal control-based autonomous navigation and collision avoidance for multiple fixed-wing aircraft (FWA) or for multiple quadrotor drones that also takes into account the physical constraints of the aircraft. The proposed method is implemented in a cooperative framework, analogous to the Reciprocal Velocity Obstacle (RVO) framework for autonomous navigation and collision avoidance for FWA swarm moving in a three-dimensional (3D) space. As a result of predictive optimal control and cooperation, we get less conservative maneuvers, reduced path lengths, and less deviation from the shortest path when compared to the conventional method without prediction and cooperation. Also, the change in performance with respect to varying prediction horizons is analyzed through numerical simulations. Simulation results are provided, highlighting the advantages of the proposed method when compared to a popular method (Optimal reciprocal collision avoidance) for collision avoidance in 3D for FWA swarm and also against the FGA algorithm (Fast geometric avoidance algorithm). Finally, in a typical UTM scenario, we can encounter multiple vehicles of different kinds (heterogeneous agents) trying to navigate and avoid collisions in a 3D space. So, we try to formalize a navigation and collision avoidance scheme for this multi-agent heterogeneous system of UAVs.

Abstract

The use of cell phones is embedded in our lives. We use it for various purposes by finding answers to our curious questions, ordering services for our homes and self, social media and many more. So when we are asked to stowaway our mobile phones or use Airplane Mode when travelling by air, it feels like a violation of our freedom to use our devices. Such a policy exists because aviation authorities such as Federal Aviation Authority (FAA), Internation Civil Aviation Authority (ICAO) etc. found that these devices could pose a safety risk to airplane systems and operations. Such devices are referred to as Personal Electronic Devices (PED). There are multiple malfunctions of aircraft systems that could be caused by usage of PEDs inside the aircraft which pose a safety risk to the flight and its passengers. Airplane Mode of cell phones was introduced to help aviation industry enforce the policy of no PED usage when in flight. In this thesis we focus on studying the existing policies for usage of cell phones inside an aircraft and provide a prototype solution where we could enable cell phone’s Airplane Mode automatically without any human interaction. We studied policies enforced by different aviation authorities in different countries, how they differ from each other and their common aspects. We found most of the aviation authorities have a policy of either stowing away PEDs or using Airplane Mode during Takeoff and Landing. We also studied reports presenting technical data on why such a policy is required. We also went through reports submitted by flight crew and cabin crew related to some instances where they observed disturbances during flight due to usage of PEDs inside the aircraft. We have also included individual experiences reported by passengers while travelling on how people react and behave when asked to turn on Airplane Mode or switch off their electronic devices. After establishing the problem, we discussed solutions which could be used to either switch off cell phones in flight or enable Airplane Mode automatically without any human interaction. Three solutions proposed were, use of PICO cells inside the aircraft, use of mobile phone jammer and develop a cell phone application to enable Airplane Mode automatically based on a trigger. We discussed the pros and cons of each solution proposed. We decided to develop a cell phone application to enable or disable Airplane Mode based on a trigger. In this thesis, we also discussed technical challenges which we faced to develop such an application. We also studied social challenges which already exist due to existing policy as well as potential social challenges once our auto Airplane Mode system is used. In this thesis we also present data on how we created this application and its limitations. We have also identified some other scenarios and places where such an application can be used. At the end, we conclude our work and list out some future scope activities

Abstract

Environmental sounds convey a large amount of information regarding day-to-day activities in nature.These sounds also provide powerful means of communication. The acoustic scene of environmentalsounds contains a set of audio events, which occur over a certain duration. Human hearing enablesus recognition of specific sounds and allows the processing of these sounds continuously without anyeffort. Therefore, even in the absence of visual cues, humans can identify most of the times, events, andsounds with acoustic cues. In this regard, it is necessary to develop an Acoustic Scene Classification(ASC) system which makes the machine to behave like human beings in the context of soundscapes. AnASC task mainly focuses on portrayal of the acoustic surroundings from an audio track by choosing atextual tag for it. The motivation for this work is to make some of the electronic gadgets to be moreintelligent. This is done by incorporating the acoustic scene knowledge in context-awareness devices,listening robots, hearing aids, automatic data tagging, and so on. The main objectives of this researchwork are to address: (a) To extract desired information that characterizes the environmental sounds in anacoustic signal using known standard acoustic features and unknown features, (b) To effectively classifythe features using the standard DNN model, (c) To develop an ASC system for the dataset containingboth the clean and channel mismatch conditions, and (d) To improve the ASC system performance bycombining evidence from multiple sources.In this regard, this research work aims to address some of the issues in the development of a sys-tem that analyses and classifies acoustic scenes. We have investigated the effect of the standard featuresnamely Mel-Frequency Cepstral Coefficients (MFCC), Log-Mel Band Energies (LOGMEL), Linear Pre-diction Cepstral Coefficients (LPCC), and All-Pole Group Delay (APGD) for the representation of ASC.We have also proposed new features namely Inverted Mel-Frequency Cepstral Coefficients (IMFCC),Spectral Centriod Magnitude Coefficients (SCMC), Spectral Subband Flux Coefficients (SSFC), andSingle Frequency Filtering Cepstral Coefficients (SFFCC). The effect of these features on the acousticscene classification of Detection and Classification of Acoustic Scenes and Events (DCASE) 2017 and2018 datasets has been studied using DNN classification modelsFrom our studies, it has been observed that, no single system can perform better for all the acousticscenes. Therefore, to reduce the confusion, we have combined the evidences for the acoustic scenesby using a late fusion mechanism. From our studies on DCASE 2017 and 2018 datasets, it is observedthat combining evidence suitably from the pair of classifiers has improved the performance of the ASCsystem.When the number of acoustic scenes are more it may be difficult for a single classifier to discriminatethe classes better. In this regard, we have proposed two-level hierarchical classification approach, inwhich initially identifying the meta category of acoustic scene is followed by fine-grained classificationbelonging to a particular meta category.Keywords:Acoustic Scene Classification, Mel-Frequency Cepstral Coefficients, Inverted Mel-FrequencyCepstral Coefficients, Spectral Centroid Magnitude Coefficients, Subband Spectral Flux Coefficients,Single Frequency Filtering Cepstral Coefficients, Deep Neural Networks, Late Fusion Mechanism, Hier-archical Classification, Detection and Classification of Acoustic Scenes and Events.

Abstract

Two-wheeler electric vehicles have proliferated in India in large numbers. In India alone in 2021 approximately 1,44,000 two wheeler EVs were sold whereas there were only 447 charging infrastructure establishment for two-wheeler EVs. This number of sales of twowheeler EVs is expected to reach to 1 million by 2026. The number of charging infrastructure required to quench to such high charging needs is around 4,00,000. Well the statistics above sounds like a close ended problem statement. A simple solution is installing the required number of charging stations. The problem underlying in this solution is installing charging stations in the middle of the city rather than on the highways, outskirts of a city etc as the most two-wheeler commuters are travelling from one place to another around the middle of the city. The issue now stands it is very hard to find a huge spaces of land in the middle of the city. Moreover the cities designed ages ago were not built with any concept of charging infrastructure. Hence there is a need of compact charging infrastructure for two-wheelers in the middle of the city. An Electric Vehicle (EV) charger, also called Electric Vehicle Charging Station (EVCS) is an infrastructure element that supplies electrical energy for recharging EVs. The thesis focuses on the design and fabrication of two-wheeler (escooter) electric vehicle charging equipment. Considering the future scenario of mass privatization of EV two-wheelers on Indian roads, the outline discusses the product design based on fabricability, affordability, and ability to mass manufacture. The proposed architecture follows the Level 2 charging standards (240 Volts), and is based on the open charge point protocol (OCPP). The suggested EVCS is constructed considering the safety prerequisites of system administrators, installers, consumers, government agencies and others. The design of EVCS links to three industries: equipment manufacturers, software industry and electric power networks. This paper presents design considerations by elaborating the hardware, software, and protocols followed to design the Level 2 charging standard EVCS. A smart city is a collection of a variety of smart devices representing several disparate infrastructural verticals. The data gathered by these smart devices helps bring a strong integration of human, collective, and artificial intelligence within the city operation. With the advent and proliferation of electric vehicles (EVs), the EV charge point is a relatively new addition to this plethora of smart devices. We believe that the EV charge point can acquire enhanced functionality if it can be seamlessly integrated with the rest of the smart city infrastructure. For example, charging services can be stopped if smart grid or smart fire alarms sense an electrical overload or fire hazard in the vicinity. However, the interaction of an EV charge point with the smart city infrastructure requires seamless information exchange across various verticals. This horizontal flow of information is enabled by making the charge point complaint with the oneM2M platform. To this end, we present the design and fabrication of an EV charge point based on the OCPP communication standard and complaint with the oneM2M platform. Further, we discuss various use cases showing an increased functional capability of the EV charger due to access to data from other IoT devices. The interaction of charging equipment with the IoT devices of the city will help us understand if there is any overloading issues in the building and not an appropriate place to install chargers.This inspired us to build a charging infrastructure which is grid free. This can be installed at any place in buildings with overloading issues, in remote places, villages with issue of constant power etc.

Abstract

Discourse parsing is a sub-field of natural language processing which involves understanding the structure, information flow, and modeling the coherence of a given text. It forms the basis of several natural language processing tasks, including, but not limited to, question-answering, text summarization, and sentiment analysis. One of the fundamental tasks in discourse parsing is discourse unit segmentation and connective identification. Discourse unit segmentation refers to identifying the elementary units of text that combine to form a coherent text. Connectives signal the presence of explicit discourse relations in text. Connective identification is the task of identifying these discourse connectives. Language has always played a significant role in human interaction and the evolution of society. With the increasing amount of text data being generated every day on social media platforms such as Facebook, Twitter, WhatsApp, Reddit, etc., helping machines understand and analyse this data is going to be the fundamental task which will further enable us to improve the performance of systems for downstream NLP tasks. In this thesis, we explore the sub-field of shallow discourse parsing, compare approaches towards segmentation and connective identification, and build a dataset and connective identification system for Hindi data. First, we look at approaches towards shallow discourse parsing to identify individual discourse relations that are present in text. This involves given a text, identifying the span of the explicit discourse connective, labelling the two text spans that act as the arguments of the connective and predicting the sense of the discourse relation. We compare and analyse several approaches for these tasks. We then look at an approach towards shallow discourse parsing in Hindi and analyse the tasks of the identification of explicit discourse connectives and their arguments. Further, we work on building a multilingual model for discourse unit segmentation and connective identification. Early approaches towards segmentation and connective detection relied on rule-based systems using POS tags and other syntactic information to identify discourse segments. Recently, transformer based neural systems have shown promising results in this domain. We establish a baseline using a bidirectional LSTM model. We then look at transformer based neural systems and train our model on 16 datasets encompassing 11 languages and 3 discourse annotation frameworks. This model gives state of the art performance for the English dataset. We then present a curated dataset and model for connective identification in Hindi. We experiment with different Indian language specific models and compare and analyse the model performance.

Abstract

Flash is a multimedia software platform used to produce animations, desktop applications, mobile games and applications. Flash graphics and animations are viewed in a browser through use of Flash Player, AIR and some third-party players are used for viewing desktop and mobile apps. For years, this free (but not open source) Flash plugin was a de- facto standard installation on nearly a billion desktop browsers. However, over the past few years, there is a significant shift from web to mobile devices. In addition, the success of HTML5 has negatively impacted the usage and support for Adobe Flash. To top this in July 2017, the official statement from Adobe stated, ”In collaboration with several of our technology partners - including Apple, Facebook, Google, Microsoft and Mozilla, Adobe is planning to end life of Flash. Specifically, we will stop updating and distributing the Flash Player at the end of 2020 and encourage content creators to migrate any existing Flash content to these new open formats.” This has caused developers to migrated towards open source platforms like HTML5 thereby resulting in unmaintained Flash assets and code bases. In our work, we propose SWF2JS - a tool that enables semi-automated migration of SWF files to JavsScript. We also discuss its feasibility to transform Flash based animations to HTML5 and JavaScript against re-writing the same animations from scratch.

Abstract

Cancer is becoming a leading cause of death due to increased life expectancy and lifestyle changes. With the evolving understanding of the complexities of the disease, oncologists and researchers attempt to determine the treatment and prevention strategies effective for specific groups of people. This approach, called precision medicine, requires redefining the practices throughout the different stages of cancer care, including accurate diagnosis and prognosis. The tissue specimens from the tumor are viewed on glass slides under the microscope by the pathologists for visual interpretation of cellular biology. This examination, known as histopathology, is the gold standard for cancer diagnosis. The slides can be digitalized as high-resolution images, called Whole Slide Images. Computer vision algorithms on these images can automate routine pathology tasks. While histopathology enables clinicians to detect and monitor tumors to make precise decisions on therapies, cancer is primarily a disease of the genome. Individual tumors harbor thousands of point mutations. Genomic data helps to understand the genetic mechanisms of cancers. With the high-throughput genotyping becoming increasingly affordable, thousands of individuals worldwide are undergoing genomic sequencing. Genome profiling of tumor cells has led to better categorizing cancers, discovering previously unknown subtypes and sub-classifications. On the other hand, studies have revealed unexpected genetic similarities across different types of tumors. Precise and extensive characterization of tumors is the basis of precision medicine. A proper understanding of its subtype, stage, and course of progression guides the therapy decisions. Making efficient use of data observed at different scales gives a more comprehensive picture of complex diseases such as cancer. Genome profiles combined with high-resolution imaging techniques provide fine-grained measures to provide intra, inter, and peri-tumor information. Imaging-omics is an emerging field as a novel method of associating imaging features with genomic data. This thesis presents two works on accurate diagnosis and prognosis of breast cancer. The first work explores the association between genomic biomarkers and histopathology images. The second work uses multiple modalities - gene expression, clinical parameters, and images for cancer prognostication. Finally, the challenges and prospects of the field of imaging-omics are discussed. Through these works, we demonstrate that the histopathology images are predictive of genomic biomarkers such as mutations, hormone receptor status, and molecular subtypes. Also, the integration of multiple modalities gives a much better prediction of survival. Further insights are presented in terms of the contributing features for diagnosis and prognosis.