Abstract

Unmanned Aerial Vehicles (UAVs) continue to penetrate diverse sectors, including agriculture, disaster management, communication, transportation, and defense. The robustness and reliability of their operation have become paramount due to the increasing ubiquity of these systems across numerous sectors. UAVs can undergo faults in their sensors, actuators (motors), and even structure - all of these have different levels of severity and effects on the system. Motor faults can be of three main types, namely, Motor Locking, Loss of Efficiency (LoE), and Loss of Actuation (LoA). Motor locking is a case where the actuator keeps spinning at a given RPM and cannot be changed, whereas LoE is a case where the actuator degrades over time and is not able to generate enough torque due to lower RPM. In this thesis, however, we focus on LoA, where an actuator stops working completely (gets stuck at zero RPM) and can result in catastrophic failure in the UAV. This work delves into the dynamics of UAVs under actuator fault conditions, examines the resulting instability issues, and explores potential methods for identification and control reconfiguration for safe operation. Once an actuator failure occurs, the Fault Detection & Isolation (FDI) module should be able to locate the fault and pinpoint it with high accuracy and in a very short time, to allow the UAV to stabilize the fault. Various approaches can be used for the FDI system, namely, rule-based, model-based, and data-driven. Implementation of these methods can be relying on RPM sensors, battery monitors, and software-only solutions. We propose a data-driven software module for this purpose, since it is UAV frame agnostic, does not require any additional hardware (software-only), and can be run with minimal setup. The proposed Rotation Forest based classifier can detect, classify, and report the fault within 120-250 milliseconds of occurrence of the fault. This is an acceptable delay, since we have observed that the vehicle cannot handle delays upwards of 500 milliseconds in simulations. Once a fault is reported by the FDI module, the Fault Tolerant Control (FTC) module reconfigures the control system to stabilize the vehicle and continue the mission, or prevent a crash by peforming a safe landing. This thesis focuses on cases of complete actuator failure in Hexacopter UAVs, specifically, for single motor failure scenarios. The proposed FTC module performs UAV stabilization using control reconfiguration, after fault occurrence. We perform a thorough analysis for the FDI and FTC modules separately, extensively in simulation, and demonstrate the same in real flight tests. We also combine the two modules and analyze the response in the simulation. In real flights, the classifier (FDI module) responds to the fault in 2-5 sensor data samples (at a 60ms rate per sample) and has a high true positive rate of 92.6%. Also, in real flights, the performance of the FTC module is measured in terms of tracking error, percentage overshoot, and settling time. Integration of the FDI and FTC modules is performed and simulation results are presented showing satisfactory operation of each of these modules with guaranteed stable flight.

Abstract

Automatic Speech Recognition (ASR) systems are increasingly prevalent in our daily lives, with commercial applications such as Siri, Alexa, and Google Assistant. However, the focus of these systems has been largely angled towards English, leaving a considerable portion of non-English speakers underserved. This is particularly evident in India, a linguistically diverse country with many languages classified as low-resource in the context of ASR due to the scarcity of annotated speech data. This thesis aims to bridge this gap, focusing on enhancing ASR systems for Indian languages using deep learning methodologies. India is a land of language diversity. There are approximately 2000 languages spoken around, and among those officially registered are 23. Of those, very few have ASR capability. This is because building an ASR system requires thousands of hours of annotated speech data, a vast amount of text, and a lexicon that can span all the words in the languages. The necessity for a comprehensive presence in the diverse Indian markets demands the development of multilingual Automatic Speech Recognition (ASR) systems. It’s a common scenario where ASR systems for Indian languages have to be implemented in low-resourced contexts. Furthermore, the complexity of the linguistic landscape is amplified due to the high prevalence of bilingualism in the Indian population, leading to frequent instances of code-switching and linguistic borrowing between languages. Operating concurrent ASR systems that can handle code-switching in the Indian context presents a considerable challenge. This predicament has spurred our research endeavors, driving us to focus on constructing a large corpus for one language and leveraging its phonetic space on other language families in monolingual and multilingual ASR scenarios. This thesis incorporates a crowd-sourcing strategy to collect an extensive speech corpus, particularly for Telugu. Using this approach, around 2000 hours of Telugu speech data, capturing regional variations through three modes: spontaneous, conversational, and read, under various background conditions, has been collected. This data served as a foundation for developing and evaluating neural network architectures tailored to the characteristics of Indian languages. We also explored the potential of self-supervised learning to understand and enhance learned representations, fine-tuning them to suit different language families and data sizes. This approach led to insights into the shared phonetic space among Indian languages, allowing for the development of a multilingual ASR system utilizing a joint acoustic model approach. These studies mark a significant stride towards overcoming the challenges of multilingualism in the Indian context, setting a path for creating more inclusive and effective ASR systems. The research findings presented in this thesis not only contribute towards building efficient and accurate ASR systems for low-resource Indian languages but also underscore the power of deep learning approaches in linguistic technology. It is our hope that this work will motivate and aid further research in this direction, promoting linguistic diversity and broadening access to information and communication technologies for speakers of low-resource languages.

Abstract

In this thesis, we try to tackle the problem of erroneous English-Hindi machine translation (MT) outputs due to the presence of the Verb Phrase Ellipsis (VPE) in English. The phenomenon of VPE is prominent in spoken English, and the antecedent to the ellipsis can come from previous sentences in a conversation as well. MT systems translate sentences as a whole and ignore the contextual information from the previous sentences. For these two reasons, spoken English-Hindi translations suffer. We approached this problem by manually annotating 1200 two-person conversations that contain VPE and by studying how their resolution affects the translation qualities. Based on this analysis, we designed a rule-based system for the detection and resolution of VPE in English with the goal of improving their subsequent Hindi translation qualities. Our rule-based system is capable of the following: 1) Detection of VPE, 2) Resolution of Elided Head verb, 3) Resolution of Elided Head verb’s children, 4) Resolution of non-verbal predicates of a copula or a ’be’ main verb, 5) Modifying original sentence in the conversation with the resolved verb phrase. In order to assess the scalability of our rule-based system, we also tested the system’s performance on VPE datasets outside of our annotated data. The system’s performance was also tested on nonconversational data in this research. The scalability test allowed us to utilize our rule-based system for the purpose of data augmentation as well. The augmented data was utilized for the training of a Transformer-Based Seq2Seq generation model. This model helped us investigate whether a sequenceto-sequence model can be designed to handle instances of conversational English VPE, which could be utilized as a pre-processing step before carrying out English-Hindi Machine Translation. Furthermore, a comparative study between the rule-based and the Transformer-based model was carried out as a part of the research supporting this thesis.

Abstract

Multi-armed bandits (MABs) have various applications in real life, however, most of these applications require certain variations to the classic MAB problem. In this thesis, we focus on one such variant - budgeted combinatorial MAB (BCMAB). A BCMAB allows for multiple arms to be pulled in each round with no restrictions on the number of arms selected per round or the budget consumed per round, as long as the arms pulled do not exceed the total budget. The reward structure is taken to be additive, i.e., the reward obtained on pulling a set of arms in a round is the sum of the rewards obtained by the individual arms. We study relevant problems and develop our solutions to BCMAB. We come up with the algorithms CBwK-Greedy-UCB and CBwK-LP-UCB. We mathematically prove regret bound for CBwKLP-UCB. We experimentally analyze our algorithms and compare them with each other and with the previous most suited algorithm.

Abstract

Cooperative human-human communication becomes challenging when restrictions such as difference in communication modality and limited time are imposed. In this thesis, we present the popular cooperative social game Pictionary as an online multimodal test bed to explore the dynamics of humanhuman interactions in such settings. Pictionary is a multiplayer game where the players attempt to convey a word or phrase through drawing. The restriction imposed on the mode of communication gives rise to intriguing diversity and creativity in the players’ responses. To explore the player activity in Pictionary, an online browser-based Pictionary application is developed and utilized to collect a Pictionary dataset. We conduct an exploratory analysis of the dataset, examining the data across three domains: global session-related statistics, target word-related statistics, and user-related statistics. We also present our interactive dashboard to visualize the analysis results. We identify attributes of player interactions that characterize cooperative gameplay. Using these attributes, we find stable role-specific playing style components independent of game difficulty. In terms of gameplay and the larger context of cooperative partially observable communication, our results suggest that too much interaction or unbalanced interaction negatively impacts game success. Additionally, the playing style components discovered via our analysis align with select player personality types proposed in existing frameworks for multiplayer games. Furthermore, this thesis explores atypical sketch content within the Pictionary dataset. We present various baseline models for detecting such atypical content. We conduct a comparative analysis of three baseline models, namely BiLSTM+CRF, SketchsegNet+, and modified CRAFT. Results indicate that the image segmentation-based deep neural network outperforms recurrent models that rely on stroke features or stroke coordinates as input.

Abstract

Ultrasound guided regional anesthesia (UGRA) involves approaching target nerves through a needle in real-time, enabling precise deposition of drug with increased success rates and fewer complications. Development of autonomous robotic systems capable of administering UGRA is desirable for remote settings and localities where anesthesiologists are unavailable. Real-time segmentation of nerves, needle tip localization and needle trajectory extrapolation are required for developing such a system. In the first part of this thesis, we developed models to localize nerves in the ultrasound domain using a large dataset. Our prospective study enrolled 227 subjects who were systematically scanned for brachial plexus nerves in various settings using three different ultrasound machines to create a dataset of 227 unique videos. In total, 41,000 video frames were annotated by experienced anaesthesiologists using partial automation with object tracking and active contour algorithms. Four baseline neural network models were trained on the dataset and their performance was evaluated for object detection and segmentation tasks. Generalizability of the best suited model was then tested on the datasets constructed from separate ultrasound scanners with and without fine-tuning. The results demonstrate that deep learning models can be leveraged for real time segmentation of brachial plexus in neck ultrasonography videos with high accuracy and reliability. Using these nerve segmentation predictions, we define automated anesthesia needle targets by fitting an ellipse to the nerve contours. The second part of this thesis focuses on localization of the needles and development of a framework to guide the needles toward their targets. For the segmentation of the needle, a natural RGB pre-trained neural network is first fine-tuned on a large ultrasound dataset for domain transfer and then adapted for the needle using a small dataset. The segmented needle’s trajectory angle is calculated using Radon transformation and the trajectory is extrapolated from the needle tip. The intersection of extrapolated trajectory with the needle target guides the needle navigation for drug delivery. The needle trajectory’s average angle error was 2 o , average error in trajectory’s distance from center of the image was 10 pixels (2 mm) and the average error in needle tip was 19 pixels (3.8 mm) which is within acceptable range of 5 mm as per experienced anesthesiologists. The entire dataset has been released publicly for further study by the research community.

Abstract

Object Detection and Tracking computer vision algorithms have remarkably progressed in the past few years, owing to the rapid progress in the field of deep learning. These algorithms find numerous applications in surveillance, robotics, autonomous driving, sports analytics, and human-computer interaction. They achieve near-perfect performance on the benchmark datasets they have been evaluated on. However, deploying such algorithms in real-world poses a large number of challenges, and they do not work as well as we expect them to work by looking at their performance on benchmark datasets. In this thesis, we present details of deploying one such Multi Camera Multi-Object Detection and Tracking system to track players in the bird’s eye (top) view in a sports event, specifically in the game of cricket. Our system is able to generate the top-view map of the fielders from cameras placed on crowd stands of a stadium, making it extremely cost-effective compared to using spider cameras. We present details on the challenges and hurdles we faced while deploying our systems and the solutions we designed to overcome these challenges. Ultimately, we tailored a neatly engineered end-to-end system that went live in the Asia Cup of 2022. Deploying such a multi-camera detection and tracking system poses many challenges. The first of them is related to camera placement. We had to devise strategies to place cameras optimally so that all the objects of interest could be captured by one or more of the cameras placed, at the same time ensuring that they do not appear so small that it becomes difficult for a detector to localize them. Constructing the bird’s eye view of the detected players required camera calibration. In the real world, we may not find reliable point correspondences to calibrate a camera. Even if we might find point correspondences, sometimes they may be really hard to see to accurately calibrate a camera. Detecting players in a setup like this proved to be really challenging because the objects of interest appeared very small in the camera views. Moreover, since the detections were coming from multiple cameras, algorithms had to be devised to associate them correctly across camera views. Tracking in a setting like this was even harder because we had to rely only on motion cues to track the objects of interest. Appearance features did not add any value because of the fact all the players being tracked wore jerseys of the same color. Each of these challenges became even more, harder to solve because of the real-time requirements of our use case. Finally, setting up the hardware architecture to receive the real-time live feeds from each camera in a synchronized manner and implementing a fast communication protocol for transmitting data between the various system components required careful design choices to be made, all of which have been presented in detail in this thesis.

Abstract

In the past decade, India witnessed a surge in concentrated urban growth to manifolds. The same trend is visible in hilly regions where the seismic safety of buildings is partially answered. Also, the peak ground accelerations observed in the past earthquakes are in accordance with the design PGA, the associated damages observed are brittle, which is undesirable. This can be mainly attributed to the current design codes not providing sufficient recommendations for the safety of buildings on hill slopes. For example, the code suggests modifications to consider the height of buildings resting on slopes in calculating the lateral forces, but do not discuss the ambiguity in the shear force distribution that is inevitable at the shorter column. The parameters responsible for their ill behaviour must be well understood to improve the safety of hill buildings. Therefore, a methodology is formulated for understanding the effect of varying building dimensions on (i) design stress ratios, (ii) dynamic response, i.e., drifts, and (iii) dynamic characteristics, i.e., modal properties. Correlation matrices are plotted to identify the parameters influencing the behaviour. Further, nonlinear dynamic analysis is performed using a reference structure to detect the failure pattern. It is observed that the predominant failure is due to shear in all uphill columns, followed by the yielding of an immediate story. Based on the parameters identified, a framework is proposed to (i) restrict the shear failure in the uphill columns and (ii) improve the base shear distribution, flexural deformations, and modal properties along and across the valley. A similar nonlinear analysis is performed to confirm the improvement in the behaviour of buildings resting on slopes

Abstract

Despite its successful predictions, the Standard Model (SM) is not a complete theory. There are theoretical issues and experimental observations that it cannot explain. Over the years, experimental measurements of several observables have shown deviations from their SM-predicted values. For instance, the latest measurement of the anomalous magnetic moment of the muon (g − 2)µ by Fermilab shows a deviation of 4.2σ. There is a discrepancy of 7σ between the SM value and the experimentally observed value of the W boson mass as reported by the CDF collaboration. The SM shows lepton-flavour universality (LFU), i.e., the couplings of leptons to the gauge bosons are flavour independent in the SM. However, the measurements of the leptonic decays of B-mesons at BaBar, Belle, and LHCb indicate some LFU violations. These deviations are popularly referred to as B-anomalies—those seen in the RD(∗) and RK(∗) observables are some of the well-known examples. Until recently, the experimental values of RD and RD∗ exceeded their SM predictions by 1.4σ and 2.5σ, respectively. On the other hand, the deviations in the RK and RK∗ measurements were smaller than the theoretical predictions by about 3.1σ. The current measurement of RD still stands 2σ away from the SM, putting the global average of RD(∗) (combined) at 3.2σ. The deviations in the measurement of (g −2)µ, W boson mass anomaly and the B-anomalies led to numerous beyond the Standard Model (BSM) explanations in the literature. Among these, the most popular ones involve a class of hypothetical particles known as the Leptoquarks (LQs). LQs are scalar (sLQ) or vector (vLQ) bosons that are electromagnetically charged. They are colour triplets with nonzero lepton numbers. They can form various weak representations—singlet, doublet or triplet. They appear in theories such as R-parity violating Supersymmetry, SU(5) Grand Unified Theories, Pati-Salam models, Technicolour models, etc. The LHC has an active LQ-search program. Their connections to the lepton and colour sector make them ideal candidates for resolving the above anomalies. In this thesis, we study the phenomenology of TeV-scale LQs, especially of the models proposed to address the anomalies and investigate their current status. We obtain the latest bounds from low-energy observables and the latest LHC data. Many of these LQ models involve sizable cross-generational LQ-lepton-quark couplings, leading to exotic decay modes of LQs. We estimate the discovery/exclusion prospects of all possible LQs in some interesting or new channels at the high-luminosity LHC (HL-LHC)

Abstract

A large number of important mathematical problems do not have perfect solutions which can be computed in a polynomial time. The solution to these problems is often calculated by using polynomial time heuristic algorithms which produce solve the problem with some approximation. If the matrix algorithm can be effectively formulated as a game, reinforcement learning can be used instead of the usual heuristics to improve the quality of the approximation. One such problem is to reduce the fill-in produced during matrix decomposition. A large number of computational and scientific methods commonly require decomposing a sparse matrix into triangular factors as LU decomposition. A common problem that is faced during this decomposition is that even though the given matrix may be very sparse, the decomposition may lead to a denser triangular factors due to fill-in. A significant fill-in may lead to prohibitively larger computational costs and memory requirement during decomposition as well as during the solve phase. To this end, several heuristic sparse matrix reordering methods have been proposed to reduce fill-in before the decomposition. However, finding an optimal reordering algorithm that leads to minimal fill-in during such decomposition is known to be a NP-hard problem. A reinforcement learning based approach is proposed for this problem. The sparse matrix reordering problem is formulated as a single player game. More specifically, Monte-Carlo tree search in combination with a neural network is used as a decision making algorithm to search for the best move in our game. The proposed method, Alpha Elimination was found to produce significantly lesser non-zeros in the LU decomposition as compared to existing state-of-the-art heuristic algorithms with little to no increase in overall running time of the algorithm. In addition to the main problem of the thesis, work was also conducted in the field of extreme classification. In this area, a fast algorithm called LightDXML was developed to solve the problem of extreme classification at scale