Abstract

In this thesis, we try to tackle the problem of erroneous English-Hindi machine translation (MT) outputs due to the presence of the Verb Phrase Ellipsis (VPE) in English. The phenomenon of VPE is prominent in spoken English, and the antecedent to the ellipsis can come from previous sentences in a conversation as well. MT systems translate sentences as a whole and ignore the contextual information from the previous sentences. For these two reasons, spoken English-Hindi translations suffer. We approached this problem by manually annotating 1200 two-person conversations that contain VPE and by studying how their resolution affects the translation qualities. Based on this analysis, we designed a rule-based system for the detection and resolution of VPE in English with the goal of improving their subsequent Hindi translation qualities. Our rule-based system is capable of the following: 1) Detection of VPE, 2) Resolution of Elided Head verb, 3) Resolution of Elided Head verb’s children, 4) Resolution of non-verbal predicates of a copula or a ’be’ main verb, 5) Modifying original sentence in the conversation with the resolved verb phrase. In order to assess the scalability of our rule-based system, we also tested the system’s performance on VPE datasets outside of our annotated data. The system’s performance was also tested on nonconversational data in this research. The scalability test allowed us to utilize our rule-based system for the purpose of data augmentation as well. The augmented data was utilized for the training of a Transformer-Based Seq2Seq generation model. This model helped us investigate whether a sequenceto-sequence model can be designed to handle instances of conversational English VPE, which could be utilized as a pre-processing step before carrying out English-Hindi Machine Translation. Furthermore, a comparative study between the rule-based and the Transformer-based model was carried out as a part of the research supporting this thesis.

Abstract

Multi-armed bandits (MABs) have various applications in real life, however, most of these applications require certain variations to the classic MAB problem. In this thesis, we focus on one such variant - budgeted combinatorial MAB (BCMAB). A BCMAB allows for multiple arms to be pulled in each round with no restrictions on the number of arms selected per round or the budget consumed per round, as long as the arms pulled do not exceed the total budget. The reward structure is taken to be additive, i.e., the reward obtained on pulling a set of arms in a round is the sum of the rewards obtained by the individual arms. We study relevant problems and develop our solutions to BCMAB. We come up with the algorithms CBwK-Greedy-UCB and CBwK-LP-UCB. We mathematically prove regret bound for CBwKLP-UCB. We experimentally analyze our algorithms and compare them with each other and with the previous most suited algorithm.

Abstract

Cooperative human-human communication becomes challenging when restrictions such as difference in communication modality and limited time are imposed. In this thesis, we present the popular cooperative social game Pictionary as an online multimodal test bed to explore the dynamics of humanhuman interactions in such settings. Pictionary is a multiplayer game where the players attempt to convey a word or phrase through drawing. The restriction imposed on the mode of communication gives rise to intriguing diversity and creativity in the players’ responses. To explore the player activity in Pictionary, an online browser-based Pictionary application is developed and utilized to collect a Pictionary dataset. We conduct an exploratory analysis of the dataset, examining the data across three domains: global session-related statistics, target word-related statistics, and user-related statistics. We also present our interactive dashboard to visualize the analysis results. We identify attributes of player interactions that characterize cooperative gameplay. Using these attributes, we find stable role-specific playing style components independent of game difficulty. In terms of gameplay and the larger context of cooperative partially observable communication, our results suggest that too much interaction or unbalanced interaction negatively impacts game success. Additionally, the playing style components discovered via our analysis align with select player personality types proposed in existing frameworks for multiplayer games. Furthermore, this thesis explores atypical sketch content within the Pictionary dataset. We present various baseline models for detecting such atypical content. We conduct a comparative analysis of three baseline models, namely BiLSTM+CRF, SketchsegNet+, and modified CRAFT. Results indicate that the image segmentation-based deep neural network outperforms recurrent models that rely on stroke features or stroke coordinates as input.

Abstract

Ultrasound guided regional anesthesia (UGRA) involves approaching target nerves through a needle in real-time, enabling precise deposition of drug with increased success rates and fewer complications. Development of autonomous robotic systems capable of administering UGRA is desirable for remote settings and localities where anesthesiologists are unavailable. Real-time segmentation of nerves, needle tip localization and needle trajectory extrapolation are required for developing such a system. In the first part of this thesis, we developed models to localize nerves in the ultrasound domain using a large dataset. Our prospective study enrolled 227 subjects who were systematically scanned for brachial plexus nerves in various settings using three different ultrasound machines to create a dataset of 227 unique videos. In total, 41,000 video frames were annotated by experienced anaesthesiologists using partial automation with object tracking and active contour algorithms. Four baseline neural network models were trained on the dataset and their performance was evaluated for object detection and segmentation tasks. Generalizability of the best suited model was then tested on the datasets constructed from separate ultrasound scanners with and without fine-tuning. The results demonstrate that deep learning models can be leveraged for real time segmentation of brachial plexus in neck ultrasonography videos with high accuracy and reliability. Using these nerve segmentation predictions, we define automated anesthesia needle targets by fitting an ellipse to the nerve contours. The second part of this thesis focuses on localization of the needles and development of a framework to guide the needles toward their targets. For the segmentation of the needle, a natural RGB pre-trained neural network is first fine-tuned on a large ultrasound dataset for domain transfer and then adapted for the needle using a small dataset. The segmented needle’s trajectory angle is calculated using Radon transformation and the trajectory is extrapolated from the needle tip. The intersection of extrapolated trajectory with the needle target guides the needle navigation for drug delivery. The needle trajectory’s average angle error was 2 o , average error in trajectory’s distance from center of the image was 10 pixels (2 mm) and the average error in needle tip was 19 pixels (3.8 mm) which is within acceptable range of 5 mm as per experienced anesthesiologists. The entire dataset has been released publicly for further study by the research community.

Abstract

Object Detection and Tracking computer vision algorithms have remarkably progressed in the past few years, owing to the rapid progress in the field of deep learning. These algorithms find numerous applications in surveillance, robotics, autonomous driving, sports analytics, and human-computer interaction. They achieve near-perfect performance on the benchmark datasets they have been evaluated on. However, deploying such algorithms in real-world poses a large number of challenges, and they do not work as well as we expect them to work by looking at their performance on benchmark datasets. In this thesis, we present details of deploying one such Multi Camera Multi-Object Detection and Tracking system to track players in the bird’s eye (top) view in a sports event, specifically in the game of cricket. Our system is able to generate the top-view map of the fielders from cameras placed on crowd stands of a stadium, making it extremely cost-effective compared to using spider cameras. We present details on the challenges and hurdles we faced while deploying our systems and the solutions we designed to overcome these challenges. Ultimately, we tailored a neatly engineered end-to-end system that went live in the Asia Cup of 2022. Deploying such a multi-camera detection and tracking system poses many challenges. The first of them is related to camera placement. We had to devise strategies to place cameras optimally so that all the objects of interest could be captured by one or more of the cameras placed, at the same time ensuring that they do not appear so small that it becomes difficult for a detector to localize them. Constructing the bird’s eye view of the detected players required camera calibration. In the real world, we may not find reliable point correspondences to calibrate a camera. Even if we might find point correspondences, sometimes they may be really hard to see to accurately calibrate a camera. Detecting players in a setup like this proved to be really challenging because the objects of interest appeared very small in the camera views. Moreover, since the detections were coming from multiple cameras, algorithms had to be devised to associate them correctly across camera views. Tracking in a setting like this was even harder because we had to rely only on motion cues to track the objects of interest. Appearance features did not add any value because of the fact all the players being tracked wore jerseys of the same color. Each of these challenges became even more, harder to solve because of the real-time requirements of our use case. Finally, setting up the hardware architecture to receive the real-time live feeds from each camera in a synchronized manner and implementing a fast communication protocol for transmitting data between the various system components required careful design choices to be made, all of which have been presented in detail in this thesis.

Abstract

In the past decade, India witnessed a surge in concentrated urban growth to manifolds. The same trend is visible in hilly regions where the seismic safety of buildings is partially answered. Also, the peak ground accelerations observed in the past earthquakes are in accordance with the design PGA, the associated damages observed are brittle, which is undesirable. This can be mainly attributed to the current design codes not providing sufficient recommendations for the safety of buildings on hill slopes. For example, the code suggests modifications to consider the height of buildings resting on slopes in calculating the lateral forces, but do not discuss the ambiguity in the shear force distribution that is inevitable at the shorter column. The parameters responsible for their ill behaviour must be well understood to improve the safety of hill buildings. Therefore, a methodology is formulated for understanding the effect of varying building dimensions on (i) design stress ratios, (ii) dynamic response, i.e., drifts, and (iii) dynamic characteristics, i.e., modal properties. Correlation matrices are plotted to identify the parameters influencing the behaviour. Further, nonlinear dynamic analysis is performed using a reference structure to detect the failure pattern. It is observed that the predominant failure is due to shear in all uphill columns, followed by the yielding of an immediate story. Based on the parameters identified, a framework is proposed to (i) restrict the shear failure in the uphill columns and (ii) improve the base shear distribution, flexural deformations, and modal properties along and across the valley. A similar nonlinear analysis is performed to confirm the improvement in the behaviour of buildings resting on slopes

Abstract

Despite its successful predictions, the Standard Model (SM) is not a complete theory. There are theoretical issues and experimental observations that it cannot explain. Over the years, experimental measurements of several observables have shown deviations from their SM-predicted values. For instance, the latest measurement of the anomalous magnetic moment of the muon (g − 2)µ by Fermilab shows a deviation of 4.2σ. There is a discrepancy of 7σ between the SM value and the experimentally observed value of the W boson mass as reported by the CDF collaboration. The SM shows lepton-flavour universality (LFU), i.e., the couplings of leptons to the gauge bosons are flavour independent in the SM. However, the measurements of the leptonic decays of B-mesons at BaBar, Belle, and LHCb indicate some LFU violations. These deviations are popularly referred to as B-anomalies—those seen in the RD(∗) and RK(∗) observables are some of the well-known examples. Until recently, the experimental values of RD and RD∗ exceeded their SM predictions by 1.4σ and 2.5σ, respectively. On the other hand, the deviations in the RK and RK∗ measurements were smaller than the theoretical predictions by about 3.1σ. The current measurement of RD still stands 2σ away from the SM, putting the global average of RD(∗) (combined) at 3.2σ. The deviations in the measurement of (g −2)µ, W boson mass anomaly and the B-anomalies led to numerous beyond the Standard Model (BSM) explanations in the literature. Among these, the most popular ones involve a class of hypothetical particles known as the Leptoquarks (LQs). LQs are scalar (sLQ) or vector (vLQ) bosons that are electromagnetically charged. They are colour triplets with nonzero lepton numbers. They can form various weak representations—singlet, doublet or triplet. They appear in theories such as R-parity violating Supersymmetry, SU(5) Grand Unified Theories, Pati-Salam models, Technicolour models, etc. The LHC has an active LQ-search program. Their connections to the lepton and colour sector make them ideal candidates for resolving the above anomalies. In this thesis, we study the phenomenology of TeV-scale LQs, especially of the models proposed to address the anomalies and investigate their current status. We obtain the latest bounds from low-energy observables and the latest LHC data. Many of these LQ models involve sizable cross-generational LQ-lepton-quark couplings, leading to exotic decay modes of LQs. We estimate the discovery/exclusion prospects of all possible LQs in some interesting or new channels at the high-luminosity LHC (HL-LHC)

Abstract

A large number of important mathematical problems do not have perfect solutions which can be computed in a polynomial time. The solution to these problems is often calculated by using polynomial time heuristic algorithms which produce solve the problem with some approximation. If the matrix algorithm can be effectively formulated as a game, reinforcement learning can be used instead of the usual heuristics to improve the quality of the approximation. One such problem is to reduce the fill-in produced during matrix decomposition. A large number of computational and scientific methods commonly require decomposing a sparse matrix into triangular factors as LU decomposition. A common problem that is faced during this decomposition is that even though the given matrix may be very sparse, the decomposition may lead to a denser triangular factors due to fill-in. A significant fill-in may lead to prohibitively larger computational costs and memory requirement during decomposition as well as during the solve phase. To this end, several heuristic sparse matrix reordering methods have been proposed to reduce fill-in before the decomposition. However, finding an optimal reordering algorithm that leads to minimal fill-in during such decomposition is known to be a NP-hard problem. A reinforcement learning based approach is proposed for this problem. The sparse matrix reordering problem is formulated as a single player game. More specifically, Monte-Carlo tree search in combination with a neural network is used as a decision making algorithm to search for the best move in our game. The proposed method, Alpha Elimination was found to produce significantly lesser non-zeros in the LU decomposition as compared to existing state-of-the-art heuristic algorithms with little to no increase in overall running time of the algorithm. In addition to the main problem of the thesis, work was also conducted in the field of extreme classification. In this area, a fast algorithm called LightDXML was developed to solve the problem of extreme classification at scale

Abstract

Recent developments in neural text-to-speech synthesis (NTTS) have been able to produce highquality human-like speech samples. However, these models require large amounts of studio-quality recordings for training. As a result, training NTTS models to serve multiple speakers and styles is resource intensive, both in terms of time and compute costs. In this thesis, we discuss the data-efficiency methods for training an NTTS acoustic model with limited data. We explore several data-efficiency techniques for speaker and style modelling with limited data. We discuss multi-speaker training for NTTS model with limited data for each speaker, and speaker adaptation for fine-tuning a pre-trained multi-speaker NTTS model with few minutes of training data for the target speaker. Controllability is TTS systems refers to the ability of explicitly controlling the prosodic variations of the synthesised speech. We explore controllability in NTTS with latent-variable conditioning with variational autoencoders (VAE). These can be used for one-shot style transfer from a reference speech sample. We further discuss improving the posterior flexibility of latent-variable from VAE using normalising flows. We adapt the multi-speaker training strategy to generate newscaster style speech with limited stylistic training data. We first analyse prosodic variations in the neutral style, newscaster style, and mixed expressive corpora. From this, we conclude that the newscaster style is more dynamic than the neutral style, however with lower dynamic range and prosodic variations than a mixed expressive corpora containing recordings with different emotions. The problem of generating newscaster style of speech with limited training data is posed as that of creating a bi-style model in which a one-hot style ID can be modified to generate either neutral or newscaster style speech. We only use a quarter of the data for newscaster style as opposed to that of neutral style. Combining the two styles gives the model enough volume of training data to learn the textual and acoustic alignments, while only a fractional amount of stylistic data is required to factorise the two styles. To further improve on the naturalness, we condition the NTTS acoustic model with contextualised word embeddings (CWE). This gives the model additional syntactic and semantic context on the input text. The proposed bi-style NTTS model conditioned is shown to improve on naturalness and styleappropriateness for newscaster speech over both neutral NTTS and concatenative systems in MUSHRA evaluations conducted with expert listeners.

Abstract

This thesis presents research on deep learning optimization, emphasizing three separate yet related topics: adaptive learning rates, first-order optimization for generative adversarial networks (GANs), and the effects of label smoothing. Firstly, we propose a novel approach for obtaining adaptive learning rates in gradientbased descent methods for classification tasks. Departing from traditional methods that rely on decayed expectations of gradient-based terms, our approach leverages the angle between the current gradient and a new gradient computed from the orthogonal direction. By incorporating angle history, we determine adaptive learning rates that lead to superior accuracy compared to existing state-of-the-art optimizers. We provide empirical evidence of convergence and evaluate our approach on diverse benchmark datasets, employing prominent image classification architectures. Secondly, we introduce a groundbreaking first-order optimization method tailored specifically for training GANs. Our method builds upon the Gauss-Newton method, approximating the min-max Hessian, and utilizes the Sherman-Morrison inversion formula to calculate the inverse. Operating as a fixed-point method that ensures necessary contraction, our approach produces high-fidelity images with enhanced diversity across multiple datasets. Notably, it outperforms state-of-the-art second-order methods, including achieving the highest inception score for CIFAR10. Additionally, our method demonstrates comparable execution times to first-order min-max methods. Furthermore, we investigate the effects of label smoothing on GAN training, examining various optimizer variants and learning rates. Our research reveals that employing label smoothing with a high learning rate and the CGD optimizer yields results surpassing the quality attained by using ADAM with the same learning rate. Importantly, we establish that label smoothing plays a vital role, as its absence fails to generate comparable results. We also explore the impact of architectural changes on the generator’s conditioning, providing valuable insights into the factors influencing GAN performance Our research advances the deep learning optimization field by delving into these interconnected areas. We present novel methodologies for adaptive learning rates, first-order optimization for GANs, and the importance of label smoothing. These advancements offer improved accuracy in classification tasks, enhanced image generation quality, and a deeper understanding of the nuances of GAN training.