Abstract

Remote labs are a technological breakthrough for accessing scientific knowledge, pushing the boundaries of traditional approaches to practical learning. They seek to provide students with unrestricted access to laboratory education and experimentation. Remote labs further prove invaluable in today’s times when pandemics such as COVID-19, geopolitical crises, and climatic conditions constantly threaten students’ access to practical learning in their institutions. Remote labs in such circumstances allow students remote access to practical equipment and laboratory resources. However, most of the developed remote labs today, either run on private networks, are specific to certain science domains, are too costly to replicate or require software installations, all of which hinder universal access to knowledge and education. With this motivation, the work presented in this thesis focuses on building a scalable software framework for real-time remote experimentation for Internet of Things (IoT)-based remote labs with lowlatency video streaming. This thesis is structured to comprise two parts. First, it explores the components and sequence flow of the developed end-to-end software architecture, emphasising the key design decisions for implementing the software solution and video streaming. It also discusses the introduction of an interoperability layer to support multiple IoT platforms and the implementation of a peer-to-peer (P2P) video streaming. The work further evaluates the developed architecture for its latency and capability of handling larger loads as the system scales. Secondly, the thesis also proposes the approach of multi-user multiplexing to address the scalability of remote labs hosting science experiments with near-instantaneous outputs based on electric currents, such as Kirchhoff’s laws. Such experiments and hardware nodes present a unique case compared to other practical experiments involving mechanical actuation. Multi-user multiplexing aims to provide users with concurrent access to the hardware nodes without impacting other users’ experience. The architecture proposed earlier can be modified slightly and adapted using WebSockets to facilitate the implementation of multi-user multiplexing. The modified WebSocket-based architecture constitutes three layers in contrast to the proposed four-layer architecture, and the two have been compared for their performance on latency and round trip time as part of this work

Abstract

A document has information emerging out of the existence of various physical entities or regions such as headings, paragraphs, figures, captions, tables, and backgrounds along with the textual content. To decipher the information in a document, a human reader uses a variety of additional cues, such as context, conventions, and information about language, script, location, and a complex reasoning process. Millions of documents are created and distributed daily over the Internet and printed media. Understanding, analyzing, sorting, and comparing a massive collection of documents in a limited time is a hectic job for humans. Here, the automatic document image understanding systems (DIUS) help humans do this tedious task within a limited time. The DIUS have typically a document image layout segmentation module and information extraction modules. This thesis focuses on the challenging problems related to document image layout segmentation in various types of documents and their applications. In this thesis, first, we analyse the various document images using deep features. The deep features are the features extracted using a pretrained deep neural network. To study deep texture features, we propose a deep network architecture that independently learns texture patterns, discriminative patches, and shapes to solve various document image analysis tasks. The considered tasks are document image classification, genre identification from book covers, scientific document figure classification, and script identification. The presented network learns global, texture, and discriminative features and combines them judicially based on the nature of the problems. We compare the performance of the proposed approach with state-of-the-art techniques on multiple publicly available datasets such as Book covers, RVL-CDIP, CVSI and DocFigure. Experiments show that our approach obtains significantly better performance over state-of-the-art for all tasks. Next, we focus on the problem of document image layout segmentation and propose a solution for a class of document images, including historical, scientific, and classroom slide document images. The historical document image segmentation problem is modeled as a pixel labeling task where each pixel in the document image is classified into one of the predefined labels, such as text, comment, decoration, and background. The method first extracts deep features from the superpixels of the document image. Then, we learn SVM classifier using these features and segment the document image. The pyramid pooling module is used to extract the logical regions of scientific document images. In the classroom slide images, the logical regions are distributed based on the location of the image. To utilize the location of the logical regions for slide image segmentation, we propose the architecture, Classroom Slide Segmentation Network (CSSN). The unique attributes of this architecture differ from most other semantic segmentation networks. We validate the performance of our segmentation architecture using publicly available benchmark datasets. Next, we analyze the output regions of document layout segmentation. Figures used in the documents are the complex regions to decipher the information by a DIUS. Hence, document figure classification (DFC) is an important stage of the DIUS. The design of a DFC system required well-defined figure categories and datasets. Existing datasets related to the classification of figures in the document images are limited in size and category. We introduce a scientific figure classification dataset named DocFigure. The dataset consists of 33K annotated figures of 28 different categories present in the document images, which correspond to scientific articles published in last several years. Manual annotation of such a large number (33K) of figures is time-consuming and cost-ineffective. We design a web-based annotation tool that can efficiently assign category labels to many figures with the minimum effort of human annotators. To benchmark our generated dataset on the classification task, we propose three baseline classification techniques using the deep feature, deep texture feature, and both. Our analysis found that the combination of both deep and texture features is more effective for document figure classification tasks than individual features. Finally, we propose the application backed by the research of this thesis. Slide presentations are an effective and efficient tool for classroom communication used by the teaching community. However, this teaching model can be challenging for blind and visually impaired (VI) students as such students require personal human assistance to understand the presented slide. This shortcoming motivates us to design a Classroom Slide Narration System (CSNS) that generates audio descriptions corresponding to the slide content. This problem poses an image-to-markup language generation task. Ext

Abstract

Unlike the classical scenario, conditional entropy in quantum systems can be negative. Such systems play a pivotal role in information-processing tasks such as super dense coding, one-way entanglement distillation, and state merging, to name a few. Quantum conditional entropy’s central role led to the formulation of a resource theoretic study, establishing the notion of free and resource states and a char- acterization of free operations. In this thesis, we delve into the study of quantum channels that affect the negative conditional entropy of a quantum system. The main focus of this thesis is the characterization of these channels from topological and information- theoretic perspectives. We consider two classes of quantum channels - Negative Conditional Entropy Breaking (NCEB) channels and Negative Conditional Entropy Annihilating (NCEA) channels. We de- fine such channels and examine their properties when combined serially. We also consider the property of parallel combinations of NCEB channels. Our exploration extends to complementary channels of NCEB, whereby we discover the information-leaking nature of NCEB channels. We exemplify NCEB and NCEA channels through standard depolarizing channels and further the characterization. Specif- ically for NCEA channels, we establish a sufficiency condition for global depolarizing channels and transpose depolarizing channels. Following this, we establish the relationship of NCEB and NCEA with newly introduced classes of quantum channels, such as Coherent Information Breaking (CIB) and Mutual Information Breaking (MIB), along with standard channels like zero capacity and entanglement breaking channels. Finally, we consider the perspective of maintaining negative conditional entropy in quantum states, which involves the identification of non-NCEB or non-NCEA channels. In this regard, we provide prescriptions to detect channels that do not affect the negativity of conditional entropy.

Abstract

The social isolation required by the pandemic further amplified the internet’s utility as a medium of communication and information for people with digital access. Social media was one of the most popular forms of communication among young people with digital access during the COVID-19 pandemic. Consequently, crucial debates and discussions about the pandemic crisis have also developed on social media platforms, making them a great primary source to study the experiences of specific groups and communities during the pandemic. Furthermore, online news media outlets were an important source of information for people with digital access during the COVID–19 pandemic. This study consisted of research on social media and online news websites in order to understand the experiences of as well as attitudes towards LGBTQ+ people during the pandemic. We used LDA topic modelling and sentiment analysis on Reddit data to understand the themes and attitudes in circulation within five subreddits devoted to LGBTQ+ experiences and issues. In the process, we attempt to make sense of the role that social media platforms may have played in the lives of LGBTQ+ people who were online during the pandemic (10 March 2020–31 August 2021) and whether this was marked by any continuities or discontinuities from before the pandemic period (January 2019–December 2019). We also examined articles from online editions of two English–language Indian newspaper websites, which differed vastly in their circulation figures—The Times of India and The Indian Express. Focusing on the period spanning March 2020 to August 2021, we analysed articles from our chosen newspaper websites using distil–RoBERTa–base and ChatGPT–3.5 for sentiment analysis and BERTopic for topic modelling. Furthermore, we manually analysed a part of our dataset obtained from Reddit and online newspaper websites to arrive at a nuanced, granular understanding of our quantitative results. The results of our research on Reddit reveal that institutions such as the health care system, justice system, and legislative system perpetuated systemic inequalities against LGBTQ+ communities, thereby adding to the pre–existing stigma against them during a global crisis. Furthermore, the results from our topic model suggest a shift in the discussions of prominent topics in the subreddits from before the pandemic period, with a preponderance of discussions on political events. Similarly, the results from our sentiment analysis show an increase in positive comments and neutral comments and a decrease in negative comments, which does not necessarily indicate a loss of users’ interest in the subreddits. While analysing online newspaper websites, we found a contrast between the quality, quantity, and attitudes prevalent across The Times of India and The Indian Express during the COVID–19 pandemic in the context of LGBTQ+ communities. Our topic modelling results indicate more meaningful coverage by The Indian Express compared to The Times of India, which, however, does not mean sufficient coverage by The Indian Express to provide a comprehensive picture. Similarly, our sentiment analysis results indicate a difference in prevailing sentiment as depicted by the two models. Both newspaper websites exhibited a greater focus on transgender communities. Our research on Reddit has demonstrated what non–cis–heteronormative spaces—virtual and real— can look or feel like. By offering a more rigorous engagement with the data available on social media platforms, this research contributes to current debates in queer and social media studies about the empowering or disempowering nature of social media platforms for LGBTQ+ voices online. Furthermore, it captures the visibility and representation of the LGBTQ+ communities during the pandemic in online Indian news media outlets. By doing so, we contribute to the larger understanding of how Indian media narrated the lived realities of LGBTQ+ communities and what attitudes such representation reveal towards vulnerable communities in India during the pandemic. Overall, this thesis expands the understanding of the kind of discursive spaces and representation offered by online platforms for LGBTQ+ communities during a global crisis, demonstrating both possibilities and potential as well as challenges and problems.

Abstract

In autonomous driving, we have multiple computer vision tasks like object detection, semantic seg- mentation, and instance segmentation which plays a crucial role in perceiving the environment around the vehicle. Understanding the behaviour and performance of such tasks helps improve and address the key issues that are inherent in the system. There can be issues which are latent in the deep learning architecture and also in the datasets on which the deep learning models are trained and tested. In this thesis, we benchmark the performance of various popular deep learning models on road scene datasets for various computer vision tasks and also formulate open-world object detection on road scenes by addressing the inherent issues present in road scene datasets. In the first part of the work, we aim to understand the performance and behaviour of various deep learning models on road scene datasets; Cityscapes, IDD, and BDD. Object detection, semantic seg- mentation, and instance segmentation form the bases for many computer vision tasks in autonomous driving. The complexity of these tasks increases as we shift from object detection to instance seg- mentation. The state-of-the-art models are evaluated on standard datasets such as PASCAL-VOC and MS-COCO, which does not consider the dynamics of road scenes. Driving datasets such as Cityscapes and Berkeley Deep Drive(BDD) are captured in a structured environment with better road markings and fewer variations in the appearance of objects and background. However, the same does not hold for Indian roads. The Indian Driving Dataset(IDD) dataset is captured in unstructured driving scenarios and is highly challenging for a model due to its diversity. This work presents a comprehensive evaluation of state-of-the-art models on object detection, semantic segmentation, and instance segmentation on road scene datasets. We present our analyses and compare their quantitative and qualitative performance on structured driving datasets(Cityscapes and BDD) and the unstructured driving dataset(IDD); under- standing the behavior on these datasets helps in addressing various practical issues and helps in creating real-life applications. In the second part of the work, we model open-world object detection on road scenes since object detection is a crucial component in autonomous navigation systems. Current object detectors are trained and tested on a fixed number of known classes. However, in real-world or open-world settings, the test set may consist of objects of unknown classes; this results in the unknown objects being falsely de- tected as known objects leading to the failure in the decision making of autonomous navigation systems. We propose Open World Object Detection on Road Scenes (ORDER) to resolve the aforementioned problem. We introduce Feature-Mix that widens the gap between known and unknown classes in latent feature space and improves the unknown object detection in the ORDER framework. We identify the inherent problems present in autonomous datasets: i) a significant proportion of the dataset comprises small objects and ii) intra-class bounding box scale variations. We address the problem of small object detection and intra-class bounding box variations by proposing a novel focal regression loss. Further, the detection of small objects is improved by curriculum learning. We present an extensive evaluation of two road scene datasets: BDD and IDD. Our experimental evaluations on BDD and IDD shows con- sistent improvement over the current state-of-the-art method. We believe that this work will lay the foundation for real-world object detection for road scenes.

Abstract

In spite of the vast work in the area of Maximal Frequent Subgraph (MFS) mining, current algorithms do not scale to large databases with numerous graphs. Even the smallest search space required by current MFS algorithms for large databases is too huge to work within the main memory alone, making the algorithms I/O bound. Added to this is the expensive cost of subgraph isomorphism operation, which is commonly used for frequency computation. This thesis explores partition-based secondary memory algorithms and proposes Partition-based Frequent Subgraph Mining using Maximum Common Subgraph (PMCS), that can make maximal frequent subgraph mining for large databases viable. The naive application of the partition-based approach is inefficient due to the need to recompute subgraph frequency and the possibility of storing intermediate subgraphs on the disk. In PMCS, we overcome these drawbacks by intelligently avoiding redundant computations and performing frequency computation operations optimally using the Maximum Common Subgraph operation. Our intermediate results are also small enough to fit in the main memory. While existing algorithms fail to scale beyond 300k, our results demonstrate that PMCS scales to databases of up to 1000k graphs with an average of 25-30 edges per graph.

Abstract

Spoken utterances have an inherent structure in the sense that some words seem to group naturally together and some words seem to have a notable break or disjuncture between them. This can be described in terms of prosodic phrasing, meaning that a spoken utterance has a prosodic phrase structure, similar to how a written utterance has a syntactic phrase structure. Phrase breaks in natural speech are important; they are physiologically essential, help emphasize content, and improve the intelligibility of speech. The process of inserting phrase breaks in an utterance is called phrasing. In the context of speech synthesis, phrasing is a crucial step. It breaks long utterances into meaningful units of information and improves the intelligibilty of the synthesized speech. Traditional methods of phrase break prediction have used discrete linguistic resources like part-ofspeech (POS) sequence information for modeling these breaks. These methods cannot be used for languages where the necessary linguistic resources are not readily available, resulting in unsupervised methods of inducing word representations which can be used as surrogates for POS tags in phrase break prediction. However, these methods are not suitable in the context of Indian languages, which are aggutinative in nature resulting in an increase in vocabulary size. This thesis presents the use of word terminal syllables (last syllable of the word) to model phrase breaks. We demonstrate the correlation between these terminal syllables and acoustic breaks found in the speech signal. We utilize these terminal syllables in building models for phrase break prediction from text in six Indian languages and demonstrate by means of objective and subjective measures that these models perform as well as traditional models using POS sequence information. The discrete linguistic representations (like POS tags, induced POS tags, word-terminal syllables) used by traditional methods of phrase break prediction require a hard classification of words into a set of predefined discrete classes, which raises issues when there is ambiguity in the linguistic representation of a word. Moreover, such a representation does not capture the co-occurrence statistics of the words, i. e., they do not take into account the distributional behaviour of words. Both these issues can be addressed by the use of continuous dimensional representations of words also known as word embeddings. This thesis presents a neural network architecture to induce task specific word embeddings for phrase break prediction. We show that our proposed architecture combines feature induction and phrase break prediction in a single framework and thus avoids the two stage process required while using features derived from Latent Semantic Analysis (LSA). We train our model on audiobook data and show that these task specific word features are better word representations as compared to those derived using LSA, for phrase break prediction. With the advent of deep learning, there have been attempts to apply deep neural networks (DNNs) to phrase break prediction. While deep neural networks are able to effectively capture dependencies across features, they lack the ability to capture long-term relations that are spread over time. On the other hand, recurrent neural networks (RNNs) are able to capture long-term temporal relations and thus are better suited for tasks where sequences have to be modeled. This thesis presents the use of RNNs for phrase break prediction, and shows by means of experimental results that they perform better than DNNs. Traditional techniques for learning word embddings learn static or context-free word embeddings, meaning that for each word in the vocabulary, one fixed embedding is learnt for all possible contexts in which that particular word appears in the text corpus. While static embeddings have proved to be effective in NLP applications, they do not take into account the fact that words have different meanings in different contexts. To address this issue, context-sensitive word representations, also called dynamic or contextual word embeddings were developed. The most popular technique for learning dynamic word embeddings is BERT (Bidirectional Encoder Representations from Transformers). This thesis presents work on phrase break prediction using bidirectional encoder representations learnt from fine-tuning a pretrained BERT model, with an additional token classification layer, on phrase break prediction. We show that representations using this model outperform task-specific static word embeddings learnt using a BLSTM token classification model trained from scratch. Finally, this thesis presents work which attempts to answer the following questions: (a) Is there any utility in incorporating an explicit external phrasing model in an End-to-End TTS system? and (b) How do you evaluate the effectiveness of a phrasing model in an End-to-End TTS system?

Abstract

Ancient manuscripts were among the first forms of written communication, offering key insights into our past, covering literature, medicine, culture, philosophy, religion, and more. It is imperative to save these writings to identify and extract their hidden knowledge. Document collections frequently exhibit overlapping components, irregular patterns, dense layouts, extremely high aspect ratios, physical and chemical degradation (evident in ink-based manuscripts), text misalignment, and more. Compounding these difficulties are issues that may arise during the digitization processes, such as improper document positioning, inadequate illumination, and scanner effects. In this academic thesis, our main emphasis is on identifying and segmenting text lines inside these documents for downstream OCR applications with utmost precision. We devise a two-stage approach - SeamFormer- to identify special text baselines in palm-leaf manuscripts using a multi-task strategy via Vision Transformers (ViTs). In the first stage, we detect text strikethroughs, namely ’scribbles,’ which act as pointers to the location of text line segments within the document. The encoder-decoder architecture is used to analyze input image patches and produce two separate maps: a scribble map and a binary map. In the second stage, we post-process the prior stage outputs and generate a diacritic-aware global energy map. To generate the final precise text line polygons, we use a modified seam generation algorithm along with customized global maps. The prior methodology is further enhanced by the proposed LineTR model, a multi-task DETR (Detection Transformer) model that reconceptualizes the scribble generation as a geometric problem. This method simply generates line parameters for each text line present in the input image patch. This design decision enables the model to exhibit zero-shot behavior across diverse historical manuscripts. The state-of-the-art approach has been proven to generate precise text-line segmentation with a single ’unified’ model with minimal post-processing efforts, making it a strong candidate for image-to-OCR integration pipelines

Abstract

The surge in robotics, IoT, machine vision, biometrics, security cameras, and smart devices has led to a rising demand for image sensors and cameras. Complementary Metal-Oxide-Semiconductor (CMOS) light sensors have become the heart of modern digital imaging, widely utilized in diverse applications ranging from consumer electronics to industrial and scientific domains, especially in highly sensitive areas, low luminescent conditions, labs on chips, etc. This thesis centers around the CMOS light sensors, discussing their architecture, performance, and integration. The research mainly focuses on CMOS-compatible photodiodes and single-photon avalanche diodes (SPADs). For the CMOS photodiodes, this work’s primary idea is to enhance their light sensitivity and photon collection efficiency and integrate these photodiodes with the circuits based on standard CMOS processes. This involves the ability to modify the structure or shape of the photodiode in order to increase the efficiency of photon collection when the photodetector is in use. This research work tackles low-light imaging by using single-photon avalanche diode detectors, which efficiently identify single photons. Another important consideration when using SPAD in such an application is using a high-speed quenching circuit to reduce afterpulsing. Different SPAD architecture simulation models are reviewed and compared to model the SPAD accurately. In this work, an active quenching circuit is designed to facilitate fast quenching and minimal dead time.The work encompasses extensive simulation and modeling using tools such as Cadence Virtuoso and COMSOL Multiphysics, allowing for precise evaluation of the proposed design’s performance metrics. The results show significant enhancements in sensor performance and the possibility of expanding its use in biomedical imaging applications, robotics, and other sensitive imaging needs.

Abstract

A Completely Automated Public Turing Test to Tell Computers and Humans Apart (CAPTCHA) is one of the primary barriers between notorious bots and legitimate human users. However, advancements in Artificial Intelligence (AI) have enabled malicious bots to circumvent CAPTCHA challenges effectively. As a result, several types of CAPTCHA have been rendered ineffective. In this work, we introduce Swiss Cheese CAPTCHA, a novel sensor-based solution designed to be easily solvable by humans while presenting multiple obstructions for bots (similar to the Swiss Cheese Model) even when the sensor outputs can be predicted and interfered with. We leverage a range of human cognitive abilities and Generic Sensor API in modern devices to provide robust protection against automated attacks by making it more computationally expensive for bots to produce a valid answer within a stipulated time. We conducted two user studies to assess our proposal’s effectiveness: one involving 116 participants to assess the likability and improvise the design, and the other, with 107 participants, to investigate the impact of improvised design changes on cognitive abilities. Our results from these studies show an average completion time of 4.76 seconds and 6.12 seconds, with a success rate of 90.3% and 83.25%, respectively. By analyzing the 2141 resultant trajectories from both user studies, we assess the learnability, error recovery rate, efficiency, and satisfaction of users using the scheme. Finally, we devise an automated attack against our proposal to analyze its security in the real world; we find the probability of attack success is low. We also make our dataset available for further research.