Abstract

Visual Question Answering (VQA) with multiple choice questions enables a vision-centric evaluation of Multimodal Large Language Models (MLLMs). Although it reliably checks the existence of specific visual abilities, it is easier for the model to select an answer from multiple choices (VQA evaluation) than to generate the answer itself. In this work, we offer a novel perspective: we evaluate how well an MLLM understands a specific visual concept by its ability to uniquely describe two extremely similar images that differ only in the targeted visual concept. Specifically, we assess the ability of MLLMs to capture specific points of visual differences using self-retrieval, i.e., by retrieving the target image using its generated caption against the other image in the pair serving as the distractor. We curate 247 highly similar image pairs as part of the D3 benchmark. For each image pair, the model is prompted to: (1) Detect a specific visual difference, and (2) Describe the target image uniquely such that it (3) Discriminates the target image from the distractor. Self-retrieval within D3 enables whitebox evaluation across six different visual patterns, revealing that current models struggle to independently discern fine-grained visual differences, with open-source models failing to outperform random guess.

Abstract

The COVID-19 pandemic wrought havoc across India, particularly during its devastating second and third waves. This study undertakes a crucial epidemiological analysis of these waves, leveraging actual variant count data. Given limited sequencing efforts, variant information is sparse, prompting a novel approach to scaling up with actual case data. Employing a multi-strain variant-level SEIRS model tailored to each Indian state, we modeled the disease’s propagation. We report the estimated parameters of the SEIRS models for the two waves separately. Notably, the transmission coefficients ( ) were estimated to be 0.12 for Kappa, 0.35 for Delta, and 0.38 for Omicron. These coefficients signify the contagiousness of each variant, offering critical information for understanding and managing the pandemic’s dynamics. The findings hold significant implications for public health strategies, emphasizing the urgency of comprehensive variant tracking and proactive measures to mitigate the impact of evolving viral strains.

Abstract

Machine Translation for low-resource languages presents significant challenges, primarily due to limited data availability. We have a baseline model and a primary model. For the baseline model, we first fine-tune the mBART model (mbart-large-50-many-to-many-mmt) for the language pairs English-Khasi, Khasi-English, English-Manipuri, and Manipuri-English. We then augment the dataset by back-translating from Indic languages to English. To enhance data quality, we fine-tune the LaBSE model specifically for Khasi and Manipuri, generating sentence embeddings and applying a cosine similarity threshold of 0.84 to filter out low-quality back-translations. The filtered data is combined with the original training data and used to further fine-tune the mBART model, creating our primary model. The results show that the primary model slightly outperforms the baseline model, with the best performance achieved by the English-to-Khasi (en-kh) primary model, which recorded a BLEU score of 0.0492, a chrF score of 0.3316, and a METEOR score of 0.2589 (on a scale of 0 to 1), with similar results for other language pairs.

Abstract

This paper presents the systems submitted by the Yes-MT team for the Low-Resource Indic Language Translation Shared Task at WMT 2024, focusing on translating between English and the Assamese, Mizo, Khasi, and Manipuri languages. The experiments explored various approaches, including fine-tuning pre-trained models like mT5 and IndicBart in both Multilingual and Monolingual settings, LoRA finetune IndicTrans2, zero-shot and few-shot prompting with large language models (LLMs) like Llama 3 and Mixtral 8x7b, LoRA Supervised Fine Tuning Llama 3, and training Transformers from scratch. The results were evaluated on the WMT23 Low-Resource Indic Language Translation Shared Task’s test data using SacreBLEU and CHRF highlighting the challenges of low-resource translation and show the potential of LLMs for these tasks, particularly with fine-tuning.

Abstract

Provided a method of detecting and tracking a traffic violation of vehicle associated with user through a trapezium shaped object based representation using a deep learning model. The method includes (i) detecting the user and the vehicle to generate a first rectangular bounding box and a second rectangular bounding box of a video, (ii) training the deep learning model by correlating historical trapezium based representations with historical rectangular bounding boxes over historical vehicles, and historical users, (ii) determining a trapezium based representation based on the bounding boxes, (iv) detecting a traffic violation of the vehicle by determining (a) triple-riding violation, if maximum number of intersection of units (IoU) are found by counting the first rectangular bounding box and the second rectangular bounding box, (b) a helmet violation by determining the user without a helmet, and (v) tracking 10 the vehicle that performs the traffic violation based on corresponding trapezium based representation.

Abstract

Scholarship on twentieth-century Asia suggests that political autonomy is a necessary condition for economic growth, allowing regions to tailor and implement policies to their contextual needs. This study will look empirically at the agriculture sector of the Himalayan state of Himachal Pradesh in India. Himachal Pradesh achieved full statehood on January 25, 1971, evolving from a Union Territory administered by the central government of India. It consistently outperforms other Indian states in economic and human development indices such as poverty alleviation, sanitation, and income per capita. The study will draw on agricultural data collected from two sub-regions, Shimla and Kullu, spanning from 1951 to 1991. It will comparatively look at the two decades preceding the transition to statehood (1951-1971) against data from 1971-1991, the two decades following the transition, to explore whether this juncture in political standing played a role in driving the change in the agriculture sector in the region. Variables such as farm prices of crops, agricultural wages, area under different food crops, use of machinery, yield of crops, and area under irrigation will be included in the assessment.

Abstract

Deep Learning (DL) finds application in several prominent fields, including computer vision, natural language processing, and bioinformatics. The proliferation of DL-based methods has brought to notice critical issues about bias (or unfairness) in classification and weak privacy guarantees of the training data. It is crucial to prioritize addressing these issues to prevent the potentially significant negative impact on users. While there has been progress, majority of the works focus on independently resolving fairness and privacy. We propose a tutorial on “Fair and Private Deep Learning” – aimed to provide an exhaustive discussion on (i) reasons behind unfair classifications and lack of privacy, (ii) fairness notions in literature and methods to ensure them, (iii) differentially private DL, and (iv) algorithms that address fair and private DL simultaneously. Moreover, in this tutorial, we not just limit our attention to classical, centralized DL models but also to the fairness and privacy challenges in distributed (or federated) DL. The code, presentation and other details are available at https://github.com/magnetar-iiith/FairPrivateDL.

Abstract

No Results Found

Abstract

The rise of smart cities and the increasing demand for drones has sparked considerable interest in the Internet of Drones (IoD) within the realms of academia and industry. IoD presents a multitude of advantages in smart city settings, facilitating services like traffic monitoring, environmental surveillance, and disaster management by harnessing the potential of IoT and Flying Ad-Hoc Networks (FANET) infrastructures. However, the transmission of sensitive messages between drones in IoD-based smart cities is disseminated over insecure channels, leaving them exposed to security vulnerabilities. Furthermore, drones operating in IoD architectures are prone to physical capture attacks as they operate in unattended environments with minimal human intervention. Moreover, the limited resources of drones pose challenges to the practicality of employing computationally intensive cryptographic methods. In response to these challenges, we introduce PAF-IoD, an authentication framework that prioritizes security and efficiency. PAF-IoD leverages physical unclonable functions (PUFs) and the AEGIS authenticated encryption scheme to guarantee trustworthy and secure communication between users and drones in smart cities. In terms of security validation, we perform both random and real model-based formal analyses. Furthermore, we employ the Scyther tool to ensure the resilience of PAF-IoD against different security vulnerabilities. Additionally, an informal analysis is conducted to demonstrate the resilience of PAF-IoD against various attacks. By introducing PAF-IoD, we offer a secure solution that addresses vulnerabilities and resource limitations associated with drone communication. The proposed framework guarantees the integrity and confidentiality of data while optimizing computational and communication resources, thereby enabling reliable and effective IoD operations in smart cities.

Abstract

—Smart Microfactories use additive manufacturing to create products with mixed materials and variable sizes. Digital twin technology enhances control of the additive manufacturing equipment in these factories, increasing productivity and minimizing errors. The digital twins communicate with the machines to furnish sensitive data and instructions, which must be protected from tampering. Authentication rescues the digital and physical twins from menacing attacks such as privileged insider, impersonation, Ephemeral Secret Leakage (ESL) and Man-in-The-Middle (MiTM) attacks. To this end, we propose lightweight authentication among the digital and physical twins with the undeniability of issued commands and deniable key agreement. It achieves perfect forward secrecy, unforgeability and anonymity. Rigorous formal verification with AVISPA and informal security analysis show the robustness of this protocol to the aforesaid attacks.