Abstract

The objective of the MTIL track in FIRE 2023 was to encourage the development of Indian Language to Indian Language (IL-IL) Neural Machine Translation models. The languages covered in the track included Hindi, Gujarati, Kannada, Odia, Punjabi, Urdu, Telugu, Kashmiri, and Sindhi. The track consists of two tasks: (i) a General Translation Task and (ii) a Domain specific Translation Task with Governance and Healthcare being the chosen domains. For the listed languages, we proposed 12 diverse language directions for the general domain translation task and 8 each for healthcare and governance domains. Participants were encouraged to submit models for one or more language pairs. We witnessed the creation of 34 distinct models spanning various language pairs and domains. Model assessments were conducted using five evaluation metrics: BLEU, CHRF, CHRF++, TER, and COMET. The submitted model outputs were ranked based on the CHRF score.

Abstract

The demand for daily form digitization requires extensive manual effort. This paper presents an efficient pipeline for digitizing Indic handwritten forms, minimizing human intervention. We validate the pipeline using textit{Hindi} and textit{Bengali} forms, creating a dedicated test set, textit{IIIT-Indic-Form-Test}. Our approach enables form capture and orientation alignment via smartphone, extracting printed and handwritten fields with OCR enhanced by template annotations. A predefined word list supports post-processing for fields like textit{name}, textit{state}, and textit{country}. We compare our pipeline with tools like Google Parser and Microsoft Azure and conduct ablation studies on form style, rotation, and handwriting variations. A GUI-based application for digitization is also developed. The code and model are publicly available at https://github.com/shaoncvit/Indic_Handwritten_Form_dataset.

Abstract

The landscape of human memory and event cognition research has witnessed a transformative journey toward the use of naturalistic contexts and tasks. In this review, we track this progression from abrupt, artificial stimuli used in extensively controlled laboratory experiments to more naturalistic tasks and stimuli that present a more faithful representation of the real world. We argue that in order to improve ecological validity, naturalistic study designs must consider the complexity of the cognitive phenomenon being studied. Then, we review the current state of “naturalistic” event segmentation studies and critically assess frequently employed movie stimuli. We evaluate recently developed tools like lifelogging and other extended reality technologies to help address the challenges we identified with existing naturalistic approaches. We conclude by offering some guidelines that can be used to design ecologically valid cognitive neuroscience studies of memory and event cognition.

Abstract

Traditional evaluation metrics like BLEU and ROUGE fall short when capturing the nuanced qualities of generated text, particularly when there is no single ground truth. In this paper, we explore the potential of Large Language Models (LLMs), specifically Google Gemini 1, to serve as automatic evaluators for non-standardized metrics in summarization and dialog-based tasks. We conduct experiments across multiple prompting strategies to examine how LLMs fare as quality evaluators when compared with human judgments on the SummEval and USR datasets, asking the model to generate both a score as well as a justification for the score. Furthermore, we explore the robustness of the LLM evaluator by using perturbed inputs. Our findings suggest that while LLMs show promise, their alignment with human evaluators is limited, they are not robust against perturbations and significant improvements are required for their standalone use as reliable evaluators for subjective metrics.

Abstract

The integration of Vision-Language Models (VLMs) into various applications has highlighted the importance of evaluating these models for inherent biases, especially along gender and racial lines. Traditional bias assessment methods in VLMs typically rely on accuracy metrics, assessing disparities in performance across different demographic groups. These methods, however, often overlook the impact of the model’s disabilities, like lack spatial reasoning, which may skew the bias assessment. In this work, we propose an approach that systematically examines how current bias evaluation metrics account for the model’s limitations. We introduce two methods that circumvent these disabilities by integrating spatial guidance from textual and visual modalities. Our experiments aim to refine bias quantification by effectively mitigating the impact of spatial reasoning limitations, offering a more accurate assessment of biases in VLMs.

Abstract

The implementation of Internet of Things (IoT) as Cyber-Physical Systems (CPS) is essential for the advancement of smart healthcare within a highly digital environment. However, the development of medical CPS (MCPS) poses various challenges, particularly in terms of dependability and security. The complex nature of medical IoT components encompasses both computational and physical aspects including secure reliability. To mitigate these issues, this article introduces a cyber engineering, an innovative approach, that combines principles from complex system design in order to establish a systematic framework for combining the cyber and physical layers of IoT and MCPS together. This integration aims to provide an unified system designing approach that can produce more secure and robust systems. By embracing the cyber engineering ideology, the healthcare industry can overcome reliability challenges and pave the way for the development of secure and dependable smart healthcare solutions, while ensuring security by design perspective. The approach also provides ideal future possibilities to the upcoming researchers.

Abstract

This paper contributes to the expanding research on toxicity in online games, exploring the relationship between game design and player behaviours in India. Using an ethnographic approach and in-depth interviews with gamers, we capture firsthand accounts of toxic gaming experiences, delving into game design, game mechanics, and player interactions intersecting to shape player behaviours. While previous research on toxicity in gaming environments has largely focused on the role of game design, aspects of gender and, to a lesser extent, race, primarily within the context of the Global North, our study shifts the focus to India and, by extension, the Global South. We seek to uncover broader dimensions of gaming toxicity that go beyond design and gender. Our analysis examines the underlying factors contributing to the prevalence and persistence of toxic behaviours in Indian gaming environments by highlighting the influence of cultural and contextual elements in producing toxic behaviours. Expanding the scope of research to include these overlooked aspects, our study aims to provide a more comprehensive understanding of the complexities of toxicity in online gaming spaces within the Indian context.

Abstract

Analysis of foot pressure, also known as plantar pressure analysis, plays a pivotal role in biomedical assessments related to posture and gait analysis. Extensive research has been conducted on leveraging this technique for clinical purposes, leading to the development of flexible pressure sensors. In this letter, we present the use of in-shoe flexible pressure sensor array for determining the nature of the walking surface. The sensor system is fabricated using eight low-cost and robust, in-shoe pressure sensors that leverage the piezoresistivity of velostat. The sensor array was characterized for four different surface types. Random Forest (RF) algorithm was used to classify the surfaces with 86% accuracy. Based on this analysis, we propose a novel method for analyzing various surfaces based on their attributes such as firmness, rigidity, and penetrability. Such a device can be used for ascertaining surface characteristics after construction, or playing surfaces in a stadium.

Abstract

Bidding in a periodic double auction (PDA) is challenging due to its sequential nature, where one needs to consider current as well as future auctions to decide the bids. Monte-Carlo Tree Search (MCTS), which is a state-of-the-art online planning algorithm for tackling sequential problems, seems a perfect fit for bidding in PDAs. However, the success stories of MCTS are largely limited to discrete action spaces, and its efficacy diminishes when dealing with continuous actions. Conventional methods often resort to overly simplistic discretizations that limit exploration and fail to provide valuable insights into unexplored actions. In this work, we propose a novel bidding strategy for PDAs, Regression-MCTS, that is built upon MCTS for a continuous action space of bid prices. Unlike conventional methods, our novel MCTS method leverages information obtained from explored actions to enhance the understanding of the larger action set within the continuous domain to place bids in the auctions, thus generalizing the information about action quality between a wider action space for faster learning. To test the efficacy of our proposed method, we design an efficient PDA simulator that closely resembles real-world PDAs. Our analysis verifies that the increase in the number of rollouts improves its performance. Furthermore, our experimental results demonstrate that our approach outperforms existing MCTS-based bidding strategies and the majority of state-of-the-art PDA bidding strategies, showcasing its superior performance in PDAs.

Abstract

Biological processes like growth, aging, and disease progression are generally studied with follow-up scans taken at different time points, i.e., image time series (TS) based analysis. Image TS represents the evolution of anatomy over time, but different anatomies may have different structural characteristics and temporal paths. Therefore, separating the time-dependent path difference and time-independent basic anatomy/shape changes is important when comparing two image TS to understand the causes of the observed differences better. A method to untangle and quantify the path and shape difference between the TS is presented in this paper. The proposed method is evaluated with simulated and adult and fetal neuro templates. Results show that the metric can separate and quantify the path and shape differences between TS.