Abstract

We explore language-agnostic deep text embeddings for severity classification of dysarthria in Amyotrophic Lateral Sclerosis (ALS). Speech recordings are transcribed by human and ASR and embeddings of the transcripts are considered. Though speech recognition accuracy has been studied for grading dysarthria severity, no effort has yet been made to utilize text embeddings of the transcripts. We perform severity classification at different granularity (2, 3, and 5-class) using data obtained from 47 ALS subjects. Experiments with dense neural network based classifiers suggest that, though text features achieve nearly equal performances as baseline speech features, like statistics of mel frequency cepstral coefficients (MFCC), for 2-class classification, speech features outperform for higher number of classes. Concatenation of text embeddings and MFCC statistics attains the best performances with mean F1 scores of 88%, 68%, and 53%, respectively, in 2, 3, and 5-class classification.

Abstract

Grammar error detection in one's speech, referred to as spoken grammar error detection (SGED), is a critical component for building computer-assisted language learning systems. Traditionally, ASR followed by text-based GED has been used, but ASR errors can limit SGED performance. In this work, we analyse SGED under three speaking conditions: spontaneous, semi-spontaneous, and memorized, using five state-of-the-art ASRs. We examine the decoded text and ASR cues such as confidence scores and aligner probabilities, out of which, for the latter one, the ground-truth grammatically correct (GGC) text is used. Experiments revealed that autoregressive ASRs show bias toward GGC text, leading to suboptimal performance. It is observed that there is an increase in performance in response to the decreased freedom of speech, with semi-spontaneous speech outperforming memorized speech. Aligner probabilities outperform confidence scores despite the alignment and overconfidence issues.

Abstract

As robotic manipulators start performing more daily tasks, striking can be a useful method for transporting objects because it significantly increases the reachable workspace. However, striking methods are underexplored compared to pick-and-place because of the difficulty of modeling and executing striking interactions. In this paper, we develop an algorithm for striking objects so that they stop at a target location. We start with an optimizer in simulation that solves for the striking velocity given the relative target position, and perform system identification to set the simulation parameters. However, real-world striking with this model does not have high accuracy because it is unclear which parameters should be considered for identification in practice. Therefore, we finally developed a residual learning approach that subsumes all unmodeled differences between the simulation and the real-world environment into action, that is, striking velocity residuals. Our real-world experiments show that the residual learning model results in 81.6% more accurate object strikes.

Abstract

Following up on our earlier study [J. Bardhan et al., Phys. Rev. D 107, 115001 (2023)], we investigate the LHC prospects of pair-produced vectorlike 𝐵 quarks decaying exotically to a new gauge-singlet (pseudo)scalar field Φ and a 𝑏 quark. After the electroweak symmetry breaking, the Φ decays predominantly to 𝑔⁡𝑔⁡/𝑏⁢𝑏 final states, leading to a fully hadronic 2⁢𝑏 +4⁢𝑗 or 6⁢𝑏 signature. Because of the large Standard Model background and the lack of leptonic handles, it is a difficult channel to probe. To overcome the challenge, we employ a hybrid deep learning model containing a graph neural network followed by a deep neural network. We estimate that such a state-of-the-art deep learning analysis pipeline can lead to a performance comparable to that in the semileptonic mode, taking the discovery (exclusion) reach up to about 𝑀_𝐵=1.8⁢(2.4)  TeV at HL-LHC when 𝐵 decays fully exotically, i.e., BR⁡(𝐵→𝑏⁢Φ)=100%.

Abstract

This study advances Kai Katahira’s Speculation Game, an agent-based model (ABM) for financial markets, by addressing its limitation in capturing order flow imbalance, a critical indicator of herd behavior. Although the original model successfully replicated key stylized facts of financial markets, it did not account for the persistence of order imbalance observed in real-world trading. Through a comprehensive analysis of two decades of BSE Sensex data, we establish the prevalence of order imbalance and its correlation with price fluctuations. To bridge this gap, we propose an extended model, Speculation Game with Information Cascade (SGIC), which integrates Self-Organized Criticality (SOC) through a sand-pile model, enabling agents to interact within a small-world network. Our proposed model not only reproduces the stylized facts captured by the original Speculation Game, but also successfully generates the additional stylized fact of order flow imbalance. These advances enhance the realism of ABMs in financial markets, providing deeper insights into the mechanisms driving herding and market fluctuations.

Abstract

The color gamut of visual scenes can encompass a multitude of hues, but how many distinct hues are individuals aware of? We examined this question in the context of texture perception, with a grid of stimuli composed of random colors chosen from color sets varying from two to six different hues or saturations. In a behavioral experiment, participants had to discriminate between differences in the number of hues present by identifying which of four color grids included a larger number of different colors. Color number discrimination was also assessed neurally, using electroencephalogram (EEG) frequency tagging, wherein a texture with an extra hue was shown as an “oddball,” once per second, in a stream of textures presented at six images per second. In both experiments, two versus three texture contrasts were readily distinguished, yet performance fell rapidly when the set size increased further. The results suggest that sensitivity to the density of the color gamut defining a texture is very restricted.

Abstract

Understanding user intent is essential for build- ing better human interaction agents, as it enables personalization, co-creation, and contextual adap- tation. However, existing approaches are either restricted to text environments, use human anno- tation, or just predict future user actions lacking the ability to reason explicitly about user goals. In this work, we introduce EARL (Early Action Reasoning for Latent intent), a theory of mind inspired inference-time algorithm that models user intent as an inverse planning problem, in- ferring latent goals from observed user actions. EARL hypothesizes potential user intent at mul- tiple stages during the course of task execution, enabling timely intervention and personalization. Evaluated on three diverse benchmarks namely Mind2Web, AiTz, and VideoGUI, and using two strong LLMs (Gemini-1.5-Pro and GPT-4o), we show that EARL consistently outperforms CoT- based LLM baselines in accurately deciphering user intent, especially under partial observations

Abstract

We introduce Sketchtopia, a large-scale dataset and AI framework designed to explore goal-driven, multimodal communication through asynchronous interactions in a Pictionary-inspired setup. Sketchtopia captures natural human interactions, including freehand sketches, open-ended guesses, and iconic feedback gestures, showcasing the complex dynamics of cooperative communication under constraints. It features over 20K gameplay sessions from 916 players, capturing 263K sketches, 10K erases, 56K guesses and 19.4K iconic feedbacks. We introduce multimodal foundational agents with capabilities for generative sketching, guess generation and asynchronous communication. Our dataset also includes 800 human-agent sessions for benchmarking the agents. We introduce novel metrics to characterize collaborative success, responsiveness to feedback and inter-agent asynchronous communication. Sketchtopia pushes the boundaries of multimodal AI, establishing a new benchmark for studying asynchronous, goal-oriented interactions between humans and AI agents.

Abstract

Modern generative models often struggle to synthesize structured objects from detailed part specifications. They frequently produce anatomically implausible outputs or hallucinated components. We introduce PLATO, a novel two-stage framework that bridges this gap by enabling precise, part-controlled object generation. The first stage is PLayGen, our novel part layout generator which takes a list of parts and object category as input and synthesizes high-fidelity layouts of part bounding boxes. To enhance PLayGen's ability to learn inter-part relationships, we introduce novel structure-based loss functions. In the second stage, PLayGen's synthesized layout is used to condition a custom-tuned ControlNet-style adapter, enforcing spatial and connectivity constraints. This results in anatomically consistent, high-fidelity object generations containing precisely the user-specified parts. We further propose new part-level evaluation metrics to rigorously quantify adherence to part specifications. Extensive experiments show that PLATO significantly outperforms state-of-the-art generative models and produces structurally coherent objects in a controllable manner — marking a step forward in modular, part-driven asset generation.

Abstract

India's gig-based food delivery platforms, such as Swiggy and Zomato, provide crucial income to marginalised communities but also entrench workers in cycles of invisible labour. Through 14 semi-structured interviews, we analyse waiting time and repetitive UI itneractions as key burdens that contribute to 'digital discomfort' for gig based food delivery agents. We find that workers employ creative strategies to navigate algorithmic management, yet remain constrained by platform-side 'gamification' and system opacity. We propose worker-centered GUI automation as a potential intervention to reduce friction while preserving agency. In conclusion, this position paper argues for rethinking HCI approaches in the Global South to prioritise worker autonomy over efficiency-driven design optimisations.