Abstract

Abstract—Building 3D maps of the environment is central to robot navigation, planning, and interaction with objects in a scene. Most existing approaches that integrate semantic concepts with 3D maps largely remain confined to the closed-set setting: they can only reason about a finite set of concepts, pre-defined at training time. Further, these maps can only be queried using class labels, or in more recent work, using text prompts. We address both these issues with ConceptFusion, a scene representation that is: (i) fundamentally open-set, enabling reasoning beyond a closed set of concepts (ii) inherently multimodal, enabling a diverse range of possible queries to the 3D map, from language, to images, to audio, to 3D geometry, all working in concert. ConceptFusion leverages the open-set capabilities of today’s foundation models that have been pretrained on internet-scale data to reason about concepts across modalities such as natural language, images, and audio. We demonstrate that pixel-aligned open-set features can be fused into 3D maps via traditional SLAM and multi-view fusion approaches. This enables effective zero-shot spatial reasoning, not needing any additional training or finetuning, and retains long-tailed concepts better than supervised approaches, outperforming them by more than 40% margin on 3D IoU. We extensively evaluate ConceptFusion on a number of real-world datasets, simulated home environments, a real-world tabletop manipulation task, and an autonomous driving platform. We showcase new avenues for blending foundation models with 3D open-set multimodal mapping. We encourage the reader to view the demos on our project page: https://concept-fusion.github.io

Abstract

Safe autonomous driving critically depends on how well the ego-vehicle can predict the trajectories of neighboring vehicles. To this end, several trajectory prediction algorithms have been presented in the existing literature. Many of these approaches output a multi-modal distribution of obstacle trajectories instead of a single deterministic prediction to account for the underlying uncertainty. However, existing planners cannot handle the multi-modality based on just sample-level information of the predictions. With this motivation, this paper proposes a trajectory optimizer that can leverage the distributional aspects of the prediction in a computationally tractable and sample-efficient manner. Our optimizer can work with arbitrarily complex distributions and thus can be used with output distribution represented as a deep neural network. The core of our approach is built on embedding distribution in Reproducing Kernel Hilbert Space (RKHS), which we leverage in two ways. First, we propose an RKHS embedding approach to select probable samples from the obstacle trajectory distribution. Second, we rephrase chance-constrained optimization as distribution matching in RKHS and propose a novel sampling-based optimizer for its solution. We validate our approach with hand-crafted and neural network-based predictors trained on real-world datasets and show improvement over the existing stochastic optimization approaches in safety metrics.

Abstract

Computer tomography (CT) have been routinely used for the diagnosis of lung diseases and recently, during the pandemic, for detecting the infectivity and severity of COVID-19 disease. One of the major concerns in using machine learning (ML) approaches for automatic processing of CT scan images in clinical setting is that these methods are trained on limited and biased subsets of publicly available COVID-19 data. This has raised concerns regarding the generalizability of these models on external datasets, not seen by the model during training. To address some of these issues, in this work CT scan images from confirmed COVID-19 data obtained from one of the largest public repositories, COVIDx CT 2A were used for training and internal validation of machine learning models. For the external validation we generated Indian-COVID-19 CT dataset, an open-source repository containing 3D CT volumes and 12096 chest CT images from 288 COVID-19 patients from India. Comparative performance evaluation of four state-of-the-art machine learning models, viz., a lightweight convolutional neural network (CNN), and three other CNN based deep learning (DL) models such as VGG-16, ResNet-50 and Inception-v3 in classifying CT images into three classes, viz., normal, non-covid pneumonia, and COVID-19 is carried out on these two datasets. Our analysis showed that the performance of all the models is comparable on the hold-out COVIDx CT 2A test set with 90%–99% accuracies (96% for CNN), while on the external Indian-COVID-19 CT dataset a drop in the performance is observed for all the models (8%–19%). The traditional machine learning model, CNN performed the best on the external dataset (accuracy 88%) in comparison to the deep learning models, indicating that a lightweight CNN is better generalizable on unseen data. The data and code are made available at https://github.com/aleesuss/c19.

Abstract

Deep learning assisted disease diagnosis using chest radiology images to assess severity of various respiratory conditions has garnered a lot of attention after the recent COVID-19 pandemic. Understanding characteristic features associated with the disease in radiology images, along with variations observed from patient-to-patient and with the progression of disease, is important while building such models. In this work, we carried out comparative analysis of various deep architectures with the proposed attention-based Convolutional Neural Network (CNN) model with customized bottleneck residual module (Attn-CNN) in classifying chest CT images into three categories, COVID-19, Normal, and Pneumonia. We show that the attention model with fewer parameters achieved better classification performance compared to state-of-the-art deep architectures such as EfficientNet-B7, Inceptionv3, ResNet-50 and VGG-16, and customized models proposed in similar studies such as COVIDNet-CT, CTnet-10, COVID-19Net, etc.

Abstract

Driving event recognition plays a crucial role in understanding and enhancing road safety. This research focuses on developing efficient time-series based models for Fall detection in two-wheelers. Traditional machine learning models proved inadequate in accurately classifying Fall scenarios due to their inability to capture temporal transitions in kinematic states. To address this limitation, time-series based Deep Learning (DL) models are proposed, utilizing Long Short-Term Memory (LSTM) networks. These networks enable direct learning from raw time series data, eliminating the need for manual feature engineering. Additionally, Bi-LSTMs were employed to capture contextual information from both past and future timesteps, further improving the model’s understanding of driving events. The architecture was enhanced with an attention mechanism to boost accuracy. Experimental results showcased that the proposed Bi-LSTM model achieved an overall accuracy of 97%, with a specific accuracy of approximately 92% in detecting Fall scenarios. This research contributes to the development of an accurate Time- series based system for Fall detection, facilitating improved road safety in the context of two-wheelers.

Abstract

The Objectifying Gaze: Impact of Sexualized Media on Viewer Gaze Behavior towards (Non)Traditional Attire Anonymous CogSci submission Abstract In this study, we investigate the priming effect of camera- induced objectifying-gaze, operationalised via sexualised mu- sic video(MV), on university students when looking at tar- get female images in traditional (saree, salwar) and western (shirt-pant, short-dress) attires in the Indian demographics. We showed participants two videos where female leads wore un- revealing Indian attire (saree) and revealing western clothing in sexualised and traditional MVs respectively. We quantified gaze as fixation-duration and visit-count (revisit frequency) metrics. Our findings report that there is a priming effect of sexualised media. Female images in all attires are equally objectified and body-biased gaze is more pronounced in sex- ualised MV priming, short-dress is more objectified in tradi- tional MV priming, and saree is less objectified overall, which confirms the social stereotypes on attire and respectability — traditionally clad (vs. non-traditional) women are seen as more moral and worthy of respect and protection compared to women in hypersexualized and objectified roles.

Abstract

In this article, a novel adaptive controller is designed for Euler-Lagrangian systems under predefined time-varying state constraints. The proposed controller could achieve this objective without a priori knowledge of system parameters and, crucially, of state-dependent uncertainties. The closed-loop stability is verified using the Lyapunov method, while the overall efficacy of the proposed scheme is verified using a simulated robotic arm compared to the state of the art.

Abstract

In this work, we describe the creation of our machine- learning-based solution for coma prognosis after cardiac arrest using longitudinal EEG and ECG recordings for the ”Predicting Neurological Recovery from Coma After Car- diac Arrest: The George B. Moody PhysioNet Challenge 2023”. Our team, “ComaToast”, had its best submission ranked 28 out of 36 teams selected worldwide, with a chal- lenge score of 0.381 on the official leaderboard for the hid- den test set. We use a combination of age and signal fea- tures from EEG and ECG recordings. Frequency domain features, specifically mean power spectral density from 4 different bands of frequencies (Delta, Theta, Alpha and Beta) and mean Burst Suppression Ratio, were extracted from pre-processed EEG recordings from the first and last available recording for a given patient. Features like mean and standard deviations were extracted along channels for ECG recordings. After imputing missing values, these fea- tures are fed to an XGBoost classifier for the final binary classification of the outcome prediction task. The features are fed to a random forest regressor to predict the CPC outcome for every patient. A solution like ours, which uses a simple model and training technique, may be more viable than deep-learning solutions in general use cases. In our final model, our approach achieved a 5-fold cross- validation score of 0.34 on the public train set.

Abstract

This paper aims to address the limitations of current water level monitoring systems, which are bulky, expensive, and difficult to maintain, resulting in limited deployment. To overcome these challenges, the study utilises Internet of Things (IoT)-enabled low-cost sensor nodes for water level monitoring. In this study, an ultrasonic sensor-based water level node is developed to send data to the cloud through GPRS (2G). Five such nodes were deployed to monitor water levels in overhead tanks and sumps on the campus of IIIT Hyderabad, India. The collected water level data were analysed for behavioural patterns and detecting faulty float switches. In addition, a deep learning algorithm was employed, which can predict future water needs. Thus, the proposed IoT-based smart water level meter offers a more accessible and cost-effective approach to water level monitoring.

Abstract

The discovery of potential therapeutic agents for life‑threatening diseases has become a significant problem. There is a requirement for fast and accurate methods to identify drug‑like molecules that can be used as potential candidates for novel targets. Existing techniques like high‑throughput screening and virtual screening are time‑consuming and inefficient. Traditional molecule generation pipelines are more efficient than virtual screening but use time‑consuming docking software. Such docking functions can be emulated using Machine Learning models with comparable accuracy and faster execution times. However, we find that when pre‑trained machine learning models are employed in generative pipelines as oracles, they suffer from model degradation in areas where data is scarce. In this study, we propose an active learning‑based model that can be added as a supplement to enhanced molecule generation architectures. The proposed method uses uncertainty sampling on the molecules created by the generator model and dynamically learns as the generator samples molecules from different regions of the chemical space. The proposed framework can generate molecules with high binding affinity with ∼a 70% improvement in runtime compared to the baseline model by labeling only ∼30% of molecules compared to the baseline oracle.