Abstract

Modern visual-inertial navigation systems (VINS) are faced with a critical challenge in real-world deployment: they need to operate reliably and robustly in highly dynamic environments. Current best solutions merely filter dynamic objects as outliers based on the semantics of the object category. Such an approach does not scale as it requires semantic classifiers to encompass all possibly-moving object classes; this is hard to define, let alone deploy. On the other hand, many realworld environments exhibit strong structural regularities in the form of planes such as walls and ground surfaces, which are also crucially static. We present RP-VIO, a monocular visual-inertial odometry system that leverages the simple geometry of these planes for improved robustness and accuracy in challenging dynamic environments. Since existing datasets have a limited number of dynamic elements, we also present a highly-dynamic, photorealistic synthetic dataset for a more effective evaluation of the capabilities of modern VINS systems. We evaluate our approach on this dataset, and three diverse sequences from standard datasets including two real-world dynamic sequences and show a significant improvement in robustness and accuracy over a state-of-the-art monocular visual-inertial odometry system. We also show in simulation an improvement over a simple dynamic-features masking approach. Our code and dataset are publicly available .

Abstract

We consider the problem of an agent/robot with non-holonomic kinematics avoiding dynamic and static obstacles. Additionally there may be bounds/constraints on the configurational space of the robot in the form of lane/corridor boundaries. State and velocity noise of the robot, the lanes, the obstacles, and the robot’s control noise are modelled as non-parametric distributions as Gaussian assumptions of noise models are violated in real-world scenarios. Under these assumptions, we formulate a robust MPC that samples robotic controls effectively in a manner that aligns the robot to the goal state while avoiding obstacles and staying within the lane bounds under the duress of such non-parametric noise. In particular, the MPC incorporates a distribution matching cost that effectively aligns the distribution of the current collision cone to a certain desired distribution whose samples are collisionfree. This cost is posed as a distance function in the Hilbert Space, whose minimization typically results in the collision cone samples becoming collision-free. We show tangible performance gains compared to methods that model the collision cone distribution by linearizing the Gaussian approximations of the original non-parametric state and obstacle distributions. We also show superior performance to methods that pose a chance constraint formulation of the Gaussian approximations of nonparametric noise without subjecting such approximations to further linearizations. The performance gain is shown both in terms of trajectory length and control costs that vindicates the efficacy of the proposed method. Finally we show the proposed method being used to navigate with a non holonomic differential drive robot in real-time in a realistic setting in Gazebo with dynamic and static obstacles. To the best of our knowledge, this is the first presentation of non-holonomic collision avoidance of stationary obstacles, moving obstacles and lane constraints in the presence of non-parametric state, velocity, actuator and lane boundary noise models

Abstract

— Humans have a natural ability to effortlessly comprehend linguistic commands such as “park next to the yellow sedan” and instinctively know which region of the road the vehicle should navigate. Extending this ability to autonomous vehicles is the next step towards creating fully autonomous agents that respond and act according to human commands. To this end, we propose the novel task of Referring Navigable Regions (RNR), i.e., grounding regions of interest for navigation based on the linguistic command. RNR is different from Referring Image Segmentation (RIS), which focuses on grounding an object referred to by the natural language expression instead of grounding a navigable region. For example, for a command “park next to the yellow sedan,” RIS will aim to segment the referred sedan, and RNR aims to segment the suggested parking region on the road. We introduce a new dataset, Talk2Car-RegSeg, which extends the existing Talk2car [1] dataset with segmentation masks for the regions described by the linguistic commands. A separate test split with concise manoeuvre-oriented commands is provided to assess the practicality of our dataset. We benchmark the proposed dataset using a novel transformer-based architecture. We present extensive ablations and show superior performance over baselines on multiple evaluation metrics. A downstream path planner generating trajectories based on RNR outputs confirms the efficacy of the proposed framework.

Abstract

— In this paper, we address the highly challenging problem of object goal navigation. The agent, in an unseen environment, has to perceive its surroundings to identify and navigate towards potential regions where the specified goal category can occur. Rather than developing goal driven exploration policies, we aim to adapt the existing exploration policies that maximize scene coverage to be goal-conditioned. Thus, we propose a standalone scene understanding module to identify potential regions where the goal occurs. We also propose Goal-Conditioned Exploration (GCExp), an algorithm that entails the integration of our novel scene understanding module with any existing exploration policy. We test our solution in photo-realistic simulation environments using stateof-the-art exploration policy, Active Neural Slam [1], and show improved performance over the same on every evaluation metric.

Abstract

— This paper presents a model predictive control (MPC) based algorithm for tracking multiple targets using a swarm of unmanned aerial vehicles (UAVs). All the UAVs belong to fixed-wing category with constraints on flight velocity, climb rate and turn rate. Each UAV carries a camera to detect and track the target. Two cases are considered where for the first case, the number of the UAVs is equal to the number of targets. For the second case, the number of UAVs is lesser than the number of targets leading to a conservative solution where the objective is to maximize the average time duration for which the targets are in the field-of-view (FOV) of any one of the UAV’s camera. A data driven Gaussian process (GP) based model is developed to relate the hyperparameters used in MPC to the mission efficiency. Bayesian optimization is performed to obtain the hyperparameters of the MPC that maximize the mission efficiency. Numerical simulations are performed for both cases using algorithm based on distributed MPC formulation. A performance comparison is provided with the centralized MPC formulation.

Abstract

The paper introduces a novel design for an in-pipe climbing robot with a flexible body, along with the successful simulation of the proposed mechanism. The presented Three-Module pipe climber can traverse a complex network of pipes and bends at various angles. It has bidirectional motion capabilities and is orientation independent. It can also negotiate the ‘Singularity Region’ encountered while turning through the Tjunction. The pipe climber has three modules which are arranged symmetrically apart from each other. These modules mount the tracks which provide the necessary traction for the robot to avoid slippage on the pipe’s surface. The modules, independently, have the ability to compress asymmetrically allowing the robot to bend in any direction. The taper in the front of the design and additional body flexibility due to passive spring compliance aid the motion at turns. The design of the robot is done on SOLIDWORKS and its motion is studied using MSC ADAMS. While simulating the proposed mechanism, differential speed was provided to the tracks for negotiating turns.

Abstract

The use of local detectors and descriptors in typical computer vision pipelines work well until variations in viewpoint and appearance change become extreme. Past research in this area has typically focused on one of two approaches to this challenge: the use of projections into spaces more suitable for feature matching under extreme viewpoint changes, and attempting to learn features that are inherently more robust to viewpoint change. In this paper we present a novel framework that combines learning of invariant descriptors through data augmentation and orthographic viewpoint projection. We propose rotation-robust local descriptors, learnt through training data augmentation based on rotation homographies, and a correspondence ensemble technique that combines vanilla feature correspondences with those obtained through rotation-robust features. Using a range of benchmark datasets as well as contributing a new bespoke dataset for this research domain, we evaluate the effectiveness of the proposed approach on key tasks including pose estimation and visual place recognition. Our system outperforms a range of baseline and state-of-the-art techniques, including enabling higher levels of place recognition precision across opposing place viewpoints, and achieves practically-useful performance levels even under extreme viewpoint changes.

Abstract

— Modern visual-inertial navigation systems (VINS) are faced with a critical challenge in real-world deployment: they need to operate reliably and robustly in highly dynamic environments. Current best solutions merely filter dynamic objects as outliers based on the semantics of the object category. Such an approach does not scale as it requires semantic classifiers to encompass all possibly-moving object classes; this is hard to define, let alone deploy. On the other hand, many realworld environments exhibit strong structural regularities in the form of planes such as walls and ground surfaces, which are also crucially static. We present RP-VIO, a monocular visual-inertial odometry system that leverages the simple geometry of these planes for improved robustness and accuracy in challenging dynamic environments. Since existing datasets have a limited number of dynamic elements, we also present a highly-dynamic, photorealistic synthetic dataset for a more effective evaluation of the capabilities of modern VINS systems. We evaluate our approach on this dataset, and three diverse sequences from standard datasets including two real-world dynamic sequences and show a significant improvement in robustness and accuracy over a state-of-the-art monocular visual-inertial odometry system. We also show in simulation an improvement over a simple dynamic-features masking approach. Our code and dataset are publicly available

Abstract

The simplicity of the visual servoing approach makes it an attractive option for tasks dealing with vision-based control of robots in many real-world applications. However, attaining precise alignment for unseen environments pose a challenge to existing visual servoing approaches. While classical approaches assume a perfect world, the recent data-driven approaches face issues when generalizing to novel environments. In this paper, we aim to combine the best of both worlds. We present a deep model predictive visual servoing framework that can achieve precise alignment with optimal trajectories and can generalize to novel environments. Our framework consists of a deep network for optical flow predictions, which are used along with a predictive model to forecast future optical flow. For generating an optimal set of velocities we present a control network that can be trained on the fly without any supervision. Through extensive simulations on photo-realistic indoor settings of the popular Habitat framework, we show significant performance gain due to the proposed formulation vis-a-vis recent state-of-the-art methods. Specifically, we show a faster convergence and an improved performance in trajectory length over recent approaches.

Abstract

Given a monocular color image of a warehouse rack, we aim to predict the bird’s-eye view layout for each shelf in the rack, which we term as ‘multi-layer’ layout prediction. To this end, we present RackLay, a deep neural network for real-time shelf layout estimation from a single image. Unlike previous layout estimation methods which provide a single layout for the dominant ground plane alone, RackLay estimates the top-view and front-view layout for each shelf in the considered rack populated with objects. RackLay’s architecture and its variants are versatile and estimate accurate layouts for diverse scenes characterized by varying number of visible shelves in an image, large range in shelf occupancy factor and varied background clutter. Given the extreme paucity of datasets in this space and the difficulty involved in acquiring real data from warehouses, we additionally release a flexible synthetic dataset generation pipeline WareSynth which allows users to control the generation process and tailor the dataset according to contingent application. The ablations across architectural variants and comparison with strong prior baselines vindicate the efficacy of RackLay as an apt architecture for the novel problem of multi-layered layout estimation. We also show that fusing the top-view and front-view enables 3D reasoning applications such as metric free space estimation for the considered rack.