Search Results for author:
Found 80 papers, 23 papers with code
NeRF and Gaussian Splatting SLAM in the Wild
Navigating outdoor environments with visual Simultaneous Localization and Mapping (SLAM) systems poses significant challenges due to dynamic scenes, lighting variations, and seasonal changes, requiring robust solutions.
Computational Efficiency
Simultaneous Localization and Mapping
ROVER: A Multi-Season Dataset for Visual SLAM
To address this gap, we present ROVER, a comprehensive benchmark dataset tailored for evaluating visual SLAM algorithms under diverse environmental conditions and spatial configurations.
The Bare Necessities: Designing Simple, Effective Open-Vocabulary Scene Graphs
3D open-vocabulary scene graph methods are a promising map representation for embodied agents, however many current approaches are computationally expensive.
Traffic and Safety Rule Compliance of Humans in Diverse Driving Situations
The increasing interest in autonomous driving systems has highlighted the need for an in-depth analysis of human driving behavior in diverse scenarios.
Neural Fields in Robotics: A Survey
Neural Fields have emerged as a transformative approach for 3D scene representation in computer vision and robotics, enabling accurate inference of geometry, 3D semantics, and dynamics from posed 2D data.
Progressive Multi-Modal Fusion for Robust 3D Object Detection
Multi-sensor fusion is crucial for accurate 3D object detection in autonomous driving, with cameras and LiDAR being the most commonly used sensors.
Entropy-Based Uncertainty Modeling for Trajectory Prediction in Autonomous Driving
In autonomous driving, accurate motion prediction is essential for safe and efficient motion planning.
Zero-Cost Whole-Body Teleoperation for Mobile Manipulation
In this work, we present MoMa-Teleop, a novel teleoperation method that delegates the base motions to a reinforcement learning agent, leaving the operator to focus fully on the task-relevant end-effector motions.
Panoptic-Depth Forecasting
Forecasting the semantics and 3D structure of scenes is essential for robots to navigate and plan actions safely.
Motion Forecasting via Model-Based Risk Minimization
In this paper, we propose a novel sampling method applicable to trajectory prediction based on the predictions of multiple models.
Visual-Inertial SLAM for Agricultural Robotics: Benchmarking the Benefits and Computational Costs of Loop Closing
The findings highlight the importance of loop closing in improving localization accuracy while managing computational resources efficiently, offering valuable insights for optimizing Visual-Inertial SLAM systems for practical outdoor applications in mobile robotics.
Taxonomy-Aware Continual Semantic Segmentation in Hyperbolic Spaces for Open-World Perception
However, existing methods impose strict rigidity on old classes, reducing their effectiveness in learning new incremental classes.
Autonomous Driving
Class-Incremental Semantic Segmentation
+3
The Art of Imitation: Learning Long-Horizon Manipulation Tasks from Few Demonstrations
Task Parametrized Gaussian Mixture Models (TP-GMM) are a sample-efficient method for learning object-centric robot manipulation tasks.
Learning Lane Graphs from Aerial Imagery Using Transformers
This work introduces a novel approach to generating successor lane graphs from aerial imagery, utilizing the advanced capabilities of transformer models.
Reinforcement Learning as a Robotics-Inspired Framework for Insect Navigation: From Spatial Representations to Neural Implementation
Bees are among the master navigators of the insect world.
A Good Foundation is Worth Many Labels: Label-Efficient Panoptic Segmentation
A key challenge for the widespread application of learning-based models for robotic perception is to significantly reduce the required amount of annotated training data while achieving accurate predictions.
LetsMap: Unsupervised Representation Learning for Semantic BEV Mapping
Our approach pretrains the network to independently reason about scene geometry and scene semantics using two disjoint neural pathways in an unsupervised manner and then finetunes it for the task of semantic BEV mapping using only a small fraction of labels in the BEV.
Imagine2touch: Predictive Tactile Sensing for Robotic Manipulation using Efficient Low-Dimensional Signals
We train Imagine2touch on two out of those shapes and validate it on the ood.
CenterArt: Joint Shape Reconstruction and 6-DoF Grasp Estimation of Articulated Objects
Precisely grasping and reconstructing articulated objects is key to enabling general robotic manipulation.
A Point-Based Approach to Efficient LiDAR Multi-Task Perception
Unlike other LiDAR-based multi-task architectures, our proposed PAttFormer does not require separate feature encoders for multiple task-specific point cloud representations, resulting in a network that is 3x smaller and 1. 4x faster while achieving competitive performance on the nuScenes and KITTI benchmarks for autonomous driving perception.
Hierarchical Open-Vocabulary 3D Scene Graphs for Language-Grounded Robot Navigation
Recent open-vocabulary robot mapping methods enrich dense geometric maps with pre-trained visual-language features.
PseudoTouch: Efficiently Imaging the Surface Feel of Objects for Robotic Manipulation
We frame this problem as the task of learning a low-dimensional visual-tactile embedding, wherein we encode a depth patch from which we decode the tactile signal.
DITTO: Demonstration Imitation by Trajectory Transformation
In the online trajectory generation stage, we first re-detect all objects, then warp the demonstration trajectory to the current scene and execute it on the robot.
BEVCar: Camera-Radar Fusion for BEV Map and Object Segmentation
Semantic scene segmentation from a bird's-eye-view (BEV) perspective plays a crucial role in facilitating planning and decision-making for mobile robots.
CMRNext: Camera to LiDAR Matching in the Wild for Localization and Extrinsic Calibration
In this paper, we present CMRNext, a novel approach for camera-LIDAR matching that is independent of sensor-specific parameters, generalizable, and can be used in the wild for monocular localization in LiDAR maps and camera-LiDAR extrinsic calibration.
CenterGrasp: Object-Aware Implicit Representation Learning for Simultaneous Shape Reconstruction and 6-DoF Grasp Estimation
It consists of an RGB-D image encoder that leverages recent advances to detect objects and infer their pose and latent code, and a decoder to predict shape and grasps for each object in the scene.
Amodal Optical Flow
Optical flow estimation is very challenging in situations with transparent or occluded objects.
Panoptic Out-of-Distribution Segmentation
Deep learning has led to remarkable strides in scene understanding with panoptic segmentation emerging as a key holistic scene interpretation task.
Compositional Servoing by Recombining Demonstrations
In this work, we present a framework that formulates the visual servoing task as graph traversal.
Few-Shot Panoptic Segmentation With Foundation Models
Concurrently, recent breakthroughs in visual representation learning have sparked a paradigm shift leading to the advent of large foundation models that can be trained with completely unlabeled images.
RaLF: Flow-based Global and Metric Radar Localization in LiDAR Maps
RaLF is composed of radar and LiDAR feature encoders, a place recognition head that generates global descriptors, and a metric localization head that predicts the 3-DoF transformation between the radar scan and the map.
Multi-camera Bird's Eye View Perception for Autonomous Driving
Surround vision systems that are pretty common in new vehicles use the IPM principle to generate a BEV image and to show it on display to the driver.
AmodalSynthDrive: A Synthetic Amodal Perception Dataset for Autonomous Driving
To address these limitations, we introduce AmodalSynthDrive, a synthetic multi-task multi-modal amodal perception dataset.
A Smart Robotic System for Industrial Plant Supervision
In today's chemical plants, human field operators perform frequent integrity checks to guarantee high safety standards, and thus are possibly the first to encounter dangerous operating conditions.
Syn-Mediverse: A Multimodal Synthetic Dataset for Intelligent Scene Understanding of Healthcare Facilities
Safety and efficiency are paramount in healthcare facilities where the lives of patients are at stake.
Learning Hierarchical Interactive Multi-Object Search for Mobile Manipulation
We present HIMOS, a hierarchical reinforcement learning approach that learns to compose exploration, navigation, and manipulation skills.
The Treachery of Images: Bayesian Scene Keypoints for Deep Policy Learning in Robotic Manipulation
We employ our method to learn challenging multi-object robot manipulation tasks from wrist camera observations and demonstrate superior utility for policy learning compared to other representation learning techniques.
Self-Supervised Multi-Object Tracking For Autonomous Driving From Consistency Across Timescales
We hypothesize that this drawback results from formulating self-supervised objectives that are limited to single frames or frame pairs.
INoD: Injected Noise Discriminator for Self-Supervised Representation Learning in Agricultural Fields
Perception datasets for agriculture are limited both in quantity and diversity which hinders effective training of supervised learning approaches.
CARTO: Category and Joint Agnostic Reconstruction of ARTiculated Objects
We present CARTO, a novel approach for reconstructing multiple articulated objects from a single stereo RGB observation.
CoDEPS: Online Continual Learning for Depth Estimation and Panoptic Segmentation
Operating a robot in the open world requires a high level of robustness with respect to previously unseen environments.
EvCenterNet: Uncertainty Estimation for Object Detection using Evidential Learning
In this work, we propose EvCenterNet, a novel uncertainty-aware 2D object detection framework using evidential learning to directly estimate both classification and regression uncertainties.
Self-Supervised Representation Learning from Temporal Ordering of Automated Driving Sequences
Self-supervised feature learning enables perception systems to benefit from the vast raw data recorded by vehicle fleets worldwide.
Learning and Aggregating Lane Graphs for Urban Automated Driving
To overcome these challenges, we propose a novel bottom-up approach to lane graph estimation from aerial imagery that aggregates multiple overlapping graphs into a single consistent graph.
SkyEye: Self-Supervised Bird's-Eye-View Semantic Mapping Using Monocular Frontal View Images
Implicit supervision trains the model by enforcing spatial consistency of the scene over time based on FV semantic sequences, while explicit supervision exploits BEV pseudolabels generated from FV semantic annotations and self-supervised depth estimates.
Fairness and Bias in Robot Learning
In this work, we present the first survey on fairness in robot learning from an interdisciplinary perspective spanning technical, ethical, and legal challenges.
N$^2$M$^2$: Learning Navigation for Arbitrary Mobile Manipulation Motions in Unseen and Dynamic Environments
Despite its importance in both industrial and service robotics, mobile manipulation remains a significant challenge as it requires a seamless integration of end-effector trajectory generation with navigation skills as well as reasoning over long-horizons.
Perceiving the Invisible: Proposal-Free Amodal Panoptic Segmentation
Amodal panoptic segmentation aims to connect the perception of the world to its cognitive understanding.
Self-Supervised Learning of Multi-Object Keypoints for Robotic Manipulation
In recent years, policy learning methods using either reinforcement or imitation have made significant progress.
3D Multi-Object Tracking Using Graph Neural Networks with Cross-Edge Modality Attention
We evaluate our approach using various sensor modalities and model configurations on the challenging nuScenes and KITTI datasets.
On Hyperbolic Embeddings in 2D Object Detection
Object detection, for the most part, has been formulated in the euclidean space, where euclidean or spherical geodesic distances measure the similarity of an image region to an object class prototype.
Amodal Panoptic Segmentation
To enable robots to reason with this capability, we formulate and propose a novel task that we name amodal panoptic segmentation.
Ranked #1 on
Amodal Panoptic Segmentation
on BDD100K val
Neural Architecture Search for Dense Prediction Tasks in Computer Vision
The success of deep learning in recent years has lead to a rising demand for neural network architecture engineering.
Doing Right by Not Doing Wrong in Human-Robot Collaboration
As robotic systems become more and more capable of assisting humans in their everyday lives, we must consider the opportunities for these artificial agents to make their human collaborators feel unsafe or to treat them unfairly.
Contrastive Object Detection Using Knowledge Graph Embeddings
Object recognition for the most part has been approached as a one-hot problem that treats classes to be discrete and unrelated.
7th AI Driving Olympics: 1st Place Report for Panoptic Tracking
In this technical report, we describe our EfficientLPT architecture that won the panoptic tracking challenge in the 7th AI Driving Olympics at NeurIPS 2021.
Catch Me If You Hear Me: Audio-Visual Navigation in Complex Unmapped Environments with Moving Sounds
Audio-visual navigation combines sight and hearing to navigate to a sound-emitting source in an unmapped environment.
Robot Skill Adaptation via Soft Actor-Critic Gaussian Mixture Models
A core challenge for an autonomous agent acting in the real world is to adapt its repertoire of skills to cope with its noisy perception and dynamics.
Unsupervised Domain Adaptation for LiDAR Panoptic Segmentation
Unsupervised Domain Adaptation (UDA) techniques are thus essential to fill this domain gap and retain the performance of models on new sensor setups without the need for additional data labeling.
Panoptic nuScenes: A Large-Scale Benchmark for LiDAR Panoptic Segmentation and Tracking
Panoptic scene understanding and tracking of dynamic agents are essential for robots and automated vehicles to navigate in urban environments.
Ranked #1 on
Panoptic Segmentation
on Panoptic nuScenes test
Bird's-Eye-View Panoptic Segmentation Using Monocular Frontal View Images
Bird's-Eye-View (BEV) maps have emerged as one of the most powerful representations for scene understanding due to their ability to provide rich spatial context while being easy to interpret and process.
Multi-Perspective Anomaly Detection
Furthermore, we introduce the dices dataset, which consists of over 2000 grayscale images of falling dices from multiple perspectives, with 5\% of the images containing rare anomalies (e. g., drill holes, sawing, or scratches).
LCDNet: Deep Loop Closure Detection and Point Cloud Registration for LiDAR SLAM
Loop closure detection is an essential component of Simultaneous Localization and Mapping (SLAM) systems, which reduces the drift accumulated over time.
There is More than Meets the Eye: Self-Supervised Multi-Object Detection and Tracking with Sound by Distilling Multimodal Knowledge
In this work, we present the novel self-supervised MM-DistillNet framework consisting of multiple teachers that leverage diverse modalities including RGB, depth and thermal images, to simultaneously exploit complementary cues and distill knowledge into a single audio student network.
EfficientLPS: Efficient LiDAR Panoptic Segmentation
Panoptic segmentation of point clouds is a crucial task that enables autonomous vehicles to comprehend their vicinity using their highly accurate and reliable LiDAR sensors.
From Learning to Relearning: A Framework for Diminishing Bias in Social Robot Navigation
The exponentially increasing advances in robotics and machine learning are facilitating the transition of robots from being confined to controlled industrial spaces to performing novel everyday tasks in domestic and urban environments.
Robust Vision Challenge 2020 -- 1st Place Report for Panoptic Segmentation
In this technical report, we present key details of our winning panoptic segmentation architecture EffPS_b1bs4_RVC.
Dynamic Object Removal and Spatio-Temporal RGB-D Inpainting via Geometry-Aware Adversarial Learning
Dynamic objects have a significant impact on the robot's perception of the environment which degrades the performance of essential tasks such as localization and mapping.
CMRNet++: Map and Camera Agnostic Monocular Visual Localization in LiDAR Maps
In this paper, we now take it a step further by introducing CMRNet++, which is a significantly more robust model that not only generalizes to new places effectively, but is also independent of the camera parameters.
MOPT: Multi-Object Panoptic Tracking
In this paper, we introduce a novel perception task denoted as multi-object panoptic tracking (MOPT), which unifies the conventionally disjoint tasks of semantic segmentation, instance segmentation, and multi-object tracking.
EfficientPS: Efficient Panoptic Segmentation
Understanding the scene in which an autonomous robot operates is critical for its competent functioning.
Ranked #1 on
Panoptic Segmentation
on Indian Driving Dataset
Self-Supervised Visual Terrain Classification from Unsupervised Acoustic Feature Learning
In this work, we propose a novel terrain classification framework leveraging an unsupervised proprioceptive classifier that learns from vehicle-terrain interaction sounds to self-supervise an exteroceptive classifier for pixel-wise semantic segmentation of images.
Vision-Based Autonomous UAV Navigation and Landing for Urban Search and Rescue
This problem is extremely challenging as pre-existing maps cannot be leveraged for navigation due to structural changes that may have occurred.
Robot Localization in Floor Plans Using a Room Layout Edge Extraction Network
Indoor localization is one of the crucial enablers for deployment of service robots.
Vision-based Autonomous Landing in Catastrophe-Struck Environments
However, these UAVs have to be able to autonomously land on debris piles in order to accurately locate the survivors.
Robotics
Multimodal Interaction-aware Motion Prediction for Autonomous Street Crossing
Learned representations from the traffic light recognition stream are fused with the estimated trajectories from the motion prediction stream to learn the crossing decision.
Self-Supervised Model Adaptation for Multimodal Semantic Segmentation
To address this limitation, we propose a mutimodal semantic segmentation framework that dynamically adapts the fusion of modality-specific features while being sensitive to the object category, spatial location and scene context in a self-supervised manner.
Ranked #1 on
Scene Recognition
on ScanNet
VLocNet++: Deep Multitask Learning for Semantic Visual Localization and Odometry
Semantic understanding and localization are fundamental enablers of robot autonomy that have for the most part been tackled as disjoint problems.
Deep Spatiotemporal Models for Robust Proprioceptive Terrain Classification
Terrain classification is a critical component of any autonomous mobile robot system operating in unknown real-world environments.
Deep Auxiliary Learning for Visual Localization and Odometry
We evaluate our proposed VLocNet on indoor as well as outdoor datasets and show that even our single task model exceeds the performance of state-of-the-art deep architectures for global localization, while achieving competitive performance for visual odometry estimation.
