Search Results for author:
Found 106 papers, 46 papers with code
Hierarchical Open-Vocabulary 3D Scene Graphs for Language-Grounded Robot Navigation
Recent open-vocabulary robot mapping methods enrich dense geometric maps with pre-trained visual-language features.
BEVCar: Camera-Radar Fusion for BEV Map and Object Segmentation
Semantic scene segmentation from a bird's-eye-view (BEV) perspective plays a crucial role in facilitating planning and decision-making for mobile robots.
Single-Agent Actor Critic for Decentralized Cooperative Driving
To bridge this gap and advance the field of active traffic management towards greater decentralization, we introduce a novel asymmetric actor-critic model aimed at learning decentralized cooperative driving policies for autonomous vehicles using single-agent reinforcement learning.
CenterGrasp: Object-Aware Implicit Representation Learning for Simultaneous Shape Reconstruction and 6-DoF Grasp Estimation
It consists of an RGB-D image encoder that leverages recent advances to detect objects and infer their pose and latent code, and a decoder to predict shape and grasps for each object in the scene.
Robot Skill Generalization via Keypoint Integrated Soft Actor-Critic Gaussian Mixture Models
A long-standing challenge for a robotic manipulation system operating in real-world scenarios is adapting and generalizing its acquired motor skills to unseen environments.
Care3D: An Active 3D Object Detection Dataset of Real Robotic-Care Environments
As labor shortage increases in the health sector, the demand for assistive robotics grows.
Few-Shot Panoptic Segmentation With Foundation Models
Concurrently, recent breakthroughs in visual representation learning have sparked a paradigm shift leading to the advent of large foundation models that can be trained with completely unlabeled images.
A Smart Robotic System for Industrial Plant Supervision
In today's chemical plants, human field operators perform frequent integrity checks to guarantee high safety standards, and thus are possibly the first to encounter dangerous operating conditions.
AutoGraph: Predicting Lane Graphs from Traffic Observations
To overcome this limitation, we propose to use the motion patterns of traffic participants as lane graph annotations.
End-to-end 2D-3D Registration between Image and LiDAR Point Cloud for Vehicle Localization
A popular approach to robot localization is based on image-to-point cloud registration, which combines illumination-invariant LiDAR-based mapping with economical image-based localization.
The Treachery of Images: Bayesian Scene Keypoints for Deep Policy Learning in Robotic Manipulation
We employ our method to learn challenging multi-object robot manipulation tasks from wrist camera observations and demonstrate superior utility for policy learning compared to other representation learning techniques.
Improving Deep Dynamics Models for Autonomous Vehicles with Multimodal Latent Mapping of Surfaces
By training everything end-to-end with the loss of the dynamics model, we enforce the latent mapper to learn an update rule for the latent map that is useful for the subsequent dynamics model.
CoDEPS: Online Continual Learning for Depth Estimation and Panoptic Segmentation
Operating a robot in the open world requires a high level of robustness with respect to previously unseen environments.
Dynamic Update-to-Data Ratio: Minimizing World Model Overfitting
Early stopping based on the validation set performance is a popular approach to find the right balance between under- and overfitting in the context of supervised learning.
Model-based Reinforcement Learning
reinforcement-learning
+1
Audio Visual Language Maps for Robot Navigation
While interacting in the world is a multi-sensory experience, many robots continue to predominantly rely on visual perception to map and navigate in their environments.
EvCenterNet: Uncertainty Estimation for Object Detection using Evidential Learning
In this work, we propose EvCenterNet, a novel uncertainty-aware 2D object detection framework using evidential learning to directly estimate both classification and regression uncertainties.
Learning and Aggregating Lane Graphs for Urban Automated Driving
To overcome these challenges, we propose a novel bottom-up approach to lane graph estimation from aerial imagery that aggregates multiple overlapping graphs into a single consistent graph.
SkyEye: Self-Supervised Bird's-Eye-View Semantic Mapping Using Monocular Frontal View Images
Implicit supervision trains the model by enforcing spatial consistency of the scene over time based on FV semantic sequences, while explicit supervision exploits BEV pseudolabels generated from FV semantic annotations and self-supervised depth estimates.
Visual Language Maps for Robot Navigation
Grounding language to the visual observations of a navigating agent can be performed using off-the-shelf visual-language models pretrained on Internet-scale data (e. g., image captions).
Uncertainty-aware LiDAR Panoptic Segmentation
Current learning-based methods typically try to achieve maximum performance for this task, while neglecting a proper estimation of the associated uncertainties.
Grounding Language with Visual Affordances over Unstructured Data
Recent works have shown that Large Language Models (LLMs) can be applied to ground natural language to a wide variety of robot skills.
Ranked #1 on
Avg. sequence length
on CALVIN
Latent Plans for Task-Agnostic Offline Reinforcement Learning
Concretely, we combine a low-level policy that learns latent skills via imitation learning and a high-level policy learned from offline reinforcement learning for skill-chaining the latent behavior priors.
T3VIP: Transformation-based 3D Video Prediction
To the best of our knowledge, our model is the first generative model that provides an RGB-D video prediction of the future for a static camera.
TrackletMapper: Ground Surface Segmentation and Mapping from Traffic Participant Trajectories
Robustly classifying ground infrastructure such as roads and street crossings is an essential task for mobile robots operating alongside pedestrians.
USegScene: Unsupervised Learning of Depth, Optical Flow and Ego-Motion with Semantic Guidance and Coupled Networks
In this paper we propose USegScene, a framework for semantically guided unsupervised learning of depth, optical flow and ego-motion estimation for stereo camera images using convolutional neural networks.
Uncertainty-aware Panoptic Segmentation
In this work, we introduce the novel task of uncertainty-aware panoptic segmentation, which aims to predict per-pixel semantic and instance segmentations, together with per-pixel uncertainty estimates.
What Matters in Language Conditioned Robotic Imitation Learning over Unstructured Data
We have open-sourced our implementation to facilitate future research in learning to perform many complex manipulation skills in a row specified with natural language.
Affordance Learning from Play for Sample-Efficient Policy Learning
Robots operating in human-centered environments should have the ability to understand how objects function: what can be done with each object, where this interaction may occur, and how the object is used to achieve a goal.
Self-Supervised Moving Vehicle Detection from Audio-Visual Cues
In extensive experiments carried out with a real-world dataset, we demonstrate that our approach provides accurate detections of moving vehicles and does not require manual annotations.
CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Tasks
We show that a baseline model based on multi-context imitation learning performs poorly on CALVIN, suggesting that there is significant room for developing innovative agents that learn to relate human language to their world models with this benchmark.
Robot Skill Adaptation via Soft Actor-Critic Gaussian Mixture Models
A core challenge for an autonomous agent acting in the real world is to adapt its repertoire of skills to cope with its noisy perception and dynamics.
Adaptively Calibrated Critic Estimates for Deep Reinforcement Learning
Accurate value estimates are important for off-policy reinforcement learning.
Robust Monocular Localization in Sparse HD Maps Leveraging Multi-Task Uncertainty Estimation
Robust localization in dense urban scenarios using a low-cost sensor setup and sparse HD maps is highly relevant for the current advances in autonomous driving, but remains a challenging topic in research.
Courteous Behavior of Automated Vehicles at Unsignalized Intersections via Reinforcement Learning
Our reinforcement learning agent learns a policy for a centralized controller to let connected autonomous vehicles at unsignalized intersections give up their right of way and yield to other vehicles to optimize traffic flow.
Lane Graph Estimation for Scene Understanding in Urban Driving
Lane-level scene annotations provide invaluable data in autonomous vehicles for trajectory planning in complex environments such as urban areas and cities.
Ranked #2 on
Lane Detection
on nuScenes
Pre-training of Deep RL Agents for Improved Learning under Domain Randomization
Visual domain randomization in simulated environments is a widely used method to transfer policies trained in simulation to real robots.
Sparse Auxiliary Networks for Unified Monocular Depth Prediction and Completion
Estimating scene geometry from data obtained with cost-effective sensors is key for robots and self-driving cars.
Learning to Track with Object Permanence
In this work, we introduce an end-to-end trainable approach for joint object detection and tracking that is capable of such reasoning.
Composing Pick-and-Place Tasks By Grounding Language
Controlling robots to perform tasks via natural language is one of the most challenging topics in human-robot interaction.
EfficientLPS: Efficient LiDAR Panoptic Segmentation
Panoptic segmentation of point clouds is a crucial task that enables autonomous vehicles to comprehend their vicinity using their highly accurate and reliable LiDAR sensors.
Modality-Buffet for Real-Time Object Detection
Real-time object detection in videos using lightweight hardware is a crucial component of many robotic tasks.
An Efficient Real-Time NMPC for Quadrotor Position Control under Communication Time-Delay
The advances in computer processor technology have enabled the application of nonlinear model predictive control (NMPC) to agile systems, such as quadrotors.
Robotics Systems and Control Systems and Control Optimization and Control
Holistic Filter Pruning for Efficient Deep Neural Networks
Deep neural networks (DNNs) are usually over-parameterized to increase the likelihood of getting adequate initial weights by random initialization.
Driving Through Ghosts: Behavioral Cloning with False Positives
In this work, we propose a behavioral cloning approach that can safely leverage imperfect perception without being conservative.
Neural Ray Surfaces for Self-Supervised Learning of Depth and Ego-motion
Self-supervised learning has emerged as a powerful tool for depth and ego-motion estimation, leading to state-of-the-art results on benchmark datasets.
PillarFlow: End-to-end Birds-eye-view Flow Estimation for Autonomous Driving
In autonomous driving, accurately estimating the state of surrounding obstacles is critical for safe and robust path planning.
Hindsight for Foresight: Unsupervised Structured Dynamics Models from Physical Interaction
A key challenge for an agent learning to interact with the world is to reason about physical properties of objects and to foresee their dynamics under the effect of applied forces.
Beyond Single Stage Encoder-Decoder Networks: Deep Decoders for Semantic Image Segmentation
In order to further improve the architecture we introduce a weight function which aims to re-balance classes to increase the attention of the networks to under-represented objects.
Scaling Imitation Learning in Minecraft
Imitation learning is a powerful family of techniques for learning sensorimotor coordination in immersive environments.
Self-supervised Transfer Learning for Instance Segmentation through Physical Interaction
To achieve this, we fine-tune an existing DeepMask network for instance segmentation on the self-labeled training data acquired by the robot.
MOPT: Multi-Object Panoptic Tracking
In this paper, we introduce a novel perception task denoted as multi-object panoptic tracking (MOPT), which unifies the conventionally disjoint tasks of semantic segmentation, instance segmentation, and multi-object tracking.
HeatNet: Bridging the Day-Night Domain Gap in Semantic Segmentation with Thermal Images
We avoid the expensive annotation of nighttime images by leveraging an existing daytime RGB-dataset and propose a teacher-student training approach that transfers the dataset's knowledge to the nighttime domain.
Efficiency and Equity are Both Essential: A Generalized Traffic Signal Controller with Deep Reinforcement Learning
In this paper we present an approach to learning policies for signal controllers using deep reinforcement learning aiming for optimized traffic flow.
SYMOG: learning symmetric mixture of Gaussian modes for improved fixed-point quantization
We propose SYMOG (symmetric mixture of Gaussian modes), which significantly decreases the complexity of DNNs through low-bit fixed-point quantization.
Learning Object Placements For Relational Instructions by Hallucinating Scene Representations
One particular requirement for such robots is that they are able to understand spatial relations and can place objects in accordance with the spatial relations expressed by their user.
Self-Supervised Visual Terrain Classification from Unsupervised Acoustic Feature Learning
In this work, we propose a novel terrain classification framework leveraging an unsupervised proprioceptive classifier that learns from vehicle-terrain interaction sounds to self-supervise an exteroceptive classifier for pixel-wise semantic segmentation of images.
Closed-Form Full Map Posteriors for Robot Localization with Lidar Sensors
A popular class of lidar-based grid mapping algorithms computes for each map cell the probability that it reflects an incident laser beam.
Long-Term Urban Vehicle Localization Using Pole Landmarks Extracted from 3-D Lidar Scans
Due to their ubiquity and long-term stability, pole-like objects are well suited to serve as landmarks for vehicle localization in urban environments.
A Maximum Likelihood Approach to Extract Finite Planes from 3-D Laser Scans
Whether it is object detection, model reconstruction, laser odometry, or point cloud registration: Plane extraction is a vital component of many robotic systems.
DCT Maps: Compact Differentiable Lidar Maps Based on the Cosine Transform
Most robot mapping techniques for lidar sensors tessellate the environment into pixels or voxels and assume uniformity of the environment within them.
An Analytical Lidar Sensor Model Based on Ray Path Information
However, many common lidar models perform poorly in unstructured, unpredictable environments, they lack a consistent physical model for both mapping and localization, and they do not exploit all the information the sensor provides, e. g. out-of-range measurements.
Adversarial Skill Networks: Unsupervised Robot Skill Learning from Video
Our method learns a general skill embedding independently from the task context by using an adversarial loss.
Self-supervised 3D Shape and Viewpoint Estimation from Single Images for Robotics
We present a convolutional neural network for joint 3D shape prediction and viewpoint estimation from a single input image.
3D Object Reconstruction From A Single Image
3D Reconstruction
+3
Adaptive Curriculum Generation from Demonstrations for Sim-to-Real Visuomotor Control
We propose Adaptive Curriculum Generation from Demonstrations (ACGD) for reinforcement learning in the presence of sparse rewards.
Learning User Preferences for Trajectories from Brain Signals
Robot motions in the presence of humans should not only be feasible and safe, but also conform to human preferences.
Learning Multimodal Fixed-Point Weights using Gradient Descent
Due to their high computational complexity, deep neural networks are still limited to powerful processing units.
CMRNet: Camera to LiDAR-Map Registration
In this paper we present CMRNet, a realtime approach based on a Convolutional Neural Network to localize an RGB image of a scene in a map built from LiDAR data.
DeepTemporalSeg: Temporally Consistent Semantic Segmentation of 3D LiDAR Scans
To make the predictions from the DCNN temporally consistent, we propose a Bayes filter based method.
Vision-Based Autonomous UAV Navigation and Landing for Urban Search and Rescue
This problem is extremely challenging as pre-existing maps cannot be leveraged for navigation due to structural changes that may have occurred.
Scheduled Intrinsic Drive: A Hierarchical Take on Intrinsically Motivated Exploration
Many state-of-the-art methods use intrinsic motivation to complement the sparse extrinsic reward signal, giving the agent more opportunities to receive feedback during exploration.
Robot Localization in Floor Plans Using a Room Layout Edge Extraction Network
Indoor localization is one of the crucial enablers for deployment of service robots.
Learning a Local Feature Descriptor for 3D LiDAR Scans
Our proposed architecture consists of a Siamese network for learning a feature descriptor and a metric learning network for matching the descriptors.
Vision-based Autonomous Landing in Catastrophe-Struck Environments
However, these UAVs have to be able to autonomously land on debris piles in order to accurately locate the survivors.
Robotics
Multimodal Interaction-aware Motion Prediction for Autonomous Street Crossing
Learned representations from the traffic light recognition stream are fused with the estimated trajectories from the motion prediction stream to learn the crossing decision.
Self-Supervised Model Adaptation for Multimodal Semantic Segmentation
To address this limitation, we propose a mutimodal semantic segmentation framework that dynamically adapts the fusion of modality-specific features while being sensitive to the object category, spatial location and scene context in a self-supervised manner.
Ranked #1 on
Scene Recognition
on ScanNet
Intracranial Error Detection via Deep Learning
Deep learning techniques have revolutionized the field of machine learning and were recently successfully applied to various classification problems in noninvasive electroencephalography (EEG).
VLocNet++: Deep Multitask Learning for Semantic Visual Localization and Odometry
Semantic understanding and localization are fundamental enablers of robot autonomy that have for the most part been tackled as disjoint problems.
The Limits and Potentials of Deep Learning for Robotics
In this paper we discuss a number of robotics-specific learning, reasoning, and embodiment challenges for deep learning.
Robotics
Curiosity-driven Exploration for Mapless Navigation with Deep Reinforcement Learning
A video of our experimental results can be found at https://goo. gl/pWbpcF.
Deep Spatiotemporal Models for Robust Proprioceptive Terrain Classification
Terrain classification is a critical component of any autonomous mobile robot system operating in unknown real-world environments.
Deep Auxiliary Learning for Visual Localization and Odometry
We evaluate our proposed VLocNet on indoor as well as outdoor datasets and show that even our single task model exceeds the performance of state-of-the-art deep architectures for global localization, while achieving competitive performance for visual odometry estimation.
3D Human Pose Estimation in RGBD Images for Robotic Task Learning
We propose an approach to estimate 3D human pose in real world units from a single RGBD image and show that it exceeds performance of monocular 3D pose estimation approaches from color as well as pose estimation exclusively from depth.
Ranked #14 on
3D Human Pose Estimation
on Total Capture
VR-Goggles for Robots: Real-to-sim Domain Adaptation for Visual Control
In this paper, we deal with the reality gap from a novel perspective, targeting transferring Deep Reinforcement Learning (DRL) policies learned in simulated environments to the real-world domain for visual control tasks.
The signature of robot action success in EEG signals of a human observer: Decoding and visualization using deep convolutional neural networks
Analysis of brain signals from a human interacting with a robot may help identifying robot errors, but accuracies of such analyses have still substantial space for improvement.
Deep Transfer Learning for Error Decoding from Non-Invasive EEG
We recorded high-density EEG in a flanker task experiment (31 subjects) and an online BCI control paradigm (4 subjects).
Socially Compliant Navigation through Raw Depth Inputs with Generative Adversarial Imitation Learning
Experiments show that our GAIL-based approach greatly improves the safety and efficiency of the behavior of mobile robots from pure behavior cloning.
From Plants to Landmarks: Time-invariant Plant Localization that uses Deep Pose Regression in Agricultural Fields
Agricultural robots are expected to increase yields in a sustainable way and automate precision tasks, such as weeding and plant monitoring.
Brain Responses During Robot-Error Observation
Our findings suggest that non-invasive recordings of brain responses elicited when observing robots indeed contain decodable information about the correctness of the robot's action and the type of observed robot.
Deep Detection of People and their Mobility Aids for a Hospital Robot
In this paper, we propose a depth-based perception pipeline that estimates the position and velocity of people in the environment and categorizes them according to the mobility aids they use: pedestrian, person in wheelchair, person in a wheelchair with a person pushing them, person with crutches and person using a walker.
Learning to Singulate Objects using a Push Proposal Network
We present a novel neural network-based approach that separates unknown objects in clutter by selecting favourable push actions.
Acting Thoughts: Towards a Mobile Robotic Service Assistant for Users with Limited Communication Skills
As our results demonstrate, our system is capable of adapting to frequent changes in the environment and reliably completing given tasks within a reasonable amount of time.
Choosing Smartly: Adaptive Multimodal Fusion for Object Detection in Changing Environments
Object detection is an essential task for autonomous robots operating in dynamic and changing environments.
Optimization Beyond the Convolution: Generalizing Spatial Relations with End-to-End Metric Learning
To operate intelligently in domestic environments, robots require the ability to understand arbitrary spatial relations between objects and to generalize them to objects of varying sizes and shapes.
Neural SLAM: Learning to Explore with External Memory
We present an approach for agents to learn representations of a global map from sensor data, to aid their exploration in new environments.
Reinforcement Learning (RL)
Simultaneous Localization and Mapping
Topometric Localization with Deep Learning
Compared to LiDAR-based localization methods, which provide high accuracy but rely on expensive sensors, visual localization approaches only require a camera and thus are more cost-effective while their accuracy and reliability typically is inferior to LiDAR-based methods.
Deep Semantic Classification for 3D LiDAR Data
To learn the distinction between movable and non-movable points in the environment, we introduce an approach based on deep neural network and for detecting the dynamic points, we estimate pointwise motion.
Deep learning with convolutional neural networks for EEG decoding and visualization
PLEASE READ AND CITE THE REVISED VERSION at Human Brain Mapping: http://onlinelibrary. wiley. com/doi/10. 1002/hbm. 23730/full Code available here: https://github. com/robintibor/braindecode
Metric Learning for Generalizing Spatial Relations to New Objects
Human-centered environments are rich with a wide variety of spatial relations between everyday objects.
A Survey of Deep Network Solutions for Learning Control in Robotics: From Reinforcement to Imitation
We carry out our discussions on the two main paradigms for learning control with deep networks: deep reinforcement learning and imitation learning.
Deep Reinforcement Learning with Successor Features for Navigation across Similar Environments
We propose a successor feature based deep reinforcement learning algorithm that can learn to transfer knowledge from previously mastered navigation tasks to new problem instances.
The Freiburg Groceries Dataset
In this paper, we address this issue and present a dataset consisting of 5, 000 images covering 25 different classes of groceries, with at least 97 images per class.
Inverse Reinforcement Learning with Simultaneous Estimation of Rewards and Dynamics
Inverse Reinforcement Learning (IRL) describes the problem of learning an unknown reward function of a Markov Decision Process (MDP) from observed behavior of an agent.
Multimodal Deep Learning for Robust RGB-D Object Recognition
Robust object recognition is a crucial ingredient of many, if not all, real-world robotics applications.
Monte Carlo Localization in Hand-Drawn Maps
In this paper, we address the localization problem when the map of the environment is not present beforehand, and the robot relies on a hand-drawn map from a non-expert user.
Metric Localization using Google Street View
In this paper, we present a novel approach that instead uses geotagged panoramas from the Google Street View as a source of global positioning.
Fast and Robust Feature Matching for RGB-D Based Localization
In this paper we present a novel approach to global localization using an RGB-D camera in maps of visual features.
