Search Results for author:
Found 58 papers, 22 papers with code
EquiBot: SIM(3)-Equivariant Diffusion Policy for Generalizable and Data Efficient Learning
Building effective imitation learning methods that enable robots to learn from limited data and still generalize across diverse real-world environments is a long-standing problem in robot learning.
Consistency Policy: Accelerated Visuomotor Policies via Consistency Distillation
Many robotic systems, such as mobile manipulators or quadrotors, cannot be equipped with high-end GPUs due to space, weight, and power constraints.
Robot Fine-Tuning Made Easy: Pre-Training Rewards and Policies for Autonomous Real-World Reinforcement Learning
We aim to enable this paradigm in robotic reinforcement learning, allowing a robot to learn a new task with little human effort by leveraging data and models from the Internet.
ShaSTA-Fuse: Camera-LiDAR Sensor Fusion to Model Shape and Spatio-Temporal Affinities for 3D Multi-Object Tracking
Our main contributions include a novel fusion approach for combining camera and LiDAR sensory signals to learn affinities, and a first-of-its-kind multimodal sequential track confidence refinement technique that fuses 2D and 3D detections.
KITE: Keypoint-Conditioned Policies for Semantic Manipulation
While natural language offers a convenient shared interface for humans and robots, enabling robots to interpret and follow language commands remains a longstanding challenge in manipulation.
The ObjectFolder Benchmark: Multisensory Learning with Neural and Real Objects
We introduce the ObjectFolder Benchmark, a benchmark suite of 10 tasks for multisensory object-centric learning, centered around object recognition, reconstruction, and manipulation with sight, sound, and touch.
TidyBot: Personalized Robot Assistance with Large Language Models
For a robot to personalize physical assistance effectively, it must learn user preferences that can be generally reapplied to future scenarios.
CARTO: Category and Joint Agnostic Reconstruction of ARTiculated Objects
We present CARTO, a novel approach for reconstructing multiple articulated objects from a single stereo RGB observation.
Development and Evaluation of a Learning-based Model for Real-time Haptic Texture Rendering
Current Virtual Reality (VR) environments lack the rich haptic signals that humans experience during real-life interactions, such as the sensation of texture during lateral movement on a surface.
Active Task Randomization: Learning Robust Skills via Unsupervised Generation of Diverse and Feasible Tasks
ATR selects suitable tasks, which consist of an initial environment state and manipulation goal, for learning robust skills by balancing the diversity and feasibility of the tasks.
ShaSTA: Modeling Shape and Spatio-Temporal Affinities for 3D Multi-Object Tracking
To address these issues in a unified framework, we propose to learn shape and spatio-temporal affinities between tracks and detections in consecutive frames.
Learning Tool Morphology for Contact-Rich Manipulation Tasks with Differentiable Simulation
We demonstrate the effectiveness of our method for designing new tools in several scenarios, such as winding ropes, flipping a box and pushing peas onto a scoop in simulation.
STAP: Sequencing Task-Agnostic Policies
We further demonstrate how STAP can be used for task and motion planning by estimating the geometric feasibility of skill sequences provided by a task planner.
Minkowski Tracker: A Sparse Spatio-Temporal R-CNN for Joint Object Detection and Tracking
In this work, we present Minkowski Tracker, a sparse spatio-temporal R-CNN that jointly solves object detection and tracking.
Rethinking Optimization with Differentiable Simulation from a Global Perspective
In this work, we study the challenges that differentiable simulation presents when it is not feasible to expect that a single descent reaches a global optimum, which is often a problem in contact-rich scenarios.
Visuomotor Control in Multi-Object Scenes Using Object-Aware Representations
Our self-supervised representations are learned by observing the agent freely interacting with different parts of the environment and is queried in two different settings: (i) policy learning and (ii) object location prediction.
Category-Independent Articulated Object Tracking with Factor Graphs
Robots deployed in human-centric environments may need to manipulate a diverse range of articulated objects, such as doors, dishwashers, and cabinets.
DiffCloud: Real-to-Sim from Point Clouds with Differentiable Simulation and Rendering of Deformable Objects
However, for highly deformable objects it is challenging to align the output of a simulator with the behavior of real objects.
ObjectFolder 2.0: A Multisensory Object Dataset for Sim2Real Transfer
We present ObjectFolder 2. 0, a large-scale, multisensory dataset of common household objects in the form of implicit neural representations that significantly enhances ObjectFolder 1. 0 in three aspects.
A Bayesian Treatment of Real-to-Sim for Deformable Object Manipulation
Deformable object manipulation remains a challenging task in robotics research.
From Machine Learning to Robotics: Challenges and Opportunities for Embodied Intelligence
Machine learning has long since become a keystone technology, accelerating science and applications in a broad range of domains.
On the Opportunities and Risks of Foundation Models
AI is undergoing a paradigm shift with the rise of models (e. g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks.
XIRL: Cross-embodiment Inverse Reinforcement Learning
We investigate the visual cross-embodiment imitation setting, in which agents learn policies from videos of other agents (such as humans) demonstrating the same task, but with stark differences in their embodiments -- shape, actions, end-effector dynamics, etc.
OmniHang: Learning to Hang Arbitrary Objects using Contact Point Correspondences and Neural Collision Estimation
In this paper, we propose a system that takes partial point clouds of an object and a supporting item as input and learns to decide where and how to hang the object stably.
How to Train Your Differentiable Filter
In many robotic applications, it is crucial to maintain a belief about the state of a system, which serves as input for planning and decision making and provides feedback during task execution.
Probabilistic 3D Multi-Modal, Multi-Object Tracking for Autonomous Driving
Second, we propose to learn a metric that combines the Mahalanobis and feature distances when comparing a track and a new detection in data association.
Detect, Reject, Correct: Crossmodal Compensation of Corrupted Sensors
Using sensor data from multiple modalities presents an opportunity to encode redundant and complementary features that can be useful when one modality is corrupted or noisy.
GRAC: Self-Guided and Self-Regularized Actor-Critic
One dominant component of recent deep reinforcement learning algorithms is the target network which mitigates the divergence when learning the Q function.
Learning User-Preferred Mappings for Intuitive Robot Control
Existing approaches to teleoperation typically assume a one-size-fits-all approach, where the designers pre-define a mapping between human inputs and robot actions, and every user must adapt to this mapping over repeated interactions.
Dynamic Multi-Robot Task Allocation under Uncertainty and Temporal Constraints
We consider the problem of dynamically allocating tasks to multiple agents under time window constraints and task completion uncertainty.
Probabilistic 3D Multi-Object Tracking for Autonomous Driving
Our method estimates the object states by adopting a Kalman Filter.
Self-Supervised Learning of State Estimation for Manipulating Deformable Linear Objects
Challenges in taking the state-space approach are the estimation of the high-dimensional state of a deformable object from raw images, where annotations are very expensive on real data, and finding a dynamics model that is both accurate, generalizable, and efficient to compute.
Object-Centric Task and Motion Planning in Dynamic Environments
We address the problem of applying Task and Motion Planning (TAMP) in real world environments.
Accurate Vision-based Manipulation through Contact Reasoning
Planning contact interactions is one of the core challenges of many robotic tasks.
Learning to Scaffold the Development of Robotic Manipulation Skills
To combat these factors and achieve more robust manipulation, humans actively exploit contact constraints in the environment.
Learning Task-Oriented Grasping from Human Activity Datasets
We develop a model that takes as input an RGB image and outputs a hand pose and configuration as well as an object pose and a shape.
Learning Hierarchical Control for Robust In-Hand Manipulation
Finally, we show how our approach generalizes to objects of other shapes.
UniGrasp: Learning a Unified Model to Grasp with Multifingered Robotic Hands
The majority of previous work has focused on developing grasp methods that generalize over novel object geometry but are specific to a certain robot hand.
MeteorNet: Deep Learning on Dynamic 3D Point Cloud Sequences
Understanding dynamic 3D environment is crucial for robotic agents and many other applications.
Learning from My Partner's Actions: Roles in Decentralized Robot Teams
When teams of robots collaborate to complete a task, communication is often necessary.
Learning an Action-Conditional Model for Haptic Texture Generation
Therefore, it is challenging to model the mapping from material and user interactions to haptic feedback in a way that generalizes over many variations of the user's input.
Learning Visual Dynamics Models of Rigid Objects using Relational Inductive Biases
The first, Graph Networks (GN) based approach, considers explicitly defined edge attributes and not only does it consistently underperform an auto-encoder baseline that we modified to predict future states, our results indicate how different edge attributes can significantly influence the predictions.
Making Sense of Vision and Touch: Learning Multimodal Representations for Contact-Rich Tasks
Contact-rich manipulation tasks in unstructured environments often require both haptic and visual feedback.
Variable Impedance Control in End-Effector Space: An Action Space for Reinforcement Learning in Contact-Rich Tasks
This paper studies the effect of different action spaces in deep RL and advocates for Variable Impedance Control in End-effector Space (VICES) as an advantageous action space for constrained and contact-rich tasks.
On Learning Heteroscedastic Noise Models within Differentiable Bayes Filters
Recursive Bayesian Filtering algorithms address the state estimation problem, but they require a model of the process dynamics and the sensory observations as well as noise estimates that quantify the accuracy of these models.
Learning to Estimate Pose and Shape of Hand-Held Objects from RGB Images
The qualitative experiments show results of pose and shape estimation of objects held by a hand "in the wild".
Making Sense of Vision and Touch: Self-Supervised Learning of Multimodal Representations for Contact-Rich Tasks
Contact-rich manipulation tasks in unstructured environments often require both haptic and visual feedback.
Leveraging Contact Forces for Learning to Grasp
While it is possible to learn grasping policies without contact sensing, our results suggest that contact feedback allows for a significant improvement of grasping robustness under object pose uncertainty and for objects with a complex shape.
ClusterNet: 3D Instance Segmentation in RGB-D Images
We show that our method generalizes well on real-world data achieving visually better segmentation results.
Motion-based Object Segmentation based on Dense RGB-D Scene Flow
Our model jointly estimates (i) the segmentation of the scene into an unknown but finite number of objects, (ii) the motion trajectories of these objects and (iii) the object scene flow.
Acquiring Target Stacking Skills by Goal-Parameterized Deep Reinforcement Learning
We created a synthetic block stacking environment with physics simulation in which the agent can learn a policy end-to-end through trial and error.
Combining Learned and Analytical Models for Predicting Action Effects from Sensory Data
In this work, we investigate the advantages and limitations of neural network based learning approaches for predicting the effects of actions based on sensory input and show how analytical and learned models can be combined to leverage the best of both worlds.
Latest Datasets and Technologies Presented in the Workshop on Grasping and Manipulation Datasets
This paper reports the activities and outcomes in the Workshop on Grasping and Manipulation Datasets that was organized under the International Conference on Robotics and Automation (ICRA) 2016.
Automatic LQR Tuning Based on Gaussian Process Global Optimization
With this framework, an initial set of controller gains is automatically improved according to a pre-defined performance objective evaluated from experimental data.
Interactive Perception: Leveraging Action in Perception and Perception in Action
Recent approaches in robotics follow the insight that perception is facilitated by interaction with the environment.
Robotics
Depth-Based Object Tracking Using a Robust Gaussian Filter
To address this issue, we show how a recently published robustification method for Gaussian filters can be applied to the problem at hand.
Robust Gaussian Filtering using a Pseudo Measurement
The contribution of this paper is to show that any Gaussian filter can be made compatible with fat-tailed sensor models by applying one simple change: Instead of filtering with the physical measurement, we propose to filter with a pseudo measurement obtained by applying a feature function to the physical measurement.
Data-Driven Grasp Synthesis - A Survey
In the case of known objects, we concentrate on the approaches that are based on object recognition and pose estimation.
Robotics
