Search Results for author:
Found 61 papers, 27 papers with code
Deep Learning for Single-View Instance Recognition
We show that feedforward neural networks outperform state-of-the-art methods for recognizing objects from novel viewpoints even when trained from just a single image per object.
Learning to Track at 100 FPS with Deep Regression Networks
We propose a method for offline training of neural networks that can track novel objects at test-time at 100 fps.
Enabling Robots to Communicate their Objectives
We show that certain approximate-inference models lead to the robot generating example behaviors that better enable users to anticipate what it will do in novel situations.
Probabilistically Safe Policy Transfer
Although learning-based methods have great potential for robotics, one concern is that a robot that updates its parameters might cause large amounts of damage before it learns the optimal policy.
Automatic Goal Generation for Reinforcement Learning Agents
Instead, we propose a method that allows an agent to automatically discover the range of tasks that it is capable of performing.
Constrained Policy Optimization
For many applications of reinforcement learning it can be more convenient to specify both a reward function and constraints, rather than trying to design behavior through the reward function.
Reverse Curriculum Generation for Reinforcement Learning
The robot is trained in reverse, gradually learning to reach the goal from a set of start states increasingly far from the goal.
PCN: Point Completion Network
Shape completion, the problem of estimating the complete geometry of objects from partial observations, lies at the core of many vision and robotics applications.
Ranked #6 on
Point Cloud Completion
on Completion3D
Iterative Transformer Network for 3D Point Cloud
Recently, neural networks operating on point clouds have shown superior performance on 3D understanding tasks such as shape classification and part segmentation.
Adaptive Variance for Changing Sparse-Reward Environments
Robots that are trained to perform a task in a fixed environment often fail when facing unexpected changes to the environment due to a lack of exploration.
Reinforcement Learning without Ground-Truth State
However, if this policy is trained with reinforcement learning, then without a state estimator, it is hard to specify a reward function based on high-dimensional observations.
Few-Shot Point Cloud Region Annotation with Human in the Loop
We propose a point cloud annotation framework that employs human-in-loop learning to enable the creation of large point cloud datasets with per-point annotations.
3D Multi-Object Tracking: A Baseline and New Evaluation Metrics
Additionally, 3D MOT datasets such as KITTI evaluate MOT methods in the 2D space and standardized 3D MOT evaluation tools are missing for a fair comparison of 3D MOT methods.
Ranked #3 on
3D Multi-Object Tracking
on KITTI
FusionMapping: Learning Depth Prediction with Monocular Images and 2D Laser Scans
In this paper, we introduce a fusion-based depth prediction method, called FusionMapping.
Just Go with the Flow: Self-Supervised Scene Flow Estimation
When interacting with highly dynamic environments, scene flow allows autonomous systems to reason about the non-rigid motion of multiple independent objects.
Adaptive Auxiliary Task Weighting for Reinforcement Learning
Reinforcement learning is known to be sample inefficient, preventing its application to many real-world problems, especially with high dimensional observations like images.
What You See is What You Get: Exploiting Visibility for 3D Object Detection
On the NuScenes 3D detection benchmark, we show that, by adding an additional stream for visibility input, we can significantly improve the overall detection accuracy of a state-of-the-art 3D detector.
Learning to Optimally Segment Point Clouds
We prove that if we score a segmentation by the worst objectness among its individual segments, there is an efficient algorithm that finds the optimal worst-case segmentation among an exponentially large number of candidate segmentations.
Combining Deep Learning and Verification for Precise Object Instance Detection
For a reliable detection system, if a high confidence detection is made, we would want high certainty that the object has indeed been detected.
Multi-modal Transfer Learning for Grasping Transparent and Specular Objects
State-of-the-art object grasping methods rely on depth sensing to plan robust grasps, but commercially available depth sensors fail to detect transparent and specular objects.
Robotics
Learning Orientation Distributions for Object Pose Estimation
Our first method, which regresses from deep learned features to an isotropic Bingham distribution, gives the best performance for orientation distribution estimation for non-symmetric objects.
Active Perception using Light Curtains for Autonomous Driving
Most real-world 3D sensors such as LiDARs perform fixed scans of the entire environment, while being decoupled from the recognition system that processes the sensor data.
Cloth Region Segmentation for Robust Grasp Selection
Our approach trains a network to segment the edges and corners of a cloth from a depth image, distinguishing such regions from wrinkles or folds.
Robotics
AB3DMOT: A Baseline for 3D Multi-Object Tracking and New Evaluation Metrics
Additionally, 3D MOT datasets such as KITTI evaluate MOT methods in 2D space and standardized 3D MOT evaluation tools are missing for a fair comparison of 3D MOT methods.
Uncertainty-aware Self-supervised 3D Data Association
Instead, we propose leveraging vast unlabeled datasets by self-supervised metric learning of 3D object trackers, with a focus on data association.
Learning Off-Policy with Online Planning
In this work, we investigate a novel instantiation of H-step lookahead with a learned model and a terminal value function learned by a model-free off-policy algorithm, named Learning Off-Policy with Online Planning (LOOP).
Robust Instance Tracking via Uncertainty Flow
Current state-of-the-art trackers often fail due to distractorsand large object appearance changes.
ROLL: Visual Self-Supervised Reinforcement Learning with Object Reasoning
Current image-based reinforcement learning (RL) algorithms typically operate on the whole image without performing object-level reasoning.
SoftGym: Benchmarking Deep Reinforcement Learning for Deformable Object Manipulation
Further, we evaluate a variety of algorithms on these tasks and highlight challenges for reinforcement learning algorithms, including dealing with a state representation that has a high intrinsic dimensionality and is partially observable.
PLAS: Latent Action Space for Offline Reinforcement Learning
The goal of offline reinforcement learning is to learn a policy from a fixed dataset, without further interactions with the environment.
PanoNet3D: Combining Semantic and Geometric Understanding for LiDARPoint Cloud Detection
Visual data in autonomous driving perception, such as camera image and LiDAR point cloud, can be interpreted as a mixture of two aspects: semantic feature and geometric structure.
Lyapunov Barrier Policy Optimization
Current RL agents explore the environment without considering these constraints, which can lead to damage to the hardware or even other agents in the environment.
ZePHyR: Zero-shot Pose Hypothesis Rating
We also demonstrate how our system can be used by quickly scanning and building a model of a novel object, which can immediately be used by our method for pose estimation.
Learning Visible Connectivity Dynamics for Cloth Smoothing
Robotic manipulation of cloth remains challenging for robotics due to the complex dynamics of the cloth, lack of a low-dimensional state representation, and self-occlusions.
Exploiting & Refining Depth Distributions With Triangulation Light Curtains
Active sensing through the use of Adaptive Depth Sensors is a nascent field, with potential in areas such as Advanced driver-assistance systems (ADAS).
Safe Local Motion Planning With Self-Supervised Freespace Forecasting
Finally, we propose future freespace as an additional source of annotation-free supervision.
Active Safety Envelopes using Light Curtains with Probabilistic Guarantees
We use light curtains to estimate the safety envelope of a scene: a hypothetical surface that separates the robot from all obstacles.
FabricFlowNet: Bimanual Cloth Manipulation with a Flow-based Policy
We introduce FabricFlowNet (FFN), a cloth manipulation policy that leverages flow as both an input and as an action representation to improve performance.
Self-Supervised Point Cloud Completion via Inpainting
The aim of this work is to learn to complete these partial point clouds, giving us a full understanding of the object's geometry using only partial observations.
OSSID: Online Self-Supervised Instance Detection by (and for) Pose Estimation
In this paper, we propose the OSSID framework, leveraging a slow zero-shot pose estimator to self-supervise the training of a fast detection algorithm.
Semi-supervised 3D Object Detection via Temporal Graph Neural Networks
However, these detectors usually require training on large amounts of annotated data that is expensive and time-consuming to collect.
Self-supervised Transparent Liquid Segmentation for Robotic Pouring
Liquid state estimation is important for robotics tasks such as pouring; however, estimating the state of transparent liquids is a challenging problem.
DiffSkill: Skill Abstraction from Differentiable Physics for Deformable Object Manipulations with Tools
We consider the problem of sequential robotic manipulation of deformable objects using tools.
FlowBot3D: Learning 3D Articulation Flow to Manipulate Articulated Objects
We propose a vision-based system that learns to predict the potential motions of the parts of a variety of articulated objects to guide downstream motion planning of the system to articulate the objects.
Mesh-based Dynamics with Occlusion Reasoning for Cloth Manipulation
We evaluate our system both on cloth flattening as well as on cloth canonicalization, in which the objective is to manipulate the cloth into a canonical pose.
Visual Haptic Reasoning: Estimating Contact Forces by Observing Deformable Object Interactions
Robotic manipulation of highly deformable cloth presents a promising opportunity to assist people with several daily tasks, such as washing dishes; folding laundry; or dressing, bathing, and hygiene assistance for individuals with severe motor impairments.
EDO-Net: Learning Elastic Properties of Deformable Objects from Graph Dynamics
We study the problem of learning graph dynamics of deformable objects that generalizes to unknown physical properties.
Differentiable Raycasting for Self-supervised Occupancy Forecasting
Self-supervised representations proposed for large-scale planning, such as ego-centric freespace, confound these two motions, making the representation difficult to use for downstream motion planners.
Planning with Spatial-Temporal Abstraction from Point Clouds for Deformable Object Manipulation
Effective planning of long-horizon deformable object manipulation requires suitable abstractions at both the spatial and temporal levels.
Deep Projective Rotation Estimation through Relative Supervision
While self-supervised learning has been used successfully for translational object keypoints, in this work, we show that naively applying relative supervision to the rotational group $SO(3)$ will often fail to converge due to the non-convexity of the rotational space.
Self-supervised Cloth Reconstruction via Action-conditioned Cloth Tracking
In this work, we propose a self-supervised method to finetune a mesh reconstruction model in the real world.
Active Velocity Estimation using Light Curtains via Self-Supervised Multi-Armed Bandits
To navigate in an environment safely and autonomously, robots must accurately estimate where obstacles are and how they move.
Point Cloud Forecasting as a Proxy for 4D Occupancy Forecasting
One promising self-supervised task is 3D point cloud forecasting from unannotated LiDAR sequences.
HACMan: Learning Hybrid Actor-Critic Maps for 6D Non-Prehensile Manipulation
In this work, we introduce Hybrid Actor-Critic Maps for Manipulation (HACMan), a reinforcement learning approach for 6D non-prehensile manipulation of objects using point cloud observations.
Learning Generalizable Tool-use Skills through Trajectory Generation
We propose to learn a generative model of the tool-use trajectories as a sequence of tool point clouds, which generalizes to different tool shapes.
Reinforcement Learning in a Safety-Embedded MDP with Trajectory Optimization
This work introduces a novel approach that combines RL with trajectory optimization to manage this trade-off effectively.
RoboGen: Towards Unleashing Infinite Data for Automated Robot Learning via Generative Simulation
We present RoboGen, a generative robotic agent that automatically learns diverse robotic skills at scale via generative simulation.
On Time-Indexing as Inductive Bias in Deep RL for Sequential Manipulation Tasks
In this paper we explore a simple structure which is conducive to skill learning required for so many of the manipulation tasks.
RL-VLM-F: Reinforcement Learning from Vision Language Foundation Model Feedback
Reward engineering has long been a challenge in Reinforcement Learning (RL) research, as it often requires extensive human effort and iterative processes of trial-and-error to design effective reward functions.
DiffTOP: Differentiable Trajectory Optimization for Deep Reinforcement and Imitation Learning
The key to our approach is to leverage the recent progress in differentiable trajectory optimization, which enables computing the gradients of the loss with respect to the parameters of trajectory optimization.
