Search Results for author:
Found 54 papers, 26 papers with code
Point Cloud Forecasting as a Proxy for 4D Occupancy Forecasting
One promising self-supervised task is 3D point cloud forecasting from unannotated LiDAR sequences.
Active Velocity Estimation using Light Curtains via Self-Supervised Multi-Armed Bandits
To navigate in an environment safely and autonomously, robots must accurately estimate where obstacles are and how they move.
Self-supervised Cloth Reconstruction via Action-conditioned Cloth Tracking
In this work, we propose a self-supervised method to finetune a mesh reconstruction model in the real world.
Deep Projective Rotation Estimation through Relative Supervision
While self-supervised learning has been used successfully for translational object keypoints, in this work, we show that naively applying relative supervision to the rotational group $SO(3)$ will often fail to converge due to the non-convexity of the rotational space.
Learning to Grasp the Ungraspable with Emergent Extrinsic Dexterity
Previous work in extrinsic dexterity usually has careful assumptions about contacts which impose restrictions on robot design, robot motions, and the variations of the physical parameters.
Planning with Spatial-Temporal Abstraction from Point Clouds for Deformable Object Manipulation
Effective planning of long-horizon deformable object manipulation requires suitable abstractions at both the spatial and temporal levels.
Differentiable Raycasting for Self-supervised Occupancy Forecasting
Self-supervised representations proposed for large-scale planning, such as ego-centric freespace, confound these two motions, making the representation difficult to use for downstream motion planners.
EDO-Net: Learning Elastic Properties of Deformable Objects from Graph Dynamics
In particular, we leverage a latent representation of elastic physical properties of cloth-like deformable objects which we explore through a pulling interaction.
Visual Haptic Reasoning: Estimating Contact Forces by Observing Deformable Object Interactions
Robotic manipulation of highly deformable cloth presents a promising opportunity to assist people with several daily tasks, such as washing dishes; folding laundry; or dressing, bathing, and hygiene assistance for individuals with severe motor impairments.
Mesh-based Dynamics with Occlusion Reasoning for Cloth Manipulation
We evaluate our system both on cloth flattening as well as on cloth canonicalization, in which the objective is to manipulate the cloth into a canonical pose.
FlowBot3D: Learning 3D Articulation Flow to Manipulate Articulated Objects
We propose a vision-based system that learns to predict the potential motions of the parts of a variety of articulated objects to guide downstream motion planning of the system to articulate the objects.
DiffSkill: Skill Abstraction from Differentiable Physics for Deformable Object Manipulations with Tools
We consider the problem of sequential robotic manipulation of deformable objects using tools.
Self-supervised Transparent Liquid Segmentation for Robotic Pouring
Liquid state estimation is important for robotics tasks such as pouring; however, estimating the state of transparent liquids is a challenging problem.
Semi-supervised 3D Object Detection via Temporal Graph Neural Networks
However, these detectors usually require training on large amounts of annotated data that is expensive and time-consuming to collect.
OSSID: Online Self-Supervised Instance Detection by (and for) Pose Estimation
In this paper, we propose the OSSID framework, leveraging a slow zero-shot pose estimator to self-supervise the training of a fast detection algorithm.
Self-Supervised Point Cloud Completion via Inpainting
The aim of this work is to learn to complete these partial point clouds, giving us a full understanding of the object's geometry using only partial observations.
FabricFlowNet: Bimanual Cloth Manipulation with a Flow-based Policy
We introduce FabricFlowNet (FFN), a cloth manipulation policy that leverages flow as both an input and as an action representation to improve performance.
Active Safety Envelopes using Light Curtains with Probabilistic Guarantees
We use light curtains to estimate the safety envelope of a scene: a hypothetical surface that separates the robot from all obstacles.
Exploiting & Refining Depth Distributions With Triangulation Light Curtains
Active sensing through the use of Adaptive Depth Sensors is a nascent field, with potential in areas such as Advanced driver-assistance systems (ADAS).
Safe Local Motion Planning With Self-Supervised Freespace Forecasting
Finally, we propose future freespace as an additional source of annotation-free supervision.
Learning Visible Connectivity Dynamics for Cloth Smoothing
Robotic manipulation of cloth remains challenging for robotics due to the complex dynamics of the cloth, lack of a low-dimensional state representation, and self-occlusions.
ZePHyR: Zero-shot Pose Hypothesis Rating
We also demonstrate how our system can be used by quickly scanning and building a model of a novel object, which can immediately be used by our method for pose estimation.
Lyapunov Barrier Policy Optimization
Current RL agents explore the environment without considering these constraints, which can lead to damage to the hardware or even other agents in the environment.
PanoNet3D: Combining Semantic and Geometric Understanding for LiDARPoint Cloud Detection
Visual data in autonomous driving perception, such as camera image and LiDAR point cloud, can be interpreted as a mixture of two aspects: semantic feature and geometric structure.
PLAS: Latent Action Space for Offline Reinforcement Learning
The goal of offline reinforcement learning is to learn a policy from a fixed dataset, without further interactions with the environment.
SoftGym: Benchmarking Deep Reinforcement Learning for Deformable Object Manipulation
Further, we evaluate a variety of algorithms on these tasks and highlight challenges for reinforcement learning algorithms, including dealing with a state representation that has a high intrinsic dimensionality and is partially observable.
ROLL: Visual Self-Supervised Reinforcement Learning with Object Reasoning
Current image-based reinforcement learning (RL) algorithms typically operate on the whole image without performing object-level reasoning.
Robust Instance Tracking via Uncertainty Flow
Current state-of-the-art trackers often fail due to distractorsand large object appearance changes.
Learning Off-Policy with Online Planning
In this work, we investigate a novel instantiation of H-step lookahead with a learned model and a terminal value function learned by a model-free off-policy algorithm, named Learning Off-Policy with Online Planning (LOOP).
AB3DMOT: A Baseline for 3D Multi-Object Tracking and New Evaluation Metrics
Additionally, 3D MOT datasets such as KITTI evaluate MOT methods in 2D space and standardized 3D MOT evaluation tools are missing for a fair comparison of 3D MOT methods.
Uncertainty-aware Self-supervised 3D Data Association
Instead, we propose leveraging vast unlabeled datasets by self-supervised metric learning of 3D object trackers, with a focus on data association.
Cloth Region Segmentation for Robust Grasp Selection
Our approach trains a network to segment the edges and corners of a cloth from a depth image, distinguishing such regions from wrinkles or folds.
Robotics
Active Perception using Light Curtains for Autonomous Driving
Most real-world 3D sensors such as LiDARs perform fixed scans of the entire environment, while being decoupled from the recognition system that processes the sensor data.
Learning Orientation Distributions for Object Pose Estimation
Our first method, which regresses from deep learned features to an isotropic Bingham distribution, gives the best performance for orientation distribution estimation for non-symmetric objects.
Multi-modal Transfer Learning for Grasping Transparent and Specular Objects
State-of-the-art object grasping methods rely on depth sensing to plan robust grasps, but commercially available depth sensors fail to detect transparent and specular objects.
Robotics
Combining Deep Learning and Verification for Precise Object Instance Detection
For a reliable detection system, if a high confidence detection is made, we would want high certainty that the object has indeed been detected.
Learning to Optimally Segment Point Clouds
We prove that if we score a segmentation by the worst objectness among its individual segments, there is an efficient algorithm that finds the optimal worst-case segmentation among an exponentially large number of candidate segmentations.
What You See is What You Get: Exploiting Visibility for 3D Object Detection
On the NuScenes 3D detection benchmark, we show that, by adding an additional stream for visibility input, we can significantly improve the overall detection accuracy of a state-of-the-art 3D detector.
Adaptive Auxiliary Task Weighting for Reinforcement Learning
Reinforcement learning is known to be sample inefficient, preventing its application to many real-world problems, especially with high dimensional observations like images.
Just Go with the Flow: Self-Supervised Scene Flow Estimation
When interacting with highly dynamic environments, scene flow allows autonomous systems to reason about the non-rigid motion of multiple independent objects.
FusionMapping: Learning Depth Prediction with Monocular Images and 2D Laser Scans
In this paper, we introduce a fusion-based depth prediction method, called FusionMapping.
3D Multi-Object Tracking: A Baseline and New Evaluation Metrics
Additionally, 3D MOT datasets such as KITTI evaluate MOT methods in the 2D space and standardized 3D MOT evaluation tools are missing for a fair comparison of 3D MOT methods.
Ranked #3 on
3D Multi-Object Tracking
on KITTI
Few-Shot Point Cloud Region Annotation with Human in the Loop
We propose a point cloud annotation framework that employs human-in-loop learning to enable the creation of large point cloud datasets with per-point annotations.
Reinforcement Learning without Ground-Truth State
However, if this policy is trained with reinforcement learning, then without a state estimator, it is hard to specify a reward function based on high-dimensional observations.
Adaptive Variance for Changing Sparse-Reward Environments
Robots that are trained to perform a task in a fixed environment often fail when facing unexpected changes to the environment due to a lack of exploration.
Iterative Transformer Network for 3D Point Cloud
Recently, neural networks operating on point clouds have shown superior performance on 3D understanding tasks such as shape classification and part segmentation.
PCN: Point Completion Network
Shape completion, the problem of estimating the complete geometry of objects from partial observations, lies at the core of many vision and robotics applications.
Ranked #4 on
Point Cloud Completion
on Completion3D
Reverse Curriculum Generation for Reinforcement Learning
The robot is trained in reverse, gradually learning to reach the goal from a set of start states increasingly far from the goal.
Constrained Policy Optimization
For many applications of reinforcement learning it can be more convenient to specify both a reward function and constraints, rather than trying to design behavior through the reward function.
Automatic Goal Generation for Reinforcement Learning Agents
Instead, we propose a method that allows an agent to automatically discover the range of tasks that it is capable of performing.
Probabilistically Safe Policy Transfer
Although learning-based methods have great potential for robotics, one concern is that a robot that updates its parameters might cause large amounts of damage before it learns the optimal policy.
Enabling Robots to Communicate their Objectives
We show that certain approximate-inference models lead to the robot generating example behaviors that better enable users to anticipate what it will do in novel situations.
Learning to Track at 100 FPS with Deep Regression Networks
We propose a method for offline training of neural networks that can track novel objects at test-time at 100 fps.
Deep Learning for Single-View Instance Recognition
We show that feedforward neural networks outperform state-of-the-art methods for recognizing objects from novel viewpoints even when trained from just a single image per object.
