Search Results for author: David Held

Found 54 papers, 26 papers with code

Point Cloud Forecasting as a Proxy for 4D Occupancy Forecasting

1 code implementation25 Feb 2023 Tarasha Khurana, Peiyun Hu, David Held, Deva Ramanan

One promising self-supervised task is 3D point cloud forecasting from unannotated LiDAR sequences.

Motion Planning

Active Velocity Estimation using Light Curtains via Self-Supervised Multi-Armed Bandits

no code implementations24 Feb 2023 Siddharth Ancha, Gaurav Pathak, Ji Zhang, Srinivasa Narasimhan, David Held

To navigate in an environment safely and autonomously, robots must accurately estimate where obstacles are and how they move.

Multi-Armed Bandits Navigate

Self-supervised Cloth Reconstruction via Action-conditioned Cloth Tracking

no code implementations19 Feb 2023 Zixuan Huang, Xingyu Lin, David Held

In this work, we propose a self-supervised method to finetune a mesh reconstruction model in the real world.

Self-Supervised Learning

Deep Projective Rotation Estimation through Relative Supervision

no code implementations21 Nov 2022 Brian Okorn, Chuer Pan, Martial Hebert, David Held

While self-supervised learning has been used successfully for translational object keypoints, in this work, we show that naively applying relative supervision to the rotational group $SO(3)$ will often fail to converge due to the non-convexity of the rotational space.

Pose Estimation Self-Supervised Learning

Learning to Grasp the Ungraspable with Emergent Extrinsic Dexterity

no code implementations2 Nov 2022 Wenxuan Zhou, David Held

Previous work in extrinsic dexterity usually has careful assumptions about contacts which impose restrictions on robot design, robot motions, and the variations of the physical parameters.

Friction Reinforcement Learning (RL)

Planning with Spatial-Temporal Abstraction from Point Clouds for Deformable Object Manipulation

no code implementations27 Oct 2022 Xingyu Lin, Carl Qi, Yunchu Zhang, Zhiao Huang, Katerina Fragkiadaki, Yunzhu Li, Chuang Gan, David Held

Effective planning of long-horizon deformable object manipulation requires suitable abstractions at both the spatial and temporal levels.

Deformable Object Manipulation

Differentiable Raycasting for Self-supervised Occupancy Forecasting

1 code implementation4 Oct 2022 Tarasha Khurana, Peiyun Hu, Achal Dave, Jason Ziglar, David Held, Deva Ramanan

Self-supervised representations proposed for large-scale planning, such as ego-centric freespace, confound these two motions, making the representation difficult to use for downstream motion planners.

Autonomous Driving Motion Planning +1

EDO-Net: Learning Elastic Properties of Deformable Objects from Graph Dynamics

no code implementations19 Sep 2022 Alberta Longhini, Marco Moletta, Alfredo Reichlin, Michael C. Welle, David Held, Zackory Erickson, Danica Kragic

In particular, we leverage a latent representation of elastic physical properties of cloth-like deformable objects which we explore through a pulling interaction.

Visual Haptic Reasoning: Estimating Contact Forces by Observing Deformable Object Interactions

no code implementations11 Aug 2022 YuFei Wang, David Held, Zackory Erickson

Robotic manipulation of highly deformable cloth presents a promising opportunity to assist people with several daily tasks, such as washing dishes; folding laundry; or dressing, bathing, and hygiene assistance for individuals with severe motor impairments.

Mesh-based Dynamics with Occlusion Reasoning for Cloth Manipulation

no code implementations6 Jun 2022 Zixuan Huang, Xingyu Lin, David Held

We evaluate our system both on cloth flattening as well as on cloth canonicalization, in which the objective is to manipulate the cloth into a canonical pose.

Pose Estimation

FlowBot3D: Learning 3D Articulation Flow to Manipulate Articulated Objects

no code implementations9 May 2022 Ben Eisner, Harry Zhang, David Held

We propose a vision-based system that learns to predict the potential motions of the parts of a variety of articulated objects to guide downstream motion planning of the system to articulate the objects.

Motion Planning

Self-supervised Transparent Liquid Segmentation for Robotic Pouring

no code implementations3 Mar 2022 Gautham Narayan Narasimhan, Kai Zhang, Ben Eisner, Xingyu Lin, David Held

Liquid state estimation is important for robotics tasks such as pouring; however, estimating the state of transparent liquids is a challenging problem.

Semi-supervised 3D Object Detection via Temporal Graph Neural Networks

no code implementations1 Feb 2022 Jianren Wang, Haiming Gang, Siddharth Ancha, Yi-Ting Chen, David Held

However, these detectors usually require training on large amounts of annotated data that is expensive and time-consuming to collect.

3D Object Detection Autonomous Driving +1

OSSID: Online Self-Supervised Instance Detection by (and for) Pose Estimation

no code implementations18 Jan 2022 Qiao Gu, Brian Okorn, David Held

In this paper, we propose the OSSID framework, leveraging a slow zero-shot pose estimator to self-supervise the training of a fast detection algorithm.

Pose Estimation Robot Manipulation

Self-Supervised Point Cloud Completion via Inpainting

1 code implementation21 Nov 2021 Himangi Mittal, Brian Okorn, Arpit Jangid, David Held

The aim of this work is to learn to complete these partial point clouds, giving us a full understanding of the object's geometry using only partial observations.

Point Cloud Completion

FabricFlowNet: Bimanual Cloth Manipulation with a Flow-based Policy

1 code implementation10 Nov 2021 Thomas Weng, Sujay Bajracharya, YuFei Wang, Khush Agrawal, David Held

We introduce FabricFlowNet (FFN), a cloth manipulation policy that leverages flow as both an input and as an action representation to improve performance.

Motion Estimation Optical Flow Estimation

Active Safety Envelopes using Light Curtains with Probabilistic Guarantees

no code implementations8 Jul 2021 Siddharth Ancha, Gaurav Pathak, Srinivasa G. Narasimhan, David Held

We use light curtains to estimate the safety envelope of a scene: a hypothetical surface that separates the robot from all obstacles.


Exploiting & Refining Depth Distributions With Triangulation Light Curtains

no code implementations CVPR 2021 Yaadhav Raaj, Siddharth Ancha, Robert Tamburo, David Held, Srinivasa G. Narasimhan

Active sensing through the use of Adaptive Depth Sensors is a nascent field, with potential in areas such as Advanced driver-assistance systems (ADAS).

Learning Visible Connectivity Dynamics for Cloth Smoothing

1 code implementation21 May 2021 Xingyu Lin, YuFei Wang, Zixuan Huang, David Held

Robotic manipulation of cloth remains challenging for robotics due to the complex dynamics of the cloth, lack of a low-dimensional state representation, and self-occlusions.

Deformable Object Manipulation Inductive Bias

ZePHyR: Zero-shot Pose Hypothesis Rating

1 code implementation28 Apr 2021 Brian Okorn, Qiao Gu, Martial Hebert, David Held

We also demonstrate how our system can be used by quickly scanning and building a model of a novel object, which can immediately be used by our method for pose estimation.

Motion Planning Pose Estimation +1

Lyapunov Barrier Policy Optimization

1 code implementation16 Mar 2021 Harshit Sikchi, Wenxuan Zhou, David Held

Current RL agents explore the environment without considering these constraints, which can lead to damage to the hardware or even other agents in the environment.

Reinforcement Learning (RL)

PanoNet3D: Combining Semantic and Geometric Understanding for LiDARPoint Cloud Detection

no code implementations17 Dec 2020 Xia Chen, Jianren Wang, David Held, Martial Hebert

Visual data in autonomous driving perception, such as camera image and LiDAR point cloud, can be interpreted as a mixture of two aspects: semantic feature and geometric structure.

Autonomous Driving Cloud Detection

PLAS: Latent Action Space for Offline Reinforcement Learning

2 code implementations14 Nov 2020 Wenxuan Zhou, Sujay Bajracharya, David Held

The goal of offline reinforcement learning is to learn a policy from a fixed dataset, without further interactions with the environment.

Continuous Control Deformable Object Manipulation +2

SoftGym: Benchmarking Deep Reinforcement Learning for Deformable Object Manipulation

2 code implementations14 Nov 2020 Xingyu Lin, YuFei Wang, Jake Olkin, David Held

Further, we evaluate a variety of algorithms on these tasks and highlight challenges for reinforcement learning algorithms, including dealing with a state representation that has a high intrinsic dimensionality and is partially observable.

Benchmarking Deformable Object Manipulation +3

ROLL: Visual Self-Supervised Reinforcement Learning with Object Reasoning

1 code implementation13 Nov 2020 YuFei Wang, Gautham Narayan Narasimhan, Xingyu Lin, Brian Okorn, David Held

Current image-based reinforcement learning (RL) algorithms typically operate on the whole image without performing object-level reasoning.

Multi-Goal Reinforcement Learning reinforcement-learning +1

Robust Instance Tracking via Uncertainty Flow

no code implementations9 Oct 2020 Jianing Qian, Junyu Nan, Siddharth Ancha, Brian Okorn, David Held

Current state-of-the-art trackers often fail due to distractorsand large object appearance changes.

Optical Flow Estimation

Learning Off-Policy with Online Planning

1 code implementation23 Aug 2020 Harshit Sikchi, Wenxuan Zhou, David Held

In this work, we investigate a novel instantiation of H-step lookahead with a learned model and a terminal value function learned by a model-free off-policy algorithm, named Learning Off-Policy with Online Planning (LOOP).

Continuous Control reinforcement-learning +1

AB3DMOT: A Baseline for 3D Multi-Object Tracking and New Evaluation Metrics

no code implementations18 Aug 2020 Xinshuo Weng, Jianren Wang, David Held, Kris Kitani

Additionally, 3D MOT datasets such as KITTI evaluate MOT methods in 2D space and standardized 3D MOT evaluation tools are missing for a fair comparison of 3D MOT methods.

3D Multi-Object Tracking Association +1

Uncertainty-aware Self-supervised 3D Data Association

1 code implementation18 Aug 2020 Jianren Wang, Siddharth Ancha, Yi-Ting Chen, David Held

Instead, we propose leveraging vast unlabeled datasets by self-supervised metric learning of 3D object trackers, with a focus on data association.

Association Metric Learning +2

Cloth Region Segmentation for Robust Grasp Selection

1 code implementation13 Aug 2020 Jianing Qian, Thomas Weng, Luxin Zhang, Brian Okorn, David Held

Our approach trains a network to segment the edges and corners of a cloth from a depth image, distinguishing such regions from wrinkles or folds.


Active Perception using Light Curtains for Autonomous Driving

no code implementations ECCV 2020 Siddharth Ancha, Yaadhav Raaj, Peiyun Hu, Srinivasa G. Narasimhan, David Held

Most real-world 3D sensors such as LiDARs perform fixed scans of the entire environment, while being decoupled from the recognition system that processes the sensor data.

3D Object Recognition Autonomous Driving

Learning Orientation Distributions for Object Pose Estimation

1 code implementation2 Jul 2020 Brian Okorn, Mengyun Xu, Martial Hebert, David Held

Our first method, which regresses from deep learned features to an isotropic Bingham distribution, gives the best performance for orientation distribution estimation for non-symmetric objects.

Pose Estimation

Multi-modal Transfer Learning for Grasping Transparent and Specular Objects

no code implementations29 May 2020 Thomas Weng, Amith Pallankize, Yimin Tang, Oliver Kroemer, David Held

State-of-the-art object grasping methods rely on depth sensing to plan robust grasps, but commercially available depth sensors fail to detect transparent and specular objects.


Combining Deep Learning and Verification for Precise Object Instance Detection

2 code implementations27 Dec 2019 Siddharth Ancha, Junyu Nan, David Held

For a reliable detection system, if a high confidence detection is made, we would want high certainty that the object has indeed been detected.

Learning to Optimally Segment Point Clouds

no code implementations10 Dec 2019 Peiyun Hu, David Held, Deva Ramanan

We prove that if we score a segmentation by the worst objectness among its individual segments, there is an efficient algorithm that finds the optimal worst-case segmentation among an exponentially large number of candidate segmentations.

Instance Segmentation Semantic Segmentation

What You See is What You Get: Exploiting Visibility for 3D Object Detection

1 code implementation CVPR 2020 Peiyun Hu, Jason Ziglar, David Held, Deva Ramanan

On the NuScenes 3D detection benchmark, we show that, by adding an additional stream for visibility input, we can significantly improve the overall detection accuracy of a state-of-the-art 3D detector.

3D Object Detection Data Augmentation +1

Adaptive Auxiliary Task Weighting for Reinforcement Learning

1 code implementation NeurIPS 2019 Xingyu Lin, Harjatin Baweja, George Kantor, David Held

Reinforcement learning is known to be sample inefficient, preventing its application to many real-world problems, especially with high dimensional observations like images.

reinforcement-learning Reinforcement Learning (RL)

Just Go with the Flow: Self-Supervised Scene Flow Estimation

1 code implementation CVPR 2020 Himangi Mittal, Brian Okorn, David Held

When interacting with highly dynamic environments, scene flow allows autonomous systems to reason about the non-rigid motion of multiple independent objects.

Autonomous Driving Scene Flow Estimation

3D Multi-Object Tracking: A Baseline and New Evaluation Metrics

1 code implementation9 Jul 2019 Xinshuo Weng, Jianren Wang, David Held, Kris Kitani

Additionally, 3D MOT datasets such as KITTI evaluate MOT methods in the 2D space and standardized 3D MOT evaluation tools are missing for a fair comparison of 3D MOT methods.

3D Multi-Object Tracking Association +2

Few-Shot Point Cloud Region Annotation with Human in the Loop

1 code implementation11 Jun 2019 Siddhant Jain, Sowmya Munukutla, David Held

We propose a point cloud annotation framework that employs human-in-loop learning to enable the creation of large point cloud datasets with per-point annotations.

Few-Shot Learning

Reinforcement Learning without Ground-Truth State

no code implementations20 May 2019 Xingyu Lin, Harjatin Singh Baweja, David Held

However, if this policy is trained with reinforcement learning, then without a state estimator, it is hard to specify a reward function based on high-dimensional observations.

reinforcement-learning Reinforcement Learning (RL) +1

Adaptive Variance for Changing Sparse-Reward Environments

no code implementations15 Mar 2019 Xingyu Lin, Pengsheng Guo, Carlos Florensa, David Held

Robots that are trained to perform a task in a fixed environment often fail when facing unexpected changes to the environment due to a lack of exploration.

Iterative Transformer Network for 3D Point Cloud

1 code implementation27 Nov 2018 Wentao Yuan, David Held, Christoph Mertz, Martial Hebert

Recently, neural networks operating on point clouds have shown superior performance on 3D understanding tasks such as shape classification and part segmentation.

General Classification

PCN: Point Completion Network

5 code implementations2 Aug 2018 Wentao Yuan, Tejas Khot, David Held, Christoph Mertz, Martial Hebert

Shape completion, the problem of estimating the complete geometry of objects from partial observations, lies at the core of many vision and robotics applications.

Point Cloud Completion

Reverse Curriculum Generation for Reinforcement Learning

no code implementations17 Jul 2017 Carlos Florensa, David Held, Markus Wulfmeier, Michael Zhang, Pieter Abbeel

The robot is trained in reverse, gradually learning to reach the goal from a set of start states increasingly far from the goal.

reinforcement-learning Reinforcement Learning (RL)

Constrained Policy Optimization

9 code implementations ICML 2017 Joshua Achiam, David Held, Aviv Tamar, Pieter Abbeel

For many applications of reinforcement learning it can be more convenient to specify both a reward function and constraints, rather than trying to design behavior through the reward function.

Reinforcement Learning (RL) Safe Reinforcement Learning

Automatic Goal Generation for Reinforcement Learning Agents

1 code implementation ICML 2018 Carlos Florensa, David Held, Xinyang Geng, Pieter Abbeel

Instead, we propose a method that allows an agent to automatically discover the range of tasks that it is capable of performing.

reinforcement-learning Reinforcement Learning (RL)

Probabilistically Safe Policy Transfer

no code implementations15 May 2017 David Held, Zoe McCarthy, Michael Zhang, Fred Shentu, Pieter Abbeel

Although learning-based methods have great potential for robotics, one concern is that a robot that updates its parameters might cause large amounts of damage before it learns the optimal policy.

Enabling Robots to Communicate their Objectives

no code implementations11 Feb 2017 Sandy H. Huang, David Held, Pieter Abbeel, Anca D. Dragan

We show that certain approximate-inference models lead to the robot generating example behaviors that better enable users to anticipate what it will do in novel situations.

Autonomous Driving

Learning to Track at 100 FPS with Deep Regression Networks

3 code implementations6 Apr 2016 David Held, Sebastian Thrun, Silvio Savarese

We propose a method for offline training of neural networks that can track novel objects at test-time at 100 fps.


Deep Learning for Single-View Instance Recognition

no code implementations29 Jul 2015 David Held, Sebastian Thrun, Silvio Savarese

We show that feedforward neural networks outperform state-of-the-art methods for recognizing objects from novel viewpoints even when trained from just a single image per object.

Cannot find the paper you are looking for? You can Submit a new open access paper.