Search Results for author:
Found 43 papers, 16 papers with code
STORM: Spatio-Temporal Reconstruction Model for Large-Scale Outdoor Scenes
We present STORM, a spatio-temporal reconstruction model designed for reconstructing dynamic outdoor scenes from sparse observations.
DreamDrive: Generative 4D Scene Modeling from Street View Images
In this paper, we present DreamDrive, a 4D spatial-temporal scene generation approach that combines the merits of generation and reconstruction, to synthesize generalizable 4D driving scenes and dynamic driving videos with 3D consistency.
LoRA3D: Low-Rank Self-Calibration of 3D Geometric Foundation Models
Emerging 3D geometric foundation models, such as DUSt3R, offer a promising approach for in-the-wild 3D vision tasks.
EgoMimic: Scaling Imitation Learning via Egocentric Video
The scale and diversity of demonstration data required for imitation learning is a significant challenge.
Large Spatial Model: End-to-end Unposed Images to Semantic 3D
To tackle the scarcity of labeled 3D semantic data and enable natural language-driven scene manipulation, we incorporate a pre-trained 2D language-based segmentation model into a 3D-consistent semantic feature field.
Deep Generative Models in Robotics: A Survey on Learning from Multimodal Demonstrations
In this survey, we aim to provide a unified and comprehensive review of the last year's progress in the use of deep generative models in robotics.
A Survey of Optimization-based Task and Motion Planning: From Classical To Learning Approaches
Task and Motion Planning (TAMP) integrates high-level task planning and low-level motion planning to equip robots with the autonomy to effectively reason over long-horizon, dynamic tasks.
InstantSplat: Sparse-view SfM-free Gaussian Splatting in Seconds
InstantSplat adopts a self-supervised framework that bridges the gap between 2D images and 3D representations using Gaussian Bundle Adjustment (GauBA) and can be optimized in an end-to-end manner.
EmerNeRF: Emergent Spatial-Temporal Scene Decomposition via Self-Supervision
We present EmerNeRF, a simple yet powerful approach for learning spatial-temporal representations of dynamic driving scenes.
NOD-TAMP: Generalizable Long-Horizon Planning with Neural Object Descriptors
Solving complex manipulation tasks in household and factory settings remains challenging due to long-horizon reasoning, fine-grained interactions, and broad object and scene diversity.
Neural Field Dynamics Model for Granular Object Piles Manipulation
We present a learning-based dynamics model for granular material manipulation.
MimicTouch: Leveraging Multi-modal Human Tactile Demonstrations for Contact-rich Manipulation
The key innovations are i) a human tactile data collection system which collects multi-modal tactile dataset for learning human's tactile-guided control strategy, ii) an imitation learning-based framework for learning human's tactile-guided control strategy through such data, and iii) an online residual RL framework to bridge the embodiment gap between the human hand and the robot gripper.
Human-in-the-Loop Task and Motion Planning for Imitation Learning
Finally, we collected 2. 1K demos with HITL-TAMP across 12 contact-rich, long-horizon tasks and show that the system often produces near-perfect agents.
Learning to Discern: Imitating Heterogeneous Human Demonstrations with Preference and Representation Learning
Furthermore, the intrinsic heterogeneity in human behavior can produce equally successful but disparate demonstrations, further exacerbating the challenge of discerning demonstration quality.
Evolutionary Curriculum Training for DRL-Based Navigation Systems
Additionally, an evolutionary training environment generates all the curriculum to improve the DRL model's inadequate skills tested in the previous evaluation.
Language-Guided Traffic Simulation via Scene-Level Diffusion
Realistic and controllable traffic simulation is a core capability that is necessary to accelerate autonomous vehicle (AV) development.
Partial-View Object View Synthesis via Filtered Inversion
At inference, given one or more views of a novel real-world object, FINV first finds a set of latent codes for the object by inverting the generative model from multiple initial seeds.
Foundation Models for Semantic Novelty in Reinforcement Learning
Effectively exploring the environment is a key challenge in reinforcement learning (RL).
Guided Conditional Diffusion for Controllable Traffic Simulation
Controllable and realistic traffic simulation is critical for developing and verifying autonomous vehicles.
LEAGUE: Guided Skill Learning and Abstraction for Long-Horizon Manipulation
We also show that the learned skills can be reused to accelerate learning in new tasks domains and transfer to a physical robot platform.
ProgPrompt: Generating Situated Robot Task Plans using Large Language Models
To ameliorate that effort, large language models (LLMs) can be used to score potential next actions during task planning, and even generate action sequences directly, given an instruction in natural language with no additional domain information.
AdvDO: Realistic Adversarial Attacks for Trajectory Prediction
Trajectory prediction is essential for autonomous vehicles (AVs) to plan correct and safe driving behaviors.
BITS: Bi-level Imitation for Traffic Simulation
We empirically validate our method, named Bi-level Imitation for Traffic Simulation (BITS), with scenarios from two large-scale driving datasets and show that BITS achieves balanced traffic simulation performance in realism, diversity, and long-horizon stability.
Robust Trajectory Prediction against Adversarial Attacks
We demonstrate that our method is able to improve the performance by 46% on adversarial data and at the cost of only 3% performance degradation on clean data, compared to the model trained with clean data.
Co-GAIL: Learning Diverse Strategies for Human-Robot Collaboration
Our method co-optimizes a human policy and a robot policy in an interactive learning process: the human policy learns to generate diverse and plausible collaborative behaviors from demonstrations while the robot policy learns to assist by estimating the unobserved latent strategy of its human collaborator.
What Matters in Learning from Offline Human Demonstrations for Robot Manipulation
Based on the study, we derive a series of lessons including the sensitivity to different algorithmic design choices, the dependence on the quality of the demonstrations, and the variability based on the stopping criteria due to the different objectives in training and evaluation.
Generalization Through Hand-Eye Coordination: An Action Space for Learning Spatially-Invariant Visuomotor Control
Key to such capability is hand-eye coordination, a cognitive ability that enables humans to adaptively direct their movements at task-relevant objects and be invariant to the objects' absolute spatial location.
Human-in-the-Loop Imitation Learning using Remote Teleoperation
We develop a simple and effective algorithm to train the policy iteratively on new data collected by the system that encourages the policy to learn how to traverse bottlenecks through the interventions.
Learning to Generalize Across Long-Horizon Tasks from Human Demonstrations
In the second stage of GTI, we collect a small set of rollouts from the unconditioned stochastic policy of the first stage, and train a goal-directed agent to generalize to novel start and goal configurations.
Positive-Unlabeled Reward Learning
Learning reward functions from data is a promising path towards achieving scalable Reinforcement Learning (RL) for robotics.
6-PACK: Category-level 6D Pose Tracker with Anchor-Based Keypoints
We present 6-PACK, a deep learning approach to category-level 6D object pose tracking on RGB-D data.
Ranked #1 on
6D Pose Estimation using RGBD
on REAL275
(Rerr metric)
Regression Planning Networks
Recent learning-to-plan methods have shown promising results on planning directly from observation space.
Situational Fusion of Visual Representation for Visual Navigation
A complex visual navigation task puts an agent in different situations which call for a diverse range of visual perception abilities.
Continuous Relaxation of Symbolic Planner for One-Shot Imitation Learning
The key technical challenge is that the symbol grounding is prone to error with limited training data and leads to subsequent symbolic planning failures.
Procedure Planning in Instructional Videos
In this paper, we study the problem of procedure planning in instructional videos, which can be seen as a step towards enabling autonomous agents to plan for complex tasks in everyday settings such as cooking.
DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusion
A key technical challenge in performing 6D object pose estimation from RGB-D image is to fully leverage the two complementary data sources.
Ranked #4 on
6D Pose Estimation
on LineMOD
Neural Task Graphs: Generalizing to Unseen Tasks from a Single Video Demonstration
We hypothesize that to successfully generalize to unseen complex tasks from a single video demonstration, it is necessary to explicitly incorporate the compositional structure of the tasks into the model.
PointFusion: Deep Sensor Fusion for 3D Bounding Box Estimation
We present PointFusion, a generic 3D object detection method that leverages both image and 3D point cloud information.
Neural Task Programming: Learning to Generalize Across Hierarchical Tasks
In this work, we propose a novel robot learning framework called Neural Task Programming (NTP), which bridges the idea of few-shot learning from demonstration and neural program induction.
Scene Graph Generation by Iterative Message Passing
In this work, we explicitly model the objects and their relationships using scene graphs, a visually-grounded graphical structure of an image.
Ranked #9 on
Panoptic Scene Graph Generation
on PSG Dataset
Model-Driven Feed-Forward Prediction for Manipulation of Deformable Objects
A fully featured 3D model of the garment is constructed in real-time and volumetric features are then used to obtain the most similar model in the database to predict the object category and pose.
3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction
Inspired by the recent success of methods that employ shape priors to achieve robust 3D reconstructions, we propose a novel recurrent neural network architecture that we call the 3D Recurrent Reconstruction Neural Network (3D-R2N2).
Ranked #4 on
3D Reconstruction
on Data3D−R2N2
