Search Results for author:
Found 32 papers, 13 papers with code
Model-Agnostic Hierarchical Attention for 3D Object Detection
By plugging our proposed modules into the state-of-the-art transformer-based 3D detector, we improve the previous best results on both benchmarks, with the largest improvement margin on small objects.
ULIP: Learning a Unified Representation of Language, Images, and Point Clouds for 3D Understanding
Then, ULIP learns a 3D representation space aligned with the common image-text space, using a small number of automatically synthesized triplets.
Ranked #2 on
3D Point Cloud Classification
on ModelNet40
(using extra training data)
Retrospectives on the Embodied AI Workshop
We present a retrospective on the state of Embodied AI research.
MaskViT: Masked Visual Pre-Training for Video Prediction
This work shows that we can create good video prediction models by pre-training transformers via masked visual modeling.
BEHAVIOR in Habitat 2.0: Simulator-Independent Logical Task Description for Benchmarking Embodied AI Agents
Robots excel in performing repetitive and precision-sensitive tasks in controlled environments such as warehouses and factories, but have not been yet extended to embodied AI agents providing assistance in household tasks.
Error-Aware Imitation Learning from Teleoperation Data for Mobile Manipulation
Doing this is challenging for two reasons: on the data side, current interfaces make collecting high-quality human demonstrations difficult, and on the learning side, policies trained on limited data can suffer from covariate shift when deployed.
iGibson 2.0: Object-Centric Simulation for Robot Learning of Everyday Household Tasks
We evaluate the new capabilities of iGibson 2. 0 to enable robot learning of novel tasks, in the hope of demonstrating the potential of this new simulator to support new research in embodied AI.
What Matters in Learning from Offline Human Demonstrations for Robot Manipulation
Based on the study, we derive a series of lessons including the sensitivity to different algorithmic design choices, the dependence on the quality of the demonstrations, and the variability based on the stopping criteria due to the different objectives in training and evaluation.
BEHAVIOR: Benchmark for Everyday Household Activities in Virtual, Interactive, and Ecological Environments
We introduce BEHAVIOR, a benchmark for embodied AI with 100 activities in simulation, spanning a range of everyday household chores such as cleaning, maintenance, and food preparation.
LASER: Learning a Latent Action Space for Efficient Reinforcement Learning
Additionally, similar tasks or instances of the same task family impose latent manifold constraints on the most effective action space: the task family can be best solved with actions in a manifold of the entire action space of the robot.
Human-in-the-Loop Imitation Learning using Remote Teleoperation
We develop a simple and effective algorithm to train the policy iteratively on new data collected by the system that encourages the policy to learn how to traverse bottlenecks through the interventions.
Learning Multi-Arm Manipulation Through Collaborative Teleoperation
To address these challenges, we present Multi-Arm RoboTurk (MART), a multi-user data collection platform that allows multiple remote users to simultaneously teleoperate a set of robotic arms and collect demonstrations for multi-arm tasks.
Semantic and Geometric Modeling with Neural Message Passing in 3D Scene Graphs for Hierarchical Mechanical Search
We propose to use a 3D scene graph representation to capture the hierarchical, semantic, and geometric aspects of this problem.
IGibson 1.0: a Simulation Environment for Interactive Tasks in Large Realistic Scenes
We present iGibson 1. 0, a novel simulation environment to develop robotic solutions for interactive tasks in large-scale realistic scenes.
Robot Navigation in Constrained Pedestrian Environments using Reinforcement Learning
Navigating fluently around pedestrians is a necessary capability for mobile robots deployed in human environments, such as buildings and homes.
robosuite: A Modular Simulation Framework and Benchmark for Robot Learning
robosuite is a simulation framework for robot learning powered by the MuJoCo physics engine.
ReLMoGen: Leveraging Motion Generation in Reinforcement Learning for Mobile Manipulation
To validate our method, we apply ReLMoGen to two types of tasks: 1) Interactive Navigation tasks, navigation problems where interactions with the environment are required to reach the destination, and 2) Mobile Manipulation tasks, manipulation tasks that require moving the robot base.
Visuomotor Mechanical Search: Learning to Retrieve Target Objects in Clutter
When searching for objects in cluttered environments, it is often necessary to perform complex interactions in order to move occluding objects out of the way and fully reveal the object of interest and make it graspable.
Learning to Generalize Across Long-Horizon Tasks from Human Demonstrations
In the second stage of GTI, we collect a small set of rollouts from the unconditioned stochastic policy of the first stage, and train a goal-directed agent to generalize to novel start and goal configurations.
JRMOT: A Real-Time 3D Multi-Object Tracker and a New Large-Scale Dataset
In this work we present JRMOT, a novel 3D MOT system that integrates information from RGB images and 3D point clouds to achieve real-time, state-of-the-art tracking performance.
Ranked #7 on
Multiple Object Tracking
on KITTI Tracking test
Leveraging Pretrained Image Classifiers for Language-Based Segmentation
Current semantic segmentation models cannot easily generalize to new object classes unseen during train time: they require additional annotated images and retraining.
Interactive Gibson Benchmark (iGibson 0.5): A Benchmark for Interactive Navigation in Cluttered Environments
We present Interactive Gibson Benchmark, the first comprehensive benchmark for training and evaluating Interactive Navigation: robot navigation strategies where physical interaction with objects is allowed and even encouraged to accomplish a task.
JRDB: A Dataset and Benchmark of Egocentric Robot Visual Perception of Humans in Built Environments
We present JRDB, a novel egocentric dataset collected from our social mobile manipulator JackRabbot.
6-PACK: Category-level 6D Pose Tracker with Anchor-Based Keypoints
We present 6-PACK, a deep learning approach to category-level 6D object pose tracking on RGB-D data.
Ranked #1 on
6D Pose Estimation using RGBD
on REAL275
(Rerr metric)
Regression Planning Networks
Recent learning-to-plan methods have shown promising results on planning directly from observation space.
Improving Social Awareness Through DANTE: A Deep Affinity Network for Clustering Conversational Interactants
We propose a data-driven approach to detect conversational groups by identifying spatial arrangements typical of these focused social encounters.
Social-BiGAT: Multimodal Trajectory Forecasting using Bicycle-GAN and Graph Attention Networks
This problem is compounded by the presence of social interactions between humans and their physical interactions with the scene.
Ranked #17 on
Trajectory Prediction
on ETH/UCY
Variable Impedance Control in End-Effector Space: An Action Space for Reinforcement Learning in Contact-Rich Tasks
This paper studies the effect of different action spaces in deep RL and advocates for Variable Impedance Control in End-effector Space (VICES) as an advantageous action space for constrained and contact-rich tasks.
Deep Local Trajectory Replanning and Control for Robot Navigation
We present a navigation system that combines ideas from hierarchical planning and machine learning.
Mechanical Search: Multi-Step Retrieval of a Target Object Occluded by Clutter
In this paper, we formalize Mechanical Search and study a version where distractor objects are heaped over the target object in a bin.
Robotics
DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusion
A key technical challenge in performing 6D object pose estimation from RGB-D image is to fully leverage the two complementary data sources.
Ranked #4 on
6D Pose Estimation
on LineMOD
The RBO Dataset of Articulated Objects and Interactions
Each interaction with an object is annotated with the ground truth poses of its rigid parts and the kinematic state obtained by a motion capture system.
