Search Results for author:
Found 21 papers, 15 papers with code
Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agents
In this paper, we study the problem of planning in Minecraft, a popular, democratized yet challenging open-ended environment for developing multi-task embodied agents.
Open-World Multi-Task Control Through Goal-Aware Representation Learning and Adaptive Horizon Prediction
We study the problem of learning goal-conditioned policies in Minecraft, a popular, widely accessible yet challenging open-ended environment for developing human-level multi-task agents.
Perceive, Ground, Reason, and Act: A Benchmark for General-purpose Visual Representation
Current computer vision models, unlike the human visual system, cannot yet achieve general-purpose visual understanding.
SQA3D: Situated Question Answering in 3D Scenes
We propose a new task to benchmark scene understanding of embodied agents: Situated Question Answering in 3D Scenes (SQA3D).
Ranked #1 on
Question Answering
on SQA3D
(using extra training data)
Continuous-Time and Multi-Level Graph Representation Learning for Origin-Destination Demand Prediction
Traffic demand forecasting by deep neural networks has attracted widespread interest in both academia and industry society.
Latent Diffusion Energy-Based Model for Interpretable Text Modeling
Latent space Energy-Based Models (EBMs), also known as energy-based priors, have drawn growing interests in generative modeling.
Bongard-HOI: Benchmarking Few-Shot Visual Reasoning for Human-Object Interactions
A significant gap remains between today's visual pattern recognition models and human-level visual cognition especially when it comes to few-shot learning and compositional reasoning of novel concepts.
Ranked #1 on
Few-Shot Image Classification
on Bongard-HOI
RelViT: Concept-guided Vision Transformer for Visual Relational Reasoning
This task remains challenging for current deep learning algorithms since it requires addressing three key technical problems jointly: 1) identifying object entities and their properties, 2) inferring semantic relations between pairs of entities, and 3) generalizing to novel object-relation combinations, i. e., systematic generalization.
Ranked #1 on
Zero-Shot Human-Object Interaction Detection
on HICO
Unsupervised Foreground Extraction via Deep Region Competition
Foreground extraction can be viewed as a special case of generic image segmentation that focuses on identifying and disentangling objects from the background.
Adversarial Option-Aware Hierarchical Imitation Learning
In particular, we propose an Expectation-Maximization(EM)-style algorithm: an E-step that samples the options of expert conditioned on the current learned policy, and an M-step that updates the low- and high-level policies of agent simultaneously to minimize the newly proposed option-occupancy measurement between the expert and the agent.
HALMA: Humanlike Abstraction Learning Meets Affordance in Rapid Problem Solving
Leveraging these concepts, they could understand the internal structure of this task, without seeing all of the problem instances.
A Mobile Robot Hand-Arm Teleoperation System by Vision and IMU
In this paper, we present a multimodal mobile teleoperation system that consists of a novel vision-based hand pose regression network (Transteleop) and an IMU-based arm tracking method.
Robust Robotic Pouring using Audition and Haptics
Both network training results and robot experiments demonstrate that MP-Net is robust against noise and changes to the task and environment.
Theory-based Causal Transfer: Integrating Instance-level Induction and Abstract-level Structure Learning
Given these general theories, the goal is to train an agent by interactively exploring the problem space to (i) discover, form, and transfer useful abstract and structural knowledge, and (ii) induce useful knowledge from the instance-level attributes observed in the environment.
Reinforcement Learning from Imperfect Demonstrations under Soft Expert Guidance
In this paper, we study Reinforcement Learning from Demonstrations (RLfD) that improves the exploration efficiency of Reinforcement Learning (RL) by providing expert demonstrations.
Imitation Learning from Observations by Minimizing Inverse Dynamics Disagreement
This paper studies Learning from Observations (LfO) for imitation learning with access to state-only demonstrations.
Making Sense of Audio Vibration for Liquid Height Estimation in Robotic Pouring
PouringNet is trained on our collected real-world pouring dataset with multimodal sensing data, which contains more than 3000 recordings of audio, force feedback, video and trajectory data of the human hand that performs the pouring task.
Robotics Sound Audio and Speech Processing
PointNetGPD: Detecting Grasp Configurations from Point Sets
In this paper, we propose an end-to-end grasp evaluation model to address the challenging problem of localizing robot grasp configurations directly from the point cloud.
Robotics
Vision-based Teleoperation of Shadow Dexterous Hand using End-to-End Deep Neural Network
In this paper, we present TeachNet, a novel neural network architecture for intuitive and markerless vision-based teleoperation of dexterous robotic hands.
Robotics
Learning and Inferring Movement with Deep Generative Model
Learning and inference movement is a very challenging problem due to its high dimensionality and dependency to varied environments or tasks.
Task Transfer by Preference-Based Cost Learning
The goal of task transfer in reinforcement learning is migrating the action policy of an agent to the target task from the source task.
