Search Results for author:
Found 27 papers, 4 papers with code
Temporal Representation Alignment: Successor Features Enable Emergent Compositionality in Robot Instruction Following
Effective task representations should facilitate compositionality, such that after learning a variety of basic tasks, an agent can perform compound tasks consisting of multiple steps simply by composing the representations of the constituent steps together.
Planning-Guided Diffusion Policy Learning for Generalizable Contact-Rich Bimanual Manipulation
Contact-rich bimanual manipulation involves precise coordination of two arms to change object states through strategically selected contacts and motions.
Blox-Net: Generative Design-for-Robot-Assembly Using VLM Supervision, Physics Simulation, and a Robot with Reset
The output is an assembly, a spatial arrangement of these components, and instructions for a robot to build this assembly.
KALIE: Fine-Tuning Vision-Language Models for Open-World Manipulation without Robot Data
Building generalist robotic systems involves effectively endowing robots with the capabilities to handle novel objects in an open-world setting.
Policy Adaptation via Language Optimization: Decomposing Tasks for Few-Shot Imitation
Learned language-conditioned robot policies often struggle to effectively adapt to new real-world tasks even when pre-trained across a diverse set of instructions.
Affordance-Guided Reinforcement Learning via Visual Prompting
On real-world manipulation tasks specified by natural language descriptions, KAGI improves the sample efficiency of autonomous RL and enables successful task completion in 30K online fine-tuning steps.
MOKA: Open-World Robotic Manipulation through Mark-Based Visual Prompting
Central to our approach is a compact point-based representation of affordance, which bridges the VLM's predictions on observed images and the robot's actions in the physical world.
BridgeData V2: A Dataset for Robot Learning at Scale
By publicly sharing BridgeData V2 and our pre-trained models, we aim to accelerate research in scalable robot learning methods.
Multi-Stage Cable Routing through Hierarchical Imitation Learning
In such settings, learning individual primitives for each stage that succeed with a high enough rate to perform a complete temporally extended task is impractical: if each stage must be completed successfully and has a non-negligible probability of failure, the likelihood of successful completion of the entire task becomes negligible.
Goal Representations for Instruction Following: A Semi-Supervised Language Interface to Control
Our method achieves robust performance in the real world by learning an embedding from the labeled data that aligns language not to the goal image, but rather to the desired change between the start and goal images that the instruction corresponds to.
Stabilizing Contrastive RL: Techniques for Robotic Goal Reaching from Offline Data
Robotic systems that rely primarily on self-supervised learning have the potential to decrease the amount of human annotation and engineering effort required to learn control strategies.
Active Task Randomization: Learning Robust Skills via Unsupervised Generation of Diverse and Feasible Tasks
ATR selects suitable tasks, which consist of an initial environment state and manipulation goal, for learning robust skills by balancing the diversity and feasibility of the tasks.
Generalization with Lossy Affordances: Leveraging Broad Offline Data for Learning Visuomotor Tasks
The utilization of broad datasets has proven to be crucial for generalization for a wide range of fields.
Planning to Practice: Efficient Online Fine-Tuning by Composing Goals in Latent Space
Our experimental results show that PTP can generate feasible sequences of subgoals that enable the policy to efficiently solve the target tasks.
Discovering Generalizable Skills via Automated Generation of Diverse Tasks
To encourage generalizable skills to emerge, our method trains each skill to specialize in the paired task and maximizes the diversity of the generated tasks.
Synergies Between Affordance and Geometry: 6-DoF Grasp Detection via Implicit Representations
The experimental results in simulation and on the real robot have demonstrated that the use of implicit neural representations and joint learning of grasp affordance and 3D reconstruction have led to state-of-the-art grasping results.
Beyond Lexical: A Semantic Retrieval Framework for Textual SearchEngine
Search engine has become a fundamental component in various web and mobile applications.
Adaptive Procedural Task Generation for Hard-Exploration Problems
To enable curriculum learning in the absence of a direct indicator of learning progress, we propose to train the task generator by balancing the agent's performance in the generated tasks and the similarity to the target tasks.
SERank: Optimize Sequencewise Learning to Rank Using Squeeze-and-Excitation Network
Recently, there are a few methods have been proposed which focused on mining information across ranking candidates list for further improvements, such as learning multivariant scoring function or learning contextual embedding.
Dynamics Learning with Cascaded Variational Inference for Multi-Step Manipulation
The fundamental challenge of planning for multi-step manipulation is to find effective and plausible action sequences that lead to the task goal.
KETO: Learning Keypoint Representations for Tool Manipulation
For this purpose, we present KETO, a framework of learning keypoint representations of tool-based manipulation.
Robotics
Scene Memory Transformer for Embodied Agents in Long-Horizon Tasks
Many robotic applications require the agent to perform long-horizon tasks in partially observable environments.
Learning Task-Oriented Grasping for Tool Manipulation from Simulated Self-Supervision
We perform both simulated and real-world experiments on two tool-based manipulation tasks: sweeping and hammering.
Demo2Vec: Reasoning Object Affordances From Online Videos
Watching expert demonstrations is an important way for humans and robots to reason about affordances of unseen objects.
Ranked #2 on
Video-to-image Affordance Grounding
on OPRA (28x28)
Recurrent Autoregressive Networks for Online Multi-Object Tracking
The external memory explicitly stores previous inputs of each trajectory in a time window, while the internal memory learns to summarize long-term tracking history and associate detections by processing the external memory.
Multi-Task Domain Adaptation for Deep Learning of Instance Grasping from Simulation
Learning-based approaches to robotic manipulation are limited by the scalability of data collection and accessibility of labels.
DeLay: Robust Spatial Layout Estimation for Cluttered Indoor Scenes
We consider the problem of estimating the spatial layout of an indoor scene from a monocular RGB image, modeled as the projection of a 3D cuboid.
