Search Results for author:
Found 109 papers, 41 papers with code
Vision Language Models are In-Context Value Learners
Instead, GVL poses value estimation as a temporal ordering problem over shuffled video frames; this seemingly more challenging task encourages VLMs to more fully exploit their underlying semantic and temporal grounding capabilities to differentiate frames based on their perceived task progress, consequently producing significantly better value predictions.
RT-Affordance: Affordances are Versatile Intermediate Representations for Robot Manipulation
Our method, RT-Affordance, is a hierarchical model that first proposes an affordance plan given the task language, and then conditions the policy on this affordance plan to perform manipulation.
Vocal Sandbox: Continual Learning and Adaptation for Situated Human-Robot Collaboration
Similarly, users teach high-level planning behaviors through spoken dialogue, using pretrained language models to synthesize behaviors such as "packing an object away" as compositions of low-level skills $-$ concepts that can be reused and built upon.
So You Think You Can Scale Up Autonomous Robot Data Collection?
Our work suggests a negative result: that scaling up autonomous data collection for learning robot policies for real-world tasks is more challenging and impractical than what is suggested in prior work.
Gen2Act: Human Video Generation in Novel Scenarios enables Generalizable Robot Manipulation
To train the policy, we use an order of magnitude less robot interaction data compared to what the video prediction model was trained on.
MotIF: Motion Instruction Fine-tuning
MotIF assesses the success of robot motion given the image observation of the trajectory, task instruction, and motion description.
FlowRetrieval: Flow-Guided Data Retrieval for Few-Shot Imitation Learning
We propose FlowRetrieval, an approach that leverages optical flow representations for both extracting similar motions to target tasks from prior data, and for guiding learning of a policy that can maximally benefit from such data.
Re-Mix: Optimizing Data Mixtures for Large Scale Imitation Learning
Increasingly large imitation learning datasets are being collected with the goal of training foundation models for robotics.
FLAIR: Feeding via Long-horizon AcquIsition of Realistic dishes
We address this with FLAIR, a system for long-horizon feeding which leverages the commonsense and few-shot reasoning capabilities of foundation models, along with a library of parameterized skills, to plan and execute user-preferred and efficient bite sequences.
OpenVLA: An Open-Source Vision-Language-Action Model
Large policies pretrained on a combination of Internet-scale vision-language data and diverse robot demonstrations have the potential to change how we teach robots new skills: rather than training new behaviors from scratch, we can fine-tune such vision-language-action (VLA) models to obtain robust, generalizable policies for visuomotor control.
Octo: An Open-Source Generalist Robot Policy
In experiments across 9 robotic platforms, we demonstrate that Octo serves as a versatile policy initialization that can be effectively finetuned to new observation and action spaces.
Policy Learning with a Language Bottleneck
Modern AI systems such as self-driving cars and game-playing agents achieve superhuman performance, but often lack human-like features such as generalization, interpretability and human inter-operability.
Explore until Confident: Efficient Exploration for Embodied Question Answering
We consider the problem of Embodied Question Answering (EQA), which refers to settings where an embodied agent such as a robot needs to actively explore an environment to gather information until it is confident about the answer to a question.
Efficient Data Collection for Robotic Manipulation via Compositional Generalization
If robot policies can compose environmental factors from their data to succeed when encountering unseen factor combinations, we can exploit this to avoid collecting data for situations that composition would address.
RT-H: Action Hierarchies Using Language
Predicting these language motions as an intermediate step between tasks and actions forces the policy to learn the shared structure of low-level motions across seemingly disparate tasks.
Batch Active Learning of Reward Functions from Human Preferences
Our results suggest that our batch active learning algorithm requires only a few queries that are computed in a short amount of time.
Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Models
Visually-conditioned language models (VLMs) have seen growing adoption in applications such as visual dialogue, scene understanding, and robotic task planning; adoption that has fueled a wealth of new models such as LLaVa, InstructBLIP, and PaLI-3.
AutoRT: Embodied Foundation Models for Large Scale Orchestration of Robotic Agents
We experimentally show that such "in-the-wild" data collected by AutoRT is significantly more diverse, and that AutoRT's use of LLMs allows for instruction following data collection robots that can align to human preferences.
SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities
By training a VLM on such data, we significantly enhance its ability on both qualitative and quantitative spatial VQA.
Chain of Code: Reasoning with a Language Model-Augmented Code Emulator
For example, consider prompting an LM to write code that counts the number of times it detects sarcasm in an essay: the LM may struggle to write an implementation for "detect_sarcasm(string)" that can be executed by the interpreter (handling the edge cases would be insurmountable).
Distilling and Retrieving Generalizable Knowledge for Robot Manipulation via Language Corrections
DROC is able to respond to a sequence of online language corrections that address failures in both high-level task plans and low-level skill primitives.
Imitation Bootstrapped Reinforcement Learning
Despite the considerable potential of reinforcement learning (RL), robotic control tasks predominantly rely on imitation learning (IL) due to its better sample efficiency.
Diverse Conventions for Human-AI Collaboration
Conventions are crucial for strong performance in cooperative multi-agent games, because they allow players to coordinate on a shared strategy without explicit communication.
Contrastive Preference Learning: Learning from Human Feedback without RL
Thus, learning a reward function from feedback is not only based on a flawed assumption of human preference, but also leads to unwieldy optimization challenges that stem from policy gradients or bootstrapping in the RL phase.
Robots That Can See: Leveraging Human Pose for Trajectory Prediction
Anticipating the motion of all humans in dynamic environments such as homes and offices is critical to enable safe and effective robot navigation.
Learning Sequential Acquisition Policies for Robot-Assisted Feeding
A robot providing mealtime assistance must perform specialized maneuvers with various utensils in order to pick up and feed a range of food items.
Physically Grounded Vision-Language Models for Robotic Manipulation
We incorporate this physically grounded VLM in an interactive framework with a large language model-based robotic planner, and show improved planning performance on tasks that require reasoning about physical object concepts, compared to baselines that do not leverage physically grounded VLMs.
Stabilize to Act: Learning to Coordinate for Bimanual Manipulation
We counteract this challenge by drawing inspiration from humans to propose a novel role assignment framework: a stabilizing arm holds an object in place to simplify the environment while an acting arm executes the task.
Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback
Reinforcement learning from human feedback (RLHF) is a technique for training AI systems to align with human goals.
Large Language Models as General Pattern Machines
We observe that pre-trained large language models (LLMs) are capable of autoregressively completing complex token sequences -- from arbitrary ones procedurally generated by probabilistic context-free grammars (PCFG), to more rich spatial patterns found in the Abstraction and Reasoning Corpus (ARC), a general AI benchmark, prompted in the style of ASCII art.
Polybot: Training One Policy Across Robots While Embracing Variability
Reusing large datasets is crucial to scale vision-based robotic manipulators to everyday scenarios due to the high cost of collecting robotic datasets.
Robots That Ask For Help: Uncertainty Alignment for Large Language Model Planners
Large language models (LLMs) exhibit a wide range of promising capabilities -- from step-by-step planning to commonsense reasoning -- that may provide utility for robots, but remain prone to confidently hallucinated predictions.
KITE: Keypoint-Conditioned Policies for Semantic Manipulation
While natural language offers a convenient shared interface for humans and robots, enabling robots to interpret and follow language commands remains a longstanding challenge in manipulation.
Toward Grounded Commonsense Reasoning
We additionally illustrate our approach with a robot on 2 carefully designed surfaces.
Language to Rewards for Robotic Skill Synthesis
However, since low-level robot actions are hardware-dependent and underrepresented in LLM training corpora, existing efforts in applying LLMs to robotics have largely treated LLMs as semantic planners or relied on human-engineered control primitives to interface with the robot.
Generating Language Corrections for Teaching Physical Control Tasks
AI assistance continues to help advance applications in education, from language learning to intelligent tutoring systems, yet current methods for providing students feedback are still quite limited.
Strategic Reasoning with Language Models
Existing approaches to solving strategic games rely on extensive training, yielding strategies that do not generalize to new scenarios or games without retraining.
Parallel Sampling of Diffusion Models
Instead of reducing the number of denoising steps (trading quality for speed), in this paper we explore an orthogonal approach: can we run the denoising steps in parallel (trading compute for speed)?
Distance Weighted Supervised Learning for Offline Interaction Data
To bridge the gap between IL and RL, we introduce Distance Weighted Supervised Learning or DWSL, a supervised method for learning goal-conditioned policies from offline data.
Behavior Retrieval: Few-Shot Imitation Learning by Querying Unlabeled Datasets
Concretely, we propose a simple approach that uses a small amount of downstream expert data to selectively query relevant behaviors from an offline, unlabeled dataset (including many sub-optimal behaviors).
Language Instructed Reinforcement Learning for Human-AI Coordination
One of the fundamental quests of AI is to produce agents that coordinate well with humans.
Multi-agent Reinforcement Learning
reinforcement-learning
+2
Active Reward Learning from Online Preferences
Robot policies need to adapt to human preferences and/or new environments.
Reward Design with Language Models
During training, the LLM evaluates an RL agent's behavior against the desired behavior described by the prompt and outputs a corresponding reward signal.
Language-Driven Representation Learning for Robotics
First, we demonstrate that existing representations yield inconsistent results across these tasks: masked autoencoding approaches pick up on low-level spatial features at the cost of high-level semantics, while contrastive learning approaches capture the opposite.
Long Horizon Temperature Scaling
LHTS is compatible with all likelihood-based models, and optimizes for the long horizon likelihood of samples.
"No, to the Right" -- Online Language Corrections for Robotic Manipulation via Shared Autonomy
Instead of discrete turn-taking between a human and robot, LILAC splits agency between the human and robot: language is an input to a learned model that produces a meaningful, low-dimensional control space that the human can use to guide the robot.
Few-Shot Preference Learning for Human-in-the-Loop RL
Contrary to most works that focus on query selection to \emph{minimize} the amount of data required for learning reward functions, we take an opposite approach: \emph{expanding} the pool of available data by viewing human-in-the-loop RL through the more flexible lens of multi-task learning.
Learning Visuo-Haptic Skewering Strategies for Robot-Assisted Feeding
Acquiring food items with a fork poses an immense challenge to a robot-assisted feeding system, due to the wide range of material properties and visual appearances present across food groups.
Learning Bimanual Scooping Policies for Food Acquisition
In order to acquire foods with such diverse properties, we propose stabilizing food items during scooping using a second arm, for example, by pushing peas against the spoon with a flat surface to prevent dispersion.
Assistive Teaching of Motor Control Tasks to Humans
In this paper, we focus on the problem of assistive teaching of motor control tasks such as parking a car or landing an aircraft.
Learning Tool Morphology for Contact-Rich Manipulation Tasks with Differentiable Simulation
We demonstrate the effectiveness of our method for designing new tools in several scenarios, such as winding ropes, flipping a box and pushing peas onto a scoop in simulation.
Eliciting Compatible Demonstrations for Multi-Human Imitation Learning
Imitation learning from human-provided demonstrations is a strong approach for learning policies for robot manipulation.
Masked Imitation Learning: Discovering Environment-Invariant Modalities in Multimodal Demonstrations
Specifically, we design a masked policy network with a binary mask to block certain modalities.
Training and Inference on Any-Order Autoregressive Models the Right Way
Conditional inference on arbitrary subsets of variables is a core problem in probabilistic inference with important applications such as masked language modeling and image inpainting.
Leveraging Smooth Attention Prior for Multi-Agent Trajectory Prediction
At a certain time, to forecast a reasonable future trajectory, each agent needs to pay attention to the interactions with only a small group of most relevant agents instead of unnecessarily paying attention to all the other agents.
Weakly Supervised Correspondence Learning
Correspondence learning is a fundamental problem in robotics, which aims to learn a mapping between state, action pairs of agents of different dynamics or embodiments.
Learning from Imperfect Demonstrations via Adversarial Confidence Transfer
Existing learning from demonstration algorithms usually assume access to expert demonstrations.
Imitation Learning by Estimating Expertise of Demonstrators
In this work, we show that unsupervised learning over demonstrator expertise can lead to a consistent boost in the performance of imitation learning algorithms.
Conditional Imitation Learning for Multi-Agent Games
In this work, we study the problem of conditional multi-agent imitation learning, where we have access to joint trajectory demonstrations at training time, and we must interact with and adapt to new partners at test time.
PantheonRL: A MARL Library for Dynamic Training Interactions
We present PantheonRL, a multiagent reinforcement learning software package for dynamic training interactions such as round-robin, adaptive, and ad-hoc training.
HyperSPNs: Compact and Expressive Probabilistic Circuits
Probabilistic circuits (PCs) are a family of generative models which allows for the computation of exact likelihoods and marginals of its probability distributions.
LILA: Language-Informed Latent Actions
We introduce Language-Informed Latent Actions (LILA), a framework for learning natural language interfaces in the context of human-robot collaboration.
Learning Feasibility to Imitate Demonstrators with Different Dynamics
The goal of learning from demonstrations is to learn a policy for an agent (imitator) by mimicking the behavior in the demonstrations.
From Machine Learning to Robotics: Challenges and Opportunities for Embodied Intelligence
Machine learning has long since become a keystone technology, accelerating science and applications in a broad range of domains.
Confidence-Aware Imitation Learning from Demonstrations with Varying Optimality
Our results show that CAIL significantly outperforms other imitation learning methods from demonstrations with varying optimality.
Open-domain clarification question generation without question examples
An overarching goal of natural language processing is to enable machines to communicate seamlessly with humans.
Influencing Towards Stable Multi-Agent Interactions
Instead of reactively adapting to the other agent's (opponent or partner) behavior, we propose an algorithm to proactively influence the other agent's strategy to stabilize -- which can restrain the non-stationarity caused by the other agent.
Partner-Aware Algorithms in Decentralized Cooperative Bandit Teams
Our results show that the proposed partner-aware strategy outperforms other known methods, and our human subject studies suggest humans prefer to collaborate with AI agents implementing our partner-aware strategy.
Learning Reward Functions from Scale Feedback
Today's robots are increasingly interacting with people and need to efficiently learn inexperienced user's preferences.
Learning Multimodal Rewards from Rankings
However, expert feedback is often assumed to be drawn from an underlying unimodal reward function.
APReL: A Library for Active Preference-based Reward Learning Algorithms
Reward learning is a fundamental problem in human-robot interaction to have robots that operate in alignment with what their human user wants.
On the Opportunities and Risks of Foundation Models
AI is undergoing a paradigm shift with the rise of models (e. g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks.
Targeted Data Acquisition for Evolving Negotiation Agents
This trend additionally holds when comparing agents using our targeted data acquisition framework to variants of agents trained with a mix of supervised learning and reinforcement learning, or to agents using tailored reward functions that explicitly optimize for utility and Pareto-optimality.
Emergent Prosociality in Multi-Agent Games Through Gifting
Coordination is often critical to forming prosocial behaviors -- behaviors that increase the overall sum of rewards received by all agents in a multi-agent game.
Incentivizing Efficient Equilibria in Traffic Networks with Mixed Autonomy
Traffic congestion has large economic and social costs.
Learning Visually Guided Latent Actions for Assistive Teleoperation
In this work, we develop assistive robots that condition their latent embeddings on visual inputs.
On the Critical Role of Conventions in Adaptive Human-AI Collaboration
Humans can quickly adapt to new partners in collaborative tasks (e. g. playing basketball), because they understand which fundamental skills of the task (e. g. how to dribble, how to shoot) carry over across new partners.
ELLA: Exploration through Learned Language Abstraction
Building agents capable of understanding language instructions is critical to effective and robust human-AI collaboration.
Learning from Imperfect Demonstrations from Agents with Varying Dynamics
The proposed score enables learning from more informative demonstrations, and disregarding the less relevant demonstrations.
Transfer Reinforcement Learning across Homotopy Classes
The ability for robots to transfer their learned knowledge to new tasks -- where data is scarce -- is a fundamental challenge for successful robot learning.
Transfer Reinforcement Learning
Robotics
Incentivizing Routing Choices for Safe and Efficient Transportation in the Face of the COVID-19 Pandemic
In turn, significant increases in traffic congestion are expected, since people are likely to prefer using their own vehicles or taxis as opposed to riskier and more crowded options such as the railway.
Learning Latent Representations to Influence Multi-Agent Interaction
We propose a reinforcement learning-based framework for learning latent representations of an agent's policy, where the ego agent identifies the relationship between its behavior and the other agent's future strategy.
ROIAL: Region of Interest Active Learning for Characterizing Exoskeleton Gait Preference Landscapes
ROIAL learns Bayesian posteriors that predict each exoskeleton user's utility landscape across four exoskeleton gait parameters.
Learning Adaptive Language Interfaces through Decomposition
Our goal is to create an interactive natural language interface that efficiently and reliably learns from users to complete tasks in simulated robotics settings.
Multi-Agent Safe Planning with Gaussian Processes
Multi-agent safe systems have become an increasingly important area of study as we can now easily have multiple AI-powered systems operating together.
Learning User-Preferred Mappings for Intuitive Robot Control
Existing approaches to teleoperation typically assume a one-size-fits-all approach, where the designers pre-define a mapping between human inputs and robot actions, and every user must adapt to this mapping over repeated interactions.
Reinforcement Learning based Control of Imitative Policies for Near-Accident Driving
To address driving in near-accident scenarios, we propose a hierarchical reinforcement and imitation learning (H-ReIL) approach that consists of low-level policies learned by IL for discrete driving modes, and a high-level policy learned by RL that switches between different driving modes.
Learning Reward Functions from Diverse Sources of Human Feedback: Optimally Integrating Demonstrations and Preferences
As designing reward functions can be extremely challenging, a more promising approach is to directly learn reward functions from human teachers.
Dynamic Multi-Robot Task Allocation under Uncertainty and Temporal Constraints
We consider the problem of dynamically allocating tasks to multiple agents under time window constraints and task completion uncertainty.
Active Preference-Based Gaussian Process Regression for Reward Learning
Our results in simulations and a user study suggest that our approach can efficiently learn expressive reward functions for robotics tasks.
BLEU Neighbors: A Reference-less Approach to Automatic Evaluation
To this end, we propose BLEU Neighbors, a nearest neighbors model for estimating language quality by using the BLEU score as a kernel function.
Exchangeable Input Representations for Reinforcement Learning
We show that our proposed representation results in an input space that is a factor of $m!$ smaller for inputs of $m$ objects.
When Humans Aren't Optimal: Robots that Collaborate with Risk-Aware Humans
Overall, we extend existing rational human models so that collaborative robots can anticipate and plan around suboptimal human behavior during HRI.
Continual adaptation for efficient machine communication
To communicate with new partners in new contexts, humans rapidly form new linguistic conventions.
Learning from My Partner's Actions: Roles in Decentralized Robot Teams
When teams of robots collaborate to complete a task, communication is often necessary.
Asking Easy Questions: A User-Friendly Approach to Active Reward Learning
Robots can learn the right reward function by querying a human expert.
Controlling Assistive Robots with Learned Latent Actions
Our insight is that we can make assistive robots easier for humans to control by leveraging latent actions.
Robotics
Learning Reward Functions by Integrating Human Demonstrations and Preferences
In a user study, we compare our method to a standard IRL method; we find that users rated the robot trained with DemPref as being more successful at learning their desired behavior, and preferred to use the DemPref system (over IRL) to train the robot.
Batch Active Learning Using Determinantal Point Processes
While active learning methods attempt to tackle this issue by labeling only the data samples that give high information, they generally suffer from large computational costs and are impractical in settings where data can be collected in parallel.
Deep Local Trajectory Replanning and Control for Robot Navigation
We present a navigation system that combines ideas from hierarchical planning and machine learning.
Object Exchangeability in Reinforcement Learning: Extended Abstract
Although deep reinforcement learning has advanced significantly over the past several years, sample efficiency remains a major challenge.
Efficient and Safe Exploration in Deterministic Markov Decision Processes with Unknown Transition Models
We propose a safe exploration algorithm for deterministic Markov Decision Processes with unknown transition models.
Unsupervised Visuomotor Control through Distributional Planning Networks
While reinforcement learning (RL) has the potential to enable robots to autonomously acquire a wide range of skills, in practice, RL usually requires manual, per-task engineering of reward functions, especially in real world settings where aspects of the environment needed to compute progress are not directly accessible.
Hierarchical Game-Theoretic Planning for Autonomous Vehicles
This mutual dependence, best captured by dynamic game theory, creates a strong coupling between the vehicle's planning and its predictions of other drivers' behavior, and constitutes an open problem with direct implications on the safety and viability of autonomous driving technology.
Batch Active Preference-Based Learning of Reward Functions
Data generation and labeling are usually an expensive part of learning for robotics.
Multi-Agent Generative Adversarial Imitation Learning
Imitation learning algorithms can be used to learn a policy from expert demonstrations without access to a reward signal.
Towards Verified Artificial Intelligence
Verified artificial intelligence (AI) is the goal of designing AI-based systems that that have strong, ideally provable, assurances of correctness with respect to mathematically-specified requirements.
Safe Control under Uncertainty
In this paper, we propose a new logic, Probabilistic Signal Temporal Logic (PrSTL), as an expressive language to define the stochastic properties, and enforce probabilistic guarantees on them.
Robust Subspace System Identification via Weighted Nuclear Norm Optimization
As in robust PCA, it can be problematic to find a suitable regularization parameter.
