Search Results for author:
Found 86 papers, 33 papers with code
Learning Sequential Acquisition Policies for Robot-Assisted Feeding
A robot providing mealtime assistance must perform specialized maneuvers with various utensils in order to pick up and feed a range of food items.
Physically Grounded Vision-Language Models for Robotic Manipulation
However, current VLMs are limited in their understanding of the physical concepts (e. g., material, fragility) of common objects, which restricts their usefulness for robotic manipulation tasks that involve interaction and physical reasoning about such objects.
Stabilize to Act: Learning to Coordinate for Bimanual Manipulation
We counteract this challenge by drawing inspiration from humans to propose a novel role assignment framework: a stabilizing arm holds an object in place to simplify the environment while an acting arm executes the task.
Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback
Reinforcement learning from human feedback (RLHF) is a technique for training AI systems to align with human goals.
Large Language Models as General Pattern Machines
We observe that pre-trained large language models (LLMs) are capable of autoregressively completing complex token sequences -- from arbitrary ones procedurally generated by probabilistic context-free grammars (PCFG), to more rich spatial patterns found in the Abstract Reasoning Corpus (ARC), a general AI benchmark, prompted in the style of ASCII art.
Polybot: Training One Policy Across Robots While Embracing Variability
Reusing large datasets is crucial to scale vision-based robotic manipulators to everyday scenarios due to the high cost of collecting robotic datasets.
Robots That Ask For Help: Uncertainty Alignment for Large Language Model Planners
Large language models (LLMs) exhibit a wide range of promising capabilities -- from step-by-step planning to commonsense reasoning -- that may provide utility for robots, but remain prone to confidently hallucinated predictions.
KITE: Keypoint-Conditioned Policies for Semantic Manipulation
While natural language offers a convenient shared interface for humans and robots, enabling robots to interpret and follow language commands remains a longstanding challenge in manipulation.
Language to Rewards for Robotic Skill Synthesis
However, since low-level robot actions are hardware-dependent and underrepresented in LLM training corpora, existing efforts in applying LLMs to robotics have largely treated LLMs as semantic planners or relied on human-engineered control primitives to interface with the robot.
Toward Grounded Social Reasoning
We additionally illustrate our approach with a robot on 2 carefully designed surfaces.
Generating Language Corrections for Teaching Physical Control Tasks
AI assistance continues to help advance applications in education, from language learning to intelligent tutoring systems, yet current methods for providing students feedback are still quite limited.
Data Quality in Imitation Learning
In this work, we take the first step toward formalizing data quality for imitation learning through the lens of distribution shift: a high quality dataset encourages the policy to stay in distribution at test time.
Strategic Reasoning with Language Models
Existing approaches to solving strategic games rely on extensive training, yielding strategies that do not generalize to new scenarios or games without retraining.
Parallel Sampling of Diffusion Models
Instead of reducing the number of denoising steps (trading quality for speed), in this paper we explore an orthogonal approach: can we run the denoising steps in parallel (trading compute for speed)?
Inverse Preference Learning: Preference-based RL without a Reward Function
Using this insight, we completely eliminate the need for a learned reward function.
Distance Weighted Supervised Learning for Offline Interaction Data
To bridge the gap between IL and RL, we introduce Distance Weighted Supervised Learning or DWSL, a supervised method for learning goal-conditioned policies from offline data.
Behavior Retrieval: Few-Shot Imitation Learning by Querying Unlabeled Datasets
Concretely, we propose a simple approach that uses a small amount of downstream expert data to selectively query relevant behaviors from an offline, unlabeled dataset (including many sub-optimal behaviors).
Language Instructed Reinforcement Learning for Human-AI Coordination
One of the fundamental quests of AI is to produce agents that coordinate well with humans.
Multi-agent Reinforcement Learning
reinforcement-learning
+1
Reward Design with Language Models
During training, the LLM evaluates an RL agent's behavior against the desired behavior described by the prompt and outputs a corresponding reward signal.
Active Reward Learning from Online Preferences
Robot policies need to adapt to human preferences and/or new environments.
Language-Driven Representation Learning for Robotics
First, we demonstrate that existing representations yield inconsistent results across these tasks: masked autoencoding approaches pick up on low-level spatial features at the cost of high-level semantics, while contrastive learning approaches capture the opposite.
Long Horizon Temperature Scaling
LHTS is compatible with all likelihood-based models, and optimizes for the long-horizon likelihood of samples.
"No, to the Right" -- Online Language Corrections for Robotic Manipulation via Shared Autonomy
Instead of discrete turn-taking between a human and robot, LILAC splits agency between the human and robot: language is an input to a learned model that produces a meaningful, low-dimensional control space that the human can use to guide the robot.
Few-Shot Preference Learning for Human-in-the-Loop RL
Contrary to most works that focus on query selection to \emph{minimize} the amount of data required for learning reward functions, we take an opposite approach: \emph{expanding} the pool of available data by viewing human-in-the-loop RL through the more flexible lens of multi-task learning.
Learning Visuo-Haptic Skewering Strategies for Robot-Assisted Feeding
Acquiring food items with a fork poses an immense challenge to a robot-assisted feeding system, due to the wide range of material properties and visual appearances present across food groups.
Learning Bimanual Scooping Policies for Food Acquisition
In order to acquire foods with such diverse properties, we propose stabilizing food items during scooping using a second arm, for example, by pushing peas against the spoon with a flat surface to prevent dispersion.
Assistive Teaching of Motor Control Tasks to Humans
In this paper, we focus on the problem of assistive teaching of motor control tasks such as parking a car or landing an aircraft.
Learning Tool Morphology for Contact-Rich Manipulation Tasks with Differentiable Simulation
We demonstrate the effectiveness of our method for designing new tools in several scenarios, such as winding ropes, flipping a box and pushing peas onto a scoop in simulation.
Eliciting Compatible Demonstrations for Multi-Human Imitation Learning
Imitation learning from human-provided demonstrations is a strong approach for learning policies for robot manipulation.
Masked Imitation Learning: Discovering Environment-Invariant Modalities in Multimodal Demonstrations
Specifically, we design a masked policy network with a binary mask to block certain modalities.
Training and Inference on Any-Order Autoregressive Models the Right Way
Conditional inference on arbitrary subsets of variables is a core problem in probabilistic inference with important applications such as masked language modeling and image inpainting.
Leveraging Smooth Attention Prior for Multi-Agent Trajectory Prediction
At a certain time, to forecast a reasonable future trajectory, each agent needs to pay attention to the interactions with only a small group of most relevant agents instead of unnecessarily paying attention to all the other agents.
Weakly Supervised Correspondence Learning
Correspondence learning is a fundamental problem in robotics, which aims to learn a mapping between state, action pairs of agents of different dynamics or embodiments.
Learning from Imperfect Demonstrations via Adversarial Confidence Transfer
Existing learning from demonstration algorithms usually assume access to expert demonstrations.
Imitation Learning by Estimating Expertise of Demonstrators
In this work, we show that unsupervised learning over demonstrator expertise can lead to a consistent boost in the performance of imitation learning algorithms.
Conditional Imitation Learning for Multi-Agent Games
In this work, we study the problem of conditional multi-agent imitation learning, where we have access to joint trajectory demonstrations at training time, and we must interact with and adapt to new partners at test time.
PantheonRL: A MARL Library for Dynamic Training Interactions
We present PantheonRL, a multiagent reinforcement learning software package for dynamic training interactions such as round-robin, adaptive, and ad-hoc training.
HyperSPNs: Compact and Expressive Probabilistic Circuits
Probabilistic circuits (PCs) are a family of generative models which allows for the computation of exact likelihoods and marginals of its probability distributions.
LILA: Language-Informed Latent Actions
We introduce Language-Informed Latent Actions (LILA), a framework for learning natural language interfaces in the context of human-robot collaboration.
From Machine Learning to Robotics: Challenges and Opportunities for Embodied Intelligence
Machine learning has long since become a keystone technology, accelerating science and applications in a broad range of domains.
Learning Feasibility to Imitate Demonstrators with Different Dynamics
The goal of learning from demonstrations is to learn a policy for an agent (imitator) by mimicking the behavior in the demonstrations.
Confidence-Aware Imitation Learning from Demonstrations with Varying Optimality
Our results show that CAIL significantly outperforms other imitation learning methods from demonstrations with varying optimality.
Open-domain clarification question generation without question examples
An overarching goal of natural language processing is to enable machines to communicate seamlessly with humans.
Influencing Towards Stable Multi-Agent Interactions
Instead of reactively adapting to the other agent's (opponent or partner) behavior, we propose an algorithm to proactively influence the other agent's strategy to stabilize -- which can restrain the non-stationarity caused by the other agent.
Partner-Aware Algorithms in Decentralized Cooperative Bandit Teams
Our results show that the proposed partner-aware strategy outperforms other known methods, and our human subject studies suggest humans prefer to collaborate with AI agents implementing our partner-aware strategy.
Learning Reward Functions from Scale Feedback
Today's robots are increasingly interacting with people and need to efficiently learn inexperienced user's preferences.
Learning Multimodal Rewards from Rankings
However, expert feedback is often assumed to be drawn from an underlying unimodal reward function.
APReL: A Library for Active Preference-based Reward Learning Algorithms
Reward learning is a fundamental problem in human-robot interaction to have robots that operate in alignment with what their human user wants.
On the Opportunities and Risks of Foundation Models
AI is undergoing a paradigm shift with the rise of models (e. g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks.
Targeted Data Acquisition for Evolving Negotiation Agents
This trend additionally holds when comparing agents using our targeted data acquisition framework to variants of agents trained with a mix of supervised learning and reinforcement learning, or to agents using tailored reward functions that explicitly optimize for utility and Pareto-optimality.
Emergent Prosociality in Multi-Agent Games Through Gifting
Coordination is often critical to forming prosocial behaviors -- behaviors that increase the overall sum of rewards received by all agents in a multi-agent game.
Incentivizing Efficient Equilibria in Traffic Networks with Mixed Autonomy
Traffic congestion has large economic and social costs.
Learning Visually Guided Latent Actions for Assistive Teleoperation
In this work, we develop assistive robots that condition their latent embeddings on visual inputs.
On the Critical Role of Conventions in Adaptive Human-AI Collaboration
Humans can quickly adapt to new partners in collaborative tasks (e. g. playing basketball), because they understand which fundamental skills of the task (e. g. how to dribble, how to shoot) carry over across new partners.
Learning from Imperfect Demonstrations from Agents with Varying Dynamics
The proposed score enables learning from more informative demonstrations, and disregarding the less relevant demonstrations.
ELLA: Exploration through Learned Language Abstraction
Building agents capable of understanding language instructions is critical to effective and robust human-AI collaboration.
Transfer Reinforcement Learning across Homotopy Classes
The ability for robots to transfer their learned knowledge to new tasks -- where data is scarce -- is a fundamental challenge for successful robot learning.
Transfer Reinforcement Learning
Robotics
Incentivizing Routing Choices for Safe and Efficient Transportation in the Face of the COVID-19 Pandemic
In turn, significant increases in traffic congestion are expected, since people are likely to prefer using their own vehicles or taxis as opposed to riskier and more crowded options such as the railway.
Learning Latent Representations to Influence Multi-Agent Interaction
We propose a reinforcement learning-based framework for learning latent representations of an agent's policy, where the ego agent identifies the relationship between its behavior and the other agent's future strategy.
ROIAL: Region of Interest Active Learning for Characterizing Exoskeleton Gait Preference Landscapes
ROIAL learns Bayesian posteriors that predict each exoskeleton user's utility landscape across four exoskeleton gait parameters.
Learning Adaptive Language Interfaces through Decomposition
Our goal is to create an interactive natural language interface that efficiently and reliably learns from users to complete tasks in simulated robotics settings.
Multi-Agent Safe Planning with Gaussian Processes
Multi-agent safe systems have become an increasingly important area of study as we can now easily have multiple AI-powered systems operating together.
Learning User-Preferred Mappings for Intuitive Robot Control
Existing approaches to teleoperation typically assume a one-size-fits-all approach, where the designers pre-define a mapping between human inputs and robot actions, and every user must adapt to this mapping over repeated interactions.
Reinforcement Learning based Control of Imitative Policies for Near-Accident Driving
To address driving in near-accident scenarios, we propose a hierarchical reinforcement and imitation learning (H-ReIL) approach that consists of low-level policies learned by IL for discrete driving modes, and a high-level policy learned by RL that switches between different driving modes.
Learning Reward Functions from Diverse Sources of Human Feedback: Optimally Integrating Demonstrations and Preferences
As designing reward functions can be extremely challenging, a more promising approach is to directly learn reward functions from human teachers.
Dynamic Multi-Robot Task Allocation under Uncertainty and Temporal Constraints
We consider the problem of dynamically allocating tasks to multiple agents under time window constraints and task completion uncertainty.
Active Preference-Based Gaussian Process Regression for Reward Learning
Our results in simulations and a user study suggest that our approach can efficiently learn expressive reward functions for robotics tasks.
BLEU Neighbors: A Reference-less Approach to Automatic Evaluation
To this end, we propose BLEU Neighbors, a nearest neighbors model for estimating language quality by using the BLEU score as a kernel function.
Exchangeable Input Representations for Reinforcement Learning
We show that our proposed representation results in an input space that is a factor of $m!$ smaller for inputs of $m$ objects.
When Humans Aren't Optimal: Robots that Collaborate with Risk-Aware Humans
Overall, we extend existing rational human models so that collaborative robots can anticipate and plan around suboptimal human behavior during HRI.
Continual adaptation for efficient machine communication
To communicate with new partners in new contexts, humans rapidly form new linguistic conventions.
Learning from My Partner's Actions: Roles in Decentralized Robot Teams
When teams of robots collaborate to complete a task, communication is often necessary.
Asking Easy Questions: A User-Friendly Approach to Active Reward Learning
Robots can learn the right reward function by querying a human expert.
Controlling Assistive Robots with Learned Latent Actions
Our insight is that we can make assistive robots easier for humans to control by leveraging latent actions.
Robotics
Learning Reward Functions by Integrating Human Demonstrations and Preferences
In a user study, we compare our method to a standard IRL method; we find that users rated the robot trained with DemPref as being more successful at learning their desired behavior, and preferred to use the DemPref system (over IRL) to train the robot.
Batch Active Learning Using Determinantal Point Processes
While active learning methods attempt to tackle this issue by labeling only the data samples that give high information, they generally suffer from large computational costs and are impractical in settings where data can be collected in parallel.
Deep Local Trajectory Replanning and Control for Robot Navigation
We present a navigation system that combines ideas from hierarchical planning and machine learning.
Object Exchangeability in Reinforcement Learning: Extended Abstract
Although deep reinforcement learning has advanced significantly over the past several years, sample efficiency remains a major challenge.
Efficient and Safe Exploration in Deterministic Markov Decision Processes with Unknown Transition Models
We propose a safe exploration algorithm for deterministic Markov Decision Processes with unknown transition models.
Unsupervised Visuomotor Control through Distributional Planning Networks
While reinforcement learning (RL) has the potential to enable robots to autonomously acquire a wide range of skills, in practice, RL usually requires manual, per-task engineering of reward functions, especially in real world settings where aspects of the environment needed to compute progress are not directly accessible.
Hierarchical Game-Theoretic Planning for Autonomous Vehicles
This mutual dependence, best captured by dynamic game theory, creates a strong coupling between the vehicle's planning and its predictions of other drivers' behavior, and constitutes an open problem with direct implications on the safety and viability of autonomous driving technology.
Batch Active Preference-Based Learning of Reward Functions
Data generation and labeling are usually an expensive part of learning for robotics.
Multi-Agent Generative Adversarial Imitation Learning
Imitation learning algorithms can be used to learn a policy from expert demonstrations without access to a reward signal.
Towards Verified Artificial Intelligence
Verified artificial intelligence (AI) is the goal of designing AI-based systems that that have strong, ideally provable, assurances of correctness with respect to mathematically-specified requirements.
Safe Control under Uncertainty
In this paper, we propose a new logic, Probabilistic Signal Temporal Logic (PrSTL), as an expressive language to define the stochastic properties, and enforce probabilistic guarantees on them.
Robust Subspace System Identification via Weighted Nuclear Norm Optimization
As in robust PCA, it can be problematic to find a suitable regularization parameter.
