Search Results for author:
Found 33 papers, 10 papers with code
Reward Learning from Suboptimal Demonstrations with Applications in Surgical Electrocautery
This paper introduces a sample-efficient method that learns a robust reward function from a limited amount of ranked suboptimal demonstrations consisting of partial-view point cloud observations.
Exploring Behavior Discovery Methods for Heterogeneous Swarms of Limited-Capability Robots
We study the problem of determining the emergent behaviors that are possible given a functionally heterogeneous swarm of robots with limited capabilities.
Quantifying Assistive Robustness Via the Natural-Adversarial Frontier
We propose that capturing robustness in these interactive settings requires constructing and analyzing the entire natural-adversarial frontier: the Pareto-frontier of human policies that are the best trade-offs between naturalness and low robot performance.
Contextual Reliability: When Different Features Matter in Different Contexts
We formalize a new setting called contextual reliability which accounts for the fact that the "right" features to use may vary depending on the context.
Can Differentiable Decision Trees Learn Interpretable Reward Functions?
There is an increasing interest in learning reward functions that model human preferences.
Leveraging Human Feedback to Evolve and Discover Novel Emergent Behaviors in Robot Swarms
We combine our learned similarity metric with novelty search and clustering to explore and categorize the space of possible swarm behaviors.
Efficient Preference-Based Reinforcement Learning Using Learned Dynamics Models
In particular, we provide evidence that a learned dynamics model offers the following benefits when performing PbRL: (1) preference elicitation and policy optimization require significantly fewer environment interactions than model-free PbRL, (2) diverse preference queries can be synthesized safely and efficiently as a byproduct of standard model-based RL, and (3) reward pre-training based on suboptimal demonstrations can be performed without any environmental interaction.
Benchmarks and Algorithms for Offline Preference-Based Reward Learning
Learning a reward function from human preferences is challenging as it typically requires having a high-fidelity simulator or using expensive and potentially unsafe actual physical rollouts in the environment.
SIRL: Similarity-based Implicit Representation Learning
This, in turn, is what enables the robot to disambiguate between what needs to go into the representation versus what is spurious, as well as what aspects of behavior can be compressed together versus not.
Learning Representations that Enable Generalization in Assistive Tasks
We advocate that generalization to such OOD policies benefits from (1) learning a good latent representation for human policies that test-time humans can accurately be mapped to, and (2) making that representation adaptable with test-time interaction data, instead of relying on it to perfectly capture the space of human policies based on the simulated population only.
Autonomous Assessment of Demonstration Sufficiency via Bayesian Inverse Reinforcement Learning
We evaluate our approach in simulation for both discrete and continuous state-space domains and illustrate the feasibility of developing a robotic system that can accurately evaluate demonstration sufficiency.
Monte Carlo Augmented Actor-Critic for Sparse Reward Deep Reinforcement Learning from Suboptimal Demonstrations
Providing densely shaped reward functions for RL algorithms is often exceedingly challenging, motivating the development of RL algorithms that can learn from easier-to-specify sparse reward functions.
The Effect of Modeling Human Rationality Level on Learning Rewards from Multiple Feedback Types
In this work, we advocate that grounding the rationality coefficient in real data for each feedback type, rather than assuming a default value, has a significant positive effect on reward learning.
Causal Confusion and Reward Misidentification in Preference-Based Reward Learning
While much prior work focuses on causal confusion in reinforcement learning and behavioral cloning, we focus on a systematic study of causal confusion and reward misidentification when learning from preferences.
Teaching Robots to Span the Space of Functional Expressive Motion
As a result 1) all user feedback can contribute to learning about every emotion; 2) the robot can generate trajectories for any emotion in the space instead of only a few predefined ones; and 3) the robot can respond emotively to user-generated natural language by mapping it to a target VAD.
ThriftyDAgger: Budget-Aware Novelty and Risk Gating for Interactive Imitation Learning
Effective robot learning often requires online human feedback and interventions that can cost significant human time, giving rise to the central challenge in interactive imitation learning: is it possible to control the timing and length of interventions to both facilitate learning and limit burden on the human supervisor?
Offline Preference-Based Apprenticeship Learning
Learning a reward function from human preferences is challenging as it typically requires having a high-fidelity simulator or using expensive and potentially unsafe actual physical rollouts in the environment.
Kit-Net: Self-Supervised Learning to Kit Novel 3D Objects into Novel 3D Cavities
In industrial part kitting, 3D objects are inserted into cavities for transportation or subsequent assembly.
Policy Gradient Bayesian Robust Optimization for Imitation Learning
Results suggest that PG-BROIL can produce a family of behaviors ranging from risk-neutral to risk-averse and outperforms state-of-the-art imitation learning algorithms when learning from ambiguous demonstrations by hedging against uncertainty, rather than seeking to uniquely identify the demonstrator's reward function.
Optimal Cost Design for Model Predictive Control
We test our approach in an autonomous driving domain where we find costs different from the ground truth that implicitly compensate for replanning, short horizon, incorrect dynamics models, and local minima issues.
Situational Confidence Assistance for Lifelong Shared Autonomy
Shared autonomy enables robots to infer user intent and assist in accomplishing it.
LazyDAgger: Reducing Context Switching in Interactive Imitation Learning
Corrective interventions while a robot is learning to automate a task provide an intuitive method for a human supervisor to assist the robot and convey information about desired behavior.
Dynamically Switching Human Prediction Models for Efficient Planning
As environments involving both robots and humans become increasingly common, so does the need to account for people during planning.
Value Alignment Verification
In this paper we formalize and theoretically analyze the problem of efficient value alignment verification: how to efficiently test whether the behavior of another agent is aligned with a human's values.
Exploratory Grasping: Asymptotically Optimal Algorithms for Grasping Challenging Polyhedral Objects
However, these policies can consistently fail to grasp challenging objects which are significantly out of the distribution of objects in the training data or which have very few high quality grasps.
Bayesian Robust Optimization for Imitation Learning
Existing safe imitation learning approaches based on IRL deal with this uncertainty using a maxmin framework that optimizes a policy under the assumption of an adversarial reward function, whereas risk-neutral IRL approaches either optimize a policy for the mean or MAP reward function.
Safe Imitation Learning via Fast Bayesian Reward Inference from Preferences
Bayesian REX can learn to play Atari games from demonstrations, without access to the game score and can generate 100, 000 samples from the posterior over reward functions in only 5 minutes on a personal laptop.
Deep Bayesian Reward Learning from Preferences
Bayesian inverse reinforcement learning (IRL) methods are ideal for safe imitation learning, as they allow a learning agent to reason about reward uncertainty and the safety of a learned policy.
Better-than-Demonstrator Imitation Learning via Automatically-Ranked Demonstrations
The performance of imitation learning is typically upper-bounded by the performance of the demonstrator.
Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from Observations
A critical flaw of existing inverse reinforcement learning (IRL) methods is their inability to significantly outperform the demonstrator.
Risk-Aware Active Inverse Reinforcement Learning
Active learning from demonstration allows a robot to query a human for specific types of input to achieve efficient learning.
Machine Teaching for Inverse Reinforcement Learning: Algorithms and Applications
Inverse reinforcement learning (IRL) infers a reward function from demonstrations, allowing for policy improvement and generalization.
Efficient Probabilistic Performance Bounds for Inverse Reinforcement Learning
In the field of reinforcement learning there has been recent progress towards safety and high-confidence bounds on policy performance.
