Search Results for author:
Found 49 papers, 26 papers with code
Automated Discovery of Functional Actual Causes in Complex Environments
This paper introduces functional actual cause (FAC), a framework that uses context-specific independencies in the environment to restrict the set of actual causes.
Learning Action-based Representations Using Invariance
Robust reinforcement learning agents using high-dimensional observations must be able to identify relevant state features amidst many exogeneous distractors.
SMORE: Score Models for Offline Goal-Conditioned Reinforcement Learning
Offline Goal-Conditioned Reinforcement Learning (GCRL) is tasked with learning to achieve multiple goals in an environment purely from offline datasets using sparse reward functions.
Contrastive Preference Learning: Learning from Human Feedback without RL
Thus, learning a reward function from feedback is not only based on a flawed assumption of human preference, but also leads to unwieldy optimization challenges that stem from policy gradients or bootstrapping in the RL phase.
Learning Optimal Advantage from Preferences and Mistaking it for Reward
Most recent work assumes that human preferences are generated based only upon the reward accrued within those segments, or their partial return.
Hierarchical Empowerment: Towards Tractable Empowerment-Based Skill Learning
Our framework makes two specific contributions.
Granger-Causal Hierarchical Skill Discovery
Reinforcement Learning (RL) has demonstrated promising results in learning policies for complex tasks, but it often suffers from low sample efficiency and limited transferability.
Dual RL: Unification and New Methods for Reinforcement and Imitation Learning
For offline RL, our analysis frames a recent offline RL method XQL in the dual framework, and we further propose a new method f-DVL that provides alternative choices to the Gumbel regression loss that fixes the known training instability issue of XQL.
Language-guided Task Adaptation for Imitation Learning
We introduce a novel setting, wherein an agent needs to learn a task from a demonstration of a related task with the difference between the tasks communicated in natural language.
Models of human preference for learning reward functions
We empirically show that our proposed regret preference model outperforms the partial return preference model with finite training data in otherwise the same setting.
Know Your Boundaries: The Necessity of Explicit Behavioral Cloning in Offline RL
In this work, we argue that it is not only viable but beneficial to explicitly model the behavior policy for offline RL because the constraint can be realized in a stable way with the trained model.
Can Foundation Models Perform Zero-Shot Task Specification For Robot Manipulation?
Task specification is at the core of programming autonomous robots.
A Ranking Game for Imitation Learning
We propose a new framework for imitation learning -- treating imitation as a two-player ranking-based game between a policy and a reward.
SOPE: Spectrum of Off-Policy Estimators
In this paper, we present a new perspective on this bias-variance trade-off and show the existence of a spectrum of estimators whose endpoints are SIS and IS.
You Only Evaluate Once: a Simple Baseline Algorithm for Offline RL
The goal of offline reinforcement learning (RL) is to find an optimal policy given prerecorded trajectories.
Fairness Guarantees under Demographic Shift
Recent studies have demonstrated that using machine learning for social applications can lead to injustice in the form of racist, sexist, and otherwise unfair and discriminatory outcomes.
Distributional Depth-Based Estimation of Object Articulation Models
Our core contributions include a novel representation for distributions over rigid body transformations and articulation model parameters based on screw theory, von Mises-Fisher distributions, and Stiefel manifolds.
On the Benefits of Inducing Local Lipschitzness for Robust Generative Adversarial Imitation Learning
We explore methodologies to improve the robustness of generative adversarial imitation learning (GAIL) algorithms to observation noise.
Zero-shot Task Adaptation using Natural Language
Imitation learning and instruction-following are two common approaches to communicate a user's intent to a learning agent.
Adversarial Intrinsic Motivation for Reinforcement Learning
In this paper, we investigate whether one such objective, the Wasserstein-1 distance between a policy's state visitation distribution and a target distribution, can be utilized effectively for reinforcement learning (RL) tasks.
Universal Off-Policy Evaluation
When faced with sequential decision-making problems, it is often useful to be able to predict what would happen if decisions were made using a new policy.
Self-Supervised Online Reward Shaping in Sparse-Reward Environments
We introduce Self-supervised Online Reward Shaping (SORS), which aims to improve the sample efficiency of any RL algorithm in sparse-reward environments by automatically densifying rewards.
Value Alignment Verification
In this paper we formalize and theoretically analyze the problem of efficient value alignment verification: how to efficiently test whether the behavior of another agent is aligned with a human's values.
The EMPATHIC Framework for Task Learning from Implicit Human Feedback
We train a deep neural network on this data and demonstrate its ability to (1) infer relative reward ranking of events in the training task from prerecorded human facial reactions; (2) improve the policy of an agent in the training task using live human facial reactions; and (3) transfer to a novel domain in which it evaluates robot manipulation trajectories.
Human-Computer Interaction Robotics
ScrewNet: Category-Independent Articulation Model Estimation From Depth Images Using Screw Theory
Robots in human environments will need to interact with a wide variety of articulated objects such as cabinets, drawers, and dishwashers while assisting humans in performing day-to-day tasks.
PixL2R: Guiding Reinforcement Learning Using Natural Language by Mapping Pixels to Rewards
Reinforcement learning (RL), particularly in sparse reward settings, often requires prohibitively large numbers of interactions with the environment, thereby limiting its applicability to complex problems.
Bayesian Robust Optimization for Imitation Learning
Existing safe imitation learning approaches based on IRL deal with this uncertainty using a maxmin framework that optimizes a policy under the assumption of an adversarial reward function, whereas risk-neutral IRL approaches either optimize a policy for the mean or MAP reward function.
Efficiently Guiding Imitation Learning Agents with Human Gaze
Based on similarities between the attention of reinforcement learning agents and human gaze, we propose a novel approach for utilizing gaze data in a computationally efficient manner, as part of an auxiliary loss function, which guides a network to have higher activations in image regions where the human's gaze fixated.
Safe Imitation Learning via Fast Bayesian Reward Inference from Preferences
Bayesian REX can learn to play Atari games from demonstrations, without access to the game score and can generate 100, 000 samples from the posterior over reward functions in only 5 minutes on a personal laptop.
Local Nonparametric Meta-Learning
A central goal of meta-learning is to find a learning rule that enables fast adaptation across a set of tasks, by learning the appropriate inductive bias for that set.
Deep Bayesian Reward Learning from Preferences
Bayesian inverse reinforcement learning (IRL) methods are ideal for safe imitation learning, as they allow a learning agent to reason about reward uncertainty and the safety of a learned policy.
Better-than-Demonstrator Imitation Learning via Automatically-Ranked Demonstrations
The performance of imitation learning is typically upper-bounded by the performance of the demonstrator.
A Review of Robot Learning for Manipulation: Challenges, Representations, and Algorithms
A key challenge in intelligent robotics is creating robots that are capable of directly interacting with the world around them to achieve their goals.
Robotics
Hypothesis-Driven Skill Discovery for Hierarchical Deep Reinforcement Learning
Deep reinforcement learning (DRL) is capable of learning high-performing policies on a variety of complex high-dimensional tasks, ranging from video games to robotic manipulation.
Uncertainty-Aware Data Aggregation for Deep Imitation Learning
Our analysis shows that UAIL outperforms existing data aggregation algorithms on a series of benchmark tasks.
Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from Observations
A critical flaw of existing inverse reinforcement learning (IRL) methods is their inability to significantly outperform the demonstrator.
Using Natural Language for Reward Shaping in Reinforcement Learning
A common approach to reduce interaction time with the environment is to use reward shaping, which involves carefully designing reward functions that provide the agent intermediate rewards for progress towards the goal.
Risk-Aware Active Inverse Reinforcement Learning
Active learning from demonstration allows a robot to query a human for specific types of input to achieve efficient learning.
One-Shot Learning of Multi-Step Tasks from Observation via Activity Localization in Auxiliary Video
Due to burdensome data requirements, learning from demonstration often falls short of its promise to allow users to quickly and naturally program robots.
Importance Sampling Policy Evaluation with an Estimated Behavior Policy
We find that this estimator often lowers the mean squared error of off-policy evaluation compared to importance sampling with the true behavior policy or using a behavior policy that is estimated from a separate data set.
Machine Teaching for Inverse Reinforcement Learning: Algorithms and Applications
Inverse reinforcement learning (IRL) infers a reward function from demonstrations, allowing for policy improvement and generalization.
Efficient Hierarchical Robot Motion Planning Under Uncertainty and Hybrid Dynamics
This hierarchical planning approach results in a decomposition of the POMDP planning problem into smaller sub-parts that can be solved with significantly lower computational costs.
Safe Reinforcement Learning via Shielding
In the first one, the shield acts each time the learning agent is about to make a decision and provides a list of safe actions.
Efficient Probabilistic Performance Bounds for Inverse Reinforcement Learning
In the field of reinforcement learning there has been recent progress towards safety and high-confidence bounds on policy performance.
Data-Efficient Policy Evaluation Through Behavior Policy Search
The standard unbiased technique for evaluating a policy is to deploy the policy and observe its performance.
Bootstrapping with Models: Confidence Intervals for Off-Policy Evaluation
In this context, we propose two bootstrapping off-policy evaluation methods which use learned MDP transition models in order to estimate lower confidence bounds on policy performance with limited data in both continuous and discrete state spaces.
Policy Evaluation Using the Ω-Return
The benefit of the Ω-return is that it accounts for the correlation of different length returns.
Clustering via Dirichlet Process Mixture Models for Portable Skill Discovery
Skill discovery algorithms in reinforcement learning typically identify single states or regions in state space that correspond to task-specific subgoals.
TD_gamma: Re-evaluating Complex Backups in Temporal Difference Learning
We show that the lambda-return target used in the TD(lambda) family of algorithms is the maximum likelihood estimator for a specific model of how the variance of an n-step return estimate increases with n. We introduce the gamma-return estimator, an alternative target based on a more accurate model of variance, which defines the TD_gamma family of complex-backup temporal difference learning algorithms.
