Search Results for author:
Found 31 papers, 17 papers with code
When Should We Prefer Offline Reinforcement Learning Over Behavioral Cloning?
To answer this question, we characterize the properties of environments that allow offline RL methods to perform better than BC methods, even when only provided with expert data.
Design-Bench: Benchmarks for Data-Driven Offline Model-Based Optimization
To address this, we present Design-Bench, a benchmark for offline MBO with a unified evaluation protocol and reference implementations of recent methods.
How to Leverage Unlabeled Data in Offline Reinforcement Learning
One natural solution is to learn a reward function from the labeled data and use it to label the unlabeled data.
DR3: Value-Based Deep Reinforcement Learning Requires Explicit Regularization
In this paper, we discuss how the implicit regularization effect of SGD seen in supervised learning could in fact be harmful in the offline deep RL setting, leading to poor generalization and degenerate feature representations.
Data-Driven Offline Optimization For Architecting Hardware Accelerators
An alternative paradigm is to use a "data-driven", offline approach that utilizes logged simulation data, to architect hardware accelerators, without needing any form of simulations.
Should I Run Offline Reinforcement Learning or Behavioral Cloning?
In this paper, our goal is to characterize environments and dataset compositions where offline RL leads to better performance than BC.
Data Sharing without Rewards in Multi-Task Offline Reinforcement Learning
However, these benefits come at a cost -- for data to be shared between tasks, each transition must be annotated with reward labels corresponding to other tasks.
A Workflow for Offline Model-Free Robotic Reinforcement Learning
To this end, we devise a set of metrics and conditions that can be tracked over the course of offline training, and can inform the practitioner about how the algorithm and model architecture should be adjusted to improve final performance.
Conservative Data Sharing for Multi-Task Offline Reinforcement Learning
We argue that a natural use case of offline RL is in settings where we can pool large amounts of data collected in various scenarios for solving different tasks, and utilize all of this data to learn behaviors for all the tasks more effectively rather than training each one in isolation.
Conservative Objective Models for Effective Offline Model-Based Optimization
Computational design problems arise in a number of settings, from synthetic biology to computer architectures.
Why Generalization in RL is Difficult: Epistemic POMDPs and Implicit Partial Observability
Generalization is a central challenge for the deployment of reinforcement learning (RL) systems in the real world.
Benchmarks for Deep Off-Policy Evaluation
Off-policy evaluation (OPE) holds the promise of being able to leverage large, offline datasets for both evaluating and selecting complex policies for decision making.
COMBO: Conservative Offline Model-Based Policy Optimization
We overcome this limitation by developing a new model-based offline RL algorithm, COMBO, that regularizes the value function on out-of-support state-action tuples generated via rollouts under the learned model.
One Solution is Not All You Need: Few-Shot Extrapolation via Structured MaxEnt RL
While reinforcement learning algorithms can learn effective policies for complex tasks, these policies are often brittle to even minor task variations, especially when variations are not explicitly provided during training.
Conservative Safety Critics for Exploration
Safe exploration presents a major challenge in reinforcement learning (RL): when active data collection requires deploying partially trained policies, we must ensure that these policies avoid catastrophically unsafe regions, while still enabling trial and error learning.
Implicit Under-Parameterization Inhibits Data-Efficient Deep Reinforcement Learning
We identify an implicit under-parameterization phenomenon in value-based deep RL methods that use bootstrapping: when value functions, approximated using deep neural networks, are trained with gradient descent using iterated regression onto target values generated by previous instances of the value network, more gradient updates decrease the expressivity of the current value network.
COG: Connecting New Skills to Past Experience with Offline Reinforcement Learning
Reinforcement learning has been applied to a wide variety of robotics problems, but most of such applications involve collecting data from scratch for each new task.
OPAL: Offline Primitive Discovery for Accelerating Offline Reinforcement Learning
Reinforcement learning (RL) has achieved impressive performance in a variety of online settings in which an agent's ability to query the environment for transitions and rewards is effectively unlimited.
Conservative Q-Learning for Offline Reinforcement Learning
We theoretically show that CQL produces a lower bound on the value of the current policy and that it can be incorporated into a policy learning procedure with theoretical improvement guarantees.
Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems
In this tutorial article, we aim to provide the reader with the conceptual tools needed to get started on research on offline reinforcement learning algorithms: reinforcement learning algorithms that utilize previously collected data, without additional online data collection.
D4RL: Datasets for Deep Data-Driven Reinforcement Learning
In this work, we introduce benchmarks specifically designed for the offline setting, guided by key properties of datasets relevant to real-world applications of offline RL.
DisCor: Corrective Feedback in Reinforcement Learning via Distribution Correction
We show that bootstrapping-based Q-learning algorithms do not necessarily benefit from this corrective feedback, and training on the experience collected by the algorithm is not sufficient to correct errors in the Q-function.
Ranked #3 on
Meta-Learning
on MT50
Model Inversion Networks for Model-Based Optimization
MINs can scale to high-dimensional input spaces and leverage offline logged data for both contextual and non-contextual optimization problems.
Reward-Conditioned Policies
By then conditioning the policy on the numerical value of the reward, we can obtain a policy that generalizes to larger returns.
Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learning
In this paper, we aim to develop a simple and scalable reinforcement learning algorithm that uses standard supervised learning methods as subroutines.
Ranked #1 on
OpenAI Gym
on Humanoid-v2
Advantage Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learning
In this paper, we aim to develop a simple and scalable reinforcement learning algorithm that uses standard supervised learning methods as subroutines.
Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reduction
Bootstrapping error is due to bootstrapping from actions that lie outside of the training data distribution, and it accumulates via the Bellman backup operator.
Graph Normalizing Flows
We introduce graph normalizing flows: a new, reversible graph neural network model for prediction and generation.
Calibration of Encoder Decoder Models for Neural Machine Translation
We study the calibration of several state of the art neural machine translation(NMT) systems built on attention-based encoder-decoder models.
Diagnosing Bottlenecks in Deep Q-learning Algorithms
Q-learning methods represent a commonly used class of algorithms in reinforcement learning: they are generally efficient and simple, and can be combined readily with function approximators for deep reinforcement learning (RL).
Trainable Calibration Measures for Neural Networks from Kernel Mean Embeddings
Modern neural networks have recently been found to be poorly calibrated, primarily in the direction of over-confidence.
