Search Results for author: Benjamin Eysenbach

Found 50 papers, 28 papers with code

Learning Temporal Distances: Contrastive Successor Features Can Provide a Metric Structure for Decision-Making

1 code implementation24 Jun 2024 Vivek Myers, Chongyi Zheng, Anca Dragan, Sergey Levine, Benjamin Eysenbach

In this paper, we build on prior work in contrastive learning and quasimetrics to show how successor features learned by contrastive learning (after a change of variables) form a temporal distance that does satisfy the triangle inequality, even in stochastic settings.

Contrastive Learning Decision Making

A Rate-Distortion View of Uncertainty Quantification

1 code implementation16 Jun 2024 Ifigeneia Apostolopoulou, Benjamin Eysenbach, Frank Nielsen, Artur Dubrawski

In this paper, we introduce Distance Aware Bottleneck (DAB), i. e., a new method for enriching deep neural networks with this property.

Gaussian Processes Out of Distribution (OOD) Detection +1

Closing the Gap between TD Learning and Supervised Learning -- A Generalisation Point of View

1 code implementation20 Jan 2024 Raj Ghugare, Matthieu Geist, Glen Berseth, Benjamin Eysenbach

Based on this analysis, we construct new datasets to explicitly test for this property, revealing that SL-based methods lack this stitching property and hence fail to perform combinatorial generalization.

Data Augmentation Reinforcement Learning (RL)

Contrastive Difference Predictive Coding

1 code implementation31 Oct 2023 Chongyi Zheng, Ruslan Salakhutdinov, Benjamin Eysenbach

Predicting and reasoning about the future lie at the heart of many time-series questions.

Representation Learning Time Series

Contrastive Example-Based Control

1 code implementation24 Jul 2023 Kyle Hatch, Benjamin Eysenbach, Rafael Rafailov, Tianhe Yu, Ruslan Salakhutdinov, Sergey Levine, Chelsea Finn

In this paper, we propose a method for offline, example-based control that learns an implicit model of multi-step transitions, rather than a reward function.

Offline RL

A Connection between One-Step Regularization and Critic Regularization in Reinforcement Learning

1 code implementation24 Jul 2023 Benjamin Eysenbach, Matthieu Geist, Sergey Levine, Ruslan Salakhutdinov

One-step methods perform regularization by doing just a single step of policy improvement, while critic regularization methods do many steps of policy improvement with a regularized objective.

Offline RL reinforcement-learning

Game-Theoretic Robust Reinforcement Learning Handles Temporally-Coupled Perturbations

no code implementations22 Jul 2023 Yongyuan Liang, Yanchao Sun, Ruijie Zheng, Xiangyu Liu, Benjamin Eysenbach, Tuomas Sandholm, Furong Huang, Stephen Mcaleer

To tackle this challenge, we propose GRAD, a novel game-theoretic approach that treats the temporally-coupled robust RL problem as a partially observable two-player zero-sum game.

Continuous Control reinforcement-learning +1

HIQL: Offline Goal-Conditioned RL with Latent States as Actions

1 code implementation NeurIPS 2023 Seohong Park, Dibya Ghosh, Benjamin Eysenbach, Sergey Levine

This structure can be very useful, as assessing the quality of actions for nearby goals is typically easier than for more distant goals.

Reinforcement Learning (RL) Unsupervised Pre-training

When Do Transformers Shine in RL? Decoupling Memory from Credit Assignment

2 code implementations NeurIPS 2023 Tianwei Ni, Michel Ma, Benjamin Eysenbach, Pierre-Luc Bacon

The Transformer architecture has been very successful to solve problems that involve long-term dependencies, including in the RL domain.

Reinforcement Learning (RL)

Stabilizing Contrastive RL: Techniques for Robotic Goal Reaching from Offline Data

1 code implementation6 Jun 2023 Chongyi Zheng, Benjamin Eysenbach, Homer Walke, Patrick Yin, Kuan Fang, Ruslan Salakhutdinov, Sergey Levine

Robotic systems that rely primarily on self-supervised learning have the potential to decrease the amount of human annotation and engineering effort required to learn control strategies.

Contrastive Learning Data Augmentation +2

Bitrate-Constrained DRO: Beyond Worst Case Robustness To Unknown Group Shifts

1 code implementation6 Feb 2023 Amrith Setlur, Don Dennis, Benjamin Eysenbach, aditi raghunathan, Chelsea Finn, Virginia Smith, Sergey Levine

Some robust training algorithms (e. g., Group DRO) specialize to group shifts and require group information on all training points.

Learning Options via Compression

1 code implementation8 Dec 2022 Yiding Jiang, Evan Zheran Liu, Benjamin Eysenbach, Zico Kolter, Chelsea Finn

Identifying statistical regularities in solutions to some tasks in multi-task reinforcement learning can accelerate the learning of new tasks.

Contrastive Value Learning: Implicit Models for Simple Offline RL

no code implementations3 Nov 2022 Bogdan Mazoure, Benjamin Eysenbach, Ofir Nachum, Jonathan Tompson

In this paper, we propose Contrastive Value Learning (CVL), which learns an implicit, multi-step model of the environment dynamics.

Continuous Control Model-based Reinforcement Learning +2

Simplifying Model-based RL: Learning Representations, Latent-space Models, and Policies with One Objective

no code implementations18 Sep 2022 Raj Ghugare, Homanga Bharadhwaj, Benjamin Eysenbach, Sergey Levine, Ruslan Salakhutdinov

In this work, we propose a single objective which jointly optimizes a latent-space model and policy to achieve high returns while remaining self-consistent.

Reinforcement Learning (RL) Value prediction

Contrastive Learning as Goal-Conditioned Reinforcement Learning

no code implementations15 Jun 2022 Benjamin Eysenbach, Tianjun Zhang, Ruslan Salakhutdinov, Sergey Levine

While deep RL should automatically acquire such good representations, prior work often finds that learning representations in an end-to-end fashion is unstable and instead equip RL algorithms with additional representation learning parts (e. g., auxiliary losses, data augmentation).

Contrastive Learning Data Augmentation +4

Imitating Past Successes can be Very Suboptimal

no code implementations7 Jun 2022 Benjamin Eysenbach, Soumith Udatha, Sergey Levine, Ruslan Salakhutdinov

Prior work has proposed a simple strategy for reinforcement learning (RL): label experience with the outcomes achieved in that experience, and then imitate the relabeled experience.

Imitation Learning Reinforcement Learning (RL)

Adversarial Unlearning: Reducing Confidence Along Adversarial Directions

no code implementations3 Jun 2022 Amrith Setlur, Benjamin Eysenbach, Virginia Smith, Sergey Levine

Supervised learning methods trained with maximum likelihood objectives often overfit on training data.

Data Augmentation

RvS: What is Essential for Offline RL via Supervised Learning?

1 code implementation20 Dec 2021 Scott Emmons, Benjamin Eysenbach, Ilya Kostrikov, Sergey Levine

Recent work has shown that supervised learning alone, without temporal difference (TD) learning, can be remarkably effective for offline RL.

Offline RL

C-Planning: An Automatic Curriculum for Learning Goal-Reaching Tasks

no code implementations ICLR 2022 Tianjun Zhang, Benjamin Eysenbach, Ruslan Salakhutdinov, Sergey Levine, Joseph E. Gonzalez

Goal-conditioned reinforcement learning (RL) can solve tasks in a wide range of domains, including navigation and manipulation, but learning to reach distant goals remains a central challenge to the field.

Reinforcement Learning (RL)

Recurrent Model-Free RL Can Be a Strong Baseline for Many POMDPs

2 code implementations11 Oct 2021 Tianwei Ni, Benjamin Eysenbach, Ruslan Salakhutdinov

However, prior work has found that such recurrent model-free RL methods tend to perform worse than more specialized algorithms that are designed for specific types of POMDPs.

The Information Geometry of Unsupervised Reinforcement Learning

1 code implementation ICLR 2022 Benjamin Eysenbach, Ruslan Salakhutdinov, Sergey Levine

In this work, we show that unsupervised skill discovery algorithms based on mutual information maximization do not learn skills that are optimal for every possible reward function.

Contrastive Learning reinforcement-learning +3

Mismatched No More: Joint Model-Policy Optimization for Model-Based RL

1 code implementation6 Oct 2021 Benjamin Eysenbach, Alexander Khazatsky, Sergey Levine, Ruslan Salakhutdinov

Many model-based reinforcement learning (RL) methods follow a similar template: fit a model to previously observed data, and then use data from that model for RL or planning.

Model-based Reinforcement Learning Reinforcement Learning (RL)

The Essential Elements of Offline RL via Supervised Learning

no code implementations ICLR 2022 Scott Emmons, Benjamin Eysenbach, Ilya Kostrikov, Sergey Levine

These methods, which we collectively refer to as reinforcement learning via supervised learning (RvS), involve a number of design decisions, such as policy architectures and how the conditioning variable is constructed.

Offline RL reinforcement-learning +1

Robust Predictable Control

1 code implementation NeurIPS 2021 Benjamin Eysenbach, Ruslan Salakhutdinov, Sergey Levine

Many of the challenges facing today's reinforcement learning (RL) algorithms, such as robustness, generalization, transfer, and computational efficiency are closely related to compression.

Computational Efficiency Decision Making +1

Actionable Models: Unsupervised Offline Reinforcement Learning of Robotic Skills

no code implementations15 Apr 2021 Yevgen Chebotar, Karol Hausman, Yao Lu, Ted Xiao, Dmitry Kalashnikov, Jake Varley, Alex Irpan, Benjamin Eysenbach, Ryan Julian, Chelsea Finn, Sergey Levine

We consider the problem of learning useful robotic skills from previously collected offline data without access to manually specified rewards or additional online exploration, a setting that is becoming increasingly important for scaling robot learning by reusing past robotic data.

Q-Learning reinforcement-learning +1

Rapid Exploration for Open-World Navigation with Latent Goal Models

no code implementations12 Apr 2021 Dhruv Shah, Benjamin Eysenbach, Gregory Kahn, Nicholas Rhinehart, Sergey Levine

We describe a robotic learning system for autonomous exploration and navigation in diverse, open-world environments.

Autonomous Navigation

Maximum Entropy RL (Provably) Solves Some Robust RL Problems

no code implementations ICLR 2022 Benjamin Eysenbach, Sergey Levine

Many potential applications of reinforcement learning (RL) require guarantees that the agent will perform well in the face of disturbances to the dynamics or reward function.

Reinforcement Learning (RL)

Model-Based Visual Planning with Self-Supervised Functional Distances

1 code implementation ICLR 2021 Stephen Tian, Suraj Nair, Frederik Ebert, Sudeep Dasari, Benjamin Eysenbach, Chelsea Finn, Sergey Levine

In our experiments, we find that our method can successfully learn models that perform a variety of tasks at test-time, moving objects amid distractors with a simulated robotic arm and even learning to open and close a drawer using a real-world robot.

reinforcement-learning Reinforcement Learning (RL)

ViNG: Learning Open-World Navigation with Visual Goals

no code implementations17 Dec 2020 Dhruv Shah, Benjamin Eysenbach, Gregory Kahn, Nicholas Rhinehart, Sergey Levine

We propose a learning-based navigation system for reaching visually indicated goals and demonstrate this system on a real mobile robot platform.

Navigate reinforcement-learning +1

C-Learning: Learning to Achieve Goals via Recursive Classification

no code implementations ICLR 2021 Benjamin Eysenbach, Ruslan Salakhutdinov, Sergey Levine

This problem, which can be viewed as a reframing of goal-conditioned reinforcement learning (RL), is centered around learning a conditional probability density function over future states.

Classification Density Estimation +3

f-IRL: Inverse Reinforcement Learning via State Marginal Matching

1 code implementation9 Nov 2020 Tianwei Ni, Harshit Sikchi, YuFei Wang, Tejus Gupta, Lisa Lee, Benjamin Eysenbach

Our method outperforms adversarial imitation learning methods in terms of sample efficiency and the required number of expert trajectories on IRL benchmarks.

Imitation Learning reinforcement-learning +1

Learning to be Safe: Deep RL with a Safety Critic

no code implementations27 Oct 2020 Krishnan Srinivasan, Benjamin Eysenbach, Sehoon Ha, Jie Tan, Chelsea Finn

Safety is an essential component for deploying reinforcement learning (RL) algorithms in real-world scenarios, and is critical during the learning process itself.

Reinforcement Learning (RL) Transfer Learning

Interactive Visualization for Debugging RL

no code implementations14 Aug 2020 Shuby Deshpande, Benjamin Eysenbach, Jeff Schneider

Visualization tools for supervised learning allow users to interpret, introspect, and gain an intuition for the successes and failures of their models.

Off-Dynamics Reinforcement Learning: Training for Transfer with Domain Classifiers

1 code implementation ICLR 2021 Benjamin Eysenbach, Swapnil Asawa, Shreyas Chaudhari, Sergey Levine, Ruslan Salakhutdinov

Building off of a probabilistic view of RL, we formally show that we can achieve this goal by compensating for the difference in dynamics by modifying the reward function.

Continuous Control Domain Adaptation +2

Rewriting History with Inverse RL: Hindsight Inference for Policy Improvement

1 code implementation NeurIPS 2020 Benjamin Eysenbach, Xinyang Geng, Sergey Levine, Ruslan Salakhutdinov

In this paper, we show that hindsight relabeling is inverse RL, an observation that suggests that we can use inverse RL in tandem for RL algorithms to efficiently solve many tasks.

Reinforcement Learning (RL)

Learning to Reach Goals via Iterated Supervised Learning

2 code implementations ICLR 2021 Dibya Ghosh, Abhishek Gupta, Ashwin Reddy, Justin Fu, Coline Devin, Benjamin Eysenbach, Sergey Levine

Current reinforcement learning (RL) algorithms can be brittle and difficult to use, especially when learning goal-reaching behaviors from sparse rewards.

Multi-Goal Reinforcement Learning Reinforcement Learning (RL)

If MaxEnt RL is the Answer, What is the Question?

no code implementations4 Oct 2019 Benjamin Eysenbach, Sergey Levine

In particular, we show (1) that MaxEnt RL can be used to solve a certain class of POMDPs, and (2) that MaxEnt RL is equivalent to a two-player game where an adversary chooses the reward function.

Learning to Reach Goals Without Reinforcement Learning

no code implementations25 Sep 2019 Dibya Ghosh, Abhishek Gupta, Justin Fu, Ashwin Reddy, Coline Devin, Benjamin Eysenbach, Sergey Levine

By maximizing the likelihood of good actions provided by an expert demonstrator, supervised imitation learning can produce effective policies without the algorithmic complexities and optimization challenges of reinforcement learning, at the cost of requiring an expert demonstrator -- typically a person -- to provide the demonstrations.

Imitation Learning reinforcement-learning +1

Avoiding Negative Side-Effects and Promoting Safe Exploration with Imaginative Planning

no code implementations25 Sep 2019 Dhruv Ramani, Benjamin Eysenbach

Our imaginative module can be seen as a ``plug-and-play'' approach to ensuring safety, as it is compatible with any existing RL algorithm and any task with discrete action space.

Reinforcement Learning (RL) Safe Exploration

Efficient Exploration via State Marginal Matching

1 code implementation12 Jun 2019 Lisa Lee, Benjamin Eysenbach, Emilio Parisotto, Eric Xing, Sergey Levine, Ruslan Salakhutdinov

The SMM objective can be viewed as a two-player, zero-sum game between a state density model and a parametric policy, an idea that we use to build an algorithm for optimizing the SMM objective.

Efficient Exploration Unsupervised Reinforcement Learning

Search on the Replay Buffer: Bridging Planning and Reinforcement Learning

1 code implementation NeurIPS 2019 Benjamin Eysenbach, Ruslan Salakhutdinov, Sergey Levine

We introduce a general control algorithm that combines the strengths of planning and reinforcement learning to effectively solve these tasks.

reinforcement-learning Reinforcement Learning (RL)

Unsupervised Meta-Learning for Reinforcement Learning

no code implementations ICLR 2020 Abhishek Gupta, Benjamin Eysenbach, Chelsea Finn, Sergey Levine

In the context of reinforcement learning, meta-learning algorithms acquire reinforcement learning procedures to solve new problems more efficiently by utilizing experience from prior tasks.

Meta-Learning Meta Reinforcement Learning +3

Leave no Trace: Learning to Reset for Safe and Autonomous Reinforcement Learning

1 code implementation ICLR 2018 Benjamin Eysenbach, Shixiang Gu, Julian Ibarz, Sergey Levine

In this work, we propose an autonomous method for safe and efficient reinforcement learning that simultaneously learns a forward and reset policy, with the reset policy resetting the environment for a subsequent attempt.

reinforcement-learning Reinforcement Learning (RL)

Who is Mistaken?

no code implementations4 Dec 2016 Benjamin Eysenbach, Carl Vondrick, Antonio Torralba

We then create a representation of characters' beliefs for two tasks in human action understanding: predicting who is mistaken, and when they are mistaken.

Action Understanding

Cannot find the paper you are looking for? You can Submit a new open access paper.