Search Results for author:
Found 50 papers, 20 papers with code
Video as the New Language for Real-World Decision Making
Moreover, we demonstrate how, like language models, video generation can serve as planners, agents, compute engines, and environment simulators through techniques such as in-context learning, planning and reinforcement learning.
Rainbow Teaming: Open-Ended Generation of Diverse Adversarial Prompts
As large language models (LLMs) become increasingly prevalent across many real-world applications, understanding and enhancing their robustness to user inputs is of paramount importance.
Genie: Generative Interactive Environments
We introduce Genie, the first generative interactive environment trained in an unsupervised manner from unlabelled Internet videos.
Multi-Agent Diagnostics for Robustness via Illuminated Diversity
In the rapidly advancing field of multi-agent systems, ensuring robustness in unfamiliar and adversarial settings is crucial.
Vision-Language Models as a Source of Rewards
Building generalist agents that can accomplish many goals in rich open-ended environments is one of the research frontiers for reinforcement learning.
Discovering General Reinforcement Learning Algorithms with Adversarial Environment Design
Recently, it has been shown that it is possible to meta-learn update rules, with the hope of discovering algorithms that can perform well on a wide range of RL tasks.
Stabilizing Unsupervised Environment Design with a Learned Adversary
As a result, we make it possible for PAIRED to match or exceed state-of-the-art methods, producing robust agents in several established challenging procedurally-generated environments, including a partially-observed maze navigation task and a continuous-control car racing environment.
Synthetic Experience Replay
We believe that synthetic training data could open the door to realizing the full potential of deep learning for replay-based RL algorithms from limited data.
MAESTRO: Open-Ended Environment Design for Multi-Agent Reinforcement Learning
Open-ended learning methods that automatically generate a curriculum of increasingly challenging tasks serve as a promising avenue toward generally capable reinforcement learning agents.
Human-Timescale Adaptation in an Open-Ended Task Space
Foundation models have shown impressive adaptation and scalability in supervised and self-supervised learning problems, but so far these successes have not fully translated to reinforcement learning (RL).
The Effectiveness of World Models for Continual Reinforcement Learning
World models power some of the most efficient reinforcement learning algorithms.
Learning General World Models in a Handful of Reward-Free Deployments
We then present CASCADE, a novel approach for self-supervised exploration in this new setting.
Hierarchical Kickstarting for Skill Transfer in Reinforcement Learning
In this paper, we investigate how skills can be incorporated into the training of reinforcement learning (RL) agents in complex environments with large state-action spaces and sparse rewards.
Bayesian Generational Population-Based Training
Leveraging the new highly parallelizable Brax physics engine, we show that these innovations lead to large performance gains, significantly outperforming the tuned baseline while learning entire configurations on the fly.
Grounding Aleatoric Uncertainty for Unsupervised Environment Design
Problematically, in partially-observable or stochastic settings, optimal policies may depend on the ground-truth distribution over aleatoric parameters of the environment in the intended deployment setting, while curriculum learning necessarily shifts the training distribution.
Challenges and Opportunities in Offline Reinforcement Learning from Visual Observations
Using this suite of benchmarking tasks, we show that simple modifications to two popular vision-based online reinforcement learning algorithms, DreamerV2 and DrQ-v2, suffice to outperform existing offline RL methods and establish competitive baselines for continuous control in the visual domain.
Insights From the NeurIPS 2021 NetHack Challenge
In this report, we summarize the takeaways from the first NeurIPS 2021 NetHack Challenge.
On-the-fly Strategy Adaptation for ad-hoc Agent Coordination
Training agents in cooperative settings offers the promise of AI agents able to interact effectively with humans (and other agents) in the real world.
Evolving Curricula with Regret-Based Environment Design
Our approach, which we call Adversarially Compounding Complexity by Editing Levels (ACCEL), seeks to constantly produce levels at the frontier of an agent's capabilities, resulting in curricula that start simple but become increasingly complex.
Automated Reinforcement Learning (AutoRL): A Survey and Open Problems
The combination of Reinforcement Learning (RL) with deep learning has led to a series of impressive feats, with many believing (deep) RL provides a path towards generally capable agents.
Lyapunov Exponents for Diversity in Differentiable Games
We generalize this idea to non-conservative, multi-agent gradient systems by proposing a method - denoted Generalized Ridge Rider (GRR) - for finding arbitrary bifurcation points.
Towards an Understanding of Default Policies in Multitask Policy Optimization
Much of the recent success of deep reinforcement learning has been driven by regularized policy optimization (RPO) algorithms with strong performance across multiple domains.
Revisiting Design Choices in Offline Model-Based Reinforcement Learning
Significant progress has been made recently in offline model-based reinforcement learning, approaches which leverage a learned dynamics model.
Replay-Guided Adversarial Environment Design
Furthermore, our theory suggests a highly counterintuitive improvement to PLR: by stopping the agent from updating its policy on uncurated levels (training on less data), we can improve the convergence to Nash equilibria.
That Escalated Quickly: Compounding Complexity by Editing Levels at the Frontier of Agent Capabilities
Deep Reinforcement Learning (RL) has recently produced impressive results in a series of settings such as games and robotics.
MiniHack the Planet: A Sandbox for Open-Ended Reinforcement Learning Research
By leveraging the full set of entities and environment dynamics from NetHack, one of the richest grid-based video games, MiniHack allows designing custom RL testbeds that are fast and convenient to use.
Return Dispersion as an Estimator of Learning Potential for Prioritized Level Replay
Prioritized Level Replay (PLR) has been shown to induce adaptive curricula that improve the sample-efficiency and generalization of reinforcement learning policies in environments featuring multiple tasks or levels.
From block-Toeplitz matrices to differential equations on graphs: towards a general theory for scalable masked Transformers
In this paper we provide, to the best of our knowledge, the first comprehensive approach for incorporating various masking mechanisms into Transformers architectures in a scalable way.
Tuning Mixed Input Hyperparameters on the Fly for Efficient Population Based AutoRL
Despite a series of recent successes in reinforcement learning (RL), many RL algorithms remain sensitive to hyperparameters.
Same State, Different Task: Continual Reinforcement Learning without Interference
In this paper we formalize this "interference" as distinct from the problem of forgetting.
Revisiting Design Choices in Offline Model Based Reinforcement Learning
Offline reinforcement learning enables agents to make use of large pre-collected datasets of environment transitions and learn control policies without the need for potentially expensive or unsafe online data collection.
Augmented World Models Facilitate Zero-Shot Dynamics Generalization From a Single Offline Environment
Reinforcement learning from large-scale offline datasets provides us with the ability to learn policies without potentially unsafe or impractical exploration.
Unlocking Pixels for Reinforcement Learning via Implicit Attention
There has recently been significant interest in training reinforcement learning (RL) agents in vision-based environments.
Tactical Optimism and Pessimism for Deep Reinforcement Learning
In recent years, deep off-policy actor-critic algorithms have become a dominant approach to reinforcement learning for continuous control.
ES-ENAS: Efficient Evolutionary Optimization for Large Hybrid Search Spaces
In this paper, we approach the problem of optimizing blackbox functions over large hybrid search spaces consisting of both combinatorial and continuous parameters.
Ridge Rider: Finding Diverse Solutions by Following Eigenvectors of the Hessian
In the era of ever decreasing loss functions, SGD and its various offspring have become the go-to optimization tool in machine learning and are a key component of the success of deep neural networks (DNNs).
Towards Tractable Optimism in Model-Based Reinforcement Learning
The principle of optimism in the face of uncertainty is prevalent throughout sequential decision making problems such as multi-armed bandits and reinforcement learning (RL).
Taming the Herd: Multi-Modal Meta-Learning with a Population of Agents
Meta-learning is a paradigm whereby an agent is trained with the specific goal of fast adaptation.
Stochastic Flows and Geometric Optimization on the Orthogonal Group
We present a new class of stochastic, geometrically-driven optimization algorithms on the orthogonal group $O(d)$ and naturally reductive homogeneous manifolds obtained from the action of the rotation group $SO(d)$.
Ready Policy One: World Building Through Active Learning
Model-Based Reinforcement Learning (MBRL) offers a promising direction for sample efficient learning, often achieving state of the art results for continuous control tasks.
Provably Efficient Online Hyperparameter Optimization with Population-Based Bandits
A recent solution to this problem is Population Based Training (PBT) which updates both weights and hyperparameters in a single training run of a population of agents.
Effective Diversity in Population Based Reinforcement Learning
Exploration is a key problem in reinforcement learning, since agents can only learn from data they acquire in the environment.
Behavior-Guided Reinforcement Learning
We introduce a new approach for comparing reinforcement learning policies, using Wasserstein distances (WDs) in a newly defined latent behavioral space.
Reinforcement Learning with Chromatic Networks
We present a neural architecture search algorithm to construct compact reinforcement learning (RL) policies, by combining ENAS and ES in a highly scalable and intuitive way.
Reinforcement Learning with Chromatic Networks for Compact Architecture Search
We present a neural architecture search algorithm to construct compact reinforcement learning (RL) policies, by combining ENAS and ES in a highly scalable and intuitive way.
Learning to Score Behaviors for Guided Policy Optimization
We introduce a new approach for comparing reinforcement learning policies, using Wasserstein distances (WDs) in a newly defined latent behavioral space.
Structured Monte Carlo Sampling for Nonisotropic Distributions via Determinantal Point Processes
We propose a new class of structured methods for Monte Carlo (MC) sampling, called DPPMC, designed for high-dimensional nonisotropic distributions where samples are correlated to reduce the variance of the estimator via determinantal point processes.
Provably Robust Blackbox Optimization for Reinforcement Learning
Interest in derivative-free optimization (DFO) and "evolutionary strategies" (ES) has recently surged in the Reinforcement Learning (RL) community, with growing evidence that they can match state of the art methods for policy optimization problems in Robotics.
From Complexity to Simplicity: Adaptive ES-Active Subspaces for Blackbox Optimization
ASEBO adapts to the geometry of the function and learns optimal sets of sensing directions, which are used to probe it, on-the-fly.
Compressing Deep Neural Networks: A New Hashing Pipeline Using Kac's Random Walk Matrices
The popularity of deep learning is increasing by the day.
