Search Results for author:
Found 38 papers, 24 papers with code
Conditional Augmentation for Generative Modeling
We present conditional augmentation (CondAugment), a simple and powerful method of regularizing generative models.
Scaling laws for single-agent reinforcement learning
Recent work has shown that, in generative modeling, cross-entropy loss improves smoothly with model size and training compute, following a power law plus constant scaling law.
Scaling Laws for Reward Model Overoptimization
In reinforcement learning from human feedback, it is common to optimize against a reward model trained to predict human preferences.
Efficient Training of Language Models to Fill in the Middle
To this end, we run a series of ablations on key hyperparameters, such as the data transformation frequency, the structure of the transformation, and the method of selecting the infill span.
Training language models to follow instructions with human feedback
In this paper, we show an avenue for aligning language models with user intent on a wide range of tasks by fine-tuning with human feedback.
WebGPT: Browser-assisted question-answering with human feedback
This model's answers are preferred by humans 56% of the time to those of our human demonstrators, and 69% of the time to the highest-voted answer from Reddit.
Training Verifiers to Solve Math Word Problems
State-of-the-art language models can match human performance on many tasks, but they still struggle to robustly perform multi-step mathematical reasoning.
Batch size-invariance for policy optimization
We say an algorithm is batch size-invariant if changes to the batch size can largely be compensated for by changes to other hyperparameters.
Unsolved Problems in ML Safety
Machine learning (ML) systems are rapidly increasing in size, are acquiring new capabilities, and are increasingly deployed in high-stakes settings.
Measuring Sample Efficiency and Generalization in Reinforcement Learning Benchmarks: NeurIPS 2020 Procgen Benchmark
We present the design of a centralized benchmark for Reinforcement Learning which can help measure Sample Efficiency and Generalization in Reinforcement Learning by doing end to end evaluation of the training and rollout phases of thousands of user submitted code bases in a scalable way.
The MineRL 2020 Competition on Sample Efficient Reinforcement Learning using Human Priors
Although deep reinforcement learning has led to breakthroughs in many difficult domains, these successes have required an ever-increasing number of samples, affording only a shrinking segment of the AI community access to their development.
Scaling Laws for Autoregressive Generative Modeling
The optimal model size also depends on the compute budget through a power-law, with exponents that are nearly universal across all data domains.
Phasic Policy Gradient
We introduce Phasic Policy Gradient (PPG), a reinforcement learning framework which modifies traditional on-policy actor-critic methods by separating policy and value function training into distinct phases.
Ranked #1 on
Reinforcement Learning (RL)
on ProcGen
Leveraging Procedural Generation to Benchmark Reinforcement Learning
We introduce Procgen Benchmark, a suite of 16 procedurally generated game-like environments designed to benchmark both sample efficiency and generalization in reinforcement learning.
Policy Gradient Search: Online Planning and Expert Iteration without Search Trees
Monte Carlo Tree Search (MCTS) algorithms perform simulation-based search to improve policies online.
Semi-Supervised Learning by Label Gradient Alignment
We then formulate an optimization problem whose objective is to minimize the distance between the labeled and the unlabeled data in this space, and we solve it by gradient descent on the imputed labels.
Quantifying Generalization in Reinforcement Learning
In this paper, we investigate the problem of overfitting in deep reinforcement learning.
Model-Based Reinforcement Learning via Meta-Policy Optimization
Finally, we demonstrate that our approach is able to match the asymptotic performance of model-free methods while requiring significantly less experience.
Model-based Reinforcement Learning
reinforcement-learning
+1
Gotta Learn Fast: A New Benchmark for Generalization in RL
In this report, we present a new reinforcement learning (RL) benchmark based on the Sonic the Hedgehog (TM) video game franchise.
On First-Order Meta-Learning Algorithms
This paper considers meta-learning problems, where there is a distribution of tasks, and we would like to obtain an agent that performs well (i. e., learns quickly) when presented with a previously unseen task sampled from this distribution.
Meta Learning Shared Hierarchies
We develop a metalearning approach for learning hierarchically structured policies, improving sample efficiency on unseen tasks through the use of shared primitives---policies that are executed for large numbers of timesteps.
Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations
Furthermore, deployment of DRL on physical systems remains challenging due to sample inefficiency.
Proximal Policy Optimization Algorithms
We propose a new family of policy gradient methods for reinforcement learning, which alternate between sampling data through interaction with the environment, and optimizing a "surrogate" objective function using stochastic gradient ascent.
Ranked #4 on
Continuous Control
on Lunar Lander (OpenAI Gym)
Teacher-Student Curriculum Learning
We propose Teacher-Student Curriculum Learning (TSCL), a framework for automatic curriculum learning, where the Student tries to learn a complex task and the Teacher automatically chooses subtasks from a given set for the Student to train on.
UCB Exploration via Q-Ensembles
We show how an ensemble of $Q^*$-functions can be leveraged for more effective exploration in deep reinforcement learning.
Equivalence Between Policy Gradients and Soft Q-Learning
A partial explanation may be that $Q$-learning methods are secretly implementing policy gradient updates: we show that there is a precise equivalence between $Q$-learning and policy gradient methods in the setting of entropy-regularized reinforcement learning, that "soft" (entropy-regularized) $Q$-learning is exactly equivalent to a policy gradient method.
#Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learning
In this work, we describe a surprising finding: a simple generalization of the classic count-based approach can reach near state-of-the-art performance on various high-dimensional and/or continuous deep RL benchmarks.
Ranked #1 on
Atari Games
on Atari 2600 Freeway
RL$^2$: Fast Reinforcement Learning via Slow Reinforcement Learning
The activations of the RNN store the state of the "fast" RL algorithm on the current (previously unseen) MDP.
Variational Lossy Autoencoder
Representation learning seeks to expose certain aspects of observed data in a learned representation that's amenable to downstream tasks like classification.
Concrete Problems in AI Safety
Rapid progress in machine learning and artificial intelligence (AI) has brought increasing attention to the potential impacts of AI technologies on society.
InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets
This paper describes InfoGAN, an information-theoretic extension to the Generative Adversarial Network that is able to learn disentangled representations in a completely unsupervised manner.
Ranked #3 on
Image Generation
on Stanford Dogs
VIME: Variational Information Maximizing Exploration
While there are methods with optimality guarantees in the setting of discrete state and action spaces, these methods cannot be applied in high-dimensional deep RL scenarios.
Theano: A Python framework for fast computation of mathematical expressions
Since its introduction, it has been one of the most used CPU and GPU mathematical compilers - especially in the machine learning community - and has shown steady performance improvements.
Benchmarking Deep Reinforcement Learning for Continuous Control
Recently, researchers have made significant progress combining the advances in deep learning for learning feature representations with reinforcement learning.
Ranked #1 on
Continuous Control
on Inverted Pendulum
Gradient Estimation Using Stochastic Computation Graphs
In a variety of problems originating in supervised, unsupervised, and reinforcement learning, the loss function is defined by an expectation over a collection of random variables, which might be part of a probabilistic model or the external world.
High-Dimensional Continuous Control Using Generalized Advantage Estimation
Policy gradient methods are an appealing approach in reinforcement learning because they directly optimize the cumulative reward and can straightforwardly be used with nonlinear function approximators such as neural networks.
Trust Region Policy Optimization
We describe an iterative procedure for optimizing policies, with guaranteed monotonic improvement.
