Search Results for author:
Found 34 papers, 16 papers with code
Approximate Thompson Sampling via Epistemic Neural Networks
Further, we demonstrate that the \textit{epinet} -- a small additive network that estimates uncertainty -- matches the performance of large ensembles at orders of magnitude lower computational cost.
Fine-Tuning Language Models via Epistemic Neural Networks
Language models often pre-train on large unsupervised text corpora, then fine-tune on additional task-specific data.
Robustness of Epinets against Distributional Shifts
However, these improvements are relatively small compared to the outstanding issues in distributionally-robust deep learning.
Ensembles for Uncertainty Estimation: Benefits of Prior Functions and Bootstrapping
In machine learning, an agent needs to estimate uncertainty to efficiently explore and adapt and to make effective decisions.
Evaluating High-Order Predictive Distributions in Deep Learning
Previous work has developed methods for assessing low-order predictive distributions with inputs sampled i. i. d.
The Neural Testbed: Evaluating Joint Predictions
Predictive distributions quantify uncertainties ignored by point estimates.
Evaluating Predictive Distributions: Does Bayesian Deep Learning Work?
This paper introduces \textit{The Neural Testbed}, which provides tools for the systematic evaluation of agents that generate such predictions.
From Predictions to Decisions: The Importance of Joint Predictive Distributions
A fundamental challenge for any intelligent system is prediction: given some inputs, can you predict corresponding outcomes?
Epistemic Neural Networks
We introduce the epinet: an architecture that can supplement any conventional neural network, including large pretrained models, and can be trained with modest incremental computation to estimate uncertainty.
Reinforcement Learning, Bit by Bit
To illustrate concepts, we design simple agents that build on them and present computational results that highlight data efficiency.
Hypermodels for Exploration
This generalizes and extends the use of ensembles to approximate Thompson sampling.
Matrix games with bandit feedback
We study a version of the classical zero-sum matrix game with unknown payoff matrix and bandit feedback, where the players only observe each others actions and a noisy payoff.
Making Sense of Reinforcement Learning and Probabilistic Inference
Reinforcement learning (RL) combines a control problem with statistical estimation: The system dynamics are not known to the agent, but can be learned through experience.
Behaviour Suite for Reinforcement Learning
bsuite is a collection of carefully-designed experiments that investigate core capabilities of reinforcement learning (RL) agents with two objectives.
Meta-learning of Sequential Strategies
In this report we review memory-based meta-learning as a tool for building sample-efficient strategies that learn from past experience to adapt to any task within a target class.
Randomized Prior Functions for Deep Reinforcement Learning
Dealing with uncertainty is essential for efficient reinforcement learning.
Scalable Coordinated Exploration in Concurrent Reinforcement Learning
We consider a team of reinforcement learning agents that concurrently operate in a common environment, and we develop an approach to efficient coordinated exploration that is suitable for problems of practical scale.
The Uncertainty Bellman Equation and Exploration
In this paper we consider a similar \textit{uncertainty} Bellman equation (UBE), which connects the uncertainty at any time-step to the expected uncertainties at subsequent time-steps, thereby extending the potential exploratory benefit of a policy beyond individual time-steps.
A Tutorial on Thompson Sampling
Thompson sampling is an algorithm for online decision problems where actions are taken sequentially in a manner that must balance between exploiting what is known to maximize immediate performance and investing to accumulate new information that may improve future performance.
Noisy Networks for Exploration
We introduce NoisyNet, a deep reinforcement learning agent with parametric noise added to its weights, and show that the induced stochasticity of the agent's policy can be used to aid efficient exploration.
Ranked #1 on
Atari Games
on Atari 2600 Surround
On Optimistic versus Randomized Exploration in Reinforcement Learning
We discuss the relative merits of optimistic and randomized approaches to exploration in reinforcement learning.
Deep Q-learning from Demonstrations
We present an algorithm, Deep Q-learning from Demonstrations (DQfD), that leverages small sets of demonstration data to massively accelerate the learning process even from relatively small amounts of demonstration data and is able to automatically assess the necessary ratio of demonstration data while learning thanks to a prioritized replay mechanism.
Deep Exploration via Randomized Value Functions
We study the use of randomized value functions to guide deep exploration in reinforcement learning.
Minimax Regret Bounds for Reinforcement Learning
We consider the problem of provably optimal exploration in reinforcement learning for finite horizon MDPs.
Gaussian-Dirichlet Posterior Dominance in Sequential Learning
We consider the problem of sequential learning from categorical observations bounded in [0, 1].
Posterior Sampling for Reinforcement Learning Without Episodes
- Review similar results for optimistic algorithms in infinite horizon problems (Jaksch et al 2010, Bartlett and Tewari 2009, Abbasi-Yadkori and Szepesvari 2011), with particular attention to the dynamic episode growth.
On Lower Bounds for Regret in Reinforcement Learning
This is a brief technical note to clarify the state of lower bounds on regret for reinforcement learning.
Why is Posterior Sampling Better than Optimism for Reinforcement Learning?
Computational results demonstrate that posterior sampling for reinforcement learning (PSRL) dramatically outperforms algorithms driven by optimism, such as UCRL2.
Deep Exploration via Bootstrapped DQN
Efficient exploration in complex environments remains a major challenge for reinforcement learning.
Ranked #6 on
Atari Games
on Atari 2600 Breakout
Bootstrapped Thompson Sampling and Deep Exploration
This technical note presents a new approach to carrying out the kind of exploration achieved by Thompson sampling, but without explicitly maintaining or sampling from posterior distributions.
Model-based Reinforcement Learning and the Eluder Dimension
We consider the problem of learning to optimize an unknown Markov decision process (MDP).
Model-based Reinforcement Learning
reinforcement-learning
+1
Near-optimal Reinforcement Learning in Factored MDPs
Any reinforcement learning algorithm that applies to all Markov decision processes (MDPs) will suffer $\Omega(\sqrt{SAT})$ regret on some MDP, where $T$ is the elapsed time and $S$ and $A$ are the cardinalities of the state and action spaces.
Generalization and Exploration via Randomized Value Functions
We propose randomized least-squares value iteration (RLSVI) -- a new reinforcement learning algorithm designed to explore and generalize efficiently via linearly parameterized value functions.
(More) Efficient Reinforcement Learning via Posterior Sampling
This bound is one of the first for an algorithm not based on optimism, and close to the state of the art for any reinforcement learning algorithm.
