Search Results for author:
Found 57 papers, 22 papers with code
Real-Time Recurrent Learning using Trace Units in Reinforcement Learning
Recurrent Neural Networks (RNNs) are used to learn representations in partially observable environments.
Meta-Gradient Search Control: A Method for Improving the Efficiency of Dyna-style Planning
We study how a Reinforcement Learning (RL) system can remain sample-efficient when learning from an imperfect model of the environment.
Beyond Optimism: Exploration With Partially Observable Rewards
Exploration in reinforcement learning (RL) remains an open challenge.
Monitored Markov Decision Processes
In this paper, we formalize a novel but general RL framework - Monitored MDPs - where the agent cannot always observe rewards.
Assessing the Interpretability of Programmatic Policies with Large Language Models
The LLM then formulates a natural language explanation of the program.
TacticAI: an AI assistant for football tactics
The utility of TacticAI is validated by a qualitative study conducted with football domain experts at Liverpool FC.
Proper Laplacian Representation Learning
The ability to learn good representations of states is essential for solving large reinforcement learning problems, where exploration, generalization, and transfer are particularly challenging.
Learning not to Regret
As these similar games feature similar equilibra, we investigate a way to accelerate equilibrium finding on such a distribution.
Targeted Search Control in AlphaZero for Effective Policy Improvement
AlphaZero trains upon self-play matches beginning from the initial state of a game and only samples actions over the first few moves, limiting its exploration of states deeper in the game tree.
Settling the Reward Hypothesis
The reward hypothesis posits that, "all of what we mean by goals and purposes can be well thought of as maximization of the expected value of the cumulative sum of a received scalar signal (reward)."
Over-communicate no more: Situated RL agents learn concise communication protocols
While it is known that communication facilitates cooperation in multi-agent settings, it is unclear how to design artificial agents that can learn to effectively and efficiently communicate with each other.
The Alberta Plan for AI Research
Herein we describe our approach to artificial intelligence research, which we call the Alberta Plan.
Interpolating Between Softmax Policy Gradient and Neural Replicator Dynamics with Capped Implicit Exploration
Neural replicator dynamics (NeuRD) is an alternative to the foundational softmax policy gradient (SPG) algorithm motivated by online learning and evolutionary game theory.
Efficient Deviation Types and Learning for Hindsight Rationality in Extensive-Form Games: Corrections
Hindsight rationality is an approach to playing general-sum games that prescribes no-regret learning dynamics for individual agents with respect to a set of deviations, and further describes jointly rational behavior among multiple agents with mediated equilibria.
Should Models Be Accurate?
We focus our investigations on Dyna-style planning in a prediction setting.
Student of Games: A unified learning algorithm for both perfect and imperfect information games
Games have a long history as benchmarks for progress in artificial intelligence.
The Partially Observable History Process
We introduce the partially observable history process (POHP) formalism for reinforcement learning.
Learning to Be Cautious
An agent that could learn to be cautious would overcome this challenge by discovering for itself when and how to behave cautiously.
Temporal Abstraction in Reinforcement Learning with the Successor Representation
In this paper, we argue that the successor representation (SR), which encodes states based on the pattern of state visitation that follows them, can be seen as a natural substrate for the discovery and use of temporal abstractions.
Efficient Deviation Types and Learning for Hindsight Rationality in Extensive-Form Games
Hindsight rationality is an approach to playing general-sum games that prescribes no-regret learning dynamics for individual agents with respect to a set of deviations, and further describes jointly rational behavior among multiple agents with mediated equilibria.
Solving Common-Payoff Games with Approximate Policy Iteration
While this choice precludes CAPI from scaling to games as large as Hanabi, empirical results demonstrate that, on the games to which CAPI does scale, it is capable of discovering optimal joint policies even when other modern multi-agent reinforcement learning algorithms are unable to do so.
Hindsight and Sequential Rationality of Correlated Play
This approach also leads to a game-theoretic analysis, but in the correlated play that arises from joint learning dynamics rather than factored agent behavior at equilibrium.
Useful Policy Invariant Shaping from Arbitrary Advice
Furthermore, if the reward is constructed from a potential function, the optimal policy is guaranteed to be unaltered.
The Advantage Regret-Matching Actor-Critic
In this paper, we describe a general model-free RL method for no-regret learning based on repeated reconsideration of past behavior.
Marginal Utility for Planning in Continuous or Large Discrete Action Spaces
Generating good candidate actions is critical to the success of sample-based planners, particularly in continuous or large action spaces.
Sample-Efficient Model-based Actor-Critic for an Interactive Dialogue Task
Human-computer interactive systems that rely on machine learning are becoming paramount to the lives of millions of people who use digital assistants on a daily basis.
Approximate exploitability: Learning a best response in large games
In prior games research, agent evaluation often focused on the in-practice game outcomes.
Alternative Function Approximation Parameterizations for Solving Games: An Analysis of $f$-Regression Counterfactual Regret Minimization
In contrast, the more conventional softmax parameterization is standard in the field of reinforcement learning and yields a regret bound with a better dependence on the number of actions.
Low-Variance and Zero-Variance Baselines for Extensive-Form Games
In this paper, we extend recent work that uses baseline estimates to reduce this variance.
Rethinking Formal Models of Partially Observable Multiagent Decision Making
A second issue is that while EFGs have recently seen significant algorithmic progress, their classical formalization is unsuitable for efficient presentation of the underlying ideas, such as those around decomposition.
Ease-of-Teaching and Language Structure from Emergent Communication
Artificial agents have been shown to learn to communicate when needed to complete a cooperative task.
The Hanabi Challenge: A New Frontier for AI Research
From the early days of computing, games have been important testbeds for studying how well machines can do sophisticated decision making.
Bayesian Action Decoder for Deep Multi-Agent Reinforcement Learning
We present the Bayesian action decoder (BAD), a new multi-agent learning method that uses an approximate Bayesian update to obtain a public belief that conditions on the actions taken by all agents in the environment.
Actor-Critic Policy Optimization in Partially Observable Multiagent Environments
Optimization of parameterized policies for reinforcement learning (RL) is an important and challenging problem in artificial intelligence.
Generalization and Regularization in DQN
Deep reinforcement learning algorithms have shown an impressive ability to learn complex control policies in high-dimensional tasks.
Solving Large Extensive-Form Games with Strategy Constraints
Extensive-form games are a common model for multiagent interactions with imperfect information.
Variance Reduction in Monte Carlo Counterfactual Regret Minimization (VR-MCCFR) for Extensive Form Games using Baselines
The new formulation allows estimates to be bootstrapped from other estimates within the same episode, propagating the benefits of baselines along the sampled trajectory; the estimates remain unbiased even when bootstrapping from other estimates.
Count-Based Exploration with the Successor Representation
In this paper we introduce a simple approach for exploration in reinforcement learning (RL) that allows us to develop theoretically justified algorithms in the tabular case but that is also extendable to settings where function approximation is required.
Ranked #16 on
Atari Games
on Atari 2600 Venture
The Effect of Planning Shape on Dyna-style Planning in High-dimensional State Spaces
Dyna is a fundamental approach to model-based reinforcement learning (MBRL) that interleaves planning, acting, and learning in an online setting.
Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agents
The Arcade Learning Environment (ALE) is an evaluation platform that poses the challenge of building AI agents with general competency across dozens of Atari 2600 games.
A Laplacian Framework for Option Discovery in Reinforcement Learning
Representation learning and option discovery are two of the biggest challenges in reinforcement learning (RL).
DeepStack: Expert-Level Artificial Intelligence in No-Limit Poker
Poker is the quintessential game of imperfect information, and a longstanding challenge problem in artificial intelligence.
Equilibrium Approximation Quality of Current No-Limit Poker Bots
Approximating a Nash equilibrium is currently the best performing approach for creating poker-playing programs.
Computer Science and Game Theory
AIVAT: A New Variance Reduction Technique for Agent Evaluation in Imperfect Information Games
Evaluating agent performance when outcomes are stochastic and agents use randomized strategies can be challenging when there is limited data available.
The Forget-me-not Process
We introduce the Forget-me-not Process, an efficient, non-parametric meta-algorithm for online probabilistic sequence prediction for piecewise stationary, repeating sources.
Learning Purposeful Behaviour in the Absence of Rewards
In the reinforcement learning framework, goals are encoded as reward functions that guide agent behaviour, and the sum of observed rewards provide a notion of progress.
State of the Art Control of Atari Games Using Shallow Reinforcement Learning
The recently introduced Deep Q-Networks (DQN) algorithm has gained attention as one of the first successful combinations of deep neural networks and reinforcement learning.
Solving Games with Functional Regret Estimation
We propose a novel online learning method for minimizing regret in large extensive-form games.
Domain-Independent Optimistic Initialization for Reinforcement Learning
In Reinforcement Learning (RL), it is common to use optimistic initialization of value functions to encourage exploration.
Tractable Objectives for Robust Policy Optimization
Robust policy optimization acknowledges that risk-aversion plays a vital role in real-world decision-making.
Sketch-Based Linear Value Function Approximation
Unfortunately, the typical use of hashing in value function approximation results in biased value estimates due to the possibility of collisions.
The Arcade Learning Environment: An Evaluation Platform for General Agents
We illustrate the promise of ALE by developing and benchmarking domain-independent agents designed using well-established AI techniques for both reinforcement learning and planning.
Ranked #1 on
Atari Games
on Atari 2600 Pooyan
Variance Reduction in Monte-Carlo Tree Search
Monte-Carlo Tree Search (MCTS) has proven to be a powerful, generic planning technique for decision-making in single-agent and adversarial environments.
Context Tree Switching
This paper describes the Context Tree Switching technique, a modification of Context Tree Weighting for the prediction of binary, stationary, n-Markov sources.
Information Theory Information Theory
Monte Carlo Sampling for Regret Minimization in Extensive Games
In the domain of poker, CFR has proven effective, particularly when using a domain-specific augmentation involving chance outcome sampling.
Strategy Grafting in Extensive Games
A common approach for computing strategies in these large games is to first employ an abstraction technique to reduce the original game to an abstract game that is of a manageable size.
Computing Robust Counter-Strategies
Adaptation to other initially unknown agents often requires computing an effective counter-strategy.
