Search Results for author:
Found 20 papers, 14 papers with code
Revisiting Recurrent Reinforcement Learning with Memory Monoids
Memory models such as Recurrent Neural Networks (RNNs) and Transformers address Partially Observable Markov Decision Processes (POMDPs) by mapping trajectories to latent Markov states.
Discovering Temporally-Aware Reinforcement Learning Algorithms
We propose a simple augmentation to two existing objective discovery approaches that allows the discovered algorithm to dynamically update its objective function throughout the agent's training procedure, resulting in expressive schedules and increased generalization across different training horizons.
Analysing the Sample Complexity of Opponent Shaping
Providing theoretical guarantees for M-FOS is hard because A) there is little literature on theoretical sample complexity bounds for meta-reinforcement learning B) M-FOS operates in continuous state and action spaces, so theoretical analysis is challenging.
Meta-learning the mirror map in policy mirror descent
Policy Mirror Descent (PMD) is a popular framework in reinforcement learning, serving as a unifying perspective that encompasses numerous algorithms.
Leading the Pack: N-player Opponent Shaping
We evaluate on over 4 different environments, varying the number of players from 3 to 5, and demonstrate that model-based OS methods converge to equilibrium with better global welfare than naive learning.
Scaling Opponent Shaping to High Dimensional Games
In multi-agent settings with mixed incentives, methods developed for zero-sum games have been shown to lead to detrimental outcomes.
JaxMARL: Multi-Agent RL Environments in JAX
This not only enables GPU acceleration, but also provides a more flexible MARL environment, unlocking the potential for self-play, meta-learning, and other future applications in MARL.
Discovering General Reinforcement Learning Algorithms with Adversarial Environment Design
Recently, it has been shown that it is possible to meta-learn update rules, with the hope of discovering algorithms that can perform well on a wide range of RL tasks.
JAX-LOB: A GPU-Accelerated limit order book simulator to unlock large scale reinforcement learning for trading
Financial exchanges across the world use limit order books (LOBs) to process orders and match trades.
ReLU to the Rescue: Improve Your On-Policy Actor-Critic with Positive Advantages
We show that the additive term is bounded proportional to the Lipschitz constant of the value function, which offers theoretical grounding for spectral normalization of critic weights.
Discovering Attention-Based Genetic Algorithms via Meta-Black-Box Optimization
Genetic algorithms constitute a family of black-box optimization algorithms, which take inspiration from the principles of biological evolution.
Arbitrary Order Meta-Learning with Simple Population-Based Evolution
Meta-learning, the notion of learning to learn, enables learning systems to quickly and flexibly solve new tasks.
Structured State Space Models for In-Context Reinforcement Learning
We propose a modification to a variant of S4 that enables us to initialise and reset the hidden state in parallel, allowing us to tackle reinforcement learning tasks.
Discovering Evolution Strategies via Meta-Black-Box Optimization
Optimizing functions without access to gradients is the remit of black-box methods such as evolution strategies.
Adversarial Cheap Talk
More specifically, we show that an ACT Adversary is capable of harming performance by interfering with the learner's function approximation, or instead helping the Victim's performance by outputting useful features.
Proximal Learning With Opponent-Learning Awareness
This problem is especially pronounced in the opponent modeling setting, where the opponent's policy is unknown and must be inferred from observations; in such settings, LOLA is ill-specified because behaviorally equivalent opponent policies can result in non-equivalent updates.
Discovered Policy Optimisation
We refer to the immediate result as Learnt Policy Optimisation (LPO).
Model-Free Opponent Shaping
In general-sum games, the interaction of self-interested learning agents commonly leads to collectively worst-case outcomes, such as defect-defect in the iterated prisoner's dilemma (IPD).
Centralized Model and Exploration Policy for Multi-Agent RL
We also learn a centralized exploration policy within our model that learns to collect additional data in state-action regions with high model uncertainty.
Learning to Control Self-Assembling Morphologies: A Study of Generalization via Modularity
We evaluate the performance of these dynamic and modular agents in simulated environments.
