Search Results for author:
Found 36 papers, 21 papers with code
Stop Regressing: Training Value Functions via Classification for Scalable Deep RL
Observing this discrepancy, in this paper, we investigate whether the scalability of deep RL can also be improved simply by using classification in place of regression for training value functions.
In deep reinforcement learning, a pruned network is a good network
Recent work has shown that deep reinforcement learning agents have difficulty in effectively using their network parameters.
Mixtures of Experts Unlock Parameter Scaling for Deep RL
The recent rapid progress in (self) supervised learning models is in large part predicted by empirical scaling laws: a model's performance scales proportionally to its size.
A density estimation perspective on learning from pairwise human preferences
Learning from human feedback (LHF) -- and in particular learning from pairwise preferences -- has recently become a crucial ingredient in training large language models (LLMs), and has been the subject of much research.
Learning and Controlling Silicon Dopant Transitions in Graphene using Scanning Transmission Electron Microscopy
We introduce a machine learning approach to determine the transition dynamics of silicon atoms on a single layer of carbon atoms, when stimulated by the electron beam of a scanning transmission electron microscope (STEM).
A Kernel Perspective on Behavioural Metrics for Markov Decision Processes
Behavioural metrics have been shown to be an effective mechanism for constructing representations in reinforcement learning.
Offline Reinforcement Learning with On-Policy Q-Function Regularization
In this work, we propose to regularize towards the Q-function of the behavior policy instead of the behavior policy itself, under the premise that the Q-function can be estimated more reliably and easily by a SARSA-style estimate and handles the extrapolation error more straightforwardly.
Bigger, Better, Faster: Human-level Atari with human-level efficiency
We introduce a value-based RL agent, which we call BBF, that achieves super-human performance in the Atari 100K benchmark.
Ranked #1 on
Atari Games 100k
on Atari 100k
JaxPruner: A concise library for sparsity research
This paper introduces JaxPruner, an open-source JAX-based pruning and sparse training library for machine learning research.
Proto-Value Networks: Scaling Representation Learning with Auxiliary Tasks
Combined with a suitable off-policy learning rule, the result is a representation learning algorithm that can be understood as extending Mahadevan & Maggioni (2007)'s proto-value functions to deep reinforcement learning -- accordingly, we call the resulting object proto-value networks.
The Dormant Neuron Phenomenon in Deep Reinforcement Learning
In this work we identify the dormant neuron phenomenon in deep reinforcement learning, where an agent's network suffers from an increasing number of inactive neurons, thereby affecting network expressivity.
The State of Sparse Training in Deep Reinforcement Learning
The use of sparse neural networks has seen rapid growth in recent years, particularly in computer vision.
Reincarnating Reinforcement Learning: Reusing Prior Computation to Accelerate Progress
To address these issues, we present reincarnating RL as an alternative workflow or class of problem settings, where prior computational work (e. g., learned policies) is reused or transferred between design iterations of an RL agent, or from one RL agent to another.
Losses, Dissonances, and Distortions
In this paper I present a study in using the losses and gradients obtained during the training of a simple function approximator as a mechanism for creating musical dissonance and visual distortion in a solo piano performance setting.
The Difficulty of Passive Learning in Deep Reinforcement Learning
Learning to act from observational data without active environmental interaction is a well-known challenge in Reinforcement Learning (RL).
Lifting the veil on hyper-parameters for value-based deep reinforcement learning
Successful applications of deep reinforcement learning (deep RL) combine algorithmic design and careful hyper-parameter selection.
Composing Features: Compositional Model Augmentation for Steerability of Music Transformers
To tackle the combinatorial nature of composing features, we propose a compositional approach to steering music transformers, building on lightweight fine-tuning methods such as prefix tuning and bias tuning.
Deep Reinforcement Learning at the Edge of the Statistical Precipice
Most published results on deep RL benchmarks compare point estimates of aggregate performance such as mean and median scores across tasks, ignoring the statistical uncertainty implied by the use of a finite number of training runs.
A general class of surrogate functions for stable and efficient reinforcement learning
Common policy gradient methods rely on the maximization of a sequence of surrogate functions.
MICo: Improved representations via sampling-based state similarity for Markov decision processes
We present a new behavioural distance over the state space of a Markov decision process, and demonstrate the use of this distance as an effective means of shaping the learnt representations of deep reinforcement learning agents.
LOCO: Adaptive exploration in reinforcement learning via local estimation of contraction coefficients
We offer a novel approach to balance exploration and exploitation in reinforcement learning (RL).
Metrics and continuity in reinforcement learning
In most practical applications of reinforcement learning, it is untenable to maintain direct estimates for individual states; in continuous-state systems, it is impossible.
Contrastive Behavioral Similarity Embeddings for Generalization in Reinforcement Learning
Specifically, we introduce a theoretically motivated policy similarity metric (PSM) for measuring behavioral similarity between states.
Revisiting Rainbow: Promoting more Insightful and Inclusive Deep Reinforcement Learning Research
Since the introduction of DQN, a vast majority of reinforcement learning research has focused on reinforcement learning with deep neural networks as function approximators.
GANterpretations
Since the introduction of Generative Adversarial Networks (GANs) [Goodfellow et al., 2014] there has been a regular stream of both technical advances (e. g., Arjovsky et al. [2017]) and creative uses of these generative models (e. g., [Karras et al., 2019, Zhu et al., 2017, Jin et al., 2017]).
Rigging the Lottery: Making All Tickets Winners
There is a large body of work on training dense networks to yield sparse networks for inference, but this limits the size of the largest trainable sparse model to that of the largest trainable dense model.
Ranked #1 on
Sparse Learning
on ImageNet
Scalable methods for computing state similarity in deterministic Markov Decision Processes
We present new algorithms for computing and approximating bisimulation metrics in Markov Decision Processes (MDPs).
Inverse Reinforcement Learning with Multiple Ranked Experts
We consider the problem of learning to behave optimally in a Markov Decision Process when a reward function is not specified, but instead we have access to a set of demonstrators of varying performance.
Performing Structured Improvisations with pre-trained Deep Learning Models
The quality of outputs produced by deep generative models for music have seen a dramatic improvement in the last few years.
Distributional reinforcement learning with linear function approximation
Despite many algorithmic advances, our theoretical understanding of practical distributional reinforcement learning methods remains limited.
Distributional Reinforcement Learning
reinforcement-learning
+1
Shaping the Narrative Arc: An Information-Theoretic Approach to Collaborative Dialogue
We consider the problem of designing an artificial agent capable of interacting with humans in collaborative dialogue to produce creative, engaging narratives.
A Geometric Perspective on Optimal Representations for Reinforcement Learning
We leverage this perspective to provide formal evidence regarding the usefulness of value functions as auxiliary tasks.
A Comparative Analysis of Expected and Distributional Reinforcement Learning
Since their introduction a year ago, distributional approaches to reinforcement learning (distributional RL) have produced strong results relative to the standard approach which models expected values (expected RL).
Distributional Reinforcement Learning
reinforcement-learning
+1
An Atari Model Zoo for Analyzing, Visualizing, and Comparing Deep Reinforcement Learning Agents
We lessen this friction, by (1) training several algorithms at scale and releasing trained models, (2) integrating with a previous Deep RL model release, and (3) releasing code that makes it easy for anyone to load, visualize, and analyze such models.
Dopamine: A Research Framework for Deep Reinforcement Learning
Dopamine is a research framework for fast prototyping of reinforcement learning algorithms.
Combining Learned Lyrical Structures and Vocabulary for Improved Lyric Generation
The use of language models for generating lyrics and poetry has received an increased interest in the last few years.
