Search Results for author:
Found 32 papers, 19 papers with code
Evaluation Function Approximation for Scrabble
The current state-of-the-art Scrabble agents are not learning-based but depend on truncated Monte Carlo simulations and the quality of such agents is contingent upon the time available for running the simulations.
Learning to Generalize from Sparse and Underspecified Rewards
The parameters of the auxiliary reward function are optimized with respect to the validation performance of a trained policy.
An Optimistic Perspective on Offline Reinforcement Learning
The DQN replay dataset can serve as an offline RL benchmark and is open-sourced.
Striving for Simplicity in Off-Policy Deep Reinforcement Learning
This paper advocates the use of offline (batch) reinforcement learning (RL) to help (1) isolate the contributions of exploitation vs. exploration in off-policy deep RL, (2) improve reproducibility of deep RL research, and (3) facilitate the design of simpler deep RL algorithms.
Neural Additive Models: Interpretable Machine Learning with Neural Nets
They perform similarly to existing state-of-the-art generalized additive models in accuracy, but are more flexible because they are based on neural nets instead of boosted trees.
RL Unplugged: A Suite of Benchmarks for Offline Reinforcement Learning
We hope that our suite of benchmarks will increase the reproducibility of experiments and make it possible to study challenging tasks with a limited computational budget, thus making RL research both more systematic and more accessible across the community.
Revisiting Fundamentals of Experience Replay
Experience replay is central to off-policy algorithms in deep reinforcement learning (RL), but there remain significant gaps in our understanding.
IITK at SemEval-2020 Task 10: Transformers for Emphasis Selection
This paper describes the system proposed for addressing the research problem posed in Task 10 of SemEval-2020: Emphasis Selection For Written Text in Visual Media.
Implicit Under-Parameterization Inhibits Data-Efficient Deep Reinforcement Learning
We identify an implicit under-parameterization phenomenon in value-based deep RL methods that use bootstrapping: when value functions, approximated using deep neural networks, are trained with gradient descent using iterated regression onto target values generated by previous instances of the value network, more gradient updates decrease the expressivity of the current value network.
RL Unplugged: A Collection of Benchmarks for Offline Reinforcement Learning
We hope that our suite of benchmarks will increase the reproducibility of experiments and make it possible to study challenging tasks with a limited computational budget, thus making RL research both more systematic and more accessible across the community.
Contrastive Behavioral Similarity Embeddings for Generalization in Reinforcement Learning
Specifically, we introduce a theoretically motivated policy similarity metric (PSM) for measuring behavioral similarity between states.
Control-Oriented Model-Based Reinforcement Learning with Implicit Differentiation
The shortcomings of maximum likelihood estimation in the context of model-based reinforcement learning have been highlighted by an increasing number of papers.
Model-based Reinforcement Learning
reinforcement-learning
+1
Deep Reinforcement Learning at the Edge of the Statistical Precipice
Most published results on deep RL benchmarks compare point estimates of aggregate performance such as mean and median scores across tasks, ignoring the statistical uncertainty implied by the use of a finite number of training runs.
DR3: Value-Based Deep Reinforcement Learning Requires Explicit Regularization
In this paper, we discuss how the implicit regularization effect of SGD seen in supervised learning could in fact be harmful in the offline deep RL setting, leading to poor generalization and degenerate feature representations.
On the Generalization of Representations in Reinforcement Learning
We complement our theoretical results with an empirical survey of classic representation learning methods from the literature and results on the Arcade Learning Environment, and find that the generalization behaviour of learned representations is well-explained by their effective dimension.
Reincarnating Reinforcement Learning: Reusing Prior Computation to Accelerate Progress
To address these issues, we present reincarnating RL as an alternative workflow or class of problem settings, where prior computational work (e. g., learned policies) is reused or transferred between design iterations of an RL agent, or from one RL agent to another.
Offline Q-Learning on Diverse Multi-Task Data Both Scales And Generalizes
The potential of offline reinforcement learning (RL) is that high-capacity models trained on large, heterogeneous datasets can lead to agents that generalize broadly, analogously to similar advances in vision and NLP.
A Novel Stochastic Gradient Descent Algorithm for Learning Principal Subspaces
In this paper, we derive an algorithm that learns a principal subspace from sample entries, can be applied when the approximate subspace is represented by a neural network, and hence can be scaled to datasets with an effectively infinite number of rows and columns.
Revisiting Bellman Errors for Offline Model Selection
Offline model selection (OMS), that is, choosing the best policy from a set of many policies given only logged data, is crucial for applying offline RL in real-world settings.
The Dormant Neuron Phenomenon in Deep Reinforcement Learning
In this work we identify the dormant neuron phenomenon in deep reinforcement learning, where an agent's network suffers from an increasing number of inactive neurons, thereby affecting network expressivity.
Proto-Value Networks: Scaling Representation Learning with Auxiliary Tasks
Combined with a suitable off-policy learning rule, the result is a representation learning algorithm that can be understood as extending Mahadevan & Maggioni (2007)'s proto-value functions to deep reinforcement learning -- accordingly, we call the resulting object proto-value networks.
Bigger, Better, Faster: Human-level Atari with human-level efficiency
We introduce a value-based RL agent, which we call BBF, that achieves super-human performance in the Atari 100K benchmark.
Ranked #1 on
Atari Games 100k
on Atari 100k
Bootstrapped Representations in Reinforcement Learning
In this paper, we address this gap and provide a theoretical characterization of the state representation learnt by temporal difference learning (Sutton, 1988).
On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes
Instead of solely relying on a fixed set of output sequences, GKD trains the student on its self-generated output sequences by leveraging feedback from the teacher on such sequences.
DistillSpec: Improving Speculative Decoding via Knowledge Distillation
Finally, in practical scenarios with models of varying sizes, first using distillation to boost the performance of the target model and then applying DistillSpec to train a well-aligned draft model can reduce decoding latency by 6-10x with minimal performance drop, compared to standard decoding without distillation.
Learning and Controlling Silicon Dopant Transitions in Graphene using Scanning Transmission Electron Microscopy
We introduce a machine learning approach to determine the transition dynamics of silicon atoms on a single layer of carbon atoms, when stimulated by the electron beam of a scanning transmission electron microscope (STEM).
Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Models
To do so, we investigate a simple self-training method based on expectation-maximization, which we call ReST$^{EM}$, where we (1) generate samples from the model and filter them using binary feedback, (2) fine-tune the model on these samples, and (3) repeat this process a few times.
V-STaR: Training Verifiers for Self-Taught Reasoners
Common self-improvement approaches for large language models (LLMs), such as STaR (Zelikman et al., 2022), iteratively fine-tune LLMs on self-generated solutions to improve their problem-solving ability.
Transformers Can Achieve Length Generalization But Not Robustly
We show that the success of length generalization is intricately linked to the data format and the type of position encoding.
Stop Regressing: Training Value Functions via Classification for Scalable Deep RL
Observing this discrepancy, in this paper, we investigate whether the scalability of deep RL can also be improved simply by using classification in place of regression for training value functions.
Many-Shot In-Context Learning
Finally, we demonstrate that, unlike few-shot learning, many-shot learning is effective at overriding pretraining biases and can learn high-dimensional functions with numerical inputs.
An Optimistic Perspective on Offline Deep Reinforcement Learning
The DQN replay dataset can serve as an offline RL benchmark and is open-sourced.
