Search Results for author:
Found 14 papers, 8 papers with code
Strategy Coopetition Explains the Emergence and Transience of In-Context Learning
Firstly, we find that, after the disappearance of ICL, the asymptotic strategy is a remarkable hybrid between in-weights and in-context learning, which we term "context-constrained in-weights learning" (CIWL).
HARP: A challenging human-annotated math reasoning benchmark
Math reasoning is becoming an ever increasing area of focus as we scale large language models.
What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation
By clamping subsets of activations throughout training, we then identify three underlying subcircuits that interact to drive IH formation, yielding the phase change.
The Transient Nature of Emergent In-Context Learning in Transformers
The transient nature of ICL is observed in transformers across a range of model sizes and datasets, raising the question of how much to "overtrain" transformers when seeking compact, cheaper-to-run models.
Confronting Reward Model Overoptimization with Constrained RLHF
Large language models are typically aligned with human preferences by optimizing $\textit{reward models}$ (RMs) fitted to human feedback.
ReLOAD: Reinforcement Learning with Optimistic Ascent-Descent for Last-Iterate Convergence in Constrained MDPs
Such applications often require to put constraints on the agent's behavior.
Transfer RL via the Undo Maps Formalism
Transferring knowledge across domains is one of the most fundamental problems in machine learning, but doing so effectively in the context of reinforcement learning remains largely an open problem.
A Unified Theory of Dual-Process Control
We apply a single model based on this observation to findings from research on executive control, reward-based learning, and judgment and decision making, showing that seemingly diverse dual-process phenomena can be understood as domain-specific consequences of a single underlying set of computational principles.
Minimum Description Length Control
We propose a novel framework for multitask reinforcement learning based on the minimum description length (MDL) principle.
Towards an Understanding of Default Policies in Multitask Policy Optimization
Much of the recent success of deep reinforcement learning has been driven by regularized policy optimization (RPO) algorithms with strong performance across multiple domains.
A First-Occupancy Representation for Reinforcement Learning
Both animals and artificial agents benefit from state representations that support rapid transfer of learning across tasks and which enable them to efficiently traverse their environments to reach rewarding states.
Tactical Optimism and Pessimism for Deep Reinforcement Learning
In recent years, deep off-policy actor-critic algorithms have become a dominant approach to reinforcement learning for continuous control.
Efficient Wasserstein Natural Gradients for Reinforcement Learning
A novel optimization approach is proposed for application to policy gradient methods and evolution strategies for reinforcement learning (RL).
First-Order Preconditioning via Hypergradient Descent
Standard gradient descent methods are susceptible to a range of issues that can impede training, such as high correlations and different scaling in parameter space. These difficulties can be addressed by second-order approaches that apply a pre-conditioning matrix to the gradient to improve convergence.
