Search Results for author:
Found 17 papers, 4 papers with code
RecurrentGemma: Moving Past Transformers for Efficient Open Language Models
We introduce RecurrentGemma, a family of open language models which uses Google's novel Griffin architecture.
Gemma: Open Models Based on Gemini Research and Technology
This work introduces Gemma, a family of lightweight, state-of-the art open models built from the research and technology used to create Gemini models.
Direct Language Model Alignment from Online AI Feedback
Moreover, responses in these datasets are often sampled from a language model distinct from the one being aligned, and since the model evolves over training, the alignment phase is inevitably off-policy.
A Survey of Temporal Credit Assignment in Deep Reinforcement Learning
The Credit Assignment Problem (CAP) refers to the longstanding challenge of Reinforcement Learning (RL) agents to associate actions with their long-term consequences.
RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedback
Reinforcement learning from human feedback (RLHF) has proven effective in aligning large language models (LLMs) with human preferences, but gathering high-quality preference labels is expensive.
Curiosity in Hindsight: Intrinsic Exploration in Stochastic Environments
In this work, we study a natural solution derived from structural causal models of the world: Our key idea is to learn representations of the future that capture precisely the unpredictable aspects of each outcome -- which we use as additional input for predictions, such that intrinsic rewards only reflect the predictable aspects of world dynamics.
Geometric Entropic Exploration
Exploration is essential for solving complex Reinforcement Learning (RL) tasks.
Model-Free Counterfactual Credit Assignment
Credit assignment in reinforcement learning is the problem of measuring an action’s influence on future rewards.
Counterfactual Credit Assignment in Model-Free Reinforcement Learning
Credit assignment in reinforcement learning is the problem of measuring an action's influence on future rewards.
Hindsight Credit Assignment
We consider the problem of efficient credit assignment in reinforcement learning.
Ghost Units Yield Biologically Plausible Backprop in Deep Neural Networks
This reduced system combines the essential elements to have a working biologically abstracted analogue of backpropagation with a simple formulation and proofs of the associated results.
Generalization of Equilibrium Propagation to Vector Field Dynamics
The biological plausibility of the backpropagation algorithm has long been doubted by neuroscientists.
Extending the Framework of Equilibrium Propagation to General Dynamics
The biological plausibility of the backpropagation algorithm has long been doubted by neuroscientists.
Towards deep learning with spiking neurons in energy based models with contrastive Hebbian plasticity
In machine learning, error back-propagation in multi-layer neural networks (deep learning) has been impressively successful in supervised and reinforcement learning tasks.
STDP as presynaptic activity times rate of change of postsynaptic activity
We introduce a weight update formula that is expressed only in terms of firing rates and their derivatives and that results in changes consistent with those associated with spike-timing dependent plasticity (STDP) rules and biological observations, even though the explicit timing of spikes is not needed.
Towards Biologically Plausible Deep Learning
Neuroscientists have long criticised deep learning algorithms as incompatible with current knowledge of neurobiology.
