Search Results for author:
Found 22 papers, 10 papers with code
Automating the Search for Artificial Life with Foundation Models
With the recent Nobel Prize awarded for radical advances in protein discovery, foundation models (FMs) for exploring large combinatorial spaces promise to revolutionize many scientific fields.
Adversarial Robustness of In-Context Learning in Transformers for Linear Regression
This work investigates the vulnerability of in-context learning in transformers to \textit{hijacking attacks} focusing on the setting of linear regression tasks.
Language Agents as Optimizable Graphs
Various human-designed prompt engineering techniques have been proposed to improve problem solvers based on Large Language Models (LLMs), yielding many disparate code bases.
Discovering Temporally-Aware Reinforcement Learning Algorithms
We propose a simple augmentation to two existing objective discovery approaches that allows the discovered algorithm to dynamically update its objective function throughout the agent's training procedure, resulting in expressive schedules and increased generalization across different training horizons.
The Languini Kitchen: Enabling Language Modelling Research at Different Scales of Compute
We introduce an experimental protocol that enables model comparisons based on equivalent compute, measured in accelerator hours.
Mindstorms in Natural Language-Based Societies of Mind
What should be the social structure of an NLSOM?
Brain-inspired learning in artificial neural networks: a review
Artificial neural networks (ANNs) have emerged as an essential tool in machine learning, achieving remarkable success across diverse domains, including image and speech generation, game playing, and robotics.
Eliminating Meta Optimization Through Self-Referential Meta Learning
We discuss the relationship of such systems to in-context and memory-based meta learning and show that self-referential neural networks require functionality to be reused in the form of parameter sharing.
Learning One Abstract Bit at a Time Through Self-Invented Experiments Encoded as Neural Networks
There are two important things in science: (A) Finding answers to given questions, and (B) Coming up with good questions.
General-Purpose In-Context Learning by Meta-Learning Transformers
We further show that the capabilities of meta-trained algorithms are bottlenecked by the accessible state size (memory) determining the next prediction, unlike standard models which are thought to be bottlenecked by parameter count.
Exploring through Random Curiosity with General Value Functions
Furthermore, RC-GVF significantly outperforms previous methods in the absence of ground-truth episodic counts in the partially observable MiniGrid environments.
The Benefits of Model-Based Generalization in Reinforcement Learning
First, we provide a simple theorem motivating how learning a model as an intermediate step can narrow down the set of possible value functions more than learning a value function directly from data using the Bellman equation.
Model-based Reinforcement Learning
reinforcement-learning
+2
Goal-Conditioned Generators of Deep Policies
A form of weight-sharing HyperNetworks and policy embeddings scales our method to generate deep NNs.
Introducing Symmetries to Black Box Meta Reinforcement Learning
We show that a recent successful meta RL approach that meta-learns an objective for backpropagation-based learning exhibits certain symmetries (specifically the reuse of the learning rule, and invariance to input and output permutations) that are not present in typical black-box meta RL systems.
Meta Learning Backpropagation And Improving It
Many concepts have been proposed for meta learning with neural networks (NNs), e. g., NNs that learn to reprogram fast weights, Hebbian plasticity, learned learning rules, and meta recurrent NNs.
Parameter-Based Value Functions
We introduce a class of value functions called Parameter-Based Value Functions (PBVFs) whose inputs include the policy parameters.
Improving Generalization in Meta Reinforcement Learning using Learned Objectives
Biological evolution has distilled the experiences of many learners into the general learning algorithms of humans.
Gaussian Mean Field Regularizes by Limiting Learned Information
Variational inference with a factorized Gaussian posterior estimate is a widely used approach for learning parameters and hidden variables.
Modular Networks: Learning to Decompose Neural Computation
Scaling model capacity has been vital in the success of deep learning.
Noisy Information Bottlenecks for Generalization
We propose Noisy Information Bottlenecks (NIB) to limit mutual information between learned parameters and the data through noise.
Transfer Learning for Speech Recognition on a Budget
End-to-end training of automated speech recognition (ASR) systems requires massive data and compute resources.
