Search Results for author:
Found 38 papers, 19 papers with code
Reverse Training to Nurse the Reversal Curse
Large language models (LLMs) have a surprising failure: when trained on "A has a feature B", they do not generalize to "B is a feature of A", which is termed the Reversal Curse.
Branch-Train-MiX: Mixing Expert LLMs into a Mixture-of-Experts LLM
We investigate efficient methods for training Large Language Models (LLMs) to possess capabilities in multiple specialized domains, such as coding, math reasoning and world knowledge.
Ranked #30 on
Question Answering
on TriviaQA
Teaching Large Language Models to Reason with Reinforcement Learning
Surprisingly, we find the sample complexity of Expert Iteration is similar to that of PPO, requiring at most on the order of $10^6$ samples to converge from a pretrained checkpoint.
Beyond A*: Better Planning with Transformers via Search Dynamics Bootstrapping
In this work, we demonstrate how to train Transformers to solve complex planning tasks and present Searchformer, a Transformer model that optimally solves previously unseen Sokoban puzzles 93. 7% of the time, while using up to 26. 8% fewer search steps than standard $A^*$ search.
Self-Rewarding Language Models
We posit that to achieve superhuman agents, future models require superhuman feedback in order to provide an adequate training signal.
Some things are more CRINGE than others: Iterative Preference Optimization with the Pairwise Cringe Loss
Practitioners commonly align large language models using pairwise preferences, i. e., given labels of the type response A is preferred to response B for a given input.
System 2 Attention (is something you might need too)
Soft attention in Transformer-based Large Language Models (LLMs) is susceptible to incorporating irrelevant information from the context into its latent representations, which adversely affects next token generations.
A Data Source for Reasoning Embodied Agents
In this work, to further pursue these advances, we introduce a new data generator for machine reasoning that integrates with an embodied agent.
Improving Open Language Models by Learning from Organic Interactions
We present BlenderBot 3x, an update on the conversational model BlenderBot 3, which is now trained using organic conversation and feedback data from participating users of the system in order to improve both its skills and safety.
Large Language Model Programs
In recent years, large pre-trained language models (LLMs) have demonstrated the ability to follow instructions and perform novel tasks from a few examples.
Think Before You Act: Unified Policy for Interleaving Language Reasoning with Actions
The success of transformer models trained with a language modeling objective brings a promising opportunity to the reinforcement learning framework.
MINOTAUR: Multi-task Video Grounding From Multimodal Queries
Video understanding tasks take many forms, from action detection to visual query localization and spatio-temporal grounding of sentences.
Learning Goal-Conditioned Policies Offline with Self-Supervised Reward Shaping
Developing agents that can execute multiple skills by learning from pre-collected datasets is an important problem in robotics, where online interaction with the environment is extremely time-consuming.
The CRINGE Loss: Learning what language not to model
Standard language model training employs gold human documents or human-human interaction data, and treats all training data as positive examples.
Walk the Random Walk: Learning to Discover and Reach Goals Without Supervision
Finally, we train a goal-conditioned policy network with goals sampled from the goal memory and reward it by the reachability network and the goal memory.
DIRECTOR: Generator-Classifiers For Supervised Language Modeling
Current language models achieve low perplexity but their resulting generations still suffer from toxic responses, repetitiveness and contradictions.
Temporal Abstractions-Augmented Temporally Contrastive Learning: An Alternative to the Laplacian in RL
In reinforcement learning, the graph Laplacian has proved to be a valuable tool in the task-agnostic setting, with applications ranging from skill discovery to reward shaping.
Exploration-Driven Representation Learning in Reinforcement Learning
In this work, we introduce a method that explicitly couples representation learning with exploration when the agent is not provided with a uniform prior over the state space.
Hash Layers For Large Sparse Models
We investigate the training of sparse layers that use different parameters for different inputs based on hashing in large Transformer models.
Staircase Attention for Recurrent Processing of Sequences
Attention mechanisms have become a standard tool for sequence modeling tasks, in particular by stacking self-attention layers over the entire input sequence as in the Transformer architecture.
Not All Memories are Created Equal: Learning to Forget by Expiring
We demonstrate that Expire-Span can help models identify and retain critical information and show it can achieve strong performance on reinforcement learning tasks specifically designed to challenge this functionality.
Ranked #4 on
Language Modelling
on enwik8
Memory-Augmented Reinforcement Learning for Image-Goal Navigation
In this work, we present a memory-augmented approach for image-goal navigation.
Not All Memories are Created Equal: Learning to Expire
We demonstrate that Expire-Span can help models identify and retain critical information and show it can achieve state of the art results on long-context language modeling, reinforcement learning, and algorithmic tasks.
Learning to Visually Navigate in Photorealistic Environments Without any Supervision
Learning to navigate in a realistic setting where an agent must rely solely on visual inputs is a challenging task, in part because the lack of position information makes it difficult to provide supervision during training.
Addressing Some Limitations of Transformers with Feedback Memory
Transformers have been successfully applied to sequential, auto-regressive tasks despite being feedforward networks.
Augmenting Self-attention with Persistent Memory
More precisely, we augment the self-attention layers with persistent memory vectors that play a similar role as the feed-forward layer.
Ranked #5 on
Language Modelling
on Text8
Training Hybrid Language Models by Marginalizing over Segmentations
In this paper, we study the problem of hybrid language modeling, that is using models which can predict both characters and larger units such as character ngrams or words.
Adaptive Attention Span in Transformers
We propose a novel self-attention mechanism that can learn its optimal attention span.
Ranked #4 on
Language Modelling
on Text8
Learning when to Communicate at Scale in Multiagent Cooperative and Competitive Tasks
Learning when to communicate and doing that effectively is essential in multi-agent tasks.
Learning Goal Embeddings via Self-Play for Hierarchical Reinforcement Learning
In hierarchical reinforcement learning a major challenge is determining appropriate low-level policies.
Hierarchical Reinforcement Learning
reinforcement-learning
+1
Planning with Arithmetic and Geometric Attributes
A desirable property of an intelligent agent is its ability to understand its environment to quickly generalize to novel tasks and compose simpler tasks into more complex ones.
Composable Planning with Attributes
The tasks that an agent will need to solve often are not known during training.
Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Play
When Bob is deployed on an RL task within the environment, this unsupervised training reduces the number of supervised episodes needed to learn, and in some cases converges to a higher reward.
Learning Multiagent Communication with Backpropagation
Many tasks in AI require the collaboration of multiple agents.
Simple Baseline for Visual Question Answering
We describe a very simple bag-of-words baseline for visual question answering.
MazeBase: A Sandbox for Learning from Games
This paper introduces MazeBase: an environment for simple 2D games, designed as a sandbox for machine learning approaches to reasoning and planning.
End-To-End Memory Networks
For the former our approach is competitive with Memory Networks, but with less supervision.
Ranked #6 on
Question Answering
on bAbi
Training Convolutional Networks with Noisy Labels
The availability of large labeled datasets has allowed Convolutional Network models to achieve impressive recognition results.
