Search Results for author:
Found 71 papers, 44 papers with code
Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolution
Driven by an LLM, Promptbreeder mutates a population of task-prompts, and subsequently evaluates them for fitness on a training set.
Stabilizing Unsupervised Environment Design with a Learned Adversary
As a result, we make it possible for PAIRED to match or exceed state-of-the-art methods, producing robust agents in several established challenging procedurally-generated environments, including a partially-observed maze navigation task and a continuous-control car racing environment.
MAESTRO: Open-Ended Environment Design for Multi-Agent Reinforcement Learning
Open-ended learning methods that automatically generate a curriculum of increasingly challenging tasks serve as a promising avenue toward generally capable reinforcement learning agents.
Human-Timescale Adaptation in an Open-Ended Task Space
Foundation models have shown impressive adaptation and scalability in supervised and self-supervised learning problems, but so far these successes have not fully translated to reinforcement learning (RL).
General Intelligence Requires Rethinking Exploration
We are at the cusp of a transition from "learning from data" to "learning what data to learn from" as a central focus of artificial intelligence (AI) research.
Dungeons and Data: A Large-Scale NetHack Dataset
Recent breakthroughs in the development of agents to solve challenging sequential decision making problems such as Go, StarCraft, or DOTA, have relied on both simulated environments and large-scale datasets.
Large language models are not zero-shot communicators
We present our findings as the starting point for further research into evaluating how LLMs interpret language in context and to drive the development of more pragmatic and useful models of human discourse.
Learning General World Models in a Handful of Reward-Free Deployments
We then present CASCADE, a novel approach for self-supervised exploration in this new setting.
Exploration via Elliptical Episodic Bonuses
In recent years, a number of reinforcement learning (RL) methods have been proposed to explore complex environments which differ across episodes.
Improving Policy Learning via Language Dynamics Distillation
Recent work has shown that augmenting environments with language descriptions improves policy learning.
Efficient Planning in a Compact Latent Action Space
Planning-based reinforcement learning has shown strong performance in tasks in discrete and low-dimensional continuous action spaces.
Hierarchical Kickstarting for Skill Transfer in Reinforcement Learning
In this paper, we investigate how skills can be incorporated into the training of reinforcement learning (RL) agents in complex environments with large state-action spaces and sparse rewards.
GriddlyJS: A Web IDE for Reinforcement Learning
Progress in reinforcement learning (RL) research is often driven by the design of new, challenging environments -- a costly undertaking requiring skills orthogonal to that of a typical machine learning researcher.
Grounding Aleatoric Uncertainty for Unsupervised Environment Design
Problematically, in partially-observable or stochastic settings, optimal policies may depend on the ground-truth distribution over aleatoric parameters of the environment in the intended deployment setting, while curriculum learning necessarily shifts the training distribution.
Graph Backup: Data Efficient Backup Exploiting Markovian Transitions
In this paper, we treat the transition data of the MDP as a graph, and define a novel backup operator, Graph Backup, which exploits this graph structure for better value estimation.
Insights From the NeurIPS 2021 NetHack Challenge
In this report, we summarize the takeaways from the first NeurIPS 2021 NetHack Challenge.
Evolving Curricula with Regret-Based Environment Design
Our approach, which we call Adversarially Compounding Complexity by Editing Levels (ACCEL), seeks to constantly produce levels at the frontier of an agent's capabilities, resulting in curricula that start simple but become increasingly complex.
Improving Intrinsic Exploration with Language Abstractions
Reinforcement learning (RL) agents are particularly hard to train when rewards are sparse.
Generalization in Cooperative Multi-Agent Systems
Specifically, we study generalization bounds under a linear dependence of the underlying dynamics on the agent capabilities, which can be seen as a generalization of Successor Features to MAS.
A Survey of Zero-shot Generalisation in Deep Reinforcement Learning
This survey is an overview of this nascent field.
Replay-Guided Adversarial Environment Design
Furthermore, our theory suggests a highly counterintuitive improvement to PLR: by stopping the agent from updating its policy on uncurated levels (training on less data), we can improve the convergence to Nash equilibria.
That Escalated Quickly: Compounding Complexity by Editing Levels at the Frontier of Agent Capabilities
Deep Reinforcement Learning (RL) has recently produced impressive results in a series of settings such as games and robotics.
MiniHack the Planet: A Sandbox for Open-Ended Reinforcement Learning Research
By leveraging the full set of entities and environment dynamics from NetHack, one of the richest grid-based video games, MiniHack allows designing custom RL testbeds that are fast and convenient to use.
Return Dispersion as an Estimator of Learning Potential for Prioritized Level Replay
Prioritized Level Replay (PLR) has been shown to induce adaptive curricula that improve the sample-efficiency and generalization of reinforcement learning policies in environments featuring multiple tasks or levels.
Don't Sweep your Learning Rate under the Rug: A Closer Look at Cross-modal Transfer of Pretrained Transformers
After carefully redesigning the empirical setup, we find that when tuning learning rates properly, pretrained transformers do outperform or match training from scratch in all of our tasks, but only as long as the entire model is finetuned.
Prioritized Level Replay
Environments with procedurally generated content serve as important benchmarks for testing systematic generalization in deep reinforcement learning.
My Body is a Cage: the Role of Morphology in Graph-Based Incompatible Control
They also allow practitioners to inject biases encoded in the structure of the input graph.
How to Motivate Your Dragon: Teaching Goal-Driven Agents to Speak and Act in Fantasy Worlds
We seek to create agents that both act and communicate with other agents in pursuit of a goal.
KILT: a Benchmark for Knowledge Intensive Language Tasks
We test both task-specific and general baselines, evaluating downstream performance in addition to the ability of the models to provide provenance.
Ranked #3 on
Entity Linking
on KILT: WNED-CWEB
WordCraft: An Environment for Benchmarking Commonsense Agents
This is partly due to the lack of lightweight simulation environments that sufficiently reflect the semantics of the real world and provide knowledge sources grounded with respect to observations in an RL environment.
Learning Reasoning Strategies in End-to-End Differentiable Proving
Attempts to render deep learning models interpretable, data-efficient, and robust have seen some success through hybridisation with rule-based systems, for example, in Neural Theorem Provers (NTPs).
Ranked #1 on
Relational Reasoning
on CLUTRR (k=3)
The NetHack Learning Environment
Here, we present the NetHack Learning Environment (NLE), a scalable, procedurally generated, stochastic, rich, and challenging environment for RL research based on the popular single-player terminal-based roguelike game, NetHack.
Ranked #1 on
NetHack Score
on NetHack Learning Environment
Learning with AMIGo: Adversarially Motivated Intrinsic Goals
A key challenge for reinforcement learning (RL) consists of learning in environments with sparse extrinsic rewards.
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Large pre-trained language models have been shown to store factual knowledge in their parameters, and achieve state-of-the-art results when fine-tuned on downstream NLP tasks.
Ranked #4 on
Question Answering
on WebQuestions
How Context Affects Language Models' Factual Predictions
When pre-trained on large unsupervised textual corpora, language models are able to store and retrieve factual knowledge to some extent, making it possible to use them directly for zero-shot cloze-style question answering.
RTFM: Generalising to New Environment Dynamics via Reading
In this work, we demonstrate that language understanding via a reading policy learner is a promising vehicle for generalisation to new environments.
Avoiding the Hypothesis-Only Bias in Natural Language Inference via Ensemble Adversarial Training
Natural Language Inference (NLI) datasets contain annotation artefacts resulting in spurious correlations between the natural language utterances and their respective entailment classes.
RIDE: Rewarding Impact-Driven Exploration for Procedurally-Generated Environments
However, we show that existing methods fall short in procedurally-generated environments where an agent is unlikely to visit a state more than once.
Differentiable Reasoning on Large Knowledge Bases and Natural Language
Reasoning with knowledge expressed in natural language and Knowledge Bases (KBs) is a major challenge for Artificial Intelligence, with applications in machine reading, dialogue, and question answering.
Ranked #3 on
Link Prediction
on FB122
How Decoding Strategies Affect the Verifiability of Generated Text
A generated sentence is verifiable if it can be corroborated or disproved by Wikipedia, and we find that the verifiability of generated text strongly depends on the decoding strategy.
RTFM: Generalising to Novel Environment Dynamics via Reading
In this work, we demonstrate that language understanding via a reading policy learner is a promising vehicle for generalisation to new environments.
MVFST-RL: An Asynchronous RL Framework for Congestion Control with Delayed Actions
This is largely an artifact of building on top of frameworks designed for RL in games (e. g. OpenAI Gym).
TorchBeast: A PyTorch Platform for Distributed RL
TorchBeast is a platform for reinforcement learning (RL) research in PyTorch.
Language Models as Knowledge Bases?
Recent progress in pretraining language models on large textual corpora led to a surge of improvements for downstream NLP tasks.
NLProlog: Reasoning with Weak Unification for Question Answering in Natural Language
In contrast, neural models can cope very well with ambiguity by learning distributed representations of words and their composition from data, but lead to models that are difficult to interpret.
A Survey of Reinforcement Learning Informed by Natural Language
To be successful in real-world tasks, Reinforcement Learning (RL) needs to exploit the compositional, relational, and hierarchical structure of the world, and learn to transfer it to the task at hand.
Scalable Neural Theorem Proving on Knowledge Bases and Natural Language
Reasoning over text and Knowledge Bases (KBs) is a major challenge for Artificial Intelligence, with applications in machine reading, dialogue, and question answering.
NLProlog: Reasoning with Weak Unification for Natural Language Question Answering
Currently, most work in natural language processing focuses on neural networks which learn distributed representations of words and their composition, thereby performing well in the presence of large linguistic variability.
Learning to Speak and Act in a Fantasy Text Adventure Game
We analyze the ingredients necessary for successful grounding in this setting, and how each of these factors relate to agents that can talk and act successfully.
e-SNLI: Natural Language Inference with Natural Language Explanations
In order for machine learning to garner widespread public adoption, models must be able to provide interpretable and robust explanations for their decisions, as well as learn from human-provided explanations at train time.
Ranked #1 on
Natural Language Inference
on e-SNLI
Stable Opponent Shaping in Differentiable Games
A growing number of learning methods are actually differentiable games whose players optimise multiple, interdependent objectives in parallel -- from GANs and intrinsic curiosity to multi-agent RL.
A Better Baseline for Second Order Gradient Estimation in Stochastic Computation Graphs
To improve the sample efficiency of DiCE, we propose a new baseline term for higher order gradient estimation.
Interpretation of Natural Language Rules in Conversational Machine Reading
This task requires both the interpretation of rules and the application of background knowledge.
Towards Neural Theorem Proving at Scale
Neural models combining representation learning and reasoning in an end-to-end trainable manner are receiving increasing interest.
DiCE: The Infinitely Differentiable Monte Carlo Estimator
Lastly, to match the first-order gradient under differentiation, SL treats part of the cost as a fixed sample, which we show leads to missing and wrong terms for estimators of higher-order derivatives.
Jack the Reader - A Machine Reading Framework
For example, in Question Answering, the supporting text can be newswire or Wikipedia articles; in Natural Language Inference, premises can be seen as the supporting text and hypotheses as questions.
DiCE: The Infinitely Differentiable Monte-Carlo Estimator
Lastly, to match the first-order gradient under differentiation, SL treats part of the cost as a fixed sample, which we show leads to missing and wrong terms for estimators of higher-order derivatives.
Combining Representation Learning with Logic for Language Processing
The current state-of-the-art in many natural language processing and automated knowledge base completion tasks is held by representation learning methods which learn distributed vector representations of symbols via gradient-based optimization.
TreeQN and ATreeC: Differentiable Tree-Structured Models for Deep Reinforcement Learning
To address these challenges, we propose TreeQN, a differentiable, recursive, tree-structured model that serves as a drop-in replacement for any value function network in deep RL with discrete actions.
Adversarial Sets for Regularising Neural Link Predictors
The training objective is defined as a minimax problem, where an adversary finds the most offending adversarial examples by maximising the inconsistency loss, and the model is trained by jointly minimising a supervised loss and the inconsistency loss on the adversarial examples.
End-to-End Differentiable Proving
We introduce neural networks for end-to-end differentiable proving of queries to knowledge bases by operating on dense vector representations of symbols.
Frustratingly Short Attention Spans in Neural Language Modeling
This vector is used both for predicting the next token as well as for the key and value of a differentiable memory of a token history.
Learning Python Code Suggestion with a Sparse Pointer Network
By augmenting a neural language model with a pointer network specialized in referring to predefined classes of identifiers, we obtain a much lower perplexity and a 5 percentage points increase in accuracy for code suggestion compared to an LSTM baseline.
emoji2vec: Learning Emoji Representations from their Description
Many current natural language processing applications for social media rely on representation learning and utilize pre-trained word embeddings.
Lifted Rule Injection for Relation Embeddings
Methods based on representation learning currently hold the state-of-the-art in many natural language processing and knowledge base inference tasks.
Stance Detection with Bidirectional Conditional Encoding
Stance detection is the task of classifying the attitude expressed in a text towards a target such as Hillary Clinton to be "positive", negative" or "neutral".
MuFuRU: The Multi-Function Recurrent Unit
Recurrent neural networks such as the GRU and LSTM found wide adoption in natural language processing and achieve state-of-the-art results for many tasks.
Generating Natural Language Inference Chains
We take entailment-pairs of the Stanford Natural Language Inference corpus and train an LSTM with attention.
Programming with a Differentiable Forth Interpreter
Given that in practice training data is scarce for all but a small set of problems, a core question is how to incorporate prior knowledge into a model.
Reasoning about Entailment with Neural Attention
We extend this model with a word-by-word neural attention mechanism that encourages reasoning over entailments of pairs of words and phrases.
Ranked #84 on
Natural Language Inference
on SNLI
