Search Results for author:
Found 68 papers, 36 papers with code
Debating with More Persuasive LLMs Leads to More Truthful Answers
In anticipation of this, we ask: can weaker models assess the correctness of stronger models?
Leading the Pack: N-player Opponent Shaping
We evaluate on over 4 different environments, varying the number of players from 3 to 5, and demonstrate that model-based OS methods converge to equilibrium with better global welfare than naive learning.
Scaling Opponent Shaping to High Dimensional Games
In multi-agent settings with mixed incentives, methods developed for zero-sum games have been shown to lead to detrimental outcomes.
H-GAP: Humanoid Control with a Generalist Planner
However, the extensive collection of human motion-captured data and the derived datasets of humanoid trajectories, such as MoCapAct, paves the way to tackle these challenges.
minimax: Efficient Baselines for Autocurricula in JAX
This compute requirement is a major obstacle to rapid innovation for the field.
Mechanistically analyzing the effects of fine-tuning on procedurally defined tasks
Fine-tuning large pre-trained models has become the de facto strategy for developing both task-specific and general-purpose machine learning systems, including developing models that are safe to deploy.
Understanding the Effects of RLHF on LLM Generalisation and Diversity
OOD generalisation is crucial given the wide range of real-world scenarios in which these models are being used, while output diversity refers to the model's ability to generate varied outputs and is important for a variety of use cases.
Finetuning from Offline Reinforcement Learning: Challenges, Trade-offs and Practical Solutions
While offline RL algorithms can in principle be used for finetuning, in practice, their online performance improves slowly.
Optimal Transport for Offline Imitation Learning
In this paper, we introduce Optimal Transport Reward labeling (OTR), an algorithm that assigns rewards to offline trajectories, with a few high-quality demonstrations.
General Intelligence Requires Rethinking Exploration
We are at the cusp of a transition from "learning from data" to "learning what data to learn from" as a central focus of artificial intelligence (AI) research.
The Goldilocks of Pragmatic Understanding: Fine-Tuning Strategy Matters for Implicature Resolution by LLMs
We present our findings as the starting point for further research into evaluating how LLMs interpret language in context and to drive the development of more pragmatic and useful models of human discourse.
Learning General World Models in a Handful of Reward-Free Deployments
We then present CASCADE, a novel approach for self-supervised exploration in this new setting.
Improving Policy Learning via Language Dynamics Distillation
Recent work has shown that augmenting environments with language descriptions improves policy learning.
Efficient Planning in a Compact Latent Action Space
Planning-based reinforcement learning has shown strong performance in tasks in discrete and low-dimensional continuous action spaces.
Hierarchical Kickstarting for Skill Transfer in Reinforcement Learning
In this paper, we investigate how skills can be incorporated into the training of reinforcement learning (RL) agents in complex environments with large state-action spaces and sparse rewards.
Grounding Aleatoric Uncertainty for Unsupervised Environment Design
Problematically, in partially-observable or stochastic settings, optimal policies may depend on the ground-truth distribution over aleatoric parameters of the environment in the intended deployment setting, while curriculum learning necessarily shifts the training distribution.
Graph Backup: Data Efficient Backup Exploiting Markovian Transitions
In this paper, we treat the transition data of the MDP as a graph, and define a novel backup operator, Graph Backup, which exploits this graph structure for better value estimation.
Insights From the NeurIPS 2021 NetHack Challenge
In this report, we summarize the takeaways from the first NeurIPS 2021 NetHack Challenge.
Evolving Curricula with Regret-Based Environment Design
Our approach, which we call Adversarially Compounding Complexity by Editing Levels (ACCEL), seeks to constantly produce levels at the frontier of an agent's capabilities, resulting in curricula that start simple but become increasingly complex.
Improving Intrinsic Exploration with Language Abstractions
Reinforcement learning (RL) agents are particularly hard to train when rewards are sparse.
A Survey of Zero-shot Generalisation in Deep Reinforcement Learning
This survey is an overview of this nascent field.
Replay-Guided Adversarial Environment Design
Furthermore, our theory suggests a highly counterintuitive improvement to PLR: by stopping the agent from updating its policy on uncurated levels (training on less data), we can improve the convergence to Nash equilibria.
That Escalated Quickly: Compounding Complexity by Editing Levels at the Frontier of Agent Capabilities
Deep Reinforcement Learning (RL) has recently produced impressive results in a series of settings such as games and robotics.
MiniHack the Planet: A Sandbox for Open-Ended Reinforcement Learning Research
By leveraging the full set of entities and environment dynamics from NetHack, one of the richest grid-based video games, MiniHack allows designing custom RL testbeds that are fast and convenient to use.
Return Dispersion as an Estimator of Learning Potential for Prioritized Level Replay
Prioritized Level Replay (PLR) has been shown to induce adaptive curricula that improve the sample-efficiency and generalization of reinforcement learning policies in environments featuring multiple tasks or levels.
Prioritized Level Replay
Environments with procedurally generated content serve as important benchmarks for testing systematic generalization in deep reinforcement learning.
Learning Reasoning Strategies in End-to-End Differentiable Proving
Attempts to render deep learning models interpretable, data-efficient, and robust have seen some success through hybridisation with rule-based systems, for example, in Neural Theorem Provers (NTPs).
Ranked #1 on
Relational Reasoning
on CLUTRR (k=3)
The NetHack Learning Environment
Here, we present the NetHack Learning Environment (NLE), a scalable, procedurally generated, stochastic, rich, and challenging environment for RL research based on the popular single-player terminal-based roguelike game, NetHack.
Ranked #1 on
NetHack Score
on NetHack Learning Environment
Learning with AMIGo: Adversarially Motivated Intrinsic Goals
A key challenge for reinforcement learning (RL) consists of learning in environments with sparse extrinsic rewards.
RTFM: Generalising to New Environment Dynamics via Reading
In this work, we demonstrate that language understanding via a reading policy learner is a promising vehicle for generalisation to new environments.
Differentiable Reasoning on Large Knowledge Bases and Natural Language
Reasoning with knowledge expressed in natural language and Knowledge Bases (KBs) is a major challenge for Artificial Intelligence, with applications in machine reading, dialogue, and question answering.
Ranked #3 on
Link Prediction
on FB122
RTFM: Generalising to Novel Environment Dynamics via Reading
In this work, we demonstrate that language understanding via a reading policy learner is a promising vehicle for generalisation to new environments.
TorchBeast: A PyTorch Platform for Distributed RL
TorchBeast is a platform for reinforcement learning (RL) research in PyTorch.
Generalized Inner Loop Meta-Learning
Many (but not all) approaches self-qualifying as "meta-learning" in deep learning and reinforcement learning fit a common pattern of approximating the solution to a nested optimization problem.
Meta Learning via Learned Loss
We present a meta-learning method for learning parametric loss functions that can generalize across different tasks and model architectures.
Meta-Learning via Learned Loss
This information shapes the learned loss function such that the environment does not need to provide this information during meta-test time.
A Survey of Reinforcement Learning Informed by Natural Language
To be successful in real-world tasks, Reinforcement Learning (RL) needs to exploit the compositional, relational, and hierarchical structure of the world, and learn to transfer it to the task at hand.
Scalable Neural Theorem Proving on Knowledge Bases and Natural Language
Reasoning over text and Knowledge Bases (KBs) is a major challenge for Artificial Intelligence, with applications in machine reading, dialogue, and question answering.
Knowing When to Stop: Evaluation and Verification of Conformity to Output-size Specifications
This behavior can have severe consequences such as usage of increased computation and induce faults in downstream modules that expect outputs of a certain length.
Analysing Mathematical Reasoning Abilities of Neural Models
The structured nature of the mathematics domain, covering arithmetic, algebra, probability and calculus, enables the construction of training and test splits designed to clearly illuminate the capabilities and failure-modes of different architectures, as well as evaluate their ability to compose and relate knowledge and learned processes.
Ranked #2 on
Question Answering
on Mathematics Dataset
CompILE: Compositional Imitation Learning and Execution
We introduce Compositional Imitation Learning and Execution (CompILE): a framework for learning reusable, variable-length segments of hierarchically-structured behavior from demonstration data.
Strength in Numbers: Trading-off Robustness and Computation via Adversarially-Trained Ensembles
We show that increasing the number of parameters in adversarially-trained models increases their robustness, and in particular that ensembling smaller models while adversarially training the entire ensemble as a single model is a more efficient way of spending said budget than simply using a larger single model.
Learning to Understand Goal Specifications by Modelling Reward
Recent work has shown that deep reinforcement-learning agents can learn to follow language-like instructions from infrequent environment rewards.
Can Neural Networks Understand Logical Entailment?
We introduce a new dataset of logical entailments for the purpose of measuring models' ability to capture and exploit the structure of logical expressions against an entailment prediction task.
The NarrativeQA Reading Comprehension Challenge
Reading comprehension (RC)---in contrast to information retrieval---requires integrating information and reasoning about events, entities, and their relations across a full document.
Ranked #9 on
Question Answering
on NarrativeQA
(BLEU-1 metric)
Learning Explanatory Rules from Noisy Data
Artificial Neural Networks are powerful function approximators capable of modelling solutions to a wide variety of problems, both supervised and unsupervised.
Deep Learning for Semantic Composition
Learning representation to model the meaning of text has been a core problem in NLP.
Discovering Discrete Latent Topics with Neural Variational Inference
Topic models have been widely explored as probabilistic generative models of documents.
Ranked #2 on
Topic Models
on 20NewsGroups
Learning to Compute Word Embeddings On the Fly
Words in natural language follow a Zipfian distribution whereby some words are frequent but most are rare.
Ranked #48 on
Question Answering
on SQuAD1.1 dev
Learning to Compose Words into Sentences with Reinforcement Learning
We use reinforcement learning to learn tree-structured neural networks for computing representations of natural language sentences.
The Neural Noisy Channel
We formulate sequence to sequence transduction as a noisy channel decoding problem and use recurrent neural networks to parameterise the source and channel models.
Semantic Parsing with Semi-Supervised Sequential Autoencoders
We present a novel semi-supervised approach for sequence transduction and apply it to semantic parsing.
Latent Predictor Networks for Code Generation
Many language generation tasks require the production of text conditioned on both structured and unstructured inputs.
Ranked #10 on
Code Generation
on Django
Reasoning about Entailment with Neural Attention
We extend this model with a word-by-word neural attention mechanism that encourages reasoning over entailments of pairs of words and phrases.
Ranked #83 on
Natural Language Inference
on SNLI
Teaching Machines to Read and Comprehend
Teaching machines to read natural language documents remains an elusive challenge.
Ranked #13 on
Question Answering
on CNN / Daily Mail
Learning to Transduce with Unbounded Memory
Recently, strong results have been demonstrated by Deep Recurrent Neural Networks on natural language transduction problems.
Investigating the Role of Prior Disambiguation in Deep-learning Compositional Models of Meaning
This paper aims to explore the effect of prior disambiguation on neural network- based compositional models, with the hope that better semantic representations for text compounds can be produced.
A Deep Architecture for Semantic Parsing
Many successful approaches to semantic parsing build on top of the syntactic analysis of text, and make use of distributional representations or statistical models to match parses to ontology-specific queries.
A Convolutional Neural Network for Modelling Sentences
The ability to accurately represent sentences is central to language understanding.
Ranked #48 on
Natural Language Inference
on SNLI
Category-Theoretic Quantitative Compositional Distributional Models of Natural Language Semantics
This thesis focuses on a particular approach to this compositionality problem, namely using the categorical framework developed by Coecke, Sadrzadeh, and Clark, which combines syntactic analysis formalisms with distributional semantic representations of meaning to produce syntactically motivated composition operations.
"Not not bad" is not "bad": A distributional account of negation
With the increasing empirical success of distributional models of compositional semantics, it is timely to consider the types of textual logic that such models are capable of capturing.
A quantum teleportation inspired algorithm produces sentence meaning from word meaning and grammatical structure
We discuss an algorithm which produces the meaning of a sentence given meanings of its words, and its resemblance to quantum teleportation.
Towards a Formal Distributional Semantics: Simulating Logical Calculi with Tensors
This paper seeks to bring this reconciliation one step further by showing how the mathematical constructs commonly used in compositional distributional models, such as tensors and matrices, can be used to simulate different aspects of predicate logic.
Experimental Support for a Categorical Compositional Distributional Model of Meaning
The evaluation is based on the word disambiguation task developed by Mitchell and Lapata (2008) for intransitive sentences, and on a similar new experiment designed for transitive sentences.
