Search Results for author:
Found 61 papers, 26 papers with code
What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation
By clamping subsets of activations throughout training, we then identify three underlying subcircuits that interact to drive IH formation, yielding the phase change.
Scaling Instructable Agents Across Many Simulated Worlds
Building embodied AI systems that can follow arbitrary language instructions in any 3D environment is a key challenge for creating general AI.
SODA: Bottleneck Diffusion Models for Representation Learning
We introduce SODA, a self-supervised diffusion model, designed for representation learning.
The Transient Nature of Emergent In-Context Learning in Transformers
The transient nature of ICL is observed in transformers across a range of model sizes and datasets, raising the question of how much to "overtrain" transformers when seeking compact, cheaper-to-run models.
Vision-Language Models as Success Detectors
Detecting successful behaviour is crucial for training intelligent agents.
The Edge of Orthogonality: A Simple View of What Makes BYOL Tick
With simple linear algebra, we show that when using a linear predictor, the optimal predictor is close to an orthogonal projection, and propose a general framework based on orthonormalization that enables to interpret and give intuition on why BYOL works.
Collaborating with language models for embodied reasoning
On the other hand, Large Scale Language Models (LSLMs) have exhibited strong reasoning ability and the ability to to adapt to new tasks through in-context learning.
SemPPL: Predicting pseudo-labels for better contrastive representations
We propose a new semi-supervised learning method, Semantic Positives via Pseudo-Labels (SemPPL), that combines labelled and unlabelled data to learn informative representations.
Transformers generalize differently from information stored in context vs in weights
In transformers trained on controlled stimuli, we find that generalization from weights is more rule-based whereas generalization from context is largely exemplar-based.
Meaning without reference in large language models
The widespread success of large language models (LLMs) has been met with skepticism that they possess anything like human concepts or meanings.
Language models show human-like content effects on reasoning tasks
We evaluate state of the art large language models, as well as humans, and find that the language models reflect many of the same patterns observed in humans across these tasks $\unicode{x2014}$ like humans, models answer more accurately when the semantic content of a task supports the logical inferences.
Know your audience: specializing grounded language models with listener subtraction
Through controlled experiments, we show that training a speaker with two listeners that perceive differently, using our method, allows the speaker to adapt to the idiosyncracies of the listeners.
Data Distributional Properties Drive Emergent In-Context Learning in Transformers
In further experiments, we found that naturalistic data distributions were only able to elicit in-context learning in transformers, and not in recurrent models.
Semantic Exploration from Language Abstractions and Pretrained Representations
We show that these pretrained representations drive meaningful, task-relevant exploration and improve performance on 3D simulated environments.
Can language models learn from explanations in context?
In summary, explanations can support the in-context learning of large LMs on challenging tasks.
Zipfian environments for Reinforcement Learning
As humans and animals learn in the natural world, they encounter distributions of entities, situations and events that are far from uniform.
Feature-Attending Recurrent Modules for Generalization in Reinforcement Learning
Many important tasks are defined in terms of object.
Tell me why! Explanations support learning relational and causal structure
Inferring the abstract relational and causal structure of the world is a major challenge for reinforcement-learning (RL) agents.
Creating Multimodal Interactive Agents with Imitation and Self-Supervised Learning
A common vision from science fiction is that robots will one day inhabit our physical spaces, sense the world as we do, assist our physical labours, and communicate with us through natural language.
BEAMetrics: A Benchmark for Language Generation Evaluation Evaluation
There is currently no simple, unified way to compare, analyse or evaluate metrics across a representative set of tasks.
Tell me why!—Explanations support learning relational and causal structure
Explanations play a considerable role in human learning, especially in areas that remain major challenges for AI—forming abstractions, and learning about the relational and causal structure of the world.
Task-driven Discovery of Perceptual Schemas for Generalization in Reinforcement Learning
Taking inspiration from cognitive science, we term representations for reoccurring segments of an agent's experience, "perceptual schemas".
Multimodal Few-Shot Learning with Frozen Language Models
When trained at sufficient scale, auto-regressive language models exhibit the notable ability to learn a new language task after being prompted with just a few examples.
Ranked #11 on
Visual Question Answering (VQA)
on VQA v2 val
Towards mental time travel: a hierarchical memory for reinforcement learning agents
Agents with common memory architectures struggle to recall and integrate across multiple timesteps of a past event, or even to recall the details of a single timestep that is followed by distractor tasks.
Neural spatio-temporal reasoning with object-centric self-supervised learning
Transformer-based language models have proved capable of rudimentary symbolic reasoning, underlining the effectiveness of applying self-attention computations to sets of discrete entities.
Attention over learned object embeddings enables complex visual reasoning
Neural networks have achieved success in a wide array of perceptual tasks but often fail at tasks involving both perception and higher-level reasoning.
Ranked #4 on
Video Object Tracking
on CATER
Imitating Interactive Intelligence
These evaluations convincingly demonstrate that interactive training and auxiliary losses improve agent behaviour beyond what is achieved by supervised learning of actions alone.
Grounded Language Learning Fast and Slow
Recent work has shown that large text-based neural language models, trained with conventional supervised learning objectives, acquire a surprising propensity for few- and one-shot learning.
Probing Emergent Semantics in Predictive Agents via Question Answering
Recent work has shown how predictive modeling can endow agents with rich knowledge of their surroundings, improving their ability to act in complex environments.
Human Instruction-Following with Deep Reinforcement Learning via Transfer-Learning from Text
Here, we propose a conceptually simple method for training instruction-following agents with deep RL that are robust to natural human instructions.
Extending Machine Language Models toward Human-Level Language Understanding
We take language to be a part of a system for understanding and communicating about situations.
Environmental drivers of systematicity and generalization in a situated agent
The question of whether deep neural networks are good at generalising beyond their immediate training experience is of critical importance for learning-based approaches to AI.
Robust Instruction-Following in a Situated Agent via Transfer-Learning from Text
We address this issue by integrating language encoders that are pretrained on large text corpora into a situated, instruction-following agent.
Higher-order Comparisons of Sentence Encoder Representations
Representational Similarity Analysis (RSA) is a technique developed by neuroscientists for comparing activity patterns of different measurement modalities (e. g., fMRI, electrophysiology, behavior).
SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems
In the last year, new models and methods for pretraining and transfer learning have driven striking performance improvements across a range of language understanding tasks.
Is coding a relevant metaphor for building AI? A commentary on "Is coding a relevant metaphor for the brain?", by Romain Brette
Brette contends that the neural coding metaphor is an invalid basis for theories of what the brain does.
Analysing Mathematical Reasoning Abilities of Neural Models
The structured nature of the mathematics domain, covering arithmetic, algebra, probability and calculus, enables the construction of training and test splits designed to clearly illuminate the capabilities and failure-modes of different architectures, as well as evaluate their ability to compose and relate knowledge and learned processes.
Ranked #2 on
Question Answering
on Mathematics Dataset
Learning to Make Analogies by Contrasting Abstract Relational Structure
Here, we study how analogical reasoning can be induced in neural networks that learn to perceive and reason about raw visual data.
Generating Diverse Programs with Instruction Conditioned Reinforced Adversarial Learning
In this work, we study the setting in which an agent must learn to generate programs for diverse scenes conditioned on a given symbolic instruction.
Neural Arithmetic Logic Units
Neural networks can learn to represent and manipulate numerical information, but they seldom generalize well outside of the range of numerical values encountered during training.
Measuring abstract reasoning in neural networks
To succeed at this challenge, models must cope with various generalisation `regimes' in which the training and test data differ in clearly-defined ways.
Learning to Understand Goal Specifications by Modelling Reward
Recent work has shown that deep reinforcement-learning agents can learn to follow language-like instructions from infrequent environment rewards.
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
For natural language understanding (NLU) technology to be maximally useful, both practically and as a scientific object of study, it must be general: it must be able to process language in a way that is not exclusively tailored to any one specific task or dataset.
Ranked #46 on
Natural Language Inference
on MultiNLI
Natural Language Inference
Natural Language Understanding
+2
Understanding Grounded Language Learning Agents
Neural network-based systems can now learn to locate the referents of words and phrases in images, answer questions about visual scenes, and even execute symbolic instructions as first-person actors in partially-observable worlds.
Understanding Early Word Learning in Situated Artificial Agents
Neural network-based systems can now learn to locate the referents of words and phrases in images, answer questions about visual scenes, and execute symbolic instructions as first-person actors in partially-observable worlds.
Grounded Language Learning in a Simulated 3D World
Trained via a combination of reinforcement and unsupervised learning, and beginning with minimal prior knowledge, the agent learns to relate linguistic symbols to emergent perceptual representations of its physical surroundings and to pertinent sequences of actions.
HyperLex: A Large-Scale Evaluation of Graded Lexical Entailment
We introduce HyperLex - a dataset and evaluation resource that quantifies the extent of of the semantic category membership, that is, type-of relation also known as hyponymy-hypernymy or lexical entailment (LE) relation between 2, 616 concept pairs.
SimVerb-3500: A Large-Scale Evaluation Set of Verb Similarity
Verbs play a critical role in the meaning of sentences, but these ubiquitous words have received little attention in recent distributional semantics research.
Learning Distributed Representations of Sentences from Unlabelled Data
Unsupervised methods for learning distributed representations of words are ubiquitous in today's NLP research, but far less is known about the best ways to learn distributed phrase or sentence representations from unlabelled data.
Ranked #16 on
Subjectivity Analysis
on SUBJ
The Goldilocks Principle: Reading Children's Books with Explicit Memory Representations
We introduce a new test of how well language models capture meaning in children's books.
Learning to Understand Phrases by Embedding the Dictionary
Distributional models that learn rich semantic word representations are a success story of recent NLP research.
Embedding Word Similarity with Neural Machine Translation
Here we investigate the embeddings learned by neural machine translation models, a recently-developed class of neural language model.
Not All Neural Embeddings are Born Equal
Neural language models learn word representations that capture rich linguistic and conceptual information.
SimLex-999: Evaluating Semantic Models with (Genuine) Similarity Estimation
We present SimLex-999, a gold standard resource for evaluating distributional semantic models that improves on existing resources in several important ways.
Multi-Modal Models for Concrete and Abstract Concept Meaning
Multi-modal models that learn semantic representations from both linguistic and perceptual input outperform language-only models on a range of evaluations, and better reflect human concept acquisition.
