Search Results for author:
Found 16 papers, 8 papers with code
SODA: Bottleneck Diffusion Models for Representation Learning
We introduce SODA, a self-supervised diffusion model, designed for representation learning.
Causal interventions expose implicit situation models for commonsense language understanding
Accounts of human language processing have long appealed to implicit ``situation models'' that enrich comprehension with relevant but unstated world knowledge.
Achieving and Understanding Out-of-Distribution Generalization in Systematic Reasoning in Small-Scale Transformers
As a step toward understanding how transformer-based systems generalize, we explore the question of OODG in small scale transformers trained with examples from a known distribution.
Out-of-Distribution Generalization
Systematic Generalization
Learning to Reason With Relational Abstractions
Large language models have recently shown promising progress in mathematical reasoning when fine-tuned with human-generated sequences walking through a sequence of solution steps.
Systematic Generalization and Emergent Structures in Transformers Trained on Structured Tasks
Transformer networks have seen great success in natural language processing and machine vision, where task objectives such as next word prediction and image classification benefit from nuanced context sensitivity across high-dimensional inputs.
Language models show human-like content effects on reasoning tasks
We evaluate state of the art large language models, as well as humans, and find that the language models reflect many of the same patterns observed in humans across these tasks $\unicode{x2014}$ like humans, models answer more accurately when the semantic content of a task supports the logical inferences.
Can language models learn from explanations in context?
In summary, explanations can support the in-context learning of large LMs on challenging tasks.
Tell me why! Explanations support learning relational and causal structure
Inferring the abstract relational and causal structure of the world is a major challenge for reinforcement-learning (RL) agents.
Systematic human learning and generalization from a brief tutorial with explanatory feedback
We also find that most of those who master the task can describe a valid solution strategy, and such participants perform better on transfer puzzles than those whose strategy descriptions are vague or incomplete.
Transforming task representations to perform novel tasks
We demonstrate the effectiveness of this framework across a wide variety of tasks and computational paradigms, ranging from regression to image classification and reinforcement learning.
Extending Machine Language Models toward Human-Level Language Understanding
We take language to be a part of a system for understanding and communicating about situations.
Environmental drivers of systematicity and generalization in a situated agent
The question of whether deep neural networks are good at generalising beyond their immediate training experience is of critical importance for learning-based approaches to AI.
Zero-shot task adaptation by homoiconic meta-mapping
How can deep learning systems flexibly reuse their knowledge?
A mathematical theory of semantic development in deep neural networks
An extensive body of empirical research has revealed remarkable regularities in the acquisition, organization, deployment, and neural representation of human semantic knowledge, thereby raising a fundamental conceptual question: what are the theoretical principles governing the ability of neural networks to acquire, organize, and deploy abstract knowledge by integrating across many individual experiences?
One-shot and few-shot learning of word embeddings
Standard deep learning systems require thousands or millions of examples to learn a concept, and cannot integrate new concepts easily.
Exact solutions to the nonlinear dynamics of learning in deep linear neural networks
We further exhibit a new class of random orthogonal initial conditions on weights that, like unsupervised pre-training, enjoys depth independent learning times.
