Search Results for author: Andrew K. Lampinen

Found 17 papers, 8 papers with code

Evaluating Spatial Understanding of Large Language Models

1 code implementation23 Oct 2023 Yutaro Yamada, Yihan Bao, Andrew K. Lampinen, Jungo Kasai, Ilker Yildirim

Large language models (LLMs) show remarkable capabilities across a variety of tasks.

Transformers generalize differently from information stored in context vs in weights

no code implementations11 Oct 2022 Stephanie C. Y. Chan, Ishita Dasgupta, Junkyung Kim, Dharshan Kumaran, Andrew K. Lampinen, Felix Hill

In transformers trained on controlled stimuli, we find that generalization from weights is more rule-based whereas generalization from context is largely exemplar-based.

In-Context Learning

Language models show human-like content effects on reasoning tasks

1 code implementation14 Jul 2022 Ishita Dasgupta, Andrew K. Lampinen, Stephanie C. Y. Chan, Hannah R. Sheahan, Antonia Creswell, Dharshan Kumaran, James L. McClelland, Felix Hill

We evaluate state of the art large language models, as well as humans, and find that the language models reflect many of the same patterns observed in humans across these tasks $\unicode{x2014}$ like humans, models answer more accurately when the semantic content of a task supports the logical inferences.

Language Modelling Logical Reasoning +2

Know your audience: specializing grounded language models with listener subtraction

no code implementations16 Jun 2022 Aaditya K. Singh, David Ding, Andrew Saxe, Felix Hill, Andrew K. Lampinen

Through controlled experiments, we show that training a speaker with two listeners that perceive differently, using our method, allows the speaker to adapt to the idiosyncracies of the listeners.

Language Modelling Large Language Model

Data Distributional Properties Drive Emergent In-Context Learning in Transformers

4 code implementations22 Apr 2022 Stephanie C. Y. Chan, Adam Santoro, Andrew K. Lampinen, Jane X. Wang, Aaditya Singh, Pierre H. Richemond, Jay McClelland, Felix Hill

In further experiments, we found that naturalistic data distributions were only able to elicit in-context learning in transformers, and not in recurrent models.

Few-Shot Learning In-Context Learning

Zipfian environments for Reinforcement Learning

1 code implementation15 Mar 2022 Stephanie C. Y. Chan, Andrew K. Lampinen, Pierre H. Richemond, Felix Hill

As humans and animals learn in the natural world, they encounter distributions of entities, situations and events that are far from uniform.

reinforcement-learning Reinforcement Learning (RL) +1

What shapes feature representations? Exploring datasets, architectures, and training

no code implementations NeurIPS 2020 Katherine L. Hermann, Andrew K. Lampinen

Answers to these questions are important for understanding the basis of models' decisions, as well as for building models that learn versatile, adaptable representations useful beyond the original training task.

Transforming task representations to perform novel tasks

3 code implementations8 May 2020 Andrew K. Lampinen, James L. McClelland

We demonstrate the effectiveness of this framework across a wide variety of tasks and computational paradigms, ranging from regression to image classification and reinforcement learning.

Image Classification Zero-Shot Learning

Automated curricula through setter-solver interactions

no code implementations27 Sep 2019 Sebastien Racaniere, Andrew K. Lampinen, Adam Santoro, David P. Reichert, Vlad Firoiu, Timothy P. Lillicrap

We demonstrate the success of our approach in rich but sparsely rewarding 2D and 3D environments, where an agent is tasked to achieve a single goal selected from a set of possible goals that varies between episodes, and identify challenges for future work.

An analytic theory of generalization dynamics and transfer learning in deep linear networks

no code implementations ICLR 2019 Andrew K. Lampinen, Surya Ganguli

However we lack analytic theories that can quantitatively predict how the degree of knowledge transfer depends on the relationship between the tasks.

Multi-Task Learning

One-shot and few-shot learning of word embeddings

no code implementations27 Oct 2017 Andrew K. Lampinen, James L. McClelland

Standard deep learning systems require thousands or millions of examples to learn a concept, and cannot integrate new concepts easily.

Few-Shot Learning Sentence +1

Cannot find the paper you are looking for? You can Submit a new open access paper.