Search Results for author:
Found 10 papers, 7 papers with code
What learning algorithm is in-context learning? Investigations with linear models
We investigate the hypothesis that transformer-based in-context learners implement standard learning algorithms implicitly, by encoding smaller models in their activations, and updating these implicit models as new examples appear in the context.
Compositional Semantic Parsing with Large Language Models
Humans can reason compositionally when presented with new tasks.
Towards Tracing Factual Knowledge in Language Models Back to the Training Data
In this paper, we propose the problem of fact tracing: identifying which training examples taught an LM to generate a particular factual assertion.
Pre-Trained Language Models for Interactive Decision-Making
Together, these results suggest that language modeling induces representations that are useful for modeling not just language, but also goals and plans; these representations can aid learning and generalization even outside of language processing.
Compositionality as Lexical Symmetry
Standard deep network models lack the inductive biases needed to generalize compositionally in tasks like semantic parsing, translation, and question answering.
Subspace Regularizers for Few-Shot Class Incremental Learning
The key to this approach is a new family of subspace regularization schemes that encourage weight vectors for new classes to lie close to the subspace spanned by the weights of existing classes.
Language Model Pre-training Improves Generalization in Policy Learning
Additional experiments explore the role of language-based encodings in these results; we find that it is possible to train a simple adapter layer that maps from observations and action histories to LM embeddings, and thus that language modeling provides an effective initializer even for tasks with no language as input or output.
Lexicon Learning for Few-Shot Neural Sequence Modeling
Sequence-to-sequence transduction is the core problem in language processing applications as diverse as semantic parsing, machine translation, and instruction following.
Learning to Recombine and Resample Data for Compositional Generalization
Flexible neural sequence models outperform grammar- and automaton-based counterparts on a variety of tasks.
Morphological analysis using a sequence decoder
Our Morse implementation and the TrMor2018 dataset are available online to support future research\footnote{See \url{https://github. com/ai-ku/Morse. jl} for a Morse implementation in Julia/Knet \cite{knet2016mlsys} and \url{https://github. com/ai-ku/TrMor2018} for the new Turkish dataset.
