Search Results for author:
Found 14 papers, 10 papers with code
Pre-Trained Language Models for Interactive Decision-Making
Together, these results suggest that language modeling induces representations that are useful for modeling not just language, but also goals and plans; these representations can aid learning and generalization even outside of language processing.
RL4F: Generating Natural Language Feedback with Reinforcement Learning for Repairing Model Outputs
Despite their unprecedented success, even the largest language models make mistakes.
In-Context Language Learning: Architectures and Algorithms
Finally, we show that hard-wiring these heads into neural models improves performance not just on ICLL, but natural language modeling -- improving the perplexity of 340M-parameter models by up to 1. 14 points (6. 7%) on the SlimPajama dataset.
Towards Tracing Factual Knowledge in Language Models Back to the Training Data
In this paper, we propose the problem of fact tracing: identifying which training examples taught an LM to generate a particular factual assertion.
Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks
The impressive performance of recent language models across a wide range of tasks suggests that they possess a degree of abstract reasoning skills.
Morphological analysis using a sequence decoder
Our Morse implementation and the TrMor2018 dataset are available online to support future research\footnote{See \url{https://github. com/ai-ku/Morse. jl} for a Morse implementation in Julia/Knet \cite{knet2016mlsys} and \url{https://github. com/ai-ku/TrMor2018} for the new Turkish dataset.
Subspace Regularizers for Few-Shot Class Incremental Learning
The key to this approach is a new family of subspace regularization schemes that encourage weight vectors for new classes to lie close to the subspace spanned by the weights of existing classes.
Lexicon Learning for Few-Shot Neural Sequence Modeling
Sequence-to-sequence transduction is the core problem in language processing applications as diverse as semantic parsing, machine translation, and instruction following.
Learning to Recombine and Resample Data for Compositional Generalization
Flexible neural sequence models outperform grammar- and automaton-based counterparts on a variety of tasks.
Compositionality as Lexical Symmetry
In tasks like semantic parsing, instruction following, and question answering, standard deep networks fail to generalize compositionally from small datasets.
Language Model Pre-training Improves Generalization in Policy Learning
Additional experiments explore the role of language-based encodings in these results; we find that it is possible to train a simple adapter layer that maps from observations and action histories to LM embeddings, and thus that language modeling provides an effective initializer even for tasks with no language as input or output.
Compositional Semantic Parsing with Large Language Models
Humans can reason compositionally when presented with new tasks.
Ranked #1 on
Semantic Parsing
on CFQ
What learning algorithm is in-context learning? Investigations with linear models
We investigate the hypothesis that transformer-based in-context learners implement standard learning algorithms implicitly, by encoding smaller models in their activations, and updating these implicit models as new examples appear in the context.
Deductive Closure Training of Language Models for Coherence, Accuracy, and Updatability
Given a collection of seed documents, DCT prompts LMs to generate additional text implied by these documents, reason globally about the correctness of this generated text, and finally fine-tune on text inferred to be correct.
