Search Results for author:
Found 13 papers, 9 papers with code
$100K or 100 Days: Trade-offs when Pre-Training with Academic Resources
We introduce a benchmark to measure the time to pre-train models on given GPUs and also identify ideal settings for maximizing training speed.
Talking Heads: Understanding Inter-layer Communication in Transformer Language Models
Although it is known that transformer language models (LMs) pass features from early layers to later layers, it is not well understood how this information is represented and routed by the model.
Dual Process Learning: Controlling Use of In-Context vs. In-Weights Strategies with Weight Forgetting
Hence, we study $\textbf{structural in-context learning}$, which we define as the ability of a model to execute in-context learning on arbitrary tokens -- so called because the model must generalize on the basis of e. g. sentence structure or task structure, rather than semantic content encoded in token embeddings.
Axiomatic Causal Interventions for Reverse Engineering Relevance Computation in Neural Retrieval Models
Neural models have demonstrated remarkable performance across diverse ranking tasks.
Transformer Mechanisms Mimic Frontostriatal Gating Operations When Trained on Human Working Memory Tasks
Models based on the Transformer neural network architecture have seen success on a wide variety of tasks that appear to require complex "cognitive branching" -- or the ability to maintain pursuit of one goal while accomplishing others.
Characterizing Mechanisms for Factual Recall in Language Models
By scaling up or down the value vector of these heads, we can control the likelihood of using the in-context answer on new data.
Circuit Component Reuse Across Tasks in Transformer Language Models
that it is mostly reused to solve a seemingly different task: Colored Objects (Ippolito & Callison-Burch, 2023).
Language Models Implement Simple Word2Vec-style Vector Arithmetic
A primary criticism towards language models (LMs) is their inscrutability.
Does CLIP Bind Concepts? Probing Compositionality in Large Image Models
Large-scale neural network models combining text and images have made incredible progress in recent years.
ezCoref: Towards Unifying Annotation Guidelines for Coreference Resolution
Large-scale, high-quality corpora are critical for advancing research in coreference resolution.
Linearly Mapping from Image to Text Space
Prior work has shown that pretrained LMs can be taught to caption images when a vision model's parameters are optimized to encode images in the language space.
Pretraining on Interactions for Learning Grounded Affordance Representations
Lexical semantics and cognitive science point to affordances (i. e. the actions that objects support) as critical for understanding and representing nouns and verbs.
Investigating Sports Commentator Bias within a Large Corpus of American Football Broadcasts
Sports broadcasters inject drama into play-by-play commentary by building team and player narratives through subjective analyses and anecdotes.
