Search Results for author:
Found 5 papers, 5 papers with code
Memory-efficient Transformers via Top-k Attention
Following the success of dot-product attention in Transformers, numerous approximations have been recently proposed to address its quadratic complexity with respect to the input length.
In-context Learning and Gradient Descent Revisited
Next, we explore a major discrepancy in the flow of information throughout the model between ICL and GD, which we term Layer Causality.
Analyzing Transformers in Embedding Space
In this work, we present a theoretical analysis where all parameters of a trained Transformer are interpreted by projecting them into the embedding space, that is, the space of vocabulary items they operate on.
LM-Debugger: An Interactive Tool for Inspection and Intervention in Transformer-Based Language Models
The opaque nature and unexplained behavior of transformer-based language models (LMs) have spurred a wide interest in interpreting their predictions.
Memory-efficient Transformers via Top-$k$ Attention
Following the success of dot-product attention in Transformers, numerous approximations have been recently proposed to address its quadratic complexity with respect to the input length.
