Search Results for author:
Found 6 papers, 1 papers with code
MAP's not dead yet: Uncovering true language model modes by conditioning away degeneracy
It has been widely observed that exact or approximate MAP (mode-seeking) decoding from natural language generation (NLG) models consistently leads to degenerate outputs (Stahlberg and Byrne, 2019, Holtzman et al., 2019).
NF4 Isn't Information Theoretically Optimal (and that's Good)
This note shares some simple calculations and experiments related to absmax-based blockwise quantization, as used in Dettmers et al., 2023.
Reconsidering the Past: Optimizing Hidden States in Language Models
We present Hidden-State Optimization (HSO), a gradient-based method for improving the performance of transformer language models at inference time.
Adding Recurrence to Pretrained Transformers
Fine-tuning a pretrained transformer for a downstream task has become a standard method in NLP in the last few years.
Adding Recurrence to Pretrained Transformers for Improved Efficiency and Context Size
Fine-tuning a pretrained transformer for a downstream task has become a standard method in NLP in the last few years.
Cannot find the paper you are looking for? You can
Submit a new open access paper.
