Search Results for author:
Found 6 papers, 4 papers with code
Mitigating the Impact of Outlier Channels for Language Model Quantization with Activation Regularization
We show that regularizing both the inputs and outputs is crucial for preventing a model's "migrating" the difficulty in input quantization to the weights, which makes post-training quantization (PTQ) of weights more difficult.
Hydra: Sequentially-Dependent Draft Heads for Medusa Decoding
In this work, we propose Hydra heads, a sequentially dependent, drop-in replacement for standard draft heads that significantly improves speculation accuracy.
Striped Attention: Faster Ring Attention for Causal Transformers
In experiments running Striped Attention on A100 GPUs and TPUv4s, we are able to achieve up to 1. 45x end-to-end throughput improvements over the original Ring Attention algorithm on causal transformer training at a sequence length of 256k.
Towards Verifiable Text Generation with Symbolic References
LLMs are vulnerable to hallucinations, and thus their outputs generally require laborious human verification for high-stakes applications.
Integer-only Zero-shot Quantization for Efficient Speech Recognition
End-to-end neural network models achieve improved performance on various automatic speech recognition (ASR) tasks.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Checkmate: Breaking the Memory Wall with Optimal Tensor Rematerialization
We formalize the problem of trading-off DNN training time and memory requirements as the tensor rematerialization optimization problem, a generalization of prior checkpointing strategies.
