Search Results for author:
Found 6 papers, 4 papers with code
LookupFFN: Making Transformers Compute-lite for CPU inference
While GPU clusters are the de facto choice for training large deep neural network (DNN) models today, several reasons including ease of workflow, security and cost have led to efforts investigating whether CPUs may be viable for inference in routine use in many sectors of the industry.
IM-Unpack: Training and Inference with Arbitrarily Low Precision Integers
A popular strategy is the use of low bit-width integers to approximate the original entries in a matrix.
FrameQuant: Flexible Low-Bit Quantization for Transformers
If quantization is interpreted as the addition of noise, our casting of the problem allows invoking an extensive body of known consistent recovery and noise robustness guarantees.
Multi Resolution Analysis (MRA) for Approximate Self-Attention
Transformers have emerged as a preferred model for many tasks in natural langugage processing and vision.
You Only Sample (Almost) Once: Linear Cost Self-Attention Via Bernoulli Sampling
In this paper, we show that a Bernoulli sampling attention mechanism based on Locality Sensitive Hashing (LSH), decreases the quadratic complexity of such models to linear.
Nyströmformer: A Nyström-Based Algorithm for Approximating Self-Attention
The scalability of Nystr\"{o}mformer enables application to longer sequences with thousands of tokens.
Ranked #13 on
Semantic Textual Similarity
on MRPC
(F1 metric)
