Search Results for author:
Found 7 papers, 3 papers with code
SparQ Attention: Bandwidth-Efficient LLM Inference
The computational difficulties of large language model (LLM) inference remain a significant obstacle to their widespread deployment.
PopSparse: Accelerated block sparse matrix multiplication on IPU
Furthermore, static sparsity in general outperforms dynamic sparsity.
Unit Scaling: Out-of-the-Box Low-Precision Training
We present unit scaling, a paradigm for designing deep learning models that simplifies the use of low-precision number formats.
BESS: Balanced Entity Sampling and Sharing for Large-Scale Knowledge Graph Completion
We present the award-winning submission to the WikiKG90Mv2 track of OGB-LSC@NeurIPS 2022.
Towards Structured Dynamic Sparse Pre-Training of BERT
Identifying algorithms for computational efficient unsupervised training of large language models is an important and active area of research.
GroupBERT: Enhanced Transformer Architecture with Efficient Grouped Structures
Attention based language models have become a critical component in state-of-the-art natural language processing systems.
N-grams Bayesian Differential Privacy
It first transforms the counts to log space, approximating the distribution of the public and private data as Gaussian.
