Search Results for author:
Found 21 papers, 9 papers with code
High-throughput Generative Inference of Large Language Models with a Single GPU
As a result, when running OPT-175B on a single 16GB GPU, FlexGen achieves significantly higher throughput compared to state-of-the-art offloading systems, reaching a generation throughput of 1 token/s for the first time with an effective batch size of 144.
Sample-efficient Surrogate Model for Frequency Response of Linear PDEs using Self-Attentive Complex Polynomials
Linear Partial Differential Equations (PDEs) govern the spatial-temporal dynamics of physical systems that are essential to building modern technology.
Decentralized Training of Foundation Models in Heterogeneous Environments
Our key technical contribution is a scheduling algorithm that allocates different computational "tasklets" in the training of foundation models to a group of decentralized GPU devices connected by a slow heterogeneous network.
Fine-tuning Language Models over Slow Networks using Activation Compression with Guarantees
Communication compression is a crucial technique for modern distributed learning systems to alleviate their communication bottlenecks over slower networks.
Monarch: Expressive Structured Matrices for Efficient and Accurate Training
To address these issues, we propose a class of matrices (Monarch) that is hardware-efficient (they are parameterized as products of two block-diagonal matrices for better hardware utilization) and expressive (they can represent many commonly used transforms).
Locality Sensitive Teaching
However, as one of the most influential and practical MT paradigms, iterative machine teaching (IMT) is prohibited on IoT devices due to its inefficient and unscalable algorithms.
Pixelated Butterfly: Simple and Efficient Sparse training for Neural Network Models
To address this, our main insight is to optimize over a continuous superset of sparse matrices with a fixed structure known as products of butterfly matrices.
Scatterbrain: Unifying Sparse and Low-rank Attention Approximation
Recent advances in efficient Transformers have exploited either the sparsity or low-rank properties of attention matrices to reduce the computational and memory bottlenecks of modeling long sequences.
Scatterbrain: Unifying Sparse and Low-rank Attention
Recent advances in efficient Transformers have exploited either the sparsity or low-rank properties of attention matrices to reduce the computational and memory bottlenecks of modeling long sequences.
MONGOOSE: A Learnable LSH Framework for Efficient Neural Network Training
Recent advances by practitioners in the deep learning community have breathed new life into Locality Sensitive Hashing (LSH), using it to reduce memory and time bottlenecks in neural network (NN) training.
A Truly Constant-time Distribution-aware Negative Sampling
Softmax classifiers with a very large number of classes naturally occur in many applications such as natural language processing and information retrieval.
A Tale of Two Efficient and Informative Negative Sampling Distributions
Unfortunately, due to the dynamically updated parameters and data samples, there is no sampling scheme that is provably adaptive and samples the negative classes efficiently.
SOLAR: Sparse Orthogonal Learned and Random Embeddings
The label vectors are random, sparse, and near-orthogonal by design, while the query vectors are learned and sparse.
Climbing the WOL: Training for Cheaper Inference
Efficient inference for wide output layers (WOLs) is an essential yet challenging task in large scale machine learning.
Angular Visual Hardness
We also find that AVH has a statistically significant correlation with human visual hardness.
Fast and Accurate Stochastic Gradient Estimation
In this paper, we break this barrier by providing the first demonstration of a scheme, Locality sensitive hashing (LSH) sampled Stochastic Gradient Descent (LGD), which leads to superior gradient estimation while keeping the sampling cost per iteration similar to that of the uniform sampling.
Lsh-sampling Breaks the Computation Chicken-and-egg Loop in Adaptive Stochastic Gradient Estimation
In this paper, we break this barrier by providing the first demonstration of a scheme, Locality sensitive hashing (LSH) sampled Stochastic Gradient Descent (LGD), which leads to superior gradient estimation while keeping the sampling cost per iteration similar to that of the uniform sampling.
SLIDE : In Defense of Smart Algorithms over Hardware Acceleration for Large-Scale Deep Learning Systems
On the same CPU hardware, SLIDE is over 10x faster than TF.
LSH-SAMPLING BREAKS THE COMPUTATIONAL CHICKEN-AND-EGG LOOP IN ADAPTIVE STOCHASTIC GRADIENT ESTIMATION
In this paper, we break this barrier by providing the first demonstration of a sampling scheme, which leads to superior gradient estimation, while keeping the sampling cost per iteration similar to that of the uniform sampling.
Revisiting Winner Take All (WTA) Hashing for Sparse Datasets
WTA (Winner Take All) hashing has been successfully applied in many large scale vision applications.
Sub-Linear Privacy-Preserving Near-Neighbor Search
In Near-Neighbor Search (NNS), a new client queries a database (held by a server) for the most similar data (near-neighbors) given a certain similarity metric.
