Search Results for author:
Found 21 papers, 14 papers with code
LLM2LLM: Boosting LLMs with Novel Iterative Data Enhancement
LLM2LLM (1) fine-tunes a baseline student LLM on the initial seed data, (2) evaluates and extracts data points that the model gets wrong, and (3) uses a teacher LLM to generate synthetic data based on these incorrect data points, which are then added back into the training data.
AI and Memory Wall
The availability of unprecedented unsupervised training data, along with neural scaling laws, has resulted in an unprecedented surge in model size and compute requirements for serving/training LLMs.
KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization
LLMs are seeing growing use for applications such as document analysis and summarization which require large context windows, and with these large context windows KV cache activations surface as the dominant contributor to memory consumption during inference.
Learned Best-Effort LLM Serving
Many applications must provide low-latency LLM service to users or risk unacceptable user experience.
An LLM Compiler for Parallel Function Calling
To address this, we introduce LLMCompiler, which executes functions in parallel to efficiently orchestrate multiple function calling.
SPEED: Speculative Pipelined Execution for Efficient Decoding
For Transformer decoders that employ parameter sharing, the memory operations for the tokens executing in parallel can be amortized, which allows us to accelerate generative LLM inference.
SqueezeLLM: Dense-and-Sparse Quantization
When applied to the LLaMA models, our 3-bit quantization significantly reduces the perplexity gap from the FP16 baseline by up to 2. 1x as compared to the state-of-the-art methods with the same memory requirement.
Full Stack Optimization of Transformer Inference: a Survey
In this work, we survey different approaches for efficient Transformer inference, including: (i) analysis and profiling of the bottlenecks in existing Transformer architectures and their similarities and differences with previous convolutional models; (ii) implications of Transformer architecture on hardware, including the impact of non-linear operations such as Layer Normalization, Softmax, and GELU, as well as linear operations, on hardware design; (iii) approaches for optimizing a fixed Transformer architecture; (iv) challenges in finding the right mapping and scheduling of operations for Transformer models; and (v) approaches for optimizing Transformer models by adapting the architecture using neural architecture search.
Speculative Decoding with Big Little Decoder
To address this, we propose Big Little Decoder (BiLD), a framework that can improve inference efficiency and latency for a wide range of text generation applications.
BigColor: Colorization using a Generative Color Prior for Natural Images
In this paper, we propose BigColor, a novel colorization approach that provides vivid colorization for diverse in-the-wild images with complex structures.
Squeezeformer: An Efficient Transformer for Automatic Speech Recognition
After re-examining the design choices for both the macro and micro-architecture of Conformer, we propose Squeezeformer which consistently outperforms the state-of-the-art ASR models under the same training schemes.
Ranked #30 on
Speech Recognition
on LibriSpeech test-clean
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
A Fast Post-Training Pruning Framework for Transformers
To address this, we propose a fast post-training pruning framework for Transformers that does not require any retraining.
Terra: Imperative-Symbolic Co-Execution of Imperative Deep Learning Programs
Recently, several systems have been proposed to combine the usability of imperative programming with the optimized performance of symbolic graph execution.
A Static Analyzer for Detecting Tensor Shape Errors in Deep Neural Network Training Code
We present an automatic static analyzer PyTea that detects tensor-shape errors in PyTorch code.
Applications and Techniques for Fast Machine Learning in Science
In this community review report, we discuss applications and techniques for fast machine learning (ML) in science -- the concept of integrating power ML methods into the real-time experimental data processing loop to accelerate scientific discovery.
Learned Token Pruning for Transformers
We extensively test the performance of LTP on GLUE tasks and show that our method outperforms the prior state-of-the-art token pruning methods by up to ~2. 5% higher accuracy with the same amount of FLOPs.
Integer-only Zero-shot Quantization for Efficient Speech Recognition
End-to-end neural network models achieve improved performance on various automatic speech recognition (ASR) tasks.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
A Survey of Quantization Methods for Efficient Neural Network Inference
Thus, it is not surprising that quantization has emerged recently as an important and very active sub-area of research in the efficient implementation of computations associated with Neural Networks.
Hessian-Aware Pruning and Optimal Neural Implant
To address this problem, we introduce a new Hessian Aware Pruning (HAP) method coupled with a Neural Implant approach that uses second-order sensitivity as a metric for structured pruning.
I-BERT: Integer-only BERT Quantization
Transformer based models, like BERT and RoBERTa, have achieved state-of-the-art results in many Natural Language Processing tasks.
Natural Language Inference
Natural Language Understanding
+1
Making Classical Machine Learning Pipelines Differentiable: A Neural Translation Approach
To this end, we propose a framework that translates a pre-trained ML pipeline into a neural network and fine-tunes the ML models within the pipeline jointly using backpropagation.
