Search Results for author:
Found 19 papers, 6 papers with code
A Survey on Efficient Inference for Large Language Models
This paper presents a comprehensive survey of the existing literature on efficient LLM inference.
Linear Combination of Saved Checkpoints Makes Consistency and Diffusion Models Better
For example, LCSC achieves better performance using 1 number of function evaluation (NFE) than the base model with 2 NFE on consistency distillation, and decreases the NFE of DM from 15 to 9 while maintaining the generation quality on CIFAR-10.
FlashEval: Towards Fast and Accurate Evaluation of Text-to-image Diffusion Generative Models
In recent years, there has been significant progress in the development of text-to-image generative models.
Evaluating Quantized Large Language Models
Post-training quantization (PTQ) has emerged as a promising technique to reduce the cost of large language models (LLMs).
LV-Eval: A Balanced Long-Context Benchmark with 5 Length Levels Up to 256K
In contrast, the average context lengths of mainstream benchmarks are insufficient (5k-21k), and they suffer from potential knowledge leakage and inaccurate metrics, resulting in biased evaluation.
FlightLLM: Efficient Large Language Model Inference with a Complete Mapping Flow on FPGAs
However, existing GPU and transformer-based accelerators cannot efficiently process compressed LLMs, due to the following unresolved challenges: low computational efficiency, underutilized memory bandwidth, and large compilation overheads.
Enabling Fast 2-bit LLM on GPUs: Memory Alignment and Asynchronous Dequantization
Weights are quantized by groups, while the ranges of weights are large in some groups, resulting in large quantization errors and nonnegligible accuracy loss (e. g. >3% for Llama2-7b with 2-bit quantization in GPTQ and Greenbit).
FlashDecoding++: Faster Large Language Model Inference on GPUs
A single and static dataflow may lead to a 50. 25% performance loss for GEMMs of different shapes in LLM inference.
TorchSparse++: Efficient Training and Inference Framework for Sparse Convolution on GPUs
On top of this, we design the Sparse Autotuner, which extends the design space of existing sparse convolution libraries and searches for the best dataflow configurations for training and inference workloads.
Seeking the Yield Barrier: High-Dimensional SRAM Evaluation Through Optimal Manifold
We then generalize it with infinite components and derive the novel optimal manifold concept, which bridges the surrogate-based and importance sampling (IS) yield estimation methods.
Ada3D : Exploiting the Spatial Redundancy with Adaptive Inference for Efficient 3D Object Detection
One reason for this high resource consumption is the presence of a large number of redundant background points in Lidar point clouds, resulting in spatial redundancy in both 3D voxel and dense BEV map representations.
Adam Accumulation to Reduce Memory Footprints of both Activations and Gradients for Large-scale DNN Training
We find that previous gradient accumulation reduces activation memory but fails to be compatible with gradient memory reduction due to a contradiction between preserving gradients and releasing gradients.
High-Dimensional Yield Estimation using Shrinkage Deep Features and Maximization of Integral Entropy Reduction
Despite the fast advances in high-sigma yield analysis with the help of machine learning techniques in the past decade, one of the main challenges, the curse of dimensionality, which is inevitable when dealing with modern large-scale circuits, remains unsolved.
Understanding GNN Computational Graph: A Coordinated Computation, IO, and Memory Perspective
The same data are propagated through the graph structure to perform the same neural operation multiple times in GNNs, leading to redundant computation which accounts for 92. 4% of total operators.
CogDL: A Comprehensive Library for Graph Deep Learning
In CogDL, we propose a unified design for the training and evaluation of GNN models for various graph tasks, making it unique among existing graph learning libraries.
Explore the Potential of CNN Low Bit Training
Through this dynamic precision framework, we can reduce the bit-width of convolution, which is the most computational cost, while keeping the training process close to the full precision floating-point training.
GE-SpMM: General-purpose Sparse Matrix-Matrix Multiplication on GPUs for Graph Neural Networks
GE-SpMM performs SpMM-like operation on sparse matrices represented in the most common Compressed Sparse Row (CSR) format, so it can be embedded in GNN frameworks with no preprocessing overheads and support general GNN algorithms.
Distributed, Parallel, and Cluster Computing
Exploring the Potential of Low-bit Training of Convolutional Neural Networks
For training a variety of models on CIFAR-10, using 1-bit mantissa and 2-bit exponent is adequate to keep the accuracy loss within $1\%$.
Enabling Efficient and Flexible FPGA Virtualization for Deep Learning in the Cloud
Currently, the majority of FPGA-based DNN accelerators in the cloud run in a time-division multiplexing way for multiple users sharing a single FPGA, and require re-compilation with $\sim$100 s overhead.
