Search Results for author:
Found 6 papers, 3 papers with code
FP6-LLM: Efficiently Serving Large Language Models Through FP6-Centric Algorithm-System Co-Design
However, existing systems do not provide Tensor Core support for FP6 quantization and struggle to achieve practical performance improvements during LLM inference.
ZeroQuant(4+2): Redefining LLMs Quantization with a New FP6-Centric Strategy for Diverse Generative Tasks
With our design, FP6 can become a promising solution to the current 4-bit quantization methods used in LLMs.
Flash-LLM: Enabling Cost-Effective and Highly-Efficient Large Generative Model Inference with Unstructured Sparsity
Therefore, we propose Flash-LLM for enabling low-cost and highly-efficient large generative model inference with the sophisticated support of unstructured sparsity on high-performance but highly restrictive Tensor Cores.
Auto-Parallelizing Large Models with Rhino: A Systematic Approach on Production AI Platform
We present Rhino, a system for accelerating tensor programs with automatic parallelization on AI platform for real production environment.
FusionStitching: Boosting Memory Intensive Computations for Deep Learning Workloads
We show in this work that memory intensive computations can result in severe performance problems due to off-chip memory access and CPU-GPU context switch overheads in a wide range of deep learning models.
Auto-MAP: A DQN Framework for Exploring Distributed Execution Plans for DNN Workloads
The last decade has witnessed growth in the computational requirements for training deep neural networks.
