Search Results for author:
Found 2 papers, 0 papers with code
TP-Aware Dequantization
In this paper, we present a novel method that reduces model inference latency during distributed deployment of Large Language Models (LLMs).
Accelerating a Triton Fused Kernel for W4A16 Quantized Inference with SplitK work decomposition
Our implementation shows improvement for the type of skinny matrix-matrix multiplications found in foundation model inference workloads.
Cannot find the paper you are looking for? You can
Submit a new open access paper.
