Search Results for author:
Found 4 papers, 3 papers with code
DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads
Based on this insight, we introduce DuoAttention, a framework that only applies a full KV cache to retrieval heads while using a light-weight, constant-length KV cache for streaming heads, which reduces both LLM's decoding and pre-filling memory and latency without compromising its long-context abilities.
Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference
By only loading the Top-K critical KV cache pages for attention, Quest significantly speeds up self-attention without sacrificing accuracy.
DCRMTA: Unbiased Causal Representation for Multi-touch Attribution
Multi-touch attribution (MTA) currently plays a pivotal role in achieving a fair estimation of the contributions of each advertising touchpoint to-wards conversion behavior, deeply influencing budget allocation and advertising recommenda-tion.
AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration
We propose Activation-aware Weight Quantization (AWQ), a hardware-friendly approach for LLM low-bit weight-only quantization.
