Search Results for author:
Found 12 papers, 9 papers with code
Locality-aware Parallel Decoding for Efficient Autoregressive Image Generation
Traditional autoregressive image generation relies on next-patch prediction, a memory-bound process that leads to high latency.
LServe: Efficient Long-sequence LLM Serving with Unified Sparse Attention
On average, LServe accelerates LLM prefilling by up to 2. 9x and decoding by 1. 3-2. 1x over vLLM, maintaining long-context accuracy.
NVILA: Efficient Frontier Visual Language Models
This paper introduces NVILA, a family of open VLMs designed to optimize both efficiency and accuracy.
Ranked #8 on
Video Question Answering
on NExT-QA
DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads
Based on this insight, we introduce DuoAttention, a framework that only applies a full KV cache to retrieval heads while using a light-weight, constant-length KV cache for streaming heads, which reduces both LLM's decoding and pre-filling memory and latency without compromising its long-context abilities.
HART: Efficient Visual Generation with Hybrid Autoregressive Transformer
To address these challenges, we present the hybrid tokenizer, which decomposes the continuous latents from the autoencoder into two components: discrete tokens representing the big picture and continuous tokens representing the residual components that cannot be represented by the discrete tokens.
Deep Compression Autoencoder for Efficient High-Resolution Diffusion Models
With these designs, we improve the autoencoder's spatial compression ratio up to 128 while maintaining the reconstruction quality.
Ranked #16 on
Image Generation
on ImageNet 512x512
LongVILA: Scaling Long-Context Visual Language Models for Long Videos
We introduce the long-context Multi-Modal Sequence Parallelism (MM-SP) system that efficiently parallelizes long video training and inference, enabling 2M context length training on 256 GPUs without any gradient checkpointing.
Ranked #11 on
Video Question Answering
on NExT-QA
Sparse Refinement for Efficient High-Resolution Semantic Segmentation
Finally, it leverages a gated ensembler to apply these sparse refinements to the initial coarse predictions.
QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving
The key insight driving QServe is that the efficiency of LLM serving on GPUs is critically influenced by operations on low-throughput CUDA cores.
TorchSparse++: Efficient Training and Inference Framework for Sparse Convolution on GPUs
On top of this, we design the Sparse Autotuner, which extends the design space of existing sparse convolution libraries and searches for the best dataflow configurations for training and inference workloads.
AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration
We propose Activation-aware Weight Quantization (AWQ), a hardware-friendly approach for LLM low-bit weight-only quantization.
FlatFormer: Flattened Window Attention for Efficient Point Cloud Transformer
Transformer, as an alternative to CNN, has been proven effective in many modalities (e. g., texts and images).
