Search Results for author:
Found 6 papers, 2 papers with code
RetroInfer: A Vector-Storage Approach for Scalable Long-Context LLM Inference
The growing context lengths of large language models (LLMs) pose significant challenges for efficient inference, primarily due to GPU memory and bandwidth constraints.
MoBA: Mixture of Block Attention for Long-Context LLMs
Scaling the effective context length is essential for advancing large language models (LLMs) toward artificial general intelligence (AGI).
Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving
Compared to the baseline method, Mooncake can achieve up to a 525% increase in throughput in certain simulated scenarios while adhering to SLOs.
Efficient Heterogeneous Large Language Model Decoding with Model-Attention Disaggregation
To enhance the efficiency of LLM decoding, we introduce model-attention disaggregation.
Multi-Stage Aggregated Transformer Network for Temporal Language Localization in Videos
Then for a moment candidate, we concatenate the starting/middle/ending representations of its starting/middle/ending elements respectively to form the final moment representation.
HpGAN: Sequence Search with Generative Adversarial Networks
Sequences play an important role in many engineering applications and systems.
