Search Results for author:
Found 13 papers, 10 papers with code
GSM-Infinite: How Do Your LLMs Behave over Infinitely Increasing Context Length and Reasoning Complexity?
Our GSM-Infinite benchmark provides a scalable and controllable testbed for systematically studying and advancing LLM reasoning in long and complex contexts.
AdaServe: SLO-Customized LLM Serving with Fine-Grained Speculative Decoding
This paper introduces AdaServe, the first LLM serving system to support SLO customization through fine-grained speculative decoding.
MagicPIG: LSH Sampling for Efficient LLM Generation
Large language models (LLMs) with long context windows have gained significant attention.
Sirius: Contextual Sparsity with Correction for Efficient LLMs
However, after a comprehensive evaluation of contextual sparsity methods on various complex generation tasks, we find that although CS succeeds in prompt-understanding tasks, CS significantly degrades the model performance for reasoning, deduction, and knowledge-based tasks.
MagicDec: Breaking the Latency-Throughput Tradeoff for Long Context Generation with Speculative Decoding
MagicDec first identifies the bottleneck shifts with increasing batch size and sequence length, and uses these insights to deploy SD more effectively for high throughput inference.
Mini-Sequence Transformer: Optimizing Intermediate Memory for Long Sequences Training
We introduce Mini-Sequence Transformer (MsT), a simple and effective methodology for highly efficient and accurate LLM training with extremely long sequences.
SpecExec: Massively Parallel Speculative Decoding for Interactive LLM Inference on Consumer Devices
We propose SpecExec (Speculative Execution), a simple parallel decoding method that can generate up to 20 tokens per target model iteration for popular LLM families.
TriForce: Lossless Acceleration of Long Sequence Generation with Hierarchical Speculative Decoding
However, key-value (KV) cache, which is stored to avoid re-computation, has emerged as a critical bottleneck by growing linearly in size with the sequence length.
Sequoia: Scalable, Robust, and Hardware-aware Speculative Decoding
This paper introduces Sequoia, a scalable, robust, and hardware-aware algorithm for speculative decoding.
GNNPipe: Scaling Deep GNN Training with Pipelined Model Parallelism
Communication is a key bottleneck for distributed graph neural network (GNN) training.
SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verification
Our evaluation shows that SpecInfer outperforms existing LLM serving systems by 1. 5-2. 8x for distributed LLM inference and by 2. 6-3. 5x for offloading-based LLM inference, while preserving the same generative performance.
Quark: A Gradient-Free Quantum Learning Framework for Classification Tasks
To address the limitations of existing quantum ML methods, we introduce Quark, a gradient-free quantum learning framework that optimizes quantum ML models using quantum optimization.
Quantized Training of Gradient Boosting Decision Trees
Recent years have witnessed significant success in Gradient Boosting Decision Trees (GBDT) for a wide range of machine learning applications.
