Search Results for author:
Found 3 papers, 2 papers with code
Speculative Streaming: Fast LLM Inference without Auxiliary Models
Speculative decoding is a prominent technique to speed up the inference of a large target language model based on predictions of an auxiliary draft model.
FastSeq: Make Sequence Generation Faster
Transformer-based models have made tremendous impacts in natural language generation.
EL-Attention: Memory Efficient Lossless Attention for Generation
Transformer model with multi-head attention requires caching intermediate results for efficient inference in generation tasks.
Cannot find the paper you are looking for? You can
Submit a new open access paper.
