Search Results for author:
Found 5 papers, 2 papers with code
Accelerating LLM Inference with Lossless Speculative Decoding Algorithms for Heterogeneous Vocabularies
Accelerating the inference of large language models (LLMs) is a critical challenge in generative AI.
Distributed Speculative Inference (DSI): Speculation Parallelism for Provably Faster Lossless Language Model Inference
Our simulations show that DSI is 1. 29-1. 92x faster than SI in single-node setups for various off-the-shelf LMs and tasks.
Dynamic Speculation Lookahead Accelerates Speculative Decoding of Large Language Models
Speculative decoding is commonly used for reducing the inference latency of large language models.
StarCoder: may the source be with you!
The BigCode community, an open-scientific collaboration working on the responsible development of Large Language Models for Code (Code LLMs), introduces StarCoder and StarCoderBase: 15. 5B parameter models with 8K context length, infilling capabilities and fast large-batch inference enabled by multi-query attention.
Ranked #53 on
Code Generation
on MBPP
Implicit Regularization Towards Rank Minimization in ReLU Networks
We study the conjectured relationship between the implicit regularization in neural networks, trained with gradient-based methods, and rank minimization of their weight matrices.
