Search Results for author: Andrei Panferov

Found 2 papers, 1 papers with code

Extreme Compression of Large Language Models via Additive Quantization

1 code implementation11 Jan 2024 Vage Egiazarian, Andrei Panferov, Denis Kuznedelev, Elias Frantar, Artem Babenko, Dan Alistarh

The emergence of accurate open large language models (LLMs) has led to a race towards quantization techniques for such models enabling execution on end-user devices.

Llama Quantization

Correlated Quantization for Faster Nonconvex Distributed Optimization

no code implementations10 Jan 2024 Andrei Panferov, Yury Demidovich, Ahmad Rammal, Peter Richtárik

We analyze the forefront distributed non-convex optimization algorithm MARINA (Gorbunov et al., 2022) utilizing the proposed correlated quantizers and show that it outperforms the original MARINA and distributed SGD of Suresh et al. (2022) with regard to the communication complexity.

Distributed Optimization Quantization

Cannot find the paper you are looking for? You can Submit a new open access paper.