Search Results for author: Alexander Borzunov

Found 7 papers, 6 papers with code

Distributed Inference and Fine-tuning of Large Language Models Over The Internet

no code implementations NeurIPS 2023 Alexander Borzunov, Max Ryabinin, Artem Chumachenko, Dmitry Baranchuk, Tim Dettmers, Younes Belkada, Pavel Samygin, Colin Raffel

Large language models (LLMs) are useful in many NLP tasks and become more capable with size, with the best open-source models having over 50 billion parameters.

Petals: Collaborative Inference and Fine-tuning of Large Models

1 code implementation2 Sep 2022 Alexander Borzunov, Dmitry Baranchuk, Tim Dettmers, Max Ryabinin, Younes Belkada, Artem Chumachenko, Pavel Samygin, Colin Raffel

However, these techniques have innate limitations: offloading is too slow for interactive inference, while APIs are not flexible enough for research that requires access to weights, attention or logits.

Collaborative Inference

Training Transformers Together

1 code implementation7 Jul 2022 Alexander Borzunov, Max Ryabinin, Tim Dettmers, Quentin Lhoest, Lucile Saulnier, Michael Diskin, Yacine Jernite, Thomas Wolf

The infrastructure necessary for training state-of-the-art models is becoming overly expensive, which makes training such models affordable only to large corporations and institutions.

Secure Distributed Training at Scale

3 code implementations21 Jun 2021 Eduard Gorbunov, Alexander Borzunov, Michael Diskin, Max Ryabinin

Training such models requires a lot of computational resources (e. g., HPC clusters) that are not available to small research groups and independent researchers.

Distributed Optimization Image Classification +1

Cannot find the paper you are looking for? You can Submit a new open access paper.