Search Results for author: Nicolas Boizard

Found 3 papers, 3 papers with code

Towards Cross-Tokenizer Distillation: the Universal Logit Distillation Loss for LLMs

1 code implementation19 Feb 2024 Nicolas Boizard, Kevin El Haddad, Céline Hudelot, Pierre Colombo

Deploying large language models (LLMs) of several billion parameters can be impractical in most industrial use cases due to constraints such as cost, latency limitations, and hardware accessibility.

Knowledge Distillation

CroissantLLM: A Truly Bilingual French-English Language Model

1 code implementation1 Feb 2024 Manuel Faysse, Patrick Fernandes, Nuno M. Guerreiro, António Loison, Duarte M. Alves, Caio Corro, Nicolas Boizard, João Alves, Ricardo Rei, Pedro H. Martins, Antoni Bigata Casademunt, François Yvon, André F. T. Martins, Gautier Viaud, Céline Hudelot, Pierre Colombo

We introduce CroissantLLM, a 1. 3B language model pretrained on a set of 3T English and French tokens, to bring to the research and industrial community a high-performance, fully open-sourced bilingual model that runs swiftly on consumer-grade local hardware.

Language Modelling Large Language Model

Cannot find the paper you are looking for? You can Submit a new open access paper.