The Falcon Series of Open Language Models

no code implementations • 28 Nov 2023 • Ebtesam Almazrouei, Hamza Alobeidli, Abdulaziz Alshamsi, Alessandro Cappelli, Ruxandra Cojocaru, Mérouane Debbah, Étienne Goffinet, Daniel Hesslow, Julien Launay, Quentin Malartic, Daniele Mazzotta, Badreddine Noune, Baptiste Pannier, Guilherme Penedo

We report detailed evaluations, as well as a deep dive into the methods and custom tooling employed to pretrain Falcon.

Multi-task Language Understanding Sentence Completion

8-bit Numerical Formats for Deep Neural Networks

no code implementations • 6 Jun 2022 • Badreddine Noune, Philip Jones, Daniel Justus, Dominic Masters, Carlo Luschi

Given the current trend of increasing size and complexity of machine learning architectures, it has become of critical importance to identify new approaches to improve the computational efficiency of model training.

Computational Efficiency Image Classification

Parallel Training of Deep Networks with Local Updates

1 code implementation • 7 Dec 2020 • Michael Laskin, Luke Metz, Seth Nabarro, Mark Saroufim, Badreddine Noune, Carlo Luschi, Jascha Sohl-Dickstein, Pieter Abbeel

Deep learning models trained on large data sets have been widely successful in both vision and language domains.