no code implementations • 3 Jan 2025 • Roseval Malaquias Junior, Ramon Pires, Thales Sales Almeida, Kenzo Sakiyama, Roseli Romero, Rodrigo Nogueira
To compare general and specialized training, we filtered a web-based dataset to extract legal domain data.
no code implementations • 15 Oct 2024 • Hugo Abonizio, Thales Sales Almeida, Thiago Laitz, Roseval Malaquias Junior, Giovana Kerche Bonás, Rodrigo Nogueira, Ramon Pires
This report presents Sabi\'a-3, our new flagship language model, and Sabiazinho-3, a more cost-effective sibling.
no code implementations • 26 Mar 2024 • Roseval Malaquias Junior, Ramon Pires, Roseli Romero, Rodrigo Nogueira
This study contributes to the growing body of scientific evidence showing that pretraining data selection may enhance the performance of large language models, enabling the exploration of these models at a lower cost.