1 code implementation • ACL 2022 • Fredrik Carlsson, Joey Öhman, Fangyu Liu, Severine Verlinden, Joakim Nivre, Magnus Sahlgren
We propose a resource-efficient method for converting a pre-trained CLM into this architecture, and demonstrate its potential on various experiments, including the novel task of contextualized word inclusion.
no code implementations • LREC 2022 • Ariel Ekgren, Amaru Cuba Gyllensten, Evangelia Gogoulou, Alice Heiman, Severine Verlinden, Joey Öhman, Fredrik Carlsson, Magnus Sahlgren
We present GTP-SW3, a 3. 5 billion parameter autoregressive language model, trained on a newly created 100 GB Swedish corpus.
no code implementations • 18 Oct 2023 • Felix Stollenwerk, Joey Öhman, Danila Petrelli, Emma Wallerö, Fredrik Olsson, Camilla Bengtsson, Andreas Horndahl, Gabriela Zarzar Gandler
This handbook is a hands-on guide on how to approach text annotation tasks.
no code implementations • 18 Oct 2023 • Felix Stollenwerk, Niklas Fastlund, Anna Nyqvist, Joey Öhman
We have trained a named entity recognition (NER) model that screens Swedish job ads for different kinds of useful information (e. g. skills required from a job seeker).
no code implementations • 22 May 2023 • Ariel Ekgren, Amaru Cuba Gyllensten, Felix Stollenwerk, Joey Öhman, Tim Isbister, Evangelia Gogoulou, Fredrik Carlsson, Alice Heiman, Judit Casademont, Magnus Sahlgren
This paper details the process of developing the first native large generative language model for the Nordic languages, GPT-SW3.
no code implementations • 30 Mar 2023 • Joey Öhman, Severine Verlinden, Ariel Ekgren, Amaru Cuba Gyllensten, Tim Isbister, Evangelia Gogoulou, Fredrik Carlsson, Magnus Sahlgren
Pre-training Large Language Models (LLMs) require massive amounts of text data, and the performance of the LLMs typically correlates with the scale and quality of the datasets.