Search Results for author: David Samuel

Found 15 papers, 13 papers with code

Not all layers are equally as important: Every Layer Counts BERT

no code implementations3 Nov 2023 Lucas Georges Gabriel Charpentier, David Samuel

This paper introduces a novel modification of the transformer architecture, tailored for the data-efficient pretraining of language models.

Mean BERTs make erratic language teachers: the effectiveness of latent bootstrapping in low-resource settings

1 code implementation30 Oct 2023 David Samuel

This paper explores the use of latent bootstrapping, an alternative self-supervision technique, for pretraining language models.

NoCoLA: The Norwegian Corpus of Linguistic Acceptability

1 code implementation13 Jun 2023 Matias Jentoft, David Samuel

While there has been a surge of large language models for Norwegian in recent years, we lack any tool to evaluate their understanding of grammaticality.

Binary Classification Language Modelling +1

Tokenization with Factorized Subword Encoding

1 code implementation13 Jun 2023 David Samuel, Lilja Øvrelid

In recent years, language models have become increasingly larger and more complex.

Language Modelling

NorBench -- A Benchmark for Norwegian Language Models

1 code implementation6 May 2023 David Samuel, Andrey Kutuzov, Samia Touileb, Erik Velldal, Lilja Øvrelid, Egil Rønningstad, Elina Sigdel, Anna Palatkina

We present NorBench: a streamlined suite of NLP tasks and probes for evaluating Norwegian language models (LMs) on standardized data splits and evaluation metrics.

BRENT: Bidirectional Retrieval Enhanced Norwegian Transformer

1 code implementation19 Apr 2023 Lucas Georges Gabriel Charpentier, Sondre Wold, David Samuel, Egil Rønningstad

After training, we also separate the language model, which we call the reader, from the retriever components, and show that this can be fine-tuned on a range of downstream tasks.

Dependency Parsing Extractive Question-Answering +7

Trained on 100 million words and still in shape: BERT meets British National Corpus

2 code implementations17 Mar 2023 David Samuel, Andrey Kutuzov, Lilja Øvrelid, Erik Velldal

While modern masked language models (LMs) are trained on ever larger corpora, we here explore the effects of down-scaling training to a modestly-sized but representative, well-balanced, and publicly available English text source -- the British National Corpus.

Language Modelling

EventGraph: Event Extraction as Semantic Graph Parsing

1 code implementation16 Oct 2022 Huiling You, David Samuel, Samia Touileb, Lilja Øvrelid

Event extraction therefore becomes a graph parsing problem, which provides the following advantages: 1) performing event detection and argument extraction jointly; 2) detecting and extracting multiple events from a piece of text; and 3) capturing the complicated interaction between event arguments and triggers.

Event Detection Event Extraction

Direct parsing to sentiment graphs

1 code implementation ACL 2022 David Samuel, Jeremy Barnes, Robin Kurtz, Stephan Oepen, Lilja Øvrelid, Erik Velldal

This paper demonstrates how a graph-based semantic parser can be applied to the task of structured sentiment analysis, directly predicting sentiment graphs from text.

Sentiment Analysis

ÚFAL at MultiLexNorm 2021: Improving Multilingual Lexical Normalization by Fine-tuning ByT5

1 code implementation WNUT (ACL) 2021 David Samuel, Milan Straka

We present the winning entry to the Multilingual Lexical Normalization (MultiLexNorm) shared task at W-NUT 2021 (van der Goot et al., 2021a), which evaluates lexical-normalization systems on 12 social media datasets in 11 languages.

Dependency Parsing Language Modelling +1

Meta-learning Extractors for Music Source Separation

1 code implementation17 Feb 2020 David Samuel, Aditya Ganeshan, Jason Naradowsky

We propose a hierarchical meta-learning-inspired model for music source separation (Meta-TasNet) in which a generator model is used to predict the weights of individual extractor models.

Meta-Learning Music Source Separation

Cannot find the paper you are looking for? You can Submit a new open access paper.