Search Results for author:
Found 55 papers, 12 papers with code
Toward Multilingual Identification of Online Registers
We consider cross- and multilingual text classification approaches to the identification of online registers (genres), i. e. text varieties with specific situational characteristics.
Multilingual text classification
Multilingual Word Embeddings
+2
Fine-grained Named Entity Annotation for Finnish
We introduce a corpus with fine-grained named entity annotation for Finnish, following the OntoNotes guidelines to create a resource that is cross-lingually compatible with existing annotations for other languages.
Towards better structured and less noisy Web data: Oscar with Register annotations
Web-crawled datasets are known to be noisy, as they feature a wide range of language use covering both user-generated and professionally edited content as well as noise originating from the crawling process.
Register Always Matters: Analysis of LLM Pretraining Data Through the Lens of Language Variation
Pretraining data curation is a cornerstone in Large Language Model (LLM) development, leading to growing research on quality filtering of large web corpora.
Got Compute, but No Data: Lessons From Post-training a Finnish LLM
We use a multilingual LLM to translate instruction and preference datasets from English to Finnish.
A Survey of Large Language Models for European Languages
Large Language Models (LLMs) have gained significant attention due to their high performance on a wide range of natural language tasks since the release of ChatGPT.
CoNECo: A corpus for named entity recognition and normalization of protein complexes
Motivation: Despite significant progress in biomedical information extraction, there is a lack of resources for Named Entity Recognition (NER) and Normalization (NEN) of protein-containing complexes.
Poro 34B and the Blessing of Multilinguality
The pretraining of state-of-the-art large language models now requires trillions of words of text, which is orders of magnitude more than available for the vast majority of languages.
Aurora-M: Open Source Continual Pre-training for Multilingual Language and Code
Despite these efforts, such models encounter challenges such as limited multilingual capabilities, risks of catastrophic forgetting during continual pretraining, and the high costs of training models from scratch, alongside the need to align with AI safety standards and regulatory frameworks.
A New Massive Multilingual Dataset for High-Performance Language Technologies
We present the HPLT (High Performance Language Technologies) language resources, a new massive multilingual dataset including both monolingual and bilingual corpora extracted from CommonCrawl and previously unused web crawls from the Internet Archive.
FinGPT: Large Generative Models for a Small Language
We pursue two approaches to pretrain models: 1) we train seven monolingual models from scratch (186M to 13B parameters) dubbed FinGPT, 2) we continue the pretraining of the multilingual BLOOM model on a mix of its original training data and Finnish, resulting in a 176 billion parameter model we call BLUUMI.
Scaling Data-Constrained Language Models
We find that with constrained data for a fixed compute budget, training with up to 4 epochs of repeated data yields negligible changes to loss compared to having unique data.
Multi-CrossRE A Multi-Lingual Multi-Domain Dataset for Relation Extraction
Most research in Relation Extraction (RE) involves the English language, mainly due to the lack of multi-lingual resources.
Silver Syntax Pre-training for Cross-Domain Relation Extraction
One of the main reasons for this is the limited training size of current RE datasets: obtaining high-quality (manually annotated) data is extremely expensive and cannot realistically be repeated for each new domain.
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions.
Explaining Classes through Word Attribution
In this work, we propose a method for explaining classes using deep learning models and the Integrated Gradients feature attribution technique by aggregating explanations of individual examples in text classification to general descriptions of the classes.
Quantitative Evaluation of Alternative Translations in a Corpus of Highly Dissimilar Finnish Paraphrases
In this paper, we present a quantitative evaluation of differences between alternative translations in a large recently released Finnish paraphrase corpus focusing in particular on non-trivial variation in translation.
Deep learning for sentence clustering in essay grading support
Essays as a form of assessment test student knowledge on a deeper level than short answer and multiple-choice questions.
Beyond the English Web: Zero-Shot Cross-Lingual and Lightweight Monolingual Classification of Registers
We explore cross-lingual transfer of register classification for web documents.
Towards Fully Bilingual Deep Language Modeling
Language models based on deep neural networks have facilitated great advances in natural language processing and understanding tasks in recent years.
The birth of Romanian BERT
Large-scale pretrained language models have become ubiquitous in Natural Language Processing.
Turku Enhanced Parser Pipeline: From Raw Text to Enhanced Graphs in the IWPT 2020 Shared Task
We present the approach of the TurkuNLP group to the IWPT 2020 shared task on Multilingual Parsing into Enhanced Universal Dependencies.
Exploring Cross-sentence Contexts for Named Entity Recognition with BERT
We find that adding context in the form of additional sentences to BERT input systematically increases NER performance on all of the tested languages and models.
WikiBERT models: deep transfer learning for many languages
In this paper, we introduce a simple, fully automated pipeline for creating language-specific BERT models from Wikipedia data and introduce 42 new such models, most for languages up to now lacking dedicated deep neural language models.
From Web Crawl to Clean Register-Annotated Corpora
However, two critical steps in the development of web corpora remain challenging: the identification of clean text from source HTML and the assignment of genre or register information to the documents.
A Broad-coverage Corpus for Finnish Named Entity Recognition
We present a new manually annotated corpus for broad-coverage named entity recognition for Finnish.
Universal Dependencies v2: An Evergrowing Multilingual Treebank Collection
Universal Dependencies is an open community effort to create cross-linguistically consistent treebank annotation for many languages within a dependency-based lexicalist framework.
Multilingual is not enough: BERT for Finnish
Deep learning-based language models pretrained on large unannotated text corpora have been demonstrated to allow efficient transfer learning for natural language processing, with recent approaches such as the transformer-based BERT model advancing the state of the art across a variety of tasks.
Neural Dependency Parsing of Biomedical Text: TurkuNLP entry in the CRAFT Structural Annotation Task
We present the approach taken by the TurkuNLP group in the CRAFT Structural Annotation task, a shared task on dependency parsing.
Biomedical Named Entity Recognition with Multilingual BERT
We present the approach of the Turku NLP group to the PharmaCoNER task on Spanish biomedical named entity recognition.
CRAFT Shared Tasks 2019 Overview --- Integrated Structure, Semantics, and Coreference
As part of the BioNLP Open Shared Tasks 2019, the CRAFT Shared Tasks 2019 provides a platform to gauge the state of the art for three fundamental language processing tasks {---} dependency parse construction, coreference resolution, and ontology concept identification {---} over full-text biomedical articles.
A neural network multi-task learning approach to biomedical named entity recognition
Additionally, we investigated the effect of dataset size on performance in both single- and multi-task settings.
CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies
The Conference on Computational Natural Language Learning (CoNLL) features a shared task, in which participants train and test their learning systems on the same data sets.
Cancer Hallmark Text Classification Using Convolutional Neural Networks
Methods based on deep learning approaches have recently achieved state-of-the-art performance in a range of machine learning tasks and are increasingly applied to natural language processing (NLP).
Attending to Characters in Neural Sequence Labeling Models
Sequence labeling architectures use word embeddings for capturing similarity, but suffer when handling previously unseen or rare words.
Ranked #7 on
Grammatical Error Detection
on FCE
Typed Entity and Relation Annotation on Computer Science Papers
In our scheme, mentions of entities are annotated with ontology-based types, and the roles of the entities are annotated as relations with other entities described in the text.
Universal Dependencies v1: A Multilingual Treebank Collection
Cross-linguistically consistent annotation is necessary for sound comparative evaluation and cross-lingual learning experiments.
Overview of BioNLP’09 Shared Task on Event Extraction
The paper presents the design and implementation of the BioNLP’09 Shared Task, and reports the final results with analysis.
