Search Results for author: Anna Korhonen

Found 103 papers, 23 papers with code

Semantic Data Set Construction from Human Clustering and Spatial Arrangement

no code implementations CL (ACL) 2021 Olga Majewska, Diana McCarthy, Jasper J. F. van den Bosch, Nikolaus Kriegeskorte, Ivan Vulić, Anna Korhonen

We demonstrate how the resultant data set can be used for fine-grained analyses and evaluation of representation learning models on the intrinsic tasks of semantic clustering and semantic similarity.

Representation Learning Semantic Similarity +2

Data Augmentation and Learned Layer Aggregation for Improved Multilingual Language Understanding in Dialogue

no code implementations Findings (ACL) 2022 Evgeniia Razumovskaia, Ivan Vulić, Anna Korhonen

Scaling dialogue systems to a multitude of domains, tasks and languages relies on costly and time-consuming data annotation for different domain-task-language configurations.

Data Augmentation Natural Language Understanding

Towards Better Context-aware Lexical Semantics:Adjusting Contextualized Representations through Static Anchors

no code implementations EMNLP 2020 Qianchu Liu, Diana McCarthy, Anna Korhonen

One of the most powerful features of contextualized models is their dynamic embeddings for words in context, leading to state-of-the-art representations for context-aware lexical semantics.


MAD-G: Multilingual Adapter Generation for Efficient Cross-Lingual Transfer

no code implementations Findings (EMNLP) 2021 Alan Ansell, Edoardo Maria Ponti, Jonas Pfeiffer, Sebastian Ruder, Goran Glavaš, Ivan Vulić, Anna Korhonen

While offering (1) improved fine-tuning efficiency (by a factor of around 50 in our experiments), (2) a smaller parameter budget, and (3) increased language coverage, MAD-G remains competitive with more expensive methods for language-specific adapter training across the board.

Dependency Parsing named-entity-recognition +3

Multi-SimLex: A Large-Scale Evaluation of Multilingual and Crosslingual Lexical Semantic Similarity

no code implementations CL (ACL) 2020 Ivan Vulić, Simon Baker, Edoardo Maria Ponti, Ulla Petti, Ira Leviant, Kelly Wing, Olga Majewska, Eden Bar, Matt Malone, Thierry Poibeau, Roi Reichart, Anna Korhonen

We introduce Multi-SimLex, a large-scale lexical resource and evaluation benchmark covering data sets for 12 typologically diverse languages, including major languages (e. g., Mandarin Chinese, Spanish, Russian) as well as less-resourced ones (e. g., Welsh, Kiswahili).

Semantic Similarity Semantic Textual Similarity +1

Improving Machine Translation of Rare and Unseen Word Senses

no code implementations WMT (EMNLP) 2021 Viktor Hangya, Qianchu Liu, Dario Stojanovski, Alexander Fraser, Anna Korhonen

The performance of NMT systems has improved drastically in the past few years but the translation of multi-sense words still poses a challenge.

Bilingual Lexicon Induction Translation +2

Exposing Cross-Lingual Lexical Knowledge from Multilingual Sentence Encoders

no code implementations30 Apr 2022 Ivan Vulić, Goran Glavaš, Fangyu Liu, Nigel Collier, Edoardo Maria Ponti, Anna Korhonen

Pretrained multilingual language models (LMs) can be successfully transformed into multilingual sentence encoders (SEs; e. g., LaBSE, xMPNET) via additional fine-tuning or model distillation on parallel data.

Contrastive Learning Cross-Lingual Entity Linking +5

Improving Word Translation via Two-Stage Contrastive Learning

1 code implementation ACL 2022 Yaoyiran Li, Fangyu Liu, Nigel Collier, Anna Korhonen, Ivan Vulić

At Stage C1, we propose to refine standard cross-lingual linear maps between static word embeddings (WEs) via a contrastive learning objective; we also show how to integrate it into the self-learning procedure for even more refined cross-lingual maps.

Bilingual Lexicon Induction Contrastive Learning +7

Delving Deeper into Cross-lingual Visual Question Answering

no code implementations15 Feb 2022 Chen Liu, Jonas Pfeiffer, Anna Korhonen, Ivan Vulic, Iryna Gurevych

Previous work on cross-lingual VQA has reported poor zero-shot transfer performance of current multilingual multimodal Transformers and large gaps to monolingual performance, attributed mostly to misalignment of text embeddings between the source and target languages, without providing any additional deeper analyses.

Question Answering Visual Question Answering

Measuring Context-Word Biases in Lexical Semantic Datasets

no code implementations13 Dec 2021 Qianchu Liu, Diana McCarthy, Anna Korhonen

This study demonstrates that with heavy context or target word biases, models are usually not being tested for word-in-context representations as such in these tasks and results are therefore open to misinterpretation.

Composable Sparse Fine-Tuning for Cross-Lingual Transfer

1 code implementation ACL 2022 Alan Ansell, Edoardo Maria Ponti, Anna Korhonen, Ivan Vulić

Based on an in-depth analysis, we additionally find that sparsity is crucial to prevent both 1) interference between the fine-tunings to be composed and 2) overfitting.

Language Modelling Masked Language Modeling +2

MirrorWiC: On Eliciting Word-in-Context Representations from Pretrained Language Models

1 code implementation CoNLL (EMNLP) 2021 Qianchu Liu, Fangyu Liu, Nigel Collier, Anna Korhonen, Ivan Vulić

Recent work indicated that pretrained language models (PLMs) such as BERT and RoBERTa can be transformed into effective sentence and word encoders even via simple self-supervised techniques.

Contextualised Word Representations Contrastive Learning +1

Towards Zero-shot Language Modeling

no code implementations IJCNLP 2019 Edoardo Maria Ponti, Ivan Vulić, Ryan Cotterell, Roi Reichart, Anna Korhonen

Motivated by this question, we aim at constructing an informative prior over neural weights, in order to adapt quickly to held-out languages in the task of character-level language modeling.

Language Modelling

LexFit: Lexical Fine-Tuning of Pretrained Language Models

no code implementations ACL 2021 Ivan Vuli{\'c}, Edoardo Maria Ponti, Anna Korhonen, Goran Glava{\v{s}}

Inspired by prior work on semantic specialization of static word embedding (WE) models, we show that it is possible to expose and enrich lexical knowledge from the LMs, that is, to specialize them to serve as effective and universal {``}decontextualized{''} word encoders even when fed input words {``}in isolation{''} (i. e., without any context).

Cross-Lingual Transfer Pretrained Language Models

Learning Domain-Specialised Representations for Cross-Lingual Biomedical Entity Linking

1 code implementation ACL 2021 Fangyu Liu, Ivan Vulić, Anna Korhonen, Nigel Collier

To this end, we propose and evaluate a series of cross-lingual transfer methods for the XL-BEL task, and demonstrate that general-domain bitext helps propagate the available English knowledge to languages with little to no in-domain data.

Cross-Lingual Transfer Entity Linking +1

AM2iCo: Evaluating Word Meaning in Context across Low-Resource Languages with Adversarial Examples

1 code implementation EMNLP 2021 Qianchu Liu, Edoardo M. Ponti, Diana McCarthy, Ivan Vulić, Anna Korhonen

In order to address these gaps, we present AM2iCo (Adversarial and Multilingual Meaning in Context), a wide-coverage cross-lingual and multilingual evaluation set; it aims to faithfully assess the ability of state-of-the-art (SotA) representation models to understand the identity of word meaning in cross-lingual contexts for 14 language pairs.

Crossing the Conversational Chasm: A Primer on Natural Language Processing for Multilingual Task-Oriented Dialogue Systems

no code implementations17 Apr 2021 Evgeniia Razumovskaia, Goran Glavaš, Olga Majewska, Edoardo M. Ponti, Anna Korhonen, Ivan Vulić

We find that the most critical factor preventing the creation of truly multilingual ToD systems is the lack of datasets in most languages for both training and evaluation.

Cross-Lingual Transfer Machine Translation +3

Combining Deep Generative Models and Multi-lingual Pretraining for Semi-supervised Document Classification

1 code implementation EACL 2021 Yi Zhu, Ehsan Shareghi, Yingzhen Li, Roi Reichart, Anna Korhonen

Semi-supervised learning through deep generative models and multi-lingual pretraining techniques have orchestrated tremendous success across different areas of NLP.

Classification Document Classification +1

A Closer Look at Few-Shot Crosslingual Transfer: The Choice of Shots Matters

no code implementations ACL 2021 Mengjie Zhao, Yi Zhu, Ehsan Shareghi, Ivan Vulić, Roi Reichart, Anna Korhonen, Hinrich Schütze

Few-shot crosslingual transfer has been shown to outperform its zero-shot counterpart with pretrained encoders like multilingual BERT.

Few-Shot Learning

Verb Knowledge Injection for Multilingual Event Processing

no code implementations ACL 2021 Olga Majewska, Ivan Vulić, Goran Glavaš, Edoardo M. Ponti, Anna Korhonen

We investigate whether injecting explicit information on verbs' semantic-syntactic behaviour improves the performance of LM-pretrained Transformers in event extraction tasks -- downstream tasks for which accurate verb processing is paramount.

Event Extraction Language Modelling

Manual Clustering and Spatial Arrangement of Verbs for Multilingual Evaluation and Typology Analysis

1 code implementation COLING 2020 Olga Majewska, Ivan Vuli{\'c}, Diana McCarthy, Anna Korhonen

We present the first evaluation of the applicability of a spatial arrangement method (SpAM) to a typologically diverse language sample, and its potential to produce semantic evaluation resources to support multilingual NLP, with a focus on verb semantics.

Multilingual NLP

SemEval-2020 Task 2: Predicting Multilingual and Cross-Lingual (Graded) Lexical Entailment

no code implementations SEMEVAL 2020 Goran Glava{\v{s}}, Ivan Vuli{\'c}, Anna Korhonen, Simone Paolo Ponzetto

The shared task spans three dimensions: (1) monolingual vs. cross-lingual LE, (2) binary vs. graded LE, and (3) a set of 6 diverse languages (and 15 corresponding language pairs).

Lexical Entailment Natural Language Inference

Probing Pretrained Language Models for Lexical Semantics

no code implementations EMNLP 2020 Ivan Vulić, Edoardo Maria Ponti, Robert Litschko, Goran Glavaš, Anna Korhonen

The success of large pretrained language models (LMs) such as BERT and RoBERTa has sparked interest in probing their representations, in order to unveil what types of knowledge they implicitly capture.

Pretrained Language Models

Investigating Word-Class Distributions in Word Vector Spaces

no code implementations ACL 2020 Ryohei Sasano, Anna Korhonen

This paper presents an investigation on the distribution of word vectors belonging to a certain word class in a pre-trained word vector space.

Improving Bilingual Lexicon Induction with Unsupervised Post-Processing of Monolingual Word Vector Spaces

no code implementations WS 2020 Ivan Vuli{\'c}, Anna Korhonen, Goran Glava{\v{s}}

Work on projection-based induction of cross-lingual word embedding spaces (CLWEs) predominantly focuses on the improvement of the projection (i. e., mapping) mechanisms.

Bilingual Lexicon Induction

Multidirectional Associative Optimization of Function-Specific Word Representations

1 code implementation ACL 2020 Daniela Gerz, Ivan Vulić, Marek Rei, Roi Reichart, Anna Korhonen

We present a neural framework for learning associations between interrelated groups of words such as the ones found in Subject-Verb-Object (SVO) structures.

XCOPA: A Multilingual Dataset for Causal Commonsense Reasoning

1 code implementation EMNLP 2020 Edoardo Maria Ponti, Goran Glavaš, Olga Majewska, Qianchu Liu, Ivan Vulić, Anna Korhonen

In order to simulate human language capacity, natural language processing systems must be able to reason about the dynamics of everyday situations, including their possible causes and effects.

 Ranked #1 on Cross-Lingual Transfer on XCOPA (using extra training data)

Cross-Lingual Transfer Natural Language Processing +1

Stylistic Dialogue Generation via Information-Guided Reinforcement Learning Strategy

no code implementations5 Apr 2020 Yixuan Su, Deng Cai, Yan Wang, Simon Baker, Anna Korhonen, Nigel Collier, Xiaojiang Liu

To enable better balance between the content quality and the style, we introduce a new training strategy, know as Information-Guided Reinforcement Learning (IG-RL).

Dialogue Generation reinforcement-learning +1

Prototype-to-Style: Dialogue Generation with Style-Aware Editing on Retrieval Memory

no code implementations5 Apr 2020 Yixuan Su, Yan Wang, Simon Baker, Deng Cai, Xiaojiang Liu, Anna Korhonen, Nigel Collier

A stylistic response generator then takes the prototype and the desired language style as model input to obtain a high-quality and stylistic response.

Benchmark Dialogue Generation +1

Multi-SimLex: A Large-Scale Evaluation of Multilingual and Cross-Lingual Lexical Semantic Similarity

no code implementations10 Mar 2020 Ivan Vulić, Simon Baker, Edoardo Maria Ponti, Ulla Petti, Ira Leviant, Kelly Wing, Olga Majewska, Eden Bar, Matt Malone, Thierry Poibeau, Roi Reichart, Anna Korhonen

We introduce Multi-SimLex, a large-scale lexical resource and evaluation benchmark covering datasets for 12 typologically diverse languages, including major languages (e. g., Mandarin Chinese, Spanish, Russian) as well as less-resourced ones (e. g., Welsh, Kiswahili).

Cross-Lingual Word Embeddings Semantic Similarity +2

The Secret is in the Spectra: Predicting Cross-lingual Task Performance with Spectral Similarity Measures

no code implementations EMNLP 2020 Haim Dubossarsky, Ivan Vulić, Roi Reichart, Anna Korhonen

Performance in cross-lingual NLP tasks is impacted by the (dis)similarity of languages at hand: e. g., previous work has suggested there is a connection between the expected success of bilingual lexicon induction (BLI) and the assumption of (approximate) isomorphism between monolingual embedding spaces.

Bilingual Lexicon Induction POS

Semi-supervised Bootstrapping of Dialogue State Trackers for Task Oriented Modelling

no code implementations26 Nov 2019 Bo-Hsiang Tseng, Marek Rei, Paweł Budzianowski, Richard E. Turner, Bill Byrne, Anna Korhonen

Dialogue systems benefit greatly from optimizing on detailed annotations, such as transcribed utterances, internal dialogue state representations and dialogue act labels.

Investigating Cross-Lingual Alignment Methods for Contextualized Embeddings with Token-Level Evaluation

no code implementations CONLL 2019 Qianchu Liu, Diana McCarthy, Ivan Vuli{\'c}, Anna Korhonen

In this paper, we present a thorough investigation on methods that align pre-trained contextualized embeddings into shared cross-lingual context-aware embedding space, providing strong reference benchmarks for future context-aware crosslingual models.

Word Similarity

Cross-lingual Semantic Specialization via Lexical Relation Induction

no code implementations IJCNLP 2019 Edoardo Maria Ponti, Ivan Vuli{\'c}, Goran Glava{\v{s}}, Roi Reichart, Anna Korhonen

Semantic specialization integrates structured linguistic knowledge from external resources (such as lexical relations in WordNet) into pretrained distributional vectors in the form of constraints.

Lexical Simplification Semantic Textual Similarity +2

Semi-Supervised Bootstrapping of Dialogue State Trackers for Task-Oriented Modelling

no code implementations IJCNLP 2019 Bo-Hsiang Tseng, Marek Rei, Pawe{\l} Budzianowski, Richard Turner, Bill Byrne, Anna Korhonen

Dialogue systems benefit greatly from optimizing on detailed annotations, such as transcribed utterances, internal dialogue state representations and dialogue act labels.

Specializing Unsupervised Pretraining Models for Word-Level Semantic Similarity

1 code implementation COLING 2020 Anne Lauscher, Ivan Vulić, Edoardo Maria Ponti, Anna Korhonen, Goran Glavaš

In this work, we complement such distributional knowledge with external lexical knowledge, that is, we integrate the discrete knowledge on word-level semantic similarity into pretraining.

Language Modelling Lexical Simplification +6

Do We Really Need Fully Unsupervised Cross-Lingual Embeddings?

1 code implementation IJCNLP 2019 Ivan Vulić, Goran Glavaš, Roi Reichart, Anna Korhonen

A series of bilingual lexicon induction (BLI) experiments with 15 diverse languages (210 language pairs) show that fully unsupervised CLWE methods still fail for a large number of language pairs (e. g., they yield zero BLI performance for 87/210 pairs).

Bilingual Lexicon Induction Self-Learning

Bayesian Learning for Neural Dependency Parsing

no code implementations NAACL 2019 Ehsan Shareghi, Yingzhen Li, Yi Zhu, Roi Reichart, Anna Korhonen

While neural dependency parsers provide state-of-the-art accuracy for several languages, they still rely on large amounts of costly labeled training data.

Dependency Parsing POS +1

A Systematic Study of Leveraging Subword Information for Learning Word Representations

1 code implementation NAACL 2019 Yi Zhu, Ivan Vulić, Anna Korhonen

The use of subword-level information (e. g., characters, character n-grams, morphemes) has become ubiquitous in modern word representation learning.

Dependency Parsing Entity Typing +2

Post-Specialisation: Retrofitting Vectors of Words Unseen in Lexical Resources

1 code implementation NAACL 2018 Ivan Vulić, Goran Glavaš, Nikola Mrkšić, Anna Korhonen

Word vector specialisation (also known as retrofitting) is a portable, light-weight approach to fine-tuning arbitrary distributional word vector spaces by injecting external knowledge from rich lexical resources such as WordNet.

Dialogue State Tracking Text Simplification +1

Erratum: Link prediction in drug-target interactions network using similarity indices

no code implementations1 Nov 2017 Yiding Lu, Yufan Guo, Anna Korhonen

Conclusion: This demonstrates that approaches purely based on network topology provide a more suitable approach to DTI prediction in the many real-life situations where little or no prior knowledge is available about the characteristics of drugs, targets, or their interactions.

Link Prediction

Initializing neural networks for hierarchical multi-label text classification

no code implementations WS 2017 Simon Baker, Anna Korhonen

Many tasks in the biomedical domain require the assignment of one or more predefined labels to input text, where the labels are a part of a hierarchical structure (such as a taxonomy).

Classification General Classification +3

Cross-Lingual Induction and Transfer of Verb Classes Based on Word Vector Space Specialisation

no code implementations EMNLP 2017 Ivan Vulić, Nikola Mrkšić, Anna Korhonen

Existing approaches to automatic VerbNet-style verb classification are heavily dependent on feature engineering and therefore limited to languages with mature NLP pipelines.

Cross-Lingual Transfer Feature Engineering +3

Morph-fitting: Fine-Tuning Word Vector Spaces with Simple Language-Specific Rules

no code implementations ACL 2017 Ivan Vulić, Nikola Mrkšić, Roi Reichart, Diarmuid Ó Séaghdha, Steve Young, Anna Korhonen

Morphologically rich languages accentuate two properties of distributional vector space models: 1) the difficulty of inducing accurate representations for low-frequency word forms; and 2) insensitivity to distinct lexical relations that have similar distributional signatures.

Dialogue State Tracking

Decoding Sentiment from Distributed Representations of Sentences

no code implementations SEMEVAL 2017 Edoardo Maria Ponti, Ivan Vulić, Anna Korhonen

Distributed representations of sentences have been developed recently to represent their meaning as real-valued vectors.

Evaluation by Association: A Systematic Study of Quantitative Word Association Evaluation

no code implementations EACL 2017 Ivan Vuli{\'c}, Douwe Kiela, Anna Korhonen

Recent work on evaluating representation learning architectures in NLP has established a need for evaluation protocols based on subconscious cognitive measures rather than manually tailored intrinsic similarity and relatedness tasks.

Information Retrieval Representation Learning +1

Cancer Hallmark Text Classification Using Convolutional Neural Networks

no code implementations WS 2016 Simon Baker, Anna Korhonen, Sampo Pyysalo

Methods based on deep learning approaches have recently achieved state-of-the-art performance in a range of machine learning tasks and are increasingly applied to natural language processing (NLP).

Classification General Classification +2

Survey on the Use of Typological Information in Natural Language Processing

no code implementations COLING 2016 Helen O'Horan, Yevgeni Berzak, Ivan Vulić, Roi Reichart, Anna Korhonen

In recent years linguistic typology, which classifies the world's languages according to their functional and structural properties, has been widely used to support multilingual NLP.

Multilingual NLP Natural Language Processing

Automatic Selection of Context Configurations for Improved Class-Specific Word Representations

no code implementations CONLL 2017 Ivan Vulić, Roy Schwartz, Ari Rappoport, Roi Reichart, Anna Korhonen

With our selected context configurations, we train on only 14% (A), 26. 2% (V), and 33. 6% (N) of all dependency-based contexts, resulting in a reduced training time.

Word Similarity

HyperLex: A Large-Scale Evaluation of Graded Lexical Entailment

no code implementations CL 2017 Ivan Vulić, Daniela Gerz, Douwe Kiela, Felix Hill, Anna Korhonen

We introduce HyperLex - a dataset and evaluation resource that quantifies the extent of of the semantic category membership, that is, type-of relation also known as hyponymy-hypernymy or lexical entailment (LE) relation between 2, 616 concept pairs.

Lexical Entailment Representation Learning

SimVerb-3500: A Large-Scale Evaluation Set of Verb Similarity

1 code implementation EMNLP 2016 Daniela Gerz, Ivan Vulić, Felix Hill, Roi Reichart, Anna Korhonen

Verbs play a critical role in the meaning of sentences, but these ubiquitous words have received little attention in recent distributional semantics research.

Representation Learning

Anchoring and Agreement in Syntactic Annotations

no code implementations EMNLP 2016 Yevgeni Berzak, Yan Huang, Andrei Barbu, Anna Korhonen, Boris Katz

Our agreement results control for parser bias, and are consequential in that they are on par with state of the art parsing performance for English newswire.

Decision Making Dependency Parsing

Learning Distributed Representations of Sentences from Unlabelled Data

1 code implementation NAACL 2016 Felix Hill, Kyunghyun Cho, Anna Korhonen

Unsupervised methods for learning distributed representations of words are ubiquitous in today's NLP research, but far less is known about the best ways to learn distributed phrase or sentence representations from unlabelled data.

Representation Learning

Learning to Understand Phrases by Embedding the Dictionary

2 code implementations TACL 2016 Felix Hill, Kyunghyun Cho, Anna Korhonen, Yoshua Bengio

Distributional models that learn rich semantic word representations are a success story of recent NLP research.

SimLex-999: Evaluating Semantic Models with (Genuine) Similarity Estimation

3 code implementations CL 2015 Felix Hill, Roi Reichart, Anna Korhonen

We present SimLex-999, a gold standard resource for evaluating distributional semantic models that improves on existing resources in several important ways.

Representation Learning

Native Language Identification Using Large, Longitudinal Data

no code implementations LREC 2014 Xiao Jiang, Yufan Guo, Jeroen Geertzen, Dora Alexopoulou, Lin Sun, Anna Korhonen

Native Language Identification (NLI) is a task aimed at determining the native language (L1) of learners of second language (L2) on the basis of their written texts.

Native Language Identification Text Classification

Multi-Modal Models for Concrete and Abstract Concept Meaning

no code implementations TACL 2014 Felix Hill, Roi Reichart, Anna Korhonen

Multi-modal models that learn semantic representations from both linguistic and perceptual input outperform language-only models on a range of evaluations, and better reflect human concept acquisition.

Language Acquisition

Cannot find the paper you are looking for? You can Submit a new open access paper.