Search Results for author: Anna Korhonen

Found 130 papers, 41 papers with code

Improving Machine Translation of Rare and Unseen Word Senses

no code implementations WMT (EMNLP) 2021 Viktor Hangya, Qianchu Liu, Dario Stojanovski, Alexander Fraser, Anna Korhonen

The performance of NMT systems has improved drastically in the past few years but the translation of multi-sense words still poses a challenge.

Bilingual Lexicon Induction NMT +3

Towards Better Context-aware Lexical Semantics:Adjusting Contextualized Representations through Static Anchors

no code implementations EMNLP 2020 Qianchu Liu, Diana McCarthy, Anna Korhonen

One of the most powerful features of contextualized models is their dynamic embeddings for words in context, leading to state-of-the-art representations for context-aware lexical semantics.

Data Augmentation and Learned Layer Aggregation for Improved Multilingual Language Understanding in Dialogue

no code implementations Findings (ACL) 2022 Evgeniia Razumovskaia, Ivan Vulić, Anna Korhonen

Scaling dialogue systems to a multitude of domains, tasks and languages relies on costly and time-consuming data annotation for different domain-task-language configurations.

Data Augmentation Natural Language Understanding

Multi-SimLex: A Large-Scale Evaluation of Multilingual and Crosslingual Lexical Semantic Similarity

no code implementations CL (ACL) 2020 Ivan Vulić, Simon Baker, Edoardo Maria Ponti, Ulla Petti, Ira Leviant, Kelly Wing, Olga Majewska, Eden Bar, Matt Malone, Thierry Poibeau, Roi Reichart, Anna Korhonen

We introduce Multi-SimLex, a large-scale lexical resource and evaluation benchmark covering data sets for 12 typologically diverse languages, including major languages (e. g., Mandarin Chinese, Spanish, Russian) as well as less-resourced ones (e. g., Welsh, Kiswahili).

Representation Learning Semantic Similarity +2

MAD-G: Multilingual Adapter Generation for Efficient Cross-Lingual Transfer

no code implementations Findings (EMNLP) 2021 Alan Ansell, Edoardo Maria Ponti, Jonas Pfeiffer, Sebastian Ruder, Goran Glavaš, Ivan Vulić, Anna Korhonen

While offering (1) improved fine-tuning efficiency (by a factor of around 50 in our experiments), (2) a smaller parameter budget, and (3) increased language coverage, MAD-G remains competitive with more expensive methods for language-specific adapter training across the board.

Dependency Parsing named-entity-recognition +4

BAD-X: Bilingual Adapters Improve Zero-Shot Cross-Lingual Transfer

1 code implementation NAACL 2022 Marinela Parović, Goran Glavaš, Ivan Vulić, Anna Korhonen

Adapter modules enable modular and efficient zero-shot cross-lingual transfer, where current state-of-the-art adapter-based approaches learn specialized language adapters (LAs) for individual languages.

Zero-Shot Cross-Lingual Transfer

Semantic Data Set Construction from Human Clustering and Spatial Arrangement

no code implementations CL (ACL) 2021 Olga Majewska, Diana McCarthy, Jasper J. F. van den Bosch, Nikolaus Kriegeskorte, Ivan Vulić, Anna Korhonen

We demonstrate how the resultant data set can be used for fine-grained analyses and evaluation of representation learning models on the intrinsic tasks of semantic clustering and semantic similarity.

Clustering Representation Learning +3

Analyzing and Adapting Large Language Models for Few-Shot Multilingual NLU: Are We There Yet?

no code implementations4 Mar 2024 Evgeniia Razumovskaia, Ivan Vulić, Anna Korhonen

Supervised fine-tuning (SFT), supervised instruction tuning (SIT) and in-context learning (ICL) are three alternative, de facto standard approaches to few-shot learning.

Few-Shot Learning In-Context Learning +1

Self-Augmented In-Context Learning for Unsupervised Word Translation

no code implementations15 Feb 2024 Yaoyiran Li, Anna Korhonen, Ivan Vulić

Recent work has shown that, while large language models (LLMs) demonstrate strong word translation or bilingual lexicon induction (BLI) capabilities in few-shot setups, they still cannot match the performance of 'traditional' mapping-based approaches in the unsupervised scenario where no seed translation pairs are available, especially for lower-resource languages.

Bilingual Lexicon Induction Cross-Lingual Word Embeddings +9

Scaling Sparse Fine-Tuning to Large Language Models

2 code implementations29 Jan 2024 Alan Ansell, Ivan Vulić, Hannah Sterz, Anna Korhonen, Edoardo M. Ponti

We experiment with instruction-tuning of LLMs on standard dataset mixtures, finding that SpIEL is often superior to popular parameter-efficient fine-tuning methods like LoRA (low-rank adaptation) in terms of performance and comparable in terms of run time.

Quantization

DIALIGHT: Lightweight Multilingual Development and Evaluation of Task-Oriented Dialogue Systems with Large Language Models

2 code implementations4 Jan 2024 Songbo Hu, Xiaobin Wang, Zhangdie Yuan, Anna Korhonen, Ivan Vulić

We present DIALIGHT, a toolkit for developing and evaluating multilingual Task-Oriented Dialogue (ToD) systems which facilitates systematic evaluations and comparisons between ToD systems using fine-tuning of Pretrained Language Models (PLMs) and those utilising the zero-shot and in-context learning capabilities of Large Language Models (LLMs).

In-Context Learning Task-Oriented Dialogue Systems

On Task Performance and Model Calibration with Supervised and Self-Ensembled In-Context Learning

1 code implementation21 Dec 2023 Chengzu Li, Han Zhou, Goran Glavaš, Anna Korhonen, Ivan Vulić

Following the standard supervised fine-tuning (SFT) paradigm, in-context learning (ICL) has become an efficient approach propelled by the recent advancements in large language models (LLMs), yielding promising performance across various tasks in few-shot data setups.

In-Context Learning

SQATIN: Supervised Instruction Tuning Meets Question Answering for Improved Dialogue NLU

no code implementations16 Nov 2023 Evgeniia Razumovskaia, Goran Glavaš, Anna Korhonen, Ivan Vulić

Task-oriented dialogue (ToD) systems help users execute well-defined tasks across a variety of domains (e. g., $\textit{flight booking}$ or $\textit{food ordering}$), with their Natural Language Understanding (NLU) components being dedicated to the analysis of user utterances, predicting users' intents ($\textit{Intent Detection}$, ID) and extracting values for informational slots ($\textit{Value Extraction}$, VE).

Intent Detection Natural Language Understanding +1

Are Large Language Models Temporally Grounded?

1 code implementation14 Nov 2023 Yifu Qiu, Zheng Zhao, Yftah Ziser, Anna Korhonen, Edoardo M. Ponti, Shay B. Cohen

Instead, we provide LLMs with textual narratives and probe them with respect to their common-sense knowledge of the structure and duration of events, their ability to order events along a timeline, and self-consistency within their temporal model (e. g., temporal relations such as after and before are mutually exclusive for any pair of events).

Common Sense Reasoning In-Context Learning +2

Quantifying the Dialect Gap and its Correlates Across Languages

no code implementations23 Oct 2023 Anjali Kantharuban, Ivan Vulić, Anna Korhonen

Historically, researchers and consumers have noticed a decrease in quality when applying NLP tools to minority variants of languages (i. e. Puerto Rican Spanish or Swiss German), but studies exploring this have been limited to a select few languages.

Automatic Speech Recognition Machine Translation +2

On Bilingual Lexicon Induction with Large Language Models

1 code implementation21 Oct 2023 Yaoyiran Li, Anna Korhonen, Ivan Vulić

Bilingual Lexicon Induction (BLI) is a core task in multilingual NLP that still, to a large extent, relies on calculating cross-lingual word representations.

Bilingual Lexicon Induction Cross-Lingual Word Embeddings +8

Survival of the Most Influential Prompts: Efficient Black-Box Prompt Search via Clustering and Pruning

1 code implementation19 Oct 2023 Han Zhou, Xingchen Wan, Ivan Vulić, Anna Korhonen

Prompt-based learning has been an effective paradigm for large pretrained language models (LLM), enabling few-shot or even zero-shot learning.

Combinatorial Optimization Zero-Shot Learning

Navigating Cultural Chasms: Exploring and Unlocking the Cultural POV of Text-To-Image Models

no code implementations3 Oct 2023 Mor Ventura, Eyal Ben-David, Anna Korhonen, Roi Reichart

Text-To-Image (TTI) models, such as DALL-E and StableDiffusion, have demonstrated remarkable prompt-based image generation capabilities.

Image Generation Visual Question Answering (VQA)

Multi3WOZ: A Multilingual, Multi-Domain, Multi-Parallel Dataset for Training and Evaluating Culturally Adapted Task-Oriented Dialog Systems

1 code implementation26 Jul 2023 Songbo Hu, Han Zhou, Mete Hergul, Milan Gritta, Guchun Zhang, Ignacio Iacobacci, Ivan Vulić, Anna Korhonen

Creating high-quality annotated data for task-oriented dialog (ToD) is known to be notoriously difficult, and the challenges are amplified when the goal is to create equitable, culturally adapted, and large-scale ToD datasets for multiple languages.

Translation

Cross-Lingual Transfer with Target Language-Ready Task Adapters

no code implementations5 Jun 2023 Marinela Parović, Alan Ansell, Ivan Vulić, Anna Korhonen

We address this mismatch by exposing the task adapter to the target language adapter during training, and empirically validate several variants of the idea: in the simplest form, we alternate between using the source and target language adapters during task adapter training, which can be generalized to cycling over any set of language adapters.

Zero-Shot Cross-Lingual Transfer

Distilling Efficient Language-Specific Models for Cross-Lingual Transfer

1 code implementation2 Jun 2023 Alan Ansell, Edoardo Maria Ponti, Anna Korhonen, Ivan Vulić

Specifically, we use a two-phase distillation approach, termed BiStil: (i) the first phase distils a general bilingual model from the MMT, while (ii) the second, task-specific phase sparsely fine-tunes the bilingual "student" model using a task-tuned variant of the original MMT as its "teacher".

Transfer Learning XLM-R +1

Translation-Enhanced Multilingual Text-to-Image Generation

no code implementations30 May 2023 Yaoyiran Li, Ching-Yun Chang, Stephen Rawls, Ivan Vulić, Anna Korhonen

Research on text-to-image generation (TTI) still predominantly focuses on the English language due to the lack of annotated image-caption data in other languages; in the long run, this might widen inequitable access to TTI technology.

Cross-lingual Text-to-Image Generation Crosslingual Text-to-Image Generation +6

Detecting and Mitigating Hallucinations in Multilingual Summarisation

1 code implementation23 May 2023 Yifu Qiu, Yftah Ziser, Anna Korhonen, Edoardo M. Ponti, Shay B. Cohen

With the existing faithful metrics focusing on English, even measuring the extent of this phenomenon in cross-lingual settings is hard.

Cross-Lingual Transfer

Transfer-Free Data-Efficient Multilingual Slot Labeling

no code implementations22 May 2023 Evgeniia Razumovskaia, Ivan Vulić, Anna Korhonen

It is especially effective for the most challenging transfer-free few-shot setups, paving the way for quick and data-efficient bootstrapping of multilingual slot labelers for ToD.

Contrastive Learning Cross-Lingual Transfer +3

Language-Agnostic Bias Detection in Language Models with Bias Probing

no code implementations22 May 2023 Abdullatif Köksal, Omer Faruk Yalcin, Ahmet Akbiyik, M. Tahir Kilavuz, Anna Korhonen, Hinrich Schütze

For nationality as a case study, we show that LABDet `surfaces' nationality bias by training a classifier on top of a frozen PLM on non-nationality sentiment detection.

Bias Detection

AutoPEFT: Automatic Configuration Search for Parameter-Efficient Fine-Tuning

1 code implementation28 Jan 2023 Han Zhou, Xingchen Wan, Ivan Vulić, Anna Korhonen

Large pretrained language models are widely used in downstream NLP tasks via task-specific fine-tuning, but such procedures can be costly.

Bayesian Optimisation Neural Architecture Search

MULTI3NLU++: A Multilingual, Multi-Intent, Multi-Domain Dataset for Natural Language Understanding in Task-Oriented Dialogue

no code implementations20 Dec 2022 Nikita Moghe, Evgeniia Razumovskaia, Liane Guillou, Ivan Vulić, Anna Korhonen, Alexandra Birch

We use MULTI3NLU++ to benchmark state-of-the-art multilingual models for the NLU tasks of intent detection and slot labelling for TOD systems in the multilingual setting.

Intent Detection Machine Translation +2

Reranking Overgenerated Responses for End-to-End Task-Oriented Dialogue Systems

1 code implementation7 Nov 2022 Songbo Hu, Ivan Vulić, Fangyu Liu, Anna Korhonen

At training, the high-scoring partition comprises all generated responses whose similarity to the gold response is higher than the similarity of the greedy response to the gold response.

Task-Oriented Dialogue Systems

Improving Bilingual Lexicon Induction with Cross-Encoder Reranking

1 code implementation30 Oct 2022 Yaoyiran Li, Fangyu Liu, Ivan Vulić, Anna Korhonen

This crucial step is done via 1) creating a word similarity dataset, comprising positive word pairs (i. e., true translations) and hard negative pairs induced from the original CLWE space, and then 2) fine-tuning an mPLM (e. g., mBERT or XLM-R) in a cross-encoder manner to predict the similarity scores.

Bilingual Lexicon Induction Cross-Lingual Word Embeddings +7

Can Pretrained Language Models (Yet) Reason Deductively?

1 code implementation12 Oct 2022 Zhangdie Yuan, Songbo Hu, Ivan Vulić, Anna Korhonen, Zaiqiao Meng

Acquiring factual knowledge with Pretrained Language Models (PLMs) has attracted increasing attention, showing promising performance in many knowledge-intensive tasks.

Probing Cross-Lingual Lexical Knowledge from Multilingual Sentence Encoders

no code implementations30 Apr 2022 Ivan Vulić, Goran Glavaš, Fangyu Liu, Nigel Collier, Edoardo Maria Ponti, Anna Korhonen

In this work, we probe SEs for the amount of cross-lingual lexical knowledge stored in their parameters, and compare them against the original multilingual LMs.

Contrastive Learning Cross-Lingual Entity Linking +6

Improving Word Translation via Two-Stage Contrastive Learning

1 code implementation ACL 2022 Yaoyiran Li, Fangyu Liu, Nigel Collier, Anna Korhonen, Ivan Vulić

At Stage C1, we propose to refine standard cross-lingual linear maps between static word embeddings (WEs) via a contrastive learning objective; we also show how to integrate it into the self-learning procedure for even more refined cross-lingual maps.

Bilingual Lexicon Induction Contrastive Learning +7

Delving Deeper into Cross-lingual Visual Question Answering

1 code implementation15 Feb 2022 Chen Liu, Jonas Pfeiffer, Anna Korhonen, Ivan Vulić, Iryna Gurevych

2) We analyze cross-lingual VQA across different question types of varying complexity for different multilingual multimodal Transformers, and identify question types that are the most difficult to improve on.

Inductive Bias Question Answering +1

Measuring Context-Word Biases in Lexical Semantic Datasets

no code implementations13 Dec 2021 Qianchu Liu, Diana McCarthy, Anna Korhonen

Our findings demonstrate that models are usually not being tested for word-in-context semantics in the same way as humans are in these tasks, which helps us better understand the model-human gap.

Retrieval

MirrorWiC: On Eliciting Word-in-Context Representations from Pretrained Language Models

1 code implementation CoNLL (EMNLP) 2021 Qianchu Liu, Fangyu Liu, Nigel Collier, Anna Korhonen, Ivan Vulić

Recent work indicated that pretrained language models (PLMs) such as BERT and RoBERTa can be transformed into effective sentence and word encoders even via simple self-supervised techniques.

Contextualised Word Representations Contrastive Learning +1

Towards Zero-shot Language Modeling

no code implementations IJCNLP 2019 Edoardo Maria Ponti, Ivan Vulić, Ryan Cotterell, Roi Reichart, Anna Korhonen

Motivated by this question, we aim at constructing an informative prior over neural weights, in order to adapt quickly to held-out languages in the task of character-level language modeling.

Language Modelling

LexFit: Lexical Fine-Tuning of Pretrained Language Models

no code implementations ACL 2021 Ivan Vuli{\'c}, Edoardo Maria Ponti, Anna Korhonen, Goran Glava{\v{s}}

Inspired by prior work on semantic specialization of static word embedding (WE) models, we show that it is possible to expose and enrich lexical knowledge from the LMs, that is, to specialize them to serve as effective and universal {``}decontextualized{''} word encoders even when fed input words {``}in isolation{''} (i. e., without any context).

Cross-Lingual Transfer

Learning Domain-Specialised Representations for Cross-Lingual Biomedical Entity Linking

1 code implementation ACL 2021 Fangyu Liu, Ivan Vulić, Anna Korhonen, Nigel Collier

To this end, we propose and evaluate a series of cross-lingual transfer methods for the XL-BEL task, and demonstrate that general-domain bitext helps propagate the available English knowledge to languages with little to no in-domain data.

Cross-Lingual Transfer Entity Linking

AM2iCo: Evaluating Word Meaning in Context across Low-Resource Languages with Adversarial Examples

1 code implementation EMNLP 2021 Qianchu Liu, Edoardo M. Ponti, Diana McCarthy, Ivan Vulić, Anna Korhonen

In order to address these gaps, we present AM2iCo (Adversarial and Multilingual Meaning in Context), a wide-coverage cross-lingual and multilingual evaluation set; it aims to faithfully assess the ability of state-of-the-art (SotA) representation models to understand the identity of word meaning in cross-lingual contexts for 14 language pairs.

Crossing the Conversational Chasm: A Primer on Natural Language Processing for Multilingual Task-Oriented Dialogue Systems

no code implementations17 Apr 2021 Evgeniia Razumovskaia, Goran Glavaš, Olga Majewska, Edoardo M. Ponti, Anna Korhonen, Ivan Vulić

We find that the most critical factor preventing the creation of truly multilingual ToD systems is the lack of datasets in most languages for both training and evaluation.

Cross-Lingual Transfer Machine Translation +2

Combining Deep Generative Models and Multi-lingual Pretraining for Semi-supervised Document Classification

1 code implementation EACL 2021 Yi Zhu, Ehsan Shareghi, Yingzhen Li, Roi Reichart, Anna Korhonen

Semi-supervised learning through deep generative models and multi-lingual pretraining techniques have orchestrated tremendous success across different areas of NLP.

Classification Document Classification +1

A Closer Look at Few-Shot Crosslingual Transfer: The Choice of Shots Matters

no code implementations ACL 2021 Mengjie Zhao, Yi Zhu, Ehsan Shareghi, Ivan Vulić, Roi Reichart, Anna Korhonen, Hinrich Schütze

Few-shot crosslingual transfer has been shown to outperform its zero-shot counterpart with pretrained encoders like multilingual BERT.

Few-Shot Learning

Verb Knowledge Injection for Multilingual Event Processing

no code implementations ACL 2021 Olga Majewska, Ivan Vulić, Goran Glavaš, Edoardo M. Ponti, Anna Korhonen

We investigate whether injecting explicit information on verbs' semantic-syntactic behaviour improves the performance of LM-pretrained Transformers in event extraction tasks -- downstream tasks for which accurate verb processing is paramount.

Event Extraction Language Modelling

SemEval-2020 Task 2: Predicting Multilingual and Cross-Lingual (Graded) Lexical Entailment

no code implementations SEMEVAL 2020 Goran Glava{\v{s}}, Ivan Vuli{\'c}, Anna Korhonen, Simone Paolo Ponzetto

The shared task spans three dimensions: (1) monolingual vs. cross-lingual LE, (2) binary vs. graded LE, and (3) a set of 6 diverse languages (and 15 corresponding language pairs).

Lexical Entailment Natural Language Inference +1

Manual Clustering and Spatial Arrangement of Verbs for Multilingual Evaluation and Typology Analysis

1 code implementation COLING 2020 Olga Majewska, Ivan Vuli{\'c}, Diana McCarthy, Anna Korhonen

We present the first evaluation of the applicability of a spatial arrangement method (SpAM) to a typologically diverse language sample, and its potential to produce semantic evaluation resources to support multilingual NLP, with a focus on verb semantics.

Clustering Multilingual NLP +1

Emergent Communication Pretraining for Few-Shot Machine Translation

1 code implementation COLING 2020 Yaoyiran Li, Edoardo M. Ponti, Ivan Vulić, Anna Korhonen

On the other hand, this also provides an extrinsic evaluation protocol to probe the properties of emergent languages ex vitro.

Machine Translation NMT +2

Probing Pretrained Language Models for Lexical Semantics

no code implementations EMNLP 2020 Ivan Vulić, Edoardo Maria Ponti, Robert Litschko, Goran Glavaš, Anna Korhonen

The success of large pretrained language models (LMs) such as BERT and RoBERTa has sparked interest in probing their representations, in order to unveil what types of knowledge they implicitly capture.

World Knowledge

Improving Bilingual Lexicon Induction with Unsupervised Post-Processing of Monolingual Word Vector Spaces

no code implementations WS 2020 Ivan Vuli{\'c}, Anna Korhonen, Goran Glava{\v{s}}

Work on projection-based induction of cross-lingual word embedding spaces (CLWEs) predominantly focuses on the improvement of the projection (i. e., mapping) mechanisms.

Bilingual Lexicon Induction

Investigating Word-Class Distributions in Word Vector Spaces

no code implementations ACL 2020 Ryohei Sasano, Anna Korhonen

This paper presents an investigation on the distribution of word vectors belonging to a certain word class in a pre-trained word vector space.

Multidirectional Associative Optimization of Function-Specific Word Representations

1 code implementation ACL 2020 Daniela Gerz, Ivan Vulić, Marek Rei, Roi Reichart, Anna Korhonen

We present a neural framework for learning associations between interrelated groups of words such as the ones found in Subject-Verb-Object (SVO) structures.

XCOPA: A Multilingual Dataset for Causal Commonsense Reasoning

1 code implementation EMNLP 2020 Edoardo Maria Ponti, Goran Glavaš, Olga Majewska, Qianchu Liu, Ivan Vulić, Anna Korhonen

In order to simulate human language capacity, natural language processing systems must be able to reason about the dynamics of everyday situations, including their possible causes and effects.

Ranked #3 on Cross-Lingual Transfer on XCOPA (using extra training data)

Cross-Lingual Transfer Translation +1

Prototype-to-Style: Dialogue Generation with Style-Aware Editing on Retrieval Memory

no code implementations5 Apr 2020 Yixuan Su, Yan Wang, Simon Baker, Deng Cai, Xiaojiang Liu, Anna Korhonen, Nigel Collier

A stylistic response generator then takes the prototype and the desired language style as model input to obtain a high-quality and stylistic response.

Dialogue Generation Information Retrieval +1

Stylistic Dialogue Generation via Information-Guided Reinforcement Learning Strategy

no code implementations5 Apr 2020 Yixuan Su, Deng Cai, Yan Wang, Simon Baker, Anna Korhonen, Nigel Collier, Xiaojiang Liu

To enable better balance between the content quality and the style, we introduce a new training strategy, know as Information-Guided Reinforcement Learning (IG-RL).

Dialogue Generation reinforcement-learning +2

Multi-SimLex: A Large-Scale Evaluation of Multilingual and Cross-Lingual Lexical Semantic Similarity

no code implementations10 Mar 2020 Ivan Vulić, Simon Baker, Edoardo Maria Ponti, Ulla Petti, Ira Leviant, Kelly Wing, Olga Majewska, Eden Bar, Matt Malone, Thierry Poibeau, Roi Reichart, Anna Korhonen

We introduce Multi-SimLex, a large-scale lexical resource and evaluation benchmark covering datasets for 12 typologically diverse languages, including major languages (e. g., Mandarin Chinese, Spanish, Russian) as well as less-resourced ones (e. g., Welsh, Kiswahili).

Cross-Lingual Word Embeddings Representation Learning +3

The Secret is in the Spectra: Predicting Cross-lingual Task Performance with Spectral Similarity Measures

no code implementations EMNLP 2020 Haim Dubossarsky, Ivan Vulić, Roi Reichart, Anna Korhonen

Performance in cross-lingual NLP tasks is impacted by the (dis)similarity of languages at hand: e. g., previous work has suggested there is a connection between the expected success of bilingual lexicon induction (BLI) and the assumption of (approximate) isomorphism between monolingual embedding spaces.

Bilingual Lexicon Induction POS +1

Semi-supervised Bootstrapping of Dialogue State Trackers for Task Oriented Modelling

no code implementations26 Nov 2019 Bo-Hsiang Tseng, Marek Rei, Paweł Budzianowski, Richard E. Turner, Bill Byrne, Anna Korhonen

Dialogue systems benefit greatly from optimizing on detailed annotations, such as transcribed utterances, internal dialogue state representations and dialogue act labels.

Cross-lingual Semantic Specialization via Lexical Relation Induction

no code implementations IJCNLP 2019 Edoardo Maria Ponti, Ivan Vuli{\'c}, Goran Glava{\v{s}}, Roi Reichart, Anna Korhonen

Semantic specialization integrates structured linguistic knowledge from external resources (such as lexical relations in WordNet) into pretrained distributional vectors in the form of constraints.

dialog state tracking Lexical Simplification +4

Semi-Supervised Bootstrapping of Dialogue State Trackers for Task-Oriented Modelling

no code implementations IJCNLP 2019 Bo-Hsiang Tseng, Marek Rei, Pawe{\l} Budzianowski, Richard Turner, Bill Byrne, Anna Korhonen

Dialogue systems benefit greatly from optimizing on detailed annotations, such as transcribed utterances, internal dialogue state representations and dialogue act labels.

Investigating Cross-Lingual Alignment Methods for Contextualized Embeddings with Token-Level Evaluation

no code implementations CONLL 2019 Qianchu Liu, Diana McCarthy, Ivan Vuli{\'c}, Anna Korhonen

In this paper, we present a thorough investigation on methods that align pre-trained contextualized embeddings into shared cross-lingual context-aware embedding space, providing strong reference benchmarks for future context-aware crosslingual models.

Retrieval Sentence +1

Specializing Unsupervised Pretraining Models for Word-Level Semantic Similarity

1 code implementation COLING 2020 Anne Lauscher, Ivan Vulić, Edoardo Maria Ponti, Anna Korhonen, Goran Glavaš

In this work, we complement such distributional knowledge with external lexical knowledge, that is, we integrate the discrete knowledge on word-level semantic similarity into pretraining.

Language Modelling Lexical Simplification +7

Do We Really Need Fully Unsupervised Cross-Lingual Embeddings?

1 code implementation IJCNLP 2019 Ivan Vulić, Goran Glavaš, Roi Reichart, Anna Korhonen

A series of bilingual lexicon induction (BLI) experiments with 15 diverse languages (210 language pairs) show that fully unsupervised CLWE methods still fail for a large number of language pairs (e. g., they yield zero BLI performance for 87/210 pairs).

Bilingual Lexicon Induction Self-Learning

Bayesian Learning for Neural Dependency Parsing

no code implementations NAACL 2019 Ehsan Shareghi, Yingzhen Li, Yi Zhu, Roi Reichart, Anna Korhonen

While neural dependency parsers provide state-of-the-art accuracy for several languages, they still rely on large amounts of costly labeled training data.

Dependency Parsing POS +2

A Systematic Study of Leveraging Subword Information for Learning Word Representations

1 code implementation NAACL 2019 Yi Zhu, Ivan Vulić, Anna Korhonen

The use of subword-level information (e. g., characters, character n-grams, morphemes) has become ubiquitous in modern word representation learning.

Dependency Parsing Entity Typing +3

Post-Specialisation: Retrofitting Vectors of Words Unseen in Lexical Resources

1 code implementation NAACL 2018 Ivan Vulić, Goran Glavaš, Nikola Mrkšić, Anna Korhonen

Word vector specialisation (also known as retrofitting) is a portable, light-weight approach to fine-tuning arbitrary distributional word vector spaces by injecting external knowledge from rich lexical resources such as WordNet.

Dialogue State Tracking Text Simplification +1

Erratum: Link prediction in drug-target interactions network using similarity indices

no code implementations1 Nov 2017 Yiding Lu, Yufan Guo, Anna Korhonen

Conclusion: This demonstrates that approaches purely based on network topology provide a more suitable approach to DTI prediction in the many real-life situations where little or no prior knowledge is available about the characteristics of drugs, targets, or their interactions.

Link Prediction

Initializing neural networks for hierarchical multi-label text classification

no code implementations WS 2017 Simon Baker, Anna Korhonen

Many tasks in the biomedical domain require the assignment of one or more predefined labels to input text, where the labels are a part of a hierarchical structure (such as a taxonomy).

General Classification Multi-Label Classification +4

Cross-Lingual Induction and Transfer of Verb Classes Based on Word Vector Space Specialisation

no code implementations EMNLP 2017 Ivan Vulić, Nikola Mrkšić, Anna Korhonen

Existing approaches to automatic VerbNet-style verb classification are heavily dependent on feature engineering and therefore limited to languages with mature NLP pipelines.

Clustering Cross-Lingual Transfer +4

Morph-fitting: Fine-Tuning Word Vector Spaces with Simple Language-Specific Rules

no code implementations ACL 2017 Ivan Vulić, Nikola Mrkšić, Roi Reichart, Diarmuid Ó Séaghdha, Steve Young, Anna Korhonen

Morphologically rich languages accentuate two properties of distributional vector space models: 1) the difficulty of inducing accurate representations for low-frequency word forms; and 2) insensitivity to distinct lexical relations that have similar distributional signatures.

Dialogue State Tracking MORPH

Decoding Sentiment from Distributed Representations of Sentences

no code implementations SEMEVAL 2017 Edoardo Maria Ponti, Ivan Vulić, Anna Korhonen

Distributed representations of sentences have been developed recently to represent their meaning as real-valued vectors.

Negation Sentence

Evaluation by Association: A Systematic Study of Quantitative Word Association Evaluation

no code implementations EACL 2017 Ivan Vuli{\'c}, Douwe Kiela, Anna Korhonen

Recent work on evaluating representation learning architectures in NLP has established a need for evaluation protocols based on subconscious cognitive measures rather than manually tailored intrinsic similarity and relatedness tasks.

Information Retrieval Representation Learning +2

Cancer Hallmark Text Classification Using Convolutional Neural Networks

no code implementations WS 2016 Simon Baker, Anna Korhonen, Sampo Pyysalo

Methods based on deep learning approaches have recently achieved state-of-the-art performance in a range of machine learning tasks and are increasingly applied to natural language processing (NLP).

General Classification text-classification +1

Survey on the Use of Typological Information in Natural Language Processing

no code implementations COLING 2016 Helen O'Horan, Yevgeni Berzak, Ivan Vulić, Roi Reichart, Anna Korhonen

In recent years linguistic typology, which classifies the world's languages according to their functional and structural properties, has been widely used to support multilingual NLP.

Multilingual NLP

Automatic Selection of Context Configurations for Improved Class-Specific Word Representations

no code implementations CONLL 2017 Ivan Vulić, Roy Schwartz, Ari Rappoport, Roi Reichart, Anna Korhonen

With our selected context configurations, we train on only 14% (A), 26. 2% (V), and 33. 6% (N) of all dependency-based contexts, resulting in a reduced training time.

Word Similarity

HyperLex: A Large-Scale Evaluation of Graded Lexical Entailment

no code implementations CL 2017 Ivan Vulić, Daniela Gerz, Douwe Kiela, Felix Hill, Anna Korhonen

We introduce HyperLex - a dataset and evaluation resource that quantifies the extent of of the semantic category membership, that is, type-of relation also known as hyponymy-hypernymy or lexical entailment (LE) relation between 2, 616 concept pairs.

Lexical Entailment Relation +1

SimVerb-3500: A Large-Scale Evaluation Set of Verb Similarity

1 code implementation EMNLP 2016 Daniela Gerz, Ivan Vulić, Felix Hill, Roi Reichart, Anna Korhonen

Verbs play a critical role in the meaning of sentences, but these ubiquitous words have received little attention in recent distributional semantics research.

Representation Learning

Anchoring and Agreement in Syntactic Annotations

no code implementations EMNLP 2016 Yevgeni Berzak, Yan Huang, Andrei Barbu, Anna Korhonen, Boris Katz

Our agreement results control for parser bias, and are consequential in that they are on par with state of the art parsing performance for English newswire.

Decision Making Dependency Parsing

Learning Distributed Representations of Sentences from Unlabelled Data

1 code implementation NAACL 2016 Felix Hill, Kyunghyun Cho, Anna Korhonen

Unsupervised methods for learning distributed representations of words are ubiquitous in today's NLP research, but far less is known about the best ways to learn distributed phrase or sentence representations from unlabelled data.

Representation Learning Sentence

Learning to Understand Phrases by Embedding the Dictionary

2 code implementations TACL 2016 Felix Hill, Kyunghyun Cho, Anna Korhonen, Yoshua Bengio

Distributional models that learn rich semantic word representations are a success story of recent NLP research.

General Knowledge

SimLex-999: Evaluating Semantic Models with (Genuine) Similarity Estimation

3 code implementations CL 2015 Felix Hill, Roi Reichart, Anna Korhonen

We present SimLex-999, a gold standard resource for evaluating distributional semantic models that improves on existing resources in several important ways.

Representation Learning

Native Language Identification Using Large, Longitudinal Data

no code implementations LREC 2014 Xiao Jiang, Yufan Guo, Jeroen Geertzen, Dora Alexopoulou, Lin Sun, Anna Korhonen

Native Language Identification (NLI) is a task aimed at determining the native language (L1) of learners of second language (L2) on the basis of their written texts.

BIG-bench Machine Learning Native Language Identification +1

Multi-Modal Models for Concrete and Abstract Concept Meaning

no code implementations TACL 2014 Felix Hill, Roi Reichart, Anna Korhonen

Multi-modal models that learn semantic representations from both linguistic and perceptual input outperform language-only models on a range of evaluations, and better reflect human concept acquisition.

Language Acquisition

Cannot find the paper you are looking for? You can Submit a new open access paper.