Search Results for author: Liane Guillou

Found 24 papers, 7 papers with code

Machine Translation Meta Evaluation through Translation Accuracy Challenge Sets

1 code implementation29 Jan 2024 Nikita Moghe, Arnisa Fazla, Chantal Amrhein, Tom Kocmi, Mark Steedman, Alexandra Birch, Rico Sennrich, Liane Guillou

We benchmark metric performance, assess their incremental performance over successive campaigns, and measure their sensitivity to a range of linguistic phenomena.

Benchmarking Machine Translation +3

ACES: Translation Accuracy Challenge Sets at WMT 2023

no code implementations2 Nov 2023 Chantal Amrhein, Nikita Moghe, Liane Guillou

We benchmark the performance of segmentlevel metrics submitted to WMT 2023 using the ACES Challenge Set (Amrhein et al., 2022).

Translation World Knowledge

MULTI3NLU++: A Multilingual, Multi-Intent, Multi-Domain Dataset for Natural Language Understanding in Task-Oriented Dialogue

no code implementations20 Dec 2022 Nikita Moghe, Evgeniia Razumovskaia, Liane Guillou, Ivan Vulić, Anna Korhonen, Alexandra Birch

We use MULTI3NLU++ to benchmark state-of-the-art multilingual models for the NLU tasks of intent detection and slot labelling for TOD systems in the multilingual setting.

Intent Detection Machine Translation +2

ACES: Translation Accuracy Challenge Sets for Evaluating Machine Translation Metrics

1 code implementation27 Oct 2022 Chantal Amrhein, Nikita Moghe, Liane Guillou

As machine translation (MT) metrics improve their correlation with human judgement every year, it is crucial to understand the limitations of such metrics at the segment level.

Machine Translation Translation +1

Investigating the use of Paraphrase Generation for Question Reformulation in the FRANK QA system

no code implementations6 Jun 2022 Nick Ferguson, Liane Guillou, Kwabena Nuamah, Alan Bundy

Our two main conclusions are that cleaning of LC-QuAD 2. 0 is required as the errors present can affect evaluation; and that, due to limitations of FRANK's parser, paraphrase generation is not a method which we can rely on to improve the variety of natural language questions that FRANK can answer.

Paraphrase Generation Question Answering

Cross-lingual Inference with A Chinese Entailment Graph

1 code implementation Findings (ACL) 2022 Tianyi Li, Sabine Weber, Mohammad Javad Hosseini, Liane Guillou, Mark Steedman

Predicate entailment detection is a crucial task for question-answering from text, where previous work has explored unsupervised learning of entailment graphs from typed open relation triples.

Entity Typing Question Answering +2

Blindness to Modality Helps Entailment Graph Mining

1 code implementation EMNLP (insights) 2021 Liane Guillou, Sander Bijl de Vroe, Mark Johnson, Mark Steedman

Understanding linguistic modality is widely seen as important for downstream tasks such as Question Answering and Knowledge Graph Population.

Graph Learning Graph Mining +1

Incorporating Temporal Information in Entailment Graph Mining

1 code implementation COLING (TextGraphs) 2020 Liane Guillou, Sander Bijl de Vroe, Mohammad Javad Hosseini, Mark Johnson, Mark Steedman

We present a novel method for injecting temporality into entailment graphs to address the problem of spurious entailments, which may arise from similar but temporally distinct events involving the same pair of entities.

Graph Mining

Modality and Negation in Event Extraction

1 code implementation ACL (CASE) 2021 Sander Bijl de Vroe, Liane Guillou, Miloš Stanojević, Nick McKenna, Mark Steedman

Language provides speakers with a rich system of modality for expressing thoughts about events, without being committed to their actual occurrence.

Event Extraction Fact Checking +3

A Pronoun Test Suite Evaluation of the English--German MT Systems at WMT 2018

no code implementations WS 2018 Liane Guillou, Christian Hardmeier, Ekaterina Lapshinova-Koltunski, Sharid Lo{\'a}iciga

We evaluate the output of 16 English-to-German MT systems with respect to the translation of pronouns in the context of the WMT 2018 competition.

Machine Translation NMT +1

Automatic Reference-Based Evaluation of Pronoun Translation Misses the Point

no code implementations EMNLP 2018 Liane Guillou, Christian Hardmeier

We compare the performance of the APT and AutoPRF metrics for pronoun translation against a manually annotated dataset comprising human judgements as to the correctness of translations of the PROTEST test suite.

Translation

What is it? Disambiguating the different readings of the pronoun `it'

no code implementations EMNLP 2017 Sharid Lo{\'a}iciga, Liane Guillou, Christian Hardmeier

In this paper, we address the problem of predicting one of three functions for the English pronoun {`}it{'}: anaphoric, event reference or pleonastic.

coreference-resolution Machine Translation +1

ParCor 1.0: A Parallel Pronoun-Coreference Corpus to Support Statistical MT

no code implementations LREC 2014 Liane Guillou, Christian Hardmeier, Aaron Smith, J{\"o}rg Tiedemann, Bonnie Webber

We present ParCor, a parallel corpus of texts in which pronoun coreference ― reduced coreference in which pronouns are used as referring expressions ― has been annotated.

Machine Translation Translation

Cannot find the paper you are looking for? You can Submit a new open access paper.