Search Results for author:
Found 33 papers, 19 papers with code
Unbabel’s Participation in the WMT20 Metrics Shared Task
We present the contribution of the Unbabel team to the WMT 2020 Shared Task on Metrics.
Results of the WMT21 Metrics Shared Task: Evaluating Metrics with Expert-based Human Evaluations on TED and News Domain
Contrary to previous years’ editions, this year we acquired our own human ratings based on expert-based human evaluation via Multidimensional Quality Metrics (MQM).
IST-Unbabel 2021 Submission for the Quality Estimation Shared Task
We present the joint contribution of IST and Unbabel to the WMT 2021 Shared Task on Quality Estimation.
Are References Really Needed? Unbabel-IST 2021 Submission for the Metrics Shared Task
In this paper, we present the joint contribution of Unbabel and IST to the WMT 2021 Metrics Shared Task.
Searching for COMETINHO: The Little Metric That Could
In recent years, several neural fine-tuned machine translation evaluation metrics such as COMET and BLEURT have been proposed.
QUARTZ: Quality-Aware Machine Translation
This paper presents QUARTZ, QUality-AwaRe machine Translation, a project led by Unbabel which aims at developing machine translation systems that are more robust and produce fewer critical errors.
Modeling User Preferences with Automatic Metrics: Creating a High-Quality Preference Dataset for Machine Translation
Alignment with human preferences is an important step in developing accurate and safe large language models.
Is Preference Alignment Always the Best Option to Enhance LLM-Based Translation? An Empirical Analysis
Neural metrics for machine translation (MT) evaluation have become increasingly prominent due to their superior correlation with human judgments compared to traditional lexical metrics.
EuroLLM: Multilingual Language Models for Europe
The quality of open-weight LLMs has seen significant improvement, yet they remain predominantly focused on English.
xTower: A Multilingual LLM for Explaining and Correcting Translation Errors
While machine translation (MT) systems are achieving increasingly strong performance on benchmarks, they often produce translations with errors and anomalies.
QUEST: Quality-Aware Metropolis-Hastings Sampling for Machine Translation
An important challenge in machine translation (MT) is to generate high-quality and diverse translations.
Can Automatic Metrics Assess High-Quality Translations?
Automatic metrics for evaluating translation quality are typically validated by measuring how well they correlate with human assessments.
Is Context Helpful for Chat Translation Evaluation?
Our findings show that augmenting neural learned metrics with contextual information helps improve correlation with human judgments in the reference-free scenario and when evaluating translations in out-of-English settings.
Tower: An Open Multilingual Large Language Model for Translation-Related Tasks
While general-purpose large language models (LLMs) demonstrate proficiency on multiple tasks within the domain of translation, approaches based on open LLMs are competitive only when specializing on a single task.
CroissantLLM: A Truly Bilingual French-English Language Model
We introduce CroissantLLM, a 1. 3B language model pretrained on a set of 3T English and French tokens, to bring to the research and industrial community a high-performance, fully open-sourced bilingual model that runs swiftly on consumer-grade local hardware.
AfriMTE and AfriCOMET: Enhancing COMET to Embrace Under-resourced African Languages
Despite the recent progress on scaling multilingual machine translation (MT) to several under-resourced African languages, accurately measuring this progress remains challenging, since evaluation is often performed on n-gram matching metrics such as BLEU, which typically show a weaker correlation with human judgments.
Steering Large Language Models for Machine Translation with Finetuning and In-Context Learning
Experiments on 10 language pairs show that our proposed approach recovers the original few-shot capabilities while keeping the added benefits of finetuning.
xCOMET: Transparent Machine Translation Evaluation through Fine-grained Error Detection
Widely used learned metrics for machine translation evaluation, such as COMET and BLEURT, estimate the quality of a translation hypothesis by providing a single sentence-level score.
Scaling up COMETKIWI: Unbabel-IST 2023 Submission for the Quality Estimation Shared Task
Our team participated on all tasks: sentence- and word-level quality prediction (task 1) and fine-grained error span detection (task 2).
The Inside Story: Towards Better Understanding of Machine Translation Neural Evaluation Metrics
Neural metrics for machine translation evaluation, such as COMET, exhibit significant improvements in their correlation with human judgments, as compared to traditional metrics based on lexical overlap, such as BLEU.
CometKiwi: IST-Unbabel 2022 Submission for the Quality Estimation Shared Task
We present the joint contribution of IST and Unbabel to the WMT 2022 Shared Task on Quality Estimation (QE).
Towards a Sentiment-Aware Conversational Agent
In this paper, we propose an end-to-end sentiment-aware conversational agent based on two models: a reply sentiment prediction model, which leverages the context of the dialogue to predict an appropriate sentiment for the agent to express in its reply; and a text generation model, which is conditioned on the predicted sentiment and the context of the dialogue, to produce a reply that is both context and sentiment appropriate.
Quality-Aware Decoding for Neural Machine Translation
Despite the progress in machine translation quality estimation and evaluation in the last years, decoding in neural machine translation (NMT) is mostly oblivious to this and centers around finding the most probable translation according to the model (MAP decoding), approximated with beam search.
Disentangling Uncertainty in Machine Translation Evaluation
Trainable evaluation metrics for machine translation (MT) exhibit strong correlation with human judgements, but they are often hard to interpret and might produce unreliable scores under noisy or out-of-domain data.
Onception: Active Learning with Expert Advice for Real World Machine Translation
Moreover, since we not know in advance which query strategy will be the most adequate for a certain language pair and set of Machine Translation models, we propose to dynamically combine multiple strategies using prediction with expert advice.
Uncertainty-Aware Machine Translation Evaluation
Several neural-based metrics have been recently proposed to evaluate machine translation quality.
MT-Telescope: An interactive platform for contrastive evaluation of MT systems
We present MT-Telescope, a visualization platform designed to facilitate comparative analysis of the output quality of two Machine Translation (MT) systems.
Online Learning Meets Machine Translation Evaluation: Finding the Best Systems with the Least Human Effort
In Machine Translation, assessing the quality of a large amount of automatic translations can be challenging.
Multilingual Email Zoning
The segmentation of emails into functional zones (also dubbed email zoning) is a relevant preprocessing step for most NLP tasks that deal with emails.
Unbabel's Participation in the WMT20 Metrics Shared Task
Overall, our systems achieve strong results for all language pairs on previous test sets and in many cases set a new state-of-the-art.
COMET: A Neural Framework for MT Evaluation
We present COMET, a neural framework for training multilingual machine translation evaluation models which obtains new state-of-the-art levels of correlation with human judgements.
A free web service for fast COVID-19 classification of chest X-Ray images
This work provides a fast detection system of COVID-19 characteristics in X-Ray images based on deep learning (DL) techniques.
