Search Results for author:
Found 21 papers, 8 papers with code
Déjà Vu: Multilingual LLM Evaluation through the Lens of Machine Translation Evaluation
Generation capabilities and language coverage of multilingual large language models (mLLMs) are advancing rapidly.
WMT24++: Expanding the Language Coverage of WMT24 to 55 Languages & Dialects
As large language models (LLM) become more and more capable in languages other than English, it is important to collect benchmark datasets in order to evaluate their multilingual performance, including on tasks like machine translation (MT).
Overestimation in LLM Evaluation: A Controlled Large-Scale Study on Data Contamination's Impact on Machine Translation
Data contamination -- the accidental consumption of evaluation examples within the pre-training data -- can undermine the validity of evaluation benchmarks.
On the Implications of Verbose LLM Outputs: A Case Study in Translation Evaluation
This paper investigates the impact of verbose LLM translations on evaluation.
Translating Step-by-Step: Decomposing the Translation Process for Improved Translation Quality of Long-Form Texts
In this paper we present a step-by-step approach to long-form text translation, drawing on established processes in translation studies.
Don't Throw Away Data: Better Sequence Knowledge Distillation
A critical component in knowledge distillation is the means of coupling the teacher and student.
Explaining with Contrastive Phrasal Highlighting: A Case Study in Assisting Humans to Detect Translation Differences
Explainable NLP techniques primarily explain by answering "Which tokens in the input are responsible for this prediction?''.
AfriMTE and AfriCOMET: Enhancing COMET to Embrace Under-resourced African Languages
Despite the recent progress on scaling multilingual machine translation (MT) to several under-resourced African languages, accurately measuring this progress remains challenging, since evaluation is often performed on n-gram matching metrics such as BLEU, which typically show a weaker correlation with human judgments.
What Else Do I Need to Know? The Effect of Background Information on Users' Reliance on QA Systems
In this work, we study how users interact with QA systems in the absence of sufficient information to assess their predictions.
Searching for Needles in a Haystack: On the Role of Incidental Bilingualism in PaLM's Translation Capability
We investigate the role of incidental bilingualism -- the unintentional consumption of bilingual signals, including translation examples -- in explaining the translation capabilities of large language models, taking the Pathways Language Model (PaLM) as a case study.
Understanding and Detecting Hallucinations in Neural Machine Translation via Model Introspection
Neural sequence generation models are known to "hallucinate", by producing outputs that are unrelated to the source text.
Can Synthetic Translations Improve Bitext Quality?
Synthetic translations have been used for a wide range of NLP tasks primarily as a means of data augmentation.
BitextEdit: Automatic Bitext Editing for Improved Low-Resource Machine Translation
Mined bitexts can contain imperfect translations that yield unreliable training signals for Neural Machine Translation (NMT).
Evaluating the Evaluation Metrics for Style Transfer: A Case Study in Multilingual Formality Transfer
While the field of style transfer (ST) has been growing rapidly, it has been hampered by a lack of standardized practices for automatic evaluation.
A Review of Human Evaluation for Style Transfer
However, in style transfer papers, we find that protocols for human evaluations are often underspecified and not standardized, which hampers the reproducibility of research in this field and progress toward better human and automatic evaluation methods.
Ol\'a, Bonjour, Salve! XFORMAL: A Benchmark for Multilingual Formality Style Transfer
We take the first step towards multilingual style transfer by creating and releasing XFORMAL, a benchmark of multiple formal reformulations of informal text in Brazilian Portuguese, French, and Italian.
Beyond Noise: Mitigating the Impact of Fine-grained Semantic Divergences on Neural Machine Translation
While it has been shown that Neural Machine Translation (NMT) is highly sensitive to noisy parallel training samples, prior work treats all types of mismatches between source and target as noise.
XFORMAL: A Benchmark for Multilingual Formality Style Transfer
We take the first step towards multilingual style transfer by creating and releasing XFORMAL, a benchmark of multiple formal reformulations of informal text in Brazilian Portuguese, French, and Italian.
Detecting Fine-Grained Cross-Lingual Semantic Divergences without Supervision by Learning to Rank
Detecting fine-grained differences in content conveyed in different languages matters for cross-lingual NLP and multilingual corpora analysis, but it is a challenging machine learning problem since annotation is expensive and hard to scale.
The University of Maryland's Kazakh-English Neural Machine Translation System at WMT19
This paper describes the University of Maryland{'}s submission to the WMT 2019 Kazakh-English news translation task.
Cross-topic distributional semantic representations via unsupervised mappings
In traditional Distributional Semantic Models (DSMs) the multiple senses of a polysemous word are conflated into a single vector space representation.
