Search Results for author: Rico Sennrich

Found 108 papers, 57 papers with code

The Samsung and University of Edinburgh’s submission to IWSLT17

no code implementations IWSLT 2017 Pawel Przybysz, Marcin Chochowski, Rico Sennrich, Barry Haddow, Alexandra Birch

This paper describes the joint submission of Samsung Research and Development, Warsaw, Poland and the University of Edinburgh team to the IWSLT MT task for TED talks.

Domain Adaptation Translation

Samsung and University of Edinburgh’s System for the IWSLT 2018 Low Resource MT Task

no code implementations IWSLT (EMNLP) 2018 Philip Williams, Marcin Chochowski, Pawel Przybysz, Rico Sennrich, Barry Haddow, Alexandra Birch

This paper describes the joint submission to the IWSLT 2018 Low Resource MT task by Samsung R&D Institute, Poland, and the University of Edinburgh.

Contrastive Conditioning for Assessing Disambiguation in MT: A Case Study of Distilled Bias

1 code implementation ACL ARR May 2021 Jannis Vamvas, Rico Sennrich

Lexical disambiguation is a major challenge for machine translation systems, especially if some senses of a word are trained less often than others.

Knowledge Distillation Machine Translation +2

Wino-X: Multilingual Winograd Schemas for Commonsense Reasoning and Coreference Resolution

1 code implementation EMNLP 2021 Denis Emelin, Rico Sennrich

We use this resource to investigate whether neural machine translation (NMT) models can perform CoR that requires commonsense knowledge and whether multilingual language models (MLLMs) are capable of CSR across multiple languages.

Common Sense Reasoning coreference-resolution +5

Exploring the Importance of Source Text in Automatic Post-Editing for Context-Aware Machine Translation

1 code implementation NoDaLiDa 2021 Chaojun Wang, Christian Hardmeier, Rico Sennrich

They also highlight blind spots in automatic methods for targeted evaluation and demonstrate the need for human assessment to evaluate document-level translation quality reliably.

Automatic Post-Editing Translation

Towards Unsupervised Recognition of Semantic Differences in Related Documents

1 code implementation22 May 2023 Jannis Vamvas, Rico Sennrich

Automatically highlighting words that cause semantic differences between two documents could be useful for a wide range of applications.

Contrastive Learning Language Modelling +2

Exploiting Biased Models to De-bias Text: A Gender-Fair Rewriting Model

1 code implementation18 May 2023 Chantal Amrhein, Florian Schottmann, Rico Sennrich, Samuel Läubli

We hypothesise that creating training data in the reverse direction, i. e. starting from gender-fair text, is easier for morphologically complex languages and show that it matches the performance of state-of-the-art rewriting models for English.

Fairness Machine Translation +2

What's the Meaning of Superhuman Performance in Today's NLU?

no code implementations15 May 2023 Simone Tedeschi, Johan Bos, Thierry Declerck, Jan Hajic, Daniel Hershcovich, Eduard H. Hovy, Alexander Koller, Simon Krek, Steven Schockaert, Rico Sennrich, Ekaterina Shutova, Roberto Navigli

In the last five years, there has been a significant focus in Natural Language Processing (NLP) on developing larger Pretrained Language Models (PLMs) and introducing benchmarks such as SuperGLUE and SQuAD to measure their abilities in language understanding, reasoning, and reading comprehension.

Reading Comprehension

SLTUNET: A Simple Unified Model for Sign Language Translation

1 code implementation International Conference on Learning Representations (ICLR) 2023 Biao Zhang, Mathias Müller, Rico Sennrich

We propose SLTUNET, a simple unified neural model designed to support multiple SLTrelated tasks jointly, such as sign-to-gloss, gloss-to-text and sign-to-text translation.

Machine Translation Sign Language Translation +1

SwissBERT: The Multilingual Language Model for Switzerland

1 code implementation23 Mar 2023 Jannis Vamvas, Johannes Graën, Rico Sennrich

We present SwissBERT, a masked language model created specifically for processing Switzerland-related text.

Language Modelling Natural Language Understanding

Efficient CTC Regularization via Coarse Labels for End-to-End Speech Translation

1 code implementation21 Feb 2023 Biao Zhang, Barry Haddow, Rico Sennrich

For end-to-end speech translation, regularizing the encoder with the Connectionist Temporal Classification (CTC) objective using the source transcript or target translation as labels can greatly improve quality metrics.


Improving the Cross-Lingual Generalisation in Visual Question Answering

1 code implementation7 Sep 2022 Farhad Nooralahzadeh, Rico Sennrich

While several benefits were realized for multilingual vision-language pretrained models, recent benchmarks across various tasks and languages showed poor cross-lingual generalisation when multilingually pre-trained vision-language models are applied to non-English data, with a large gap between (supervised) English performance and (zero-shot) cross-lingual transfer.

Question Answering Visual Question Answering +1

Revisiting End-to-End Speech-to-Text Translation From Scratch

1 code implementation9 Jun 2022 Biao Zhang, Barry Haddow, Rico Sennrich

Finally, we discuss neural acoustic feature modeling, where a neural model is designed to extract acoustic features from raw speech signals directly, with the goal to simplify inductive biases and add freedom to the model in describing speech.

speech-recognition Speech Recognition +2

NMTScore: A Multilingual Analysis of Translation-based Text Similarity Measures

1 code implementation28 Apr 2022 Jannis Vamvas, Rico Sennrich

Being able to rank the similarity of short text segments is an interesting bonus feature of neural machine translation.

Data-to-Text Generation Machine Translation +5

Identifying Weaknesses in Machine Translation Metrics Through Minimum Bayes Risk Decoding: A Case Study for COMET

1 code implementation10 Feb 2022 Chantal Amrhein, Rico Sennrich

Neural metrics have achieved impressive correlation with human judgements in the evaluation of machine translation systems, but before we can safely optimise towards such metrics, we should be aware of (and ideally eliminate) biases toward bad translations that receive high scores.

Machine Translation Translation

Distributionally Robust Recurrent Decoders with Random Network Distillation

no code implementations RepL4NLP (ACL) 2022 Antonio Valerio Miceli-Barone, Alexandra Birch, Rico Sennrich

Neural machine learning models can successfully model language that is similar to their training distribution, but they are highly susceptible to degradation under distribution shift, which occurs in many practical applications when processing out-of-domain (OOD) text.

Language Modelling Out of Distribution (OOD) Detection

On the Limits of Minimal Pairs in Contrastive Evaluation

1 code implementation EMNLP (BlackboxNLP) 2021 Jannis Vamvas, Rico Sennrich

Minimal sentence pairs are frequently used to analyze the behavior of language models.

Improving Zero-shot Cross-lingual Transfer between Closely Related Languages by injecting Character-level Noise

no code implementations Findings (ACL) 2022 Noëmi Aepli, Rico Sennrich

Cross-lingual transfer between a high-resource language and its dialects or closely related language varieties should be facilitated by their similarity.

POS Zero-Shot Cross-Lingual Transfer

Vision Matters When It Should: Sanity Checking Multimodal Machine Translation Models

1 code implementation EMNLP 2021 Jiaoda Li, Duygu Ataman, Rico Sennrich

Multimodal machine translation (MMT) systems have been shown to outperform their text-only neural machine translation (NMT) counterparts when visual context is available.

Image Captioning Multimodal Machine Translation +2

Language Modeling, Lexical Translation, Reordering: The Training Process of NMT through the Lens of Classical SMT

no code implementations EMNLP 2021 Elena Voita, Rico Sennrich, Ivan Titov

Differently from the traditional statistical MT that decomposes the translation task into distinct separately learned components, neural machine translation uses a single neural network to model the entire translation process.

Language Modelling Machine Translation +4

How Suitable Are Subword Segmentation Strategies for Translating Non-Concatenative Morphology?

1 code implementation Findings (EMNLP) 2021 Chantal Amrhein, Rico Sennrich

Data-driven subword segmentation has become the default strategy for open-vocabulary machine translation and other NLP tasks, but may not be sufficiently generic for optimal learning of non-concatenative morphology.

Machine Translation Translation

Beyond Sentence-Level End-to-End Speech Translation: Context Helps

1 code implementation ACL 2021 Biao Zhang, Ivan Titov, Barry Haddow, Rico Sennrich

Document-level contextual information has shown benefits to text-based machine translation, but whether and how context helps end-to-end (E2E) speech translation (ST) is still under-studied.

Machine Translation Translation

Revisiting Negation in Neural Machine Translation

1 code implementation26 Jul 2021 Gongbo Tang, Philipp Rönchen, Rico Sennrich, Joakim Nivre

In this paper, we evaluate the translation of negation both automatically and manually, in English--German (EN--DE) and English--Chinese (EN--ZH).

Machine Translation NMT +1

Understanding the Properties of Minimum Bayes Risk Decoding in Neural Machine Translation

1 code implementation ACL 2021 Mathias Müller, Rico Sennrich

Neural Machine Translation (NMT) currently exhibits biases such as producing translations that are too short and overgenerating frequent words, and shows poor robustness to copy noise in training data or domain shift.

Machine Translation NMT +1

Sparse Attention with Linear Units

2 code implementations EMNLP 2021 Biao Zhang, Ivan Titov, Rico Sennrich

Recently, it has been argued that encoder-decoder models can be made more interpretable by replacing the softmax function in the attention with its sparse variants.

Machine Translation Translation +1

On Biasing Transformer Attention Towards Monotonicity

1 code implementation NAACL 2021 Annette Rios, Chantal Amrhein, Noëmi Aepli, Rico Sennrich

Many sequence-to-sequence tasks in natural language processing are roughly monotonic in the alignment between source and target sequence, and previous work has facilitated or enforced learning of monotonic attention behavior via specialized attention functions or pretraining.

Morphological Inflection Transliteration

The Impact of Text Presentation on Translator Performance

no code implementations11 Nov 2020 Samuel Läubli, Patrick Simianer, Joern Wuebker, Geza Kovacs, Rico Sennrich, Spence Green

Widely used computer-aided translation (CAT) tools divide documents into segments such as sentences and arrange them in a side-by-side, spreadsheet-like view.


Understanding Pure Character-Based Neural Machine Translation: The Case of Translating Finnish into English

no code implementations COLING 2020 Gongbo Tang, Rico Sennrich, Joakim Nivre

The attention distribution pattern shows that separators attract a lot of attention and we explore a sparse word-level attention to enforce character hidden states to capture the full word-level information.

Machine Translation NMT +1

Subword Segmentation and a Single Bridge Language Affect Zero-Shot Neural Machine Translation

1 code implementation WMT (EMNLP) 2020 Annette Rios, Mathias Müller, Rico Sennrich

A recent trend in multilingual models is to not train on parallel data between all language pairs, but have a single bridge language, e. g. English.

Machine Translation TAG +1

Fast Interleaved Bidirectional Sequence Generation

1 code implementation WMT (EMNLP) 2020 Biao Zhang, Ivan Titov, Rico Sennrich

Instead of assuming independence between neighbouring tokens (semi-autoregressive decoding, SA), we take inspiration from bidirectional sequence generation and introduce a decoder that generates target words from the left-to-right and right-to-left directions simultaneously.

Document Summarization Machine Translation

Analyzing the Source and Target Contributions to Predictions in Neural Machine Translation

1 code implementation ACL 2021 Elena Voita, Rico Sennrich, Ivan Titov

We find that models trained with more data tend to rely on source information more and to have more sharp token contributions; the training process is non-monotonic with several stages of different nature.

Language Modelling Machine Translation +2

Adaptive Feature Selection for End-to-End Speech Translation

1 code implementation Findings of the Association for Computational Linguistics 2020 Biao Zhang, Ivan Titov, Barry Haddow, Rico Sennrich

Information in speech signals is not evenly distributed, making it an additional challenge for end-to-end (E2E) speech translation (ST) to learn to focus on informative features.

Data Augmentation Translation

On Romanization for Model Transfer Between Scripts in Neural Machine Translation

no code implementations Findings of the Association for Computational Linguistics 2020 Chantal Amrhein, Rico Sennrich

Our results show that romanization entails information loss and is thus not always superior to simpler vocabulary transfer methods, but can improve the transfer between related languages with different scripts.

Machine Translation Transfer Learning +1

On Exposure Bias, Hallucination and Domain Shift in Neural Machine Translation

1 code implementation ACL 2020 Chaojun Wang, Rico Sennrich

In experiments on three datasets with multiple test domains, we show that exposure bias is partially to blame for hallucinations, and that training with Minimum Risk Training, which avoids exposure bias, can mitigate this.

Machine Translation NMT +1

Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translation

3 code implementations ACL 2020 Biao Zhang, Philip Williams, Ivan Titov, Rico Sennrich

Massively multilingual models for neural machine translation (NMT) are theoretically attractive, but often underperform bilingual models and deliver poor zero-shot translations.

Machine Translation NMT +1

On Sparsifying Encoder Outputs in Sequence-to-Sequence Models

1 code implementation Findings (ACL) 2021 Biao Zhang, Ivan Titov, Rico Sennrich

Inspired by these observations, we explore the feasibility of specifying rule-based patterns that mask out encoder outputs based on information such as part-of-speech tags, word frequency and word position.

Document Summarization Machine Translation

A Set of Recommendations for Assessing Human-Machine Parity in Language Translation

1 code implementation3 Apr 2020 Samuel Läubli, Sheila Castilho, Graham Neubig, Rico Sennrich, Qinlan Shen, Antonio Toral

The quality of machine translation has increased remarkably over the past years, to the degree that it was found to be indistinguishable from professional human translation in a number of empirical investigations.

Machine Translation Translation

X-Stance: A Multilingual Multi-Target Dataset for Stance Detection

1 code implementation18 Mar 2020 Jannis Vamvas, Rico Sennrich

Unlike stance detection models that have specific target issues, we use the dataset to train a single model on all the issues.

Stance Detection

Domain Robustness in Neural Machine Translation

2 code implementations AMTA 2020 Mathias Müller, Annette Rios, Rico Sennrich

Domain robustness---the generalization of models to unseen test domains---is low for both statistical (SMT) and neural machine translation (NMT).

Machine Translation NMT +1

Domain, Translationese and Noise in Synthetic Data for Neural Machine Translation

no code implementations6 Nov 2019 Nikolay Bogoychev, Rico Sennrich

The quality of neural machine translation can be improved by leveraging additional monolingual resources to create synthetic training data.

Machine Translation Translation

Root Mean Square Layer Normalization

1 code implementation NeurIPS 2019 Biao Zhang, Rico Sennrich

RMSNorm regularizes the summed inputs to a neuron in one layer according to root mean square (RMS), giving the model re-scaling invariance property and implicit learning rate adaptation ability.

The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectives

no code implementations IJCNLP 2019 Elena Voita, Rico Sennrich, Ivan Titov

In this work, we use canonical correlation analysis and mutual information estimators to study how information flows across Transformer layers and how this process depends on the choice of learning objective.

Language Modelling Machine Translation +2

Encoders Help You Disambiguate Word Senses in Neural Machine Translation

no code implementations IJCNLP 2019 Gongbo Tang, Rico Sennrich, Joakim Nivre

We find that encoder hidden states outperform word embeddings significantly which indicates that encoders adequately encode relevant information for disambiguation into hidden states.

Machine Translation NMT +2

Understanding Neural Machine Translation by Simplification: The Case of Encoder-free Models

no code implementations RANLP 2019 Gongbo Tang, Rico Sennrich, Joakim Nivre

In this paper, we try to understand neural machine translation (NMT) via simplifying NMT architectures and training encoder-free NMT models.

Machine Translation NMT +2

Widening the Representation Bottleneck in Neural Machine Translation with Lexical Shortcuts

1 code implementation WS 2019 Denis Emelin, Ivan Titov, Rico Sennrich

The transformer is a state-of-the-art neural translation model that uses attention to iteratively refine lexical representations with information drawn from the surrounding context.

Machine Translation Translation

A Lightweight Recurrent Network for Sequence Modeling

1 code implementation ACL 2019 Biao Zhang, Rico Sennrich

We apply LRN as a drop-in replacement of existing recurrent units in several neural sequential models.

Revisiting Low-Resource Neural Machine Translation: A Case Study

2 code implementations ACL 2019 Rico Sennrich, Biao Zhang

It has been shown that the performance of neural machine translation (NMT) drops starkly in low-resource conditions, underperforming phrase-based statistical machine translation (PBSMT) and requiring large amounts of auxiliary data to achieve competitive results.

Low-Resource Neural Machine Translation NMT +1

When a Good Translation is Wrong in Context: Context-Aware Machine Translation Improves on Deixis, Ellipsis, and Lexical Cohesion

1 code implementation ACL 2019 Elena Voita, Rico Sennrich, Ivan Titov

Though machine translation errors caused by the lack of context beyond one sentence have long been acknowledged, the development of context-aware NMT systems is hampered by several problems.

Machine Translation NMT +1

An Analysis of Attention Mechanisms: The Case of Word Sense Disambiguation in Neural Machine Translation

no code implementations WS 2018 Gongbo Tang, Rico Sennrich, Joakim Nivre

Recent work has shown that the encoder-decoder attention mechanisms in neural machine translation (NMT) are different from the word alignment in statistical machine translation.

Machine Translation NMT +3

A Large-Scale Test Set for the Evaluation of Context-Aware Pronoun Translation in Neural Machine Translation

1 code implementation WS 2018 Mathias Müller, Annette Rios, Elena Voita, Rico Sennrich

We show that, while gains in BLEU are moderate for those systems, they outperform baselines by a large margin in terms of accuracy on our contrastive test set.

Machine Translation Translation

The Word Sense Disambiguation Test Suite at WMT18

no code implementations WS 2018 Annette Rios, Mathias M{\"u}ller, Rico Sennrich

We evaluate all German{--}English submissions to the WMT{'}18 shared translation task, plus a number of submissions from previous years, and find that performance on the task has markedly improved compared to the 2016 WMT submissions (81{\%}→93{\%} accuracy on the WSD task).

Machine Translation Translation +1

Has Machine Translation Achieved Human Parity? A Case for Document-level Evaluation

1 code implementation EMNLP 2018 Samuel Läubli, Rico Sennrich, Martin Volk

Recent research suggests that neural machine translation achieves parity with professional human translation on the WMT Chinese--English news translation task.

Machine Translation Translation

Context-Aware Neural Machine Translation Learns Anaphora Resolution

no code implementations ACL 2018 Elena Voita, Pavel Serdyukov, Rico Sennrich, Ivan Titov

Standard machine translation systems process sentences in isolation and hence ignore extra-sentential information, even though extended context can both prevent mistakes in ambiguous cases and improve translation coherence.

Machine Translation Translation

Evaluating Machine Translation Performance on Chinese Idioms with a Blacklist Method

no code implementations LREC 2018 Yutong Shao, Rico Sennrich, Bonnie Webber, Federico Fancellu

Our evaluation confirms that a sizable number of idioms in our test set are mistranslated (46. 1%), that literal translation error is a common error type, and that our blacklist method is effective at identifying literal translation errors.

Machine Translation Translation +1

Evaluating Discourse Phenomena in Neural Machine Translation

no code implementations NAACL 2018 Rachel Bawden, Rico Sennrich, Alexandra Birch, Barry Haddow

Despite gains using BLEU, multi-encoder models give limited improvement in the handling of discourse phenomena: 50% accuracy on our coreference test set and 53. 5% for coherence/cohesion (compared to a non-contextual baseline of 50%).

Machine Translation NMT +1

Regularization techniques for fine-tuning in neural machine translation

no code implementations EMNLP 2017 Antonio Valerio Miceli Barone, Barry Haddow, Ulrich Germann, Rico Sennrich

We investigate techniques for supervised domain adaptation for neural machine translation where an existing model trained on a large out-of-domain dataset is adapted to a small in-domain dataset.

Domain Adaptation L2 Regularization +4

Image Pivoting for Learning Multilingual Multimodal Representations

no code implementations EMNLP 2017 Spandana Gella, Rico Sennrich, Frank Keller, Mirella Lapata

In this paper we propose a model to learn multimodal multilingual representations for matching images and sentences in different languages, with the aim of advancing multilingual versions of image search and image understanding.

Image Retrieval Semantic Textual Similarity

A parallel corpus of Python functions and documentation strings for automated code documentation and code generation

6 code implementations IJCNLP 2017 Antonio Valerio Miceli Barone, Rico Sennrich

Automated documentation of programming source code and automated code generation from natural language are challenging tasks of both practical and scientific interest.

Code Generation Data Augmentation +2

Practical Neural Machine Translation

no code implementations EACL 2017 Rico Sennrich, Barry Haddow

Neural Machine Translation (NMT) has achieved new breakthroughs in machine translation in recent years.

Machine Translation NMT +1

Predicting Target Language CCG Supertags Improves Neural Machine Translation

no code implementations WS 2017 Maria Nadejde, Siva Reddy, Rico Sennrich, Tomasz Dwojak, Marcin Junczys-Dowmunt, Philipp Koehn, Alexandra Birch

Our results on WMT data show that explicitly modeling target-syntax improves machine translation quality for German->English, a high-resource pair, and for Romanian->English, a low-resource pair and also several syntactic phenomena including prepositional phrase attachment.

Machine Translation NMT +2

How Grammatical is Character-level Neural Machine Translation? Assessing MT Quality with Contrastive Translation Pairs

1 code implementation EACL 2017 Rico Sennrich

Analysing translation quality in regards to specific linguistic phenomena has historically been difficult and time-consuming.

NMT Translation +1

Edinburgh Neural Machine Translation Systems for WMT 16

1 code implementation WS 2016 Rico Sennrich, Barry Haddow, Alexandra Birch

We participated in the WMT 2016 shared news translation task by building neural translation systems for four language pairs, each trained in both directions: English<->Czech, English<->German, English<->Romanian and English<->Russian.

Machine Translation Translation

Linguistic Input Features Improve Neural Machine Translation

1 code implementation WS 2016 Rico Sennrich, Barry Haddow

Neural machine translation has recently achieved impressive results, while using little in the way of external linguistic information.

Machine Translation Translation

The AMU-UEDIN Submission to the WMT16 News Translation Task: Attention-based NMT Models as Feature Functions in Phrase-based SMT

1 code implementation WS 2016 Marcin Junczys-Dowmunt, Tomasz Dwojak, Rico Sennrich

For the Russian-English task, our submission achieves the top BLEU result, outperforming the best pure neural system by 1. 1 BLEU points and our own phrase-based baseline by 1. 6 BLEU.

Machine Translation NMT +1

Neural Machine Translation of Rare Words with Subword Units

25 code implementations ACL 2016 Rico Sennrich, Barry Haddow, Alexandra Birch

Neural machine translation (NMT) models typically operate with a fixed vocabulary, but translation is an open-vocabulary problem.

NMT Translation

Modelling and Optimizing on Syntactic N-Grams for Statistical Machine Translation

no code implementations TACL 2015 Rico Sennrich

The role of language models in SMT is to promote fluent translation output, but traditional n-gram language models are unable to capture fluency phenomena between distant words, such as some morphological agreement phenomena, subcategorisation, and syntactic collocations with string-level gaps.

Language Modelling Machine Translation +1

Zmorge: A German Morphological Lexicon Extracted from Wiktionary

no code implementations LREC 2014 Rico Sennrich, Beat Kunz

We describe a method to automatically extract a German lexicon from Wiktionary that is compatible with the finite-state morphological grammar SMOR.

Machine Translation Morphological Analysis

Cannot find the paper you are looking for? You can Submit a new open access paper.