Search Results for author: Rico Sennrich

Found 90 papers, 42 papers with code

Contrastive Conditioning for Assessing Disambiguation in MT: A Case Study of Distilled Bias

1 code implementation ACL ARR May 2021 Jannis Vamvas, Rico Sennrich

Lexical disambiguation is a major challenge for machine translation systems, especially if some senses of a word are trained less often than others.

Knowledge Distillation Machine Translation +2

Wino-X: Multilingual Winograd Schemas for Commonsense Reasoning and Coreference Resolution

no code implementations EMNLP 2021 Denis Emelin, Rico Sennrich

We use this resource to investigate whether neural machine translation (NMT) models can perform CoR that requires commonsense knowledge and whether multilingual language models (MLLMs) are capable of CSR across multiple languages.

Coreference Resolution Machine Translation +2

Exploring the Importance of Source Text in Automatic Post-Editing for Context-Aware Machine Translation

1 code implementation NoDaLiDa 2021 Chaojun Wang, Christian Hardmeier, Rico Sennrich

They also highlight blind spots in automatic methods for targeted evaluation and demonstrate the need for human assessment to evaluate document-level translation quality reliably.

Automatic Post-Editing Translation

Distributionally Robust Recurrent Decoders with Random Network Distillation

no code implementations25 Oct 2021 Antonio Valerio Miceli-Barone, Alexandra Birch, Rico Sennrich

Neural machine learning models can successfully model language that is similar to their training distribution, but they are highly susceptible to degradation under distribution shift, which occurs in many practical applications when processing out-of-domain (OOD) text.

Language Modelling

On the Limits of Minimal Pairs in Contrastive Evaluation

1 code implementation EMNLP (BlackboxNLP) 2021 Jannis Vamvas, Rico Sennrich

Minimal sentence pairs are frequently used to analyze the behavior of language models.

Improving Zero-shot Cross-lingual Transfer between Closely Related Languages by injecting Character-level Noise

no code implementations14 Sep 2021 Noëmi Aepli, Rico Sennrich

Cross-lingual transfer between a high-resource language and its dialects or closely related language varieties should be facilitated by their similarity, but current approaches that operate in the embedding space do not take surface similarity into account.

POS Zero-Shot Cross-Lingual Transfer

Vision Matters When It Should: Sanity Checking Multimodal Machine Translation Models

1 code implementation EMNLP 2021 Jiaoda Li, Duygu Ataman, Rico Sennrich

Multimodal machine translation (MMT) systems have been shown to outperform their text-only neural machine translation (NMT) counterparts when visual context is available.

Image Captioning Multimodal Machine Translation +1

Language Modeling, Lexical Translation, Reordering: The Training Process of NMT through the Lens of Classical SMT

no code implementations EMNLP 2021 Elena Voita, Rico Sennrich, Ivan Titov

Differently from the traditional statistical MT that decomposes the translation task into distinct separately learned components, neural machine translation uses a single neural network to model the entire translation process.

Language Modelling Machine Translation +2

How Suitable Are Subword Segmentation Strategies for Translating Non-Concatenative Morphology?

1 code implementation Findings (EMNLP) 2021 Chantal Amrhein, Rico Sennrich

Data-driven subword segmentation has become the default strategy for open-vocabulary machine translation and other NLP tasks, but may not be sufficiently generic for optimal learning of non-concatenative morphology.

Machine Translation Translation

Beyond Sentence-Level End-to-End Speech Translation: Context Helps

1 code implementation ACL 2021 Biao Zhang, Ivan Titov, Barry Haddow, Rico Sennrich

Document-level contextual information has shown benefits to text-based machine translation, but whether and how context helps end-to-end (E2E) speech translation (ST) is still under-studied.

Feature Selection Machine Translation +1

Revisiting Negation in Neural Machine Translation

1 code implementation26 Jul 2021 Gongbo Tang, Philipp Rönchen, Rico Sennrich, Joakim Nivre

In this paper, we evaluate the translation of negation both automatically and manually, in English--German (EN--DE) and English--Chinese (EN--ZH).

Machine Translation Translation

Understanding the Properties of Minimum Bayes Risk Decoding in Neural Machine Translation

1 code implementation ACL 2021 Mathias Müller, Rico Sennrich

Neural Machine Translation (NMT) currently exhibits biases such as producing translations that are too short and overgenerating frequent words, and shows poor robustness to copy noise in training data or domain shift.

Machine Translation Translation

Sparse Attention with Linear Units

2 code implementations EMNLP 2021 Biao Zhang, Ivan Titov, Rico Sennrich

Recently, it has been argued that encoder-decoder models can be made more interpretable by replacing the softmax function in the attention with its sparse variants.

Machine Translation Translation +1

On Biasing Transformer Attention Towards Monotonicity

no code implementations NAACL 2021 Annette Rios, Chantal Amrhein, Noëmi Aepli, Rico Sennrich

Many sequence-to-sequence tasks in natural language processing are roughly monotonic in the alignment between source and target sequence, and previous work has facilitated or enforced learning of monotonic attention behavior via specialized attention functions or pretraining.

Morphological Inflection Transliteration

The Impact of Text Presentation on Translator Performance

no code implementations11 Nov 2020 Samuel Läubli, Patrick Simianer, Joern Wuebker, Geza Kovacs, Rico Sennrich, Spence Green

Widely used computer-aided translation (CAT) tools divide documents into segments such as sentences and arrange them in a side-by-side, spreadsheet-like view.

Translation

Understanding Pure Character-Based Neural Machine Translation: The Case of Translating Finnish into English

no code implementations COLING 2020 Gongbo Tang, Rico Sennrich, Joakim Nivre

The attention distribution pattern shows that separators attract a lot of attention and we explore a sparse word-level attention to enforce character hidden states to capture the full word-level information.

Machine Translation Translation

Subword Segmentation and a Single Bridge Language Affect Zero-Shot Neural Machine Translation

1 code implementation WMT (EMNLP) 2020 Annette Rios, Mathias Müller, Rico Sennrich

A recent trend in multilingual models is to not train on parallel data between all language pairs, but have a single bridge language, e. g. English.

Machine Translation TAG +1

Fast Interleaved Bidirectional Sequence Generation

1 code implementation WMT (EMNLP) 2020 Biao Zhang, Ivan Titov, Rico Sennrich

Instead of assuming independence between neighbouring tokens (semi-autoregressive decoding, SA), we take inspiration from bidirectional sequence generation and introduce a decoder that generates target words from the left-to-right and right-to-left directions simultaneously.

Document Summarization Machine Translation

Analyzing the Source and Target Contributions to Predictions in Neural Machine Translation

1 code implementation ACL 2021 Elena Voita, Rico Sennrich, Ivan Titov

We find that models trained with more data tend to rely on source information more and to have more sharp token contributions; the training process is non-monotonic with several stages of different nature.

Language Modelling Machine Translation +1

Adaptive Feature Selection for End-to-End Speech Translation

1 code implementation Findings of the Association for Computational Linguistics 2020 Biao Zhang, Ivan Titov, Barry Haddow, Rico Sennrich

Information in speech signals is not evenly distributed, making it an additional challenge for end-to-end (E2E) speech translation (ST) to learn to focus on informative features.

Data Augmentation Feature Selection +1

On Romanization for Model Transfer Between Scripts in Neural Machine Translation

no code implementations Findings of the Association for Computational Linguistics 2020 Chantal Amrhein, Rico Sennrich

Our results show that romanization entails information loss and is thus not always superior to simpler vocabulary transfer methods, but can improve the transfer between related languages with different scripts.

Machine Translation Transfer Learning +1

On Exposure Bias, Hallucination and Domain Shift in Neural Machine Translation

1 code implementation ACL 2020 Chaojun Wang, Rico Sennrich

In experiments on three datasets with multiple test domains, we show that exposure bias is partially to blame for hallucinations, and that training with Minimum Risk Training, which avoids exposure bias, can mitigate this.

Machine Translation Translation

Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translation

2 code implementations ACL 2020 Biao Zhang, Philip Williams, Ivan Titov, Rico Sennrich

Massively multilingual models for neural machine translation (NMT) are theoretically attractive, but often underperform bilingual models and deliver poor zero-shot translations.

Machine Translation Translation

On Sparsifying Encoder Outputs in Sequence-to-Sequence Models

1 code implementation Findings (ACL) 2021 Biao Zhang, Ivan Titov, Rico Sennrich

Inspired by these observations, we explore the feasibility of specifying rule-based patterns that mask out encoder outputs based on information such as part-of-speech tags, word frequency and word position.

Document Summarization Machine Translation

A Set of Recommendations for Assessing Human-Machine Parity in Language Translation

1 code implementation3 Apr 2020 Samuel Läubli, Sheila Castilho, Graham Neubig, Rico Sennrich, Qinlan Shen, Antonio Toral

The quality of machine translation has increased remarkably over the past years, to the degree that it was found to be indistinguishable from professional human translation in a number of empirical investigations.

Machine Translation Translation

X-Stance: A Multilingual Multi-Target Dataset for Stance Detection

1 code implementation18 Mar 2020 Jannis Vamvas, Rico Sennrich

Unlike stance detection models that have specific target issues, we use the dataset to train a single model on all the issues.

Stance Detection

Domain Robustness in Neural Machine Translation

2 code implementations AMTA 2020 Mathias Müller, Annette Rios, Rico Sennrich

Domain robustness---the generalization of models to unseen test domains---is low for both statistical (SMT) and neural machine translation (NMT).

Machine Translation Translation

Domain, Translationese and Noise in Synthetic Data for Neural Machine Translation

no code implementations6 Nov 2019 Nikolay Bogoychev, Rico Sennrich

The quality of neural machine translation can be improved by leveraging additional monolingual resources to create synthetic training data.

Machine Translation Translation

Root Mean Square Layer Normalization

2 code implementations NeurIPS 2019 Biao Zhang, Rico Sennrich

RMSNorm regularizes the summed inputs to a neuron in one layer according to root mean square (RMS), giving the model re-scaling invariance property and implicit learning rate adaptation ability.

The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectives

no code implementations IJCNLP 2019 Elena Voita, Rico Sennrich, Ivan Titov

In this work, we use canonical correlation analysis and mutual information estimators to study how information flows across Transformer layers and how this process depends on the choice of learning objective.

Language Modelling Machine Translation +1

Encoders Help You Disambiguate Word Senses in Neural Machine Translation

no code implementations IJCNLP 2019 Gongbo Tang, Rico Sennrich, Joakim Nivre

We find that encoder hidden states outperform word embeddings significantly which indicates that encoders adequately encode relevant information for disambiguation into hidden states.

Machine Translation Translation +1

Understanding Neural Machine Translation by Simplification: The Case of Encoder-free Models

no code implementations RANLP 2019 Gongbo Tang, Rico Sennrich, Joakim Nivre

In this paper, we try to understand neural machine translation (NMT) via simplifying NMT architectures and training encoder-free NMT models.

Machine Translation Translation +1

Widening the Representation Bottleneck in Neural Machine Translation with Lexical Shortcuts

1 code implementation WS 2019 Denis Emelin, Ivan Titov, Rico Sennrich

The transformer is a state-of-the-art neural translation model that uses attention to iteratively refine lexical representations with information drawn from the surrounding context.

Machine Translation Translation

A Lightweight Recurrent Network for Sequence Modeling

1 code implementation ACL 2019 Biao Zhang, Rico Sennrich

We apply LRN as a drop-in replacement of existing recurrent units in several neural sequential models.

Revisiting Low-Resource Neural Machine Translation: A Case Study

2 code implementations ACL 2019 Rico Sennrich, Biao Zhang

It has been shown that the performance of neural machine translation (NMT) drops starkly in low-resource conditions, underperforming phrase-based statistical machine translation (PBSMT) and requiring large amounts of auxiliary data to achieve competitive results.

Low-Resource Neural Machine Translation Translation

When a Good Translation is Wrong in Context: Context-Aware Machine Translation Improves on Deixis, Ellipsis, and Lexical Cohesion

1 code implementation ACL 2019 Elena Voita, Rico Sennrich, Ivan Titov

Though machine translation errors caused by the lack of context beyond one sentence have long been acknowledged, the development of context-aware NMT systems is hampered by several problems.

Machine Translation Translation

An Analysis of Attention Mechanisms: The Case of Word Sense Disambiguation in Neural Machine Translation

no code implementations WS 2018 Gongbo Tang, Rico Sennrich, Joakim Nivre

Recent work has shown that the encoder-decoder attention mechanisms in neural machine translation (NMT) are different from the word alignment in statistical machine translation.

Machine Translation Translation +2

A Large-Scale Test Set for the Evaluation of Context-Aware Pronoun Translation in Neural Machine Translation

1 code implementation WS 2018 Mathias Müller, Annette Rios, Elena Voita, Rico Sennrich

We show that, while gains in BLEU are moderate for those systems, they outperform baselines by a large margin in terms of accuracy on our contrastive test set.

Machine Translation Translation

The Word Sense Disambiguation Test Suite at WMT18

no code implementations WS 2018 Annette Rios, Mathias M{\"u}ller, Rico Sennrich

We evaluate all German{--}English submissions to the WMT{'}18 shared translation task, plus a number of submissions from previous years, and find that performance on the task has markedly improved compared to the 2016 WMT submissions (81{\%}→93{\%} accuracy on the WSD task).

Machine Translation Translation +1

Has Machine Translation Achieved Human Parity? A Case for Document-level Evaluation

1 code implementation EMNLP 2018 Samuel Läubli, Rico Sennrich, Martin Volk

Recent research suggests that neural machine translation achieves parity with professional human translation on the WMT Chinese--English news translation task.

Machine Translation Translation

Context-Aware Neural Machine Translation Learns Anaphora Resolution

no code implementations ACL 2018 Elena Voita, Pavel Serdyukov, Rico Sennrich, Ivan Titov

Standard machine translation systems process sentences in isolation and hence ignore extra-sentential information, even though extended context can both prevent mistakes in ambiguous cases and improve translation coherence.

Machine Translation Translation

Evaluating Machine Translation Performance on Chinese Idioms with a Blacklist Method

no code implementations LREC 2018 Yutong Shao, Rico Sennrich, Bonnie Webber, Federico Fancellu

Our evaluation confirms that a sizable number of idioms in our test set are mistranslated (46. 1%), that literal translation error is a common error type, and that our blacklist method is effective at identifying literal translation errors.

Machine Translation Translation

Evaluating Discourse Phenomena in Neural Machine Translation

no code implementations NAACL 2018 Rachel Bawden, Rico Sennrich, Alexandra Birch, Barry Haddow

Despite gains using BLEU, multi-encoder models give limited improvement in the handling of discourse phenomena: 50% accuracy on our coreference test set and 53. 5% for coherence/cohesion (compared to a non-contextual baseline of 50%).

Machine Translation Translation

Regularization techniques for fine-tuning in neural machine translation

no code implementations EMNLP 2017 Antonio Valerio Miceli Barone, Barry Haddow, Ulrich Germann, Rico Sennrich

We investigate techniques for supervised domain adaptation for neural machine translation where an existing model trained on a large out-of-domain dataset is adapted to a small in-domain dataset.

Domain Adaptation L2 Regularization +3

Image Pivoting for Learning Multilingual Multimodal Representations

no code implementations EMNLP 2017 Spandana Gella, Rico Sennrich, Frank Keller, Mirella Lapata

In this paper we propose a model to learn multimodal multilingual representations for matching images and sentences in different languages, with the aim of advancing multilingual versions of image search and image understanding.

Image Retrieval Semantic Textual Similarity

A parallel corpus of Python functions and documentation strings for automated code documentation and code generation

4 code implementations IJCNLP 2017 Antonio Valerio Miceli Barone, Rico Sennrich

Automated documentation of programming source code and automated code generation from natural language are challenging tasks of both practical and scientific interest.

Code Generation Data Augmentation +2

Practical Neural Machine Translation

no code implementations EACL 2017 Rico Sennrich, Barry Haddow

Neural Machine Translation (NMT) has achieved new breakthroughs in machine translation in recent years.

Machine Translation Translation

Predicting Target Language CCG Supertags Improves Neural Machine Translation

no code implementations WS 2017 Maria Nadejde, Siva Reddy, Rico Sennrich, Tomasz Dwojak, Marcin Junczys-Dowmunt, Philipp Koehn, Alexandra Birch

Our results on WMT data show that explicitly modeling target-syntax improves machine translation quality for German->English, a high-resource pair, and for Romanian->English, a low-resource pair and also several syntactic phenomena including prepositional phrase attachment.

Machine Translation Prepositional Phrase Attachment +1

Linguistic Input Features Improve Neural Machine Translation

1 code implementation WS 2016 Rico Sennrich, Barry Haddow

Neural machine translation has recently achieved impressive results, while using little in the way of external linguistic information.

Machine Translation Translation

Edinburgh Neural Machine Translation Systems for WMT 16

1 code implementation WS 2016 Rico Sennrich, Barry Haddow, Alexandra Birch

We participated in the WMT 2016 shared news translation task by building neural translation systems for four language pairs, each trained in both directions: English<->Czech, English<->German, English<->Romanian and English<->Russian.

Machine Translation Translation

The AMU-UEDIN Submission to the WMT16 News Translation Task: Attention-based NMT Models as Feature Functions in Phrase-based SMT

1 code implementation WS 2016 Marcin Junczys-Dowmunt, Tomasz Dwojak, Rico Sennrich

For the Russian-English task, our submission achieves the top BLEU result, outperforming the best pure neural system by 1. 1 BLEU points and our own phrase-based baseline by 1. 6 BLEU.

Machine Translation Translation

Neural Machine Translation of Rare Words with Subword Units

25 code implementations ACL 2016 Rico Sennrich, Barry Haddow, Alexandra Birch

Neural machine translation (NMT) models typically operate with a fixed vocabulary, but translation is an open-vocabulary problem.

Translation

Modelling and Optimizing on Syntactic N-Grams for Statistical Machine Translation

no code implementations TACL 2015 Rico Sennrich

The role of language models in SMT is to promote fluent translation output, but traditional n-gram language models are unable to capture fluency phenomena between distant words, such as some morphological agreement phenomena, subcategorisation, and syntactic collocations with string-level gaps.

Language Modelling Machine Translation +1

Zmorge: A German Morphological Lexicon Extracted from Wiktionary

no code implementations LREC 2014 Rico Sennrich, Beat Kunz

We describe a method to automatically extract a German lexicon from Wiktionary that is compatible with the finite-state morphological grammar SMOR.

Machine Translation Morphological Analysis

Cannot find the paper you are looking for? You can Submit a new open access paper.