Search Results for author: Kenneth Heafield

Found 49 papers, 6 papers with code

Findings of the WMT 2021 Shared Task on Efficient Translation

no code implementations WMT (EMNLP) 2021 Kenneth Heafield, Qianqian Zhu, Roman Grundkiewicz

The machine translation efficiency task challenges participants to make their systems faster and smaller with minimal impact on translation quality.

Machine Translation Translation

Findings of the 2021 Conference on Machine Translation (WMT21)

no code implementations WMT (EMNLP) 2021 Farhad Akhbardeh, Arkady Arkhangorodsky, Magdalena Biesialska, Ondřej Bojar, Rajen Chatterjee, Vishrav Chaudhary, Marta R. Costa-Jussa, Cristina España-Bonet, Angela Fan, Christian Federmann, Markus Freitag, Yvette Graham, Roman Grundkiewicz, Barry Haddow, Leonie Harter, Kenneth Heafield, Christopher Homan, Matthias Huck, Kwabena Amponsah-Kaakyire, Jungo Kasai, Daniel Khashabi, Kevin Knight, Tom Kocmi, Philipp Koehn, Nicholas Lourie, Christof Monz, Makoto Morishita, Masaaki Nagata, Ajay Nagesh, Toshiaki Nakazawa, Matteo Negri, Santanu Pal, Allahsera Auguste Tapo, Marco Turchi, Valentin Vydrin, Marcos Zampieri

This paper presents the results of the newstranslation task, the multilingual low-resourcetranslation for Indo-European languages, thetriangular translation task, and the automaticpost-editing task organised as part of the Con-ference on Machine Translation (WMT) 2021. In the news task, participants were asked tobuild machine translation systems for any of10 language pairs, to be evaluated on test setsconsisting mainly of news stories.

Machine Translation Translation

Pruning Neural Machine Translation for Speed Using Group Lasso

no code implementations WMT (EMNLP) 2021 Maximiliana Behnke, Kenneth Heafield

In the WMT 2021 Efficiency Task, our pruned and quantised models are 1. 9–2. 7x faster at the cost 0. 9–1. 7 BLEU in comparison to the unoptimised baselines.

Machine Translation Translation

Approaching Neural Chinese Word Segmentation as a Low-Resource Machine Translation Task

no code implementations12 Aug 2020 Pin-zhen Chen, Kenneth Heafield

Supervised Chinese word segmentation has entered the deep learning era which reduces the hassle of feature engineering.

Chinese Word Segmentation Data Augmentation +4

Compressing Neural Machine Translation Models with 4-bit Precision

no code implementations WS 2020 Alham Fikri Aji, Kenneth Heafield

We empirically show that NMT models based on Transformer or RNN architecture can be compressed up to 4-bit precision without any noticeable quality degradation.

Machine Translation Quantization +1

Findings of the Fourth Workshop on Neural Generation and Translation

no code implementations WS 2020 Kenneth Heafield, Hiroaki Hayashi, Yusuke Oda, Ioannis Konstas, Andrew Finch, Graham Neubig, Xi-An Li, Alex Birch, ra

We describe the finding of the Fourth Workshop on Neural Generation and Translation, held in concert with the annual conference of the Association for Computational Linguistics (ACL 2020).

Machine Translation Translation

Parallel Sentence Mining by Constrained Decoding

1 code implementation ACL 2020 Pin-zhen Chen, Nikolay Bogoychev, Kenneth Heafield, Faheem Kirefu

We present a novel method to extract parallel sentences from two monolingual corpora, using neural machine translation.

Machine Translation Sentence Similarity +1

Zero-Resource Neural Machine Translation with Monolingual Pivot Data

no code implementations WS 2019 Anna Currey, Kenneth Heafield

An extension to zero-shot NMT is zero-resource NMT, which generates pseudo-parallel corpora using a zero-shot system and further trains the zero-shot system on that data.

Machine Translation Translation

From Research to Production and Back: Ludicrously Fast Neural Machine Translation

no code implementations WS 2019 Young Jin Kim, Marcin Junczys-Dowmunt, Hany Hassan, Alham Fikri Aji, Kenneth Heafield, Roman Grundkiewicz, Nikolay Bogoychev

Taking our dominating submissions to the previous edition of the shared task as a starting point, we develop improved teacher-student training via multi-agent dual-learning and noisy backward-forward translation for Transformer-based student models.

Machine Translation Translation

Neural Machine Translation with 4-Bit Precision and Beyond

no code implementations13 Sep 2019 Alham Fikri Aji, Kenneth Heafield

We empirically show that NMT models based on Transformer or RNN architecture can be compressed up to 4-bit precision without any noticeable quality degradation.

Machine Translation Quantization +1

Incorporating Source Syntax into Transformer-Based Neural Machine Translation

no code implementations WS 2019 Anna Currey, Kenneth Heafield

Transformer-based neural machine translation (NMT) has recently achieved state-of-the-art performance on many machine translation tasks.

Machine Translation Translation

Making Asynchronous Stochastic Gradient Descent Work for Transformers

no code implementations WS 2019 Alham Fikri Aji, Kenneth Heafield

Asynchronous stochastic gradient descent (SGD) is attractive from a speed perspective because workers do not wait for synchronization.

Machine Translation Translation

Findings of the WMT 2018 Shared Task on Parallel Corpus Filtering

no code implementations WS 2018 Philipp Koehn, Huda Khayrallah, Kenneth Heafield, Mikel L. Forcada

We posed the shared task of assigning sentence-level quality scores for a very noisy corpus of sentence pairs crawled from the web, with the goal of sub-selecting 1{\%} and 10{\%} of high-quality data to be used to train machine translation systems.

Machine Translation Outlier Detection +1

Multi-Source Syntactic Neural Machine Translation

no code implementations EMNLP 2018 Anna Currey, Kenneth Heafield

We introduce a novel multi-source technique for incorporating source syntax into neural machine translation using linearized parses.

Machine Translation Translation

Accelerating Asynchronous Stochastic Gradient Descent for Neural Machine Translation

no code implementations EMNLP 2018 Nikolay Bogoychev, Marcin Junczys-Dowmunt, Kenneth Heafield, Alham Fikri Aji

In order to extract the best possible performance from asynchronous stochastic gradient descent one must increase the mini-batch size and scale the learning rate accordingly.

Machine Translation Translation

Neural Machine Translation Techniques for Named Entity Transliteration

1 code implementation WS 2018 Roman Grundkiewicz, Kenneth Heafield

Transliterating named entities from one language into another can be approached as neural machine translation (NMT) problem, for which we use deep attentional RNN encoder-decoder models.

Automatic Post-Editing Grammatical Error Correction +2

Unsupervised Source Hierarchies for Low-Resource Neural Machine Translation

no code implementations WS 2018 Anna Currey, Kenneth Heafield

Incorporating source syntactic information into neural machine translation (NMT) has recently proven successful (Eriguchi et al., 2016; Luong et al., 2016).

Low-Resource Neural Machine Translation Natural Language Inference +2

Fast Neural Machine Translation Implementation

no code implementations WS 2018 Hieu Hoang, Tomasz Dwojak, Rihards Krislauks, Daniel Torregrosa, Kenneth Heafield

This paper describes the submissions to the efficiency track for GPUs at the Workshop for Neural Machine Translation and Generation by members of the University of Edinburgh, Adam Mickiewicz University, Tilde and University of Alicante.

Machine Translation Translation

Exploring Hyper-Parameter Optimization for Neural Machine Translation on GPU Architectures

no code implementations5 May 2018 Robert Lim, Kenneth Heafield, Hieu Hoang, Mark Briers, Allen Malony

Neural machine translation (NMT) has been accelerated by deep learning neural networks over statistical-based approaches, due to the plethora and programmability of commodity heterogeneous computing architectures such as FPGAs and GPUs and the massive amount of training corpuses generated from news outlets, government agencies and social media.

Language Modelling Machine Translation +1

Approaching Neural Grammatical Error Correction as a Low-Resource Machine Translation Task

1 code implementation NAACL 2018 Marcin Junczys-Dowmunt, Roman Grundkiewicz, Shubha Guha, Kenneth Heafield

Previously, neural methods in grammatical error correction (GEC) did not reach state-of-the-art results compared to phrase-based statistical machine translation (SMT) baselines.

Domain Adaptation Grammatical Error Correction +3

Marian: Fast Neural Machine Translation in C++

2 code implementations ACL 2018 Marcin Junczys-Dowmunt, Roman Grundkiewicz, Tomasz Dwojak, Hieu Hoang, Kenneth Heafield, Tom Neckermann, Frank Seide, Ulrich Germann, Alham Fikri Aji, Nikolay Bogoychev, André F. T. Martins, Alexandra Birch

We present Marian, an efficient and self-contained Neural Machine Translation framework with an integrated automatic differentiation engine based on dynamic computation graphs.

Machine Translation Translation

Sparse Communication for Distributed Gradient Descent

no code implementations EMNLP 2017 Alham Fikri Aji, Kenneth Heafield

Most configurations work on MNIST, whereas different configurations reduce convergence rate on the more complex translation task.

General Classification Image Classification +3

N-gram Counts and Language Models from the Common Crawl

no code implementations LREC 2014 Christian Buck, Kenneth Heafield, Bas van Ooyen

We contribute 5-gram counts and language models trained on the Common Crawl corpus, a collection over 9 billion web pages.

Machine Translation Speech Recognition

Cannot find the paper you are looking for? You can Submit a new open access paper.