Efficient Marginalization of Discrete and Structured Latent Variables via Sparsity

1 code implementation • NeurIPS 2020 • Gonçalo M. Correia, Vlad Niculae, Wilker Aziz, André F. T. Martins

In this paper, we propose a new training strategy which replaces these estimators by an exact yet efficient marginalization.

Adaptively Sparse Transformers

3 code implementations • IJCNLP 2019 • Gonçalo M. Correia, Vlad Niculae, André F. T. Martins

Findings of the quantitative and qualitative analysis of our approach include that heads in different layers learn different sparsity preferences and tend to be more diverse in their attention distributions than softmax Transformers.

Machine Translation Translation

A Simple and Effective Approach to Automatic Post-Editing with Transfer Learning

1 code implementation • 14 Jun 2019 • Gonçalo M. Correia, André F. T. Martins

Automatic post-editing (APE) seeks to automatically refine the output of a black-box machine translation (MT) system through human post-edits.

Automatic Post-Editing Transfer Learning +1

Unbabel's Submission to the WMT2019 APE Shared Task: BERT-based Encoder-Decoder for Automatic Post-Editing

no code implementations • WS 2019 • António V. Lopes, M. Amin Farajian, Gonçalo M. Correia, Jonay Trenous, André F. T. Martins

Analogously to dual-encoder architectures we develop a BERT-based encoder-decoder (BED) model in which a single pretrained BERT encoder receives both the source src and machine translation tgt strings.

Automatic Post-Editing NMT +1