TASK	DATASET	MODEL	METRIC NAME	METRIC VALUE	GLOBAL RANK	REMOVE
Machine Translation	WMT2014 English-French	Transformer (big) + Relative Position Representations	BLEU score	41.5	# 14
Machine Translation	WMT2014 English-German	Transformer (big) + Relative Position Representations	BLEU score	29.2	# 14

🦡 Badges

Include the markdown at the top of your GitHub README.md file to showcase the performance of the model.

Badges are live and will be dynamically updated with the latest ranking of this paper.

Badge	Markdown
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/self-attention-with-relative-position/machine-translation-on-wmt2014-english-french)](https://paperswithcode.com/sota/machine-translation-on-wmt2014-english-french?p=self-attention-with-relative-position)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/self-attention-with-relative-position/machine-translation-on-wmt2014-english-german)](https://paperswithcode.com/sota/machine-translation-on-wmt2014-english-german?p=self-attention-with-relative-position)`

Self-Attention with Relative Position Representations

NAACL 2018 • Peter Shaw • Jakob Uszkoreit • Ashish Vaswani

Relying entirely on an attention mechanism, the Transformer introduced by Vaswani et al. (2017) achieves state-of-the-art results for machine translation. In contrast to recurrent and convolutional neural networks, it does not explicitly model relative or absolute position information in its structure... (read more)

PDF Abstract NAACL 2018 PDF NAACL 2018 Abstract

Code

Add Remove Mark official

tensorflow/tensor2tensor official

↳ Quickstart in

Colab

10,999

OpenNMT/OpenNMT-tf

1,219

THUNLP-MT/THUMT

546

thumt/THUMT

546

The-AI-Summer/self-attention-cv

129

See all 9 implementations

Tasks

Add Remove

MACHINE TRANSLATION

Datasets

Add Datasets introduced or used in this paper

Results from the Paper

Edit

Ranked #14 on Machine Translation on WMT2014 English-French

Get a GitHub badge

TASK	DATASET	MODEL	METRIC NAME	METRIC VALUE	GLOBAL RANK	RESULT	BENCHMARK
Machine Translation	WMT2014 English-French	Transformer (big) + Relative Position Representations	BLEU score	41.5	# 14		Compare
Machine Translation	WMT2014 English-German	Transformer (big) + Relative Position Representations	BLEU score	29.2	# 14		Compare

Methods used in the Paper

Add Remove

METHOD	TYPE
Residual Connection	Skip Connections
BPE	Subword Segmentation
Dense Connections	Feedforward Networks
Label Smoothing	Regularization
ReLU	Activation Functions
Adam	Stochastic Optimization
Softmax	Output Functions
Dropout	Regularization
Multi-Head Attention	Attention Modules
Layer Normalization	Normalization
Scaled Dot-Product Attention	Attention Mechanisms
Transformer	Transformers

Edit Social Preview

Self-Attention with Relative Position Representations Edit social preview

Code Edit Add Remove Mark official

Tasks Edit Add Remove

Datasets Edit

Results from the Paper Edit

Methods used in the Paper Edit Add Remove