GRET: Global Representation Enhanced Transformer

24 Feb 2020 • Rongxiang Weng • Hao-Ran Wei • Shu-Jian Huang • Heng Yu • Lidong Bing • Weihua Luo • Jia-Jun Chen

Transformer, based on the encoder-decoder framework, has achieved state-of-the-art performance on several natural language generation tasks. The encoder maps the words in the input sentence into a sequence of hidden states, which are then fed into the decoder to generate the output sentence... (read more)

PDF Abstract

Results from the Paper

Edit

Submit results from this paper to get state-of-the-art GitHub badges and help the community compare results to other papers.

METHOD	TYPE
Residual Connection	Skip Connections
BPE	Subword Segmentation
Dense Connections	Feedforward Networks
Label Smoothing	Regularization
ReLU	Activation Functions
Adam	Stochastic Optimization
Softmax	Output Functions
Dropout	Regularization
Multi-Head Attention	Attention Modules
Layer Normalization	Normalization
Scaled Dot-Product Attention	Attention Mechanisms
Transformer	Transformers

METHOD

TYPE

Residual Connection