Explore Better Relative Position Embeddings from Encoding Perspective for Transformer Models

1 code implementation • EMNLP 2021 • Anlin Qu, Jianwei Niu, Shasha Mo

Relative position embedding (RPE) is a successful method to explicitly and efficaciously encode position information into Transformer models.

Position