Transformers
| METHOD | YEAR | PAPERS | |
|---|---|---|---|
Transformer
|
2017 | 1147 | |
BERT
|
2018 | 1125 | |
RoBERTa
|
2019 | 92 | |
GPT-2
|
2019 | 83 | |
XLNet
|
2019 | 45 | |
GPT
|
2018 | 38 | |
Transformer-XL
|
2019 | 20 | |
ALBERT
|
2019 | 18 | |
XLM
|
2019 | 16 | |
DistilBERT
|
2019 | 10 | |
T5
|
2019 | 7 | |
BART
|
2019 | 7 | |
Universal Transformer
|
2018 | 6 | |
Sparse Transformer
|
2019 | 5 | |
ELECTRA
|
2020 | 4 | |
Longformer
|
2020 | 2 | |
Reformer
|
2020 | 2 | |
GPT-3
|
2020 | 1 | |
Adaptively Sparse Transformer
|
2019 | 1 | |
Routing Transformer
|
2020 | 1 | |
BP-Transformer
|
2019 | 1 | |
CTRL
|
2019 | 1 | |
Adaptive Span Transformer
|
2019 | 1 | |
Sinkhorn Transformer
|
2020 | 1 | |
Compressive Transformer
|
2019 | 1 | |
Linformer
|
2020 | 1 |