Methods > Natural Language Processing

Language Models

METHOD YEAR PAPERS
BERT
2018 1975
Transformer
2017 1751
RoBERTa
2019 213
ELMo
2018 180
GPT-2
2019 164
XLNet
2019 88
ALBERT
2019 60
GPT
2018 58
CAM
2015 58
BART
2019 48
T5
2019 43
GPT-3
2020 40
DistilBERT
2019 33
Transformer-XL
2019 31
XLM
2019 29
ULMFiT
2018 28
ELECTRA
2020 28
Longformer
2020 14
ViLBERT
2019 14
Sparse Transformer
2019 9
Reformer
2020 7
Universal Transformer
2018 6
Cross-View Training
2018 5
Synthesizer
2020 5
Neural Probabilistic Language Model
2003 4
Linformer
2020 4
Levenshtein Transformer
2019 4
VideoBERT
2019 3
CTRL
2019 2
Adaptive Span Transformer
2019 2
Routing Transformer
2020 2
CharacterBERT
2020 2
SHA-RNN
2019 1
Gated Convolution Network
2016 1
Compressive Transformer
2019 1
Sinkhorn Transformer
2020 1
Adaptively Sparse Transformer
2019 1
BP-Transformer
2019 1
DeLighT
2020 1
Feedback Transformer
2020 1
Sandwich Transformer
2019 1
Nyströmformer
2021 1
TNT
2021 1
GANsformer
2021 1
T-D
2018 1