Transformers

Transformers are a type of neural network architecture that have several properties that make them effective for modeling data with long-range dependencies. They generally feature a combination of multi-headed attention mechanisms, residual connections, layer normalization, feedforward connections, and positional embeddings.

Subcategories

Method Year Papers
2017 5884
2018 4237
2020 606
2019 591
2019 420
2019 394
2018 267
2019 253
2023 242
2019 144
2019 143
2019 98
2020 89
2020 81
2020 60
2020 58
2019 53
2019 50
2020 49
2020 46
2020 45
2019 40
2022 40
2020 36
2019 29
2019 27
2019 24
2020 24
2021 21
2000 19
2019 19
2020 15
2021 14
2020 14
2018 13
2020 12
2020 12
2019 10
2021 9
2020 7
2021 6
2020 6
2021 6
2020 5
2021 5
2020 5
2021 5
2022 4
2019 3
2020 3
2019 3
2021 3
2019 3
2020 2
2021 2
2019 2
2021 2
2020 2
2020 2
2021 2
2021 2
2021 2
2021 1
2021 1
2019 1
2020 1
2020 1
2020 1
2021 1
2021 1
2020 1
2019 1
2020 1
2020 1
2020 1
2021 1
2021 1
2020 1
2020 1
2020 1
2020 1
2018 1
2020 1
2021 1
2020 1
2019 1
2020 1