Transformers

Transformers are a type of neural network architecture that have several properties that make them effective for modeling data with long-range dependencies. They generally feature a combination of multi-headed attention mechanisms, residual connections, layer normalization, feedforward connections, and positional embeddings.

Subcategories

Method Year Papers
2017 9241
2023 5517
2018 5008
2020 1317
2019 739
2018 645
2019 548
2019 540
2019 426
2019 156
2019 152
2020 129
2019 125
2020 112
2022 107
2020 105
2020 78
2020 78
2020 72
2020 63
2019 61
2020 59
2019 55
2019 48
2020 48
2019 43
2019 35
2020 29
2019 29
2000 28
2019 21
2021 18
2018 17
2021 16
2020 15
2020 14
2020 13
2019 11
2021 10
2020 9
2021 6
2020 6
2021 6
2022 6
2020 5
2021 5
2021 5
2020 5
2021 3
2019 3
2021 3
2019 3
2021 3
2019 3
2020 3
2020 3
2020 3
2020 2
2019 2
2021 2
2020 2
2021 2
2020 2
2021 2
2022 2
2021 2
2020 1
2020 1
2021 1
2021 1
2019 1
2020 1
2020 1
2021 1
2020 1
2020 1
2020 1
2020 1
2018 1
2020 1
2020 1
2021 1
2021 1
2020 1
2019 1
2021 1
2020 1
2019 1