Transformers

Transformers are a type of neural network architecture that have several properties that make them effective for modeling data with long-range dependencies. They generally feature a combination of multi-headed attention mechanisms, residual connections, layer normalization, feedforward connections, and positional embeddings.

Subcategories

Method Year Papers
2017 8906
2023 4956
2018 4934
2020 1246
2019 725
2018 603
2019 535
2019 525
2019 398
2019 156
2019 151
2019 125
2020 110
2020 105
2020 105
2022 102
2020 77
2020 74
2020 69
2020 63
2019 61
2020 58
2019 54
2019 48
2020 48
2019 43
2019 34
2020 29
2019 29
2000 28
2021 21
2019 20
2018 17
2021 16
2020 15
2020 14
2020 13
2019 11
2021 9
2020 9
2020 6
2021 6
2022 6
2021 6
2020 5
2021 5
2021 5
2020 5
2021 3
2019 3
2021 3
2019 3
2021 3
2019 3
2020 3
2020 3
2020 2
2020 2
2019 2
2021 2
2020 2
2021 2
2020 2
2021 2
2022 2
2021 2
2020 1
2020 1
2021 1
2021 1
2019 1
2020 1
2020 1
2021 1
2020 1
2020 1
2020 1
2020 1
2018 1
2020 1
2020 1
2021 1
2021 1
2020 1
2019 1
2021 1
2020 1
2019 1