Transformers

Transformers are a type of neural network architecture that have several properties that make them effective for modeling data with long-range dependencies. They generally feature a combination of multi-headed attention mechanisms, residual connections, layer normalization, feedforward connections, and positional embeddings.

Subcategories

Method Year Papers
2017 3767
2018 3539
2019 452
2019 307
2019 203
2020 195
2019 180
2018 133
2019 131
2019 126
2019 73
2020 67
2019 46
2019 45
2020 41
2020 35
2020 33
2020 31
2020 29
2020 22
2019 21
2020 19
2019 18
2020 13
2020 12
2021 11
2018 10
2020 10
2020 10
2020 9
2000 8
2019 8
2019 7
2020 6
2021 6
2021 5
2021 5
2020 4
2021 4
2020 4
2020 4
2020 3
2019 3
2019 3
2021 3
2019 2
2020 2
2019 2
2021 2
2020 2
2020 2
2021 2
2021 2
2021 2
2021 1
2021 1
2019 1
2020 1
2020 1
2020 1
2021 1
2021 1
2020 1
2019 1
2020 1
2020 1
2020 1
2021 1
2021 1
2021 1
2020 1
2020 1
2020 1
2020 1
2018 1
2020 1
2021 1
2021 1
2020 1
2019 1
2020 1