Natural Language Processing • Language Models • 92 methods
Transformers are a type of neural network architecture that have several properties that make them effective for modeling data with long-range dependencies. They generally feature a combination of multi-headed attention mechanisms, residual connections, layer normalization, feedforward connections, and positional embeddings.
Subcategories
Method | Year | Papers |
---|---|---|
2017 | 5884 | |
2018 | 4237 | |
2020 | 606 | |
2019 | 591 | |
2019 | 420 | |
2019 | 394 | |
2018 | 267 | |
2019 | 253 | |
2023 | 242 | |
2019 | 144 | |
2019 | 143 | |
2019 | 98 | |
2020 | 89 | |
2020 | 81 | |
2020 | 60 | |
2020 | 58 | |
2019 | 53 | |
2019 | 50 | |
2020 | 49 | |
2020 | 46 | |
2020 | 45 | |
2019 | 40 | |
2022 | 40 | |
2020 | 36 | |
2019 | 29 | |
2019 | 27 | |
2019 | 24 | |
2020 | 24 | |
2021 | 21 | |
2000 | 19 | |
2019 | 19 | |
2020 | 15 | |
2021 | 14 | |
2020 | 14 | |
2018 | 13 | |
2020 | 12 | |
2020 | 12 | |
2019 | 10 | |
2021 | 9 | |
2020 | 7 | |
2021 | 6 | |
2020 | 6 | |
2021 | 6 | |
2020 | 5 | |
2021 | 5 | |
2020 | 5 | |
2021 | 5 | |
2022 | 4 | |
2019 | 3 | |
2020 | 3 | |
2019 | 3 | |
2021 | 3 | |
2019 | 3 | |
2020 | 2 | |
2021 | 2 | |
2019 | 2 | |
2021 | 2 | |
2020 | 2 | |
2020 | 2 | |
2021 | 2 | |
2021 | 2 | |
2021 | 2 | |
2021 | 1 | |
2021 | 1 | |
2019 | 1 | |
2020 | 1 | |
2020 | 1 | |
2020 | 1 | |
2021 | 1 | |
2021 | 1 | |
2020 | 1 | |
2019 | 1 | |
2020 | 1 | |
2020 | 1 | |
2020 | 1 | |
2021 | 1 | |
2021 | 1 | |
2020 | 1 | |
2020 | 1 | |
2020 | 1 | |
2020 | 1 | |
2018 | 1 | |
2020 | 1 | |
2021 | 1 | |
2020 | 1 | |
2019 | 1 | |
2020 | 1 |