Natural Language Processing • Language Models • 86 methods
Transformers are a type of neural network architecture that have several properties that make them effective for modeling data with long-range dependencies. They generally feature a combination of multi-headed attention mechanisms, residual connections, layer normalization, feedforward connections, and positional embeddings.
Subcategories
Method | Year | Papers |
---|---|---|
2017 | 3767 | |
2018 | 3539 | |
2019 | 452 | |
2019 | 307 | |
2019 | 203 | |
2020 | 195 | |
2019 | 180 | |
2018 | 133 | |
2019 | 131 | |
2019 | 126 | |
2019 | 73 | |
2020 | 67 | |
2019 | 46 | |
2019 | 45 | |
2020 | 41 | |
2020 | 35 | |
2020 | 33 | |
2020 | 31 | |
2020 | 29 | |
2020 | 22 | |
2019 | 21 | |
2020 | 19 | |
2019 | 18 | |
2020 | 13 | |
2020 | 12 | |
2021 | 11 | |
2018 | 10 | |
2020 | 10 | |
2020 | 10 | |
2020 | 9 | |
2000 | 8 | |
2019 | 8 | |
2019 | 7 | |
2020 | 6 | |
2021 | 6 | |
2021 | 5 | |
2021 | 5 | |
2020 | 4 | |
2021 | 4 | |
2020 | 4 | |
2020 | 4 | |
2020 | 3 | |
2019 | 3 | |
2019 | 3 | |
2021 | 3 | |
2019 | 2 | |
2020 | 2 | |
2019 | 2 | |
2021 | 2 | |
2020 | 2 | |
2020 | 2 | |
2021 | 2 | |
2021 | 2 | |
2021 | 2 | |
2021 | 1 | |
2021 | 1 | |
2019 | 1 | |
2020 | 1 | |
2020 | 1 | |
2020 | 1 | |
2021 | 1 | |
2021 | 1 | |
2020 | 1 | |
2019 | 1 | |
2020 | 1 | |
2020 | 1 | |
2020 | 1 | |
2021 | 1 | |
2021 | 1 | |
2021 | 1 | |
2020 | 1 | |
2020 | 1 | |
2020 | 1 | |
2020 | 1 | |
2018 | 1 | |
2020 | 1 | |
2021 | 1 | |
2021 | 1 | |
2020 | 1 | |
2019 | 1 | |
2020 | 1 |