Methods > Natural Language Processing > Attention Mechanisms

Attention Patterns

METHOD YEAR PAPERS
Strided Attention
2019 41
Fixed Factorized Attention
2019 41
Sliding Window Attention
2020 14
Dilated Sliding Window Attention
2020 14
Global and Sliding Window Attention
2020 14
Routing Attention
2020 2