MaiT: integrating spatial locality into image transformers with attention masks

1 code implementation • 29 Sep 2021 • Ling Li, Ali Shafiee, Joseph H Hassoun

Moreover, attention masks regulate the training of attention maps, which facilitates the convergence and improves the accuracy of deeper transformers.