no code implementations • 3 Sep 2020 • Rui Li, Shunyi Zheng, Chenxi Duan, Ce Zhang, Jianlin Su, P. M. Atkinson
A novel attention mechanism of kernel attention with linear complexity is proposed to alleviate the large computational demand in attention.