Linear Attention Mechanism: An Efficient Attention for Semantic Segmentation

29 Jul 2020  ·  Rui Li, Jianlin Su, Chenxi Duan, Shunyi Zheng ·

In this paper, to remedy this deficiency, we propose a Linear Attention Mechanism which is approximate to dot-product attention with much less memory and computational costs. The efficient design makes the incorporation between attention mechanisms and neural networks more flexible and versatile. Experiments conducted on semantic segmentation demonstrated the effectiveness of linear attention mechanism. Code is available at

PDF Abstract


Results from the Paper

  Submit results from this paper to get state-of-the-art GitHub badges and help the community compare results to other papers.