Dual Cross-Attention for Medical Image Segmentation

30 Mar 2023  ·  Gorkem Can Ates, Prasoon Mohan, Emrah Celik ·

We propose Dual Cross-Attention (DCA), a simple yet effective attention module that is able to enhance skip-connections in U-Net-based architectures for medical image segmentation. DCA addresses the semantic gap between encoder and decoder features by sequentially capturing channel and spatial dependencies across multi-scale encoder features. First, the Channel Cross-Attention (CCA) extracts global channel-wise dependencies by utilizing cross-attention across channel tokens of multi-scale encoder features. Then, the Spatial Cross-Attention (SCA) module performs cross-attention to capture spatial dependencies across spatial tokens. Finally, these fine-grained encoder features are up-sampled and connected to their corresponding decoder parts to form the skip-connection scheme. Our proposed DCA module can be integrated into any encoder-decoder architecture with skip-connections such as U-Net and its variants. We test our DCA module by integrating it into six U-Net-based architectures such as U-Net, V-Net, R2Unet, ResUnet++, DoubleUnet and MultiResUnet. Our DCA module shows Dice Score improvements up to 2.05% on GlaS, 2.74% on MoNuSeg, 1.37% on CVC-ClinicDB, 1.12% on Kvasir-Seg and 1.44% on Synapse datasets. Our codes are available at: https://github.com/gorkemcanates/Dual-Cross-Attention

PDF Abstract

Results from the Paper

Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Medical Image Segmentation CVC-ClinicDB DoubleUnet-DCA mean Dice 0.9086 # 26
mIoU 0.8347 # 1
Medical Image Segmentation Kvasir-SEG DoubleUnet-DCA mean Dice 0.8516 # 34
mIoU 0.7434 # 34
Medical Image Segmentation MoNuSeg DoubleUnet-DCA IoU 65.97 # 7
Medical Image Segmentation Synapse multi-organ CT DoubleUnet-DCA Avg DSC 80.22 # 16