TransFuse: Fusing Transformers and CNNs for Medical Image Segmentation

16 Feb 2021  ·  Yundong Zhang, Huiye Liu, Qiang Hu ·

Medical image segmentation - the prerequisite of numerous clinical needs - has been significantly prospered by recent advances in convolutional neural networks (CNNs). However, it exhibits general limitations on modeling explicit long-range relation, and existing cures, resorting to building deep encoders along with aggressive downsampling operations, leads to redundant deepened networks and loss of localized details... Hence, the segmentation task awaits a better solution to improve the efficiency of modeling global contexts while maintaining a strong grasp of low-level details. In this paper, we propose a novel parallel-in-branch architecture, TransFuse, to address this challenge. TransFuse combines Transformers and CNNs in a parallel style, where both global dependency and low-level spatial details can be efficiently captured in a much shallower manner. Besides, a novel fusion technique - BiFusion module is created to efficiently fuse the multi-level features from both branches. Extensive experiments demonstrate that TransFuse achieves the newest state-of-the-art results on both 2D and 3D medical image sets including polyp, skin lesion, hip, and prostate segmentation, with significant parameter decrease and inference speed improvement. read more

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Benchmark
Medical Image Segmentation CVC-ClinicDB TransFuse-L mean Dice 0.934 # 4
Medical Image Segmentation CVC-ClinicDB TransFuse-S mean Dice 0.918 # 9
Medical Image Segmentation CVC-ColonDB TransFuse-S mean Dice 0.773 # 4
mIoU 0.696 # 4
Medical Image Segmentation CVC-ColonDB TransFuse-L mean Dice 0.744 # 7
mIoU 0.676 # 7
Medical Image Segmentation ETIS-LARIBPOLYPDB TransFuse-S mIoU 0.659 # 6
mean Dice 0.733 # 4
Medical Image Segmentation ETIS-LARIBPOLYPDB TransFuse-L mIoU 0.661 # 5
mean Dice 0.737 # 3
Medical Image Segmentation Kvasir-SEG TransFuse-S mean Dice 0.918 # 2
mIoU 0.868 # 2
Medical Image Segmentation Kvasir-SEG TransFuse-L mean Dice 0.918 # 2
mIoU 0.868 # 2


No methods listed for this paper. Add relevant methods here