U$^2$-Net: Going Deeper with Nested U-Structure for Salient Object Detection

18 May 2020  ·  Xuebin Qin, Zichen Zhang, Chenyang Huang, Masood Dehghan, Osmar R. Zaiane, Martin Jagersand ·

In this paper, we design a simple yet powerful deep network architecture, U$^2$-Net, for salient object detection (SOD). The architecture of our U$^2$-Net is a two-level nested U-structure. The design has the following advantages: (1) it is able to capture more contextual information from different scales thanks to the mixture of receptive fields of different sizes in our proposed ReSidual U-blocks (RSU), (2) it increases the depth of the whole architecture without significantly increasing the computational cost because of the pooling operations used in these RSU blocks. This architecture enables us to train a deep network from scratch without using backbones from image classification tasks. We instantiate two models of the proposed architecture, U$^2$-Net (176.3 MB, 30 FPS on GTX 1080Ti GPU) and U$^2$-Net$^{\dagger}$ (4.7 MB, 40 FPS), to facilitate the usage in different environments. Both models achieve competitive performance on six SOD datasets. The code is available: https://github.com/NathanUA/U-2-Net.

PDF Abstract

Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Dichotomous Image Segmentation DIS-TE1 U2Net max F-Measure 0.694 # 6
weighted F-measure 0.601 # 4
MAE 0.083 # 5
S-Measure 0.760 # 6
E-measure 0.801 # 6
HCE 224 # 7
Dichotomous Image Segmentation DIS-TE2 U2Net max F-Measure 0.756 # 6
weighted F-measure 0.668 # 5
MAE 0.085 # 6
S-Measure 0.788 # 6
E-measure 0.833 # 8
HCE 490 # 8
Dichotomous Image Segmentation DIS-TE3 U2Net max F-Measure 0.798 # 5
weighted F-measure 0.707 # 5
MAE 0.079 # 6
S-Measure 0.809 # 6
E-measure 0.858 # 7
HCE 965 # 8
Dichotomous Image Segmentation DIS-TE4 U2Net max F-Measure 0.795 # 5
weighted F-measure 0.705 # 5
MAE 0.087 # 5
S-Measure 0.807 # 5
E-measure 0.847 # 8
HCE 3653 # 8
Dichotomous Image Segmentation DIS-VD U2Net max F-Measure 0.748 # 5
weighted F-measure 0.656 # 4
MAE 0.090 # 4
S-Measure 0.781 # 5
E-measure 0.823 # 6
HCE 1413 # 8
Saliency Detection DUT-OMRON U2-Net+ MAE 0.06 # 4
{max}Fβ 0.813 # 1
Fwβ 0.731 # 1
Sm 0.837 # 1
relaxFbβ 0.676 # 1
Saliency Detection DUT-OMRON U2-Net MAE 0.054 # 3
Salient Object Detection ECSSD F3Net MAE 0.041 # 7
max_F1 0.885 # 7
S-measure 0.918 # 7
Salient Object Detection HKU-IS F3Net MAE 0.031 # 7
Saliency Detection HKU-IS U2-Net+ MAE 0.037 # 3
{max}Fβ 0.928 # 1
Fwβ 0.867 # 1
Sm 0.908 # 1
relaxFbβ 0.794 # 1
Salient Object Detection PASCAL-S F3Net MAE 0.086 # 7
max_F1 0.768 # 7
S-measure 0.831 # 7
Salient Object Detection SOD U2-Net+ {max}Fβ 0.841 # 1
MAE 0.124 # 1
Fwβ 0.697 # 1
Sm 0.759 # 1
relaxFbβ 0.559 # 1

Methods