Revisiting Image Pyramid Structure for High Resolution Salient Object Detection

20 Sep 2022  ·  Taehun Kim, Kunhee Kim, Joonyeong Lee, Dongmin Cha, Jiho Lee, Daijin Kim ·

Salient object detection (SOD) has been in the spotlight recently, yet has been studied less for high-resolution (HR) images. Unfortunately, HR images and their pixel-level annotations are certainly more labor-intensive and time-consuming compared to low-resolution (LR) images and annotations. Therefore, we propose an image pyramid-based SOD framework, Inverse Saliency Pyramid Reconstruction Network (InSPyReNet), for HR prediction without any of HR datasets. We design InSPyReNet to produce a strict image pyramid structure of saliency map, which enables to ensemble multiple results with pyramid-based image blending. For HR prediction, we design a pyramid blending method which synthesizes two different image pyramids from a pair of LR and HR scale from the same image to overcome effective receptive field (ERF) discrepancy. Our extensive evaluations on public LR and HR SOD benchmarks demonstrate that InSPyReNet surpasses the State-of-the-Art (SotA) methods on various SOD metrics and boundary accuracy.

PDF Abstract

Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Uses Extra
Training Data
Result Benchmark
RGB Salient Object Detection DAVIS-S InSPyReNet S-measure 0.962 # 5
F-measure 0.959 # 5
mBA 0.743 # 3
MAE 0.009 # 5
RGB Salient Object Detection DAVIS-S InSPyReNet (HRSOD, UHRSD) S-measure 0.973 # 1
F-measure 0.977 # 2
mBA 0.770 # 1
MAE 0.007 # 3
RGB Salient Object Detection DAVIS-S InSPyReNet (DUTS, HRSOD) S-measure 0.972 # 4
F-measure 0.976 # 4
mBA 0.770 # 1
MAE 0.007 # 3
Dichotomous Image Segmentation DIS-TE1 InSPyReNet (HR scale) max F-Measure 0.845 # 3
weighted F-measure 0.788 # 3
MAE 0.043 # 3
S-Measure 0.873 # 3
E-measure 0.894 # 3
HCE 110 # 1
Dichotomous Image Segmentation DIS-TE1 InSPyReNet max F-Measure 0.834 # 4
S-Measure 0.862 # 4
HCE 148 # 3
Dichotomous Image Segmentation DIS-TE2 InSPyReNet max F-Measure 0.881 # 4
weighted F-measure 0.834 # 3
MAE 0.038 # 3
S-Measure 0.893 # 4
E-measure 0.925 # 3
HCE 316 # 3
Dichotomous Image Segmentation DIS-TE2 InSPyReNet (HR scale) max F-Measure 0.894 # 3
S-Measure 0.905 # 3
HCE 255 # 1
Dichotomous Image Segmentation DIS-TE3 InSPyReNet (HR scale) max F-Measure 0.919 # 3
weighted F-measure 0.871 # 3
MAE 0.034 # 3
S-Measure 0.918 # 2
E-measure 0.943 # 2
HCE 522 # 1
Dichotomous Image Segmentation DIS-TE3 InSPyReNet max F-Measure 0.904 # 4
weighted F-measure 0.856 # 4
MAE 0.038 # 4
S-Measure 0.902 # 4
E-measure 0.938 # 3
HCE 582 # 2
Dichotomous Image Segmentation DIS-TE4 InSPyReNet max F-Measure 0.892 # 4
weighted F-measure 0.840 # 4
MAE 0.046 # 4
S-Measure 0.891 # 4
E-measure 0.926 # 4
HCE 2243 # 1
Dichotomous Image Segmentation DIS-TE4 InSPyReNet (HR scale) max F-Measure 0.905 # 3
weighted F-measure 0.848 # 3
MAE 0.042 # 3
S-Measure 0.905 # 1
E-measure 0.928 # 3
HCE 2336 # 2
Dichotomous Image Segmentation DIS-VD InSPyReNet max F-Measure 0.876 # 4
weighted F-measure 0.826 # 3
MAE 0.043 # 3
S-Measure 0.887 # 4
E-measure 0.921 # 3
HCE 905 # 2
Dichotomous Image Segmentation DIS-VD InSPyReNet (HR scale) max F-Measure 0.889 # 3
S-Measure 0.900 # 3
HCE 904 # 1
RGB Salient Object Detection DUT-OMRON InSPyReNet MAE 0.045 # 4
F-measure 0.832 # 4
S-Measure 0.875 # 2
MAE 0.059 # 14
F-measure 0.791 # 8
S-Measure 0.845 # 8
RGB Salient Object Detection DUTS-TE InSPyReNet MAE 0.024 # 4
max F-measure 0.927 # 4
S-Measure 0.931 # 3
MAE 0.035 # 12
max F-measure 0.892 # 8
S-Measure 0.904 # 7
RGB Salient Object Detection ECSSD InSPyReNet MAE 0.031 # 6
F-measure 0.949 # 4
S-Measure 0.936 # 3
MAE 0.023 # 2
F-measure 0.96 # 2
S-Measure 0.949 # 1
RGB Salient Object Detection HKU-IS InSPyReNet MAE 0.028 # 8
F-measure 0.938 # 3
S-Measure 0.929 # 4
MAE 0.021 # 3
F-measure 0.955 # 1
S-Measure 0.944 # 1
RGB Salient Object Detection HRSOD InSPyReNet (HRSOD, UHRSD) S-Measure 0.956 # 4
max F-Measure 0.956 # 4
MAE 0.018 # 5
mBA 0.771 # 1
RGB Salient Object Detection HRSOD InSPyReNet (DUTS, HRSOD) S-Measure 0.960 # 1
max F-Measure 0.957 # 3
MAE 0.014 # 2
mBA 0.766 # 2
RGB Salient Object Detection HRSOD InSPyReNet S-Measure 0.952 # 5
max F-Measure 0.949 # 5
MAE 0.016 # 4
mBA 0.738 # 3
RGB Salient Object Detection PASCAL-S InSPyReNet MAE 0.056 # 6
F-measure 0.869 # 5
S-Measure 0.876 # 5
MAE 0.048 # 4
F-measure 0.893 # 2
S-Measure 0.893 # 2
RGB Salient Object Detection UHRSD InSPyReNet (HRSOD, UHRSD) S-Measure 0.953 # 1
max F-Measure 0.957 # 3
MAE 0.020 # 3
mBA 0.812 # 1
RGB Salient Object Detection UHRSD InSPyReNet (DUTS, HRSOD) S-Measure 0.936 # 4
max F-Measure 0.938 # 4
MAE 0.028 # 5
mBA 0.785 # 2
RGB Salient Object Detection UHRSD InSPyReNet S-Measure 0.932 # 6
max F-Measure 0.938 # 4
MAE 0.029 # 6
mBA 0.741 # 4

Methods