Uncertainty Inspired RGB-D Saliency Detection

We propose the first stochastic framework to employ uncertainty for RGB-D saliency detection by learning from the data labeling process. Existing RGB-D saliency detection models treat this task as a point estimation problem by predicting a single saliency map following a deterministic learning pipeline... We argue that, however, the deterministic solution is relatively ill-posed. Inspired by the saliency data labeling process, we propose a generative architecture to achieve probabilistic RGB-D saliency detection which utilizes a latent variable to model the labeling variations. Our framework includes two main models: 1) a generator model, which maps the input image and latent variable to stochastic saliency prediction, and 2) an inference model, which gradually updates the latent variable by sampling it from the true or approximate posterior distribution. The generator model is an encoder-decoder saliency network. To infer the latent variable, we introduce two different solutions: i) a Conditional Variational Auto-encoder with an extra encoder to approximate the posterior distribution of the latent variable; and ii) an Alternating Back-Propagation technique, which directly samples the latent variable from the true posterior distribution. Qualitative and quantitative results on six challenging RGB-D benchmark datasets show our approach's superior performance in learning the distribution of saliency maps. The source code is publicly available via our project page: https://github.com/JingZhang617/UCNet. read more

PDF Abstract

Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
RGB-D Salient Object Detection DES UCNet-ABP S-Measure 94.0 # 2
Average MAE 0.016 # 1
RGB-D Salient Object Detection DES UCNet-CVAE S-Measure 93.7 # 3
Average MAE 0.016 # 1
RGB Salient Object Detection DUT-OMRON UCNet-CVAE MAE 0.051 # 2
S-Measure 0.839 # 2
RGB Salient Object Detection DUT-OMRON UCNet-ABP MAE 0.050 # 1
S-Measure 0.843 # 1
RGB Salient Object Detection DUTS-TE UCNet-CVAE MAE 0.034 # 3
S-Measure 0.888 # 2
mean E-Measure 0.927 # 2
mean F-Measure 0.860 # 2
RGB Salient Object Detection DUTS-TE UCNet-ABP MAE 0.034 # 3
S-Measure 0.890 # 1
mean E-Measure 0.931 # 1
mean F-Measure 0.864 # 1
RGB Salient Object Detection DUTS-test UCNet-CVAE MAE 0.034 # 1
mean F-Measure 0.773 # 1
RGB Salient Object Detection ECSSD UCNet-CVAE MAE 0.035 # 1
S-Measure 0.921 # 1
RGB Salient Object Detection HKU-IS UCNet-CVAE MAE 0.026 # 1
S-Measure 0.921 # 1
RGB Salient Object Detection HKU-IS UCNet-ABP MAE 0.027 # 2
S-Measure 0.917 # 2
RGB-D Salient Object Detection LFSD UCNet-ABP S-Measure 86.6 # 3
Average MAE 0.065 # 1
RGB-D Salient Object Detection LFSD UCNet-CVAE S-Measure 86.8 # 1
Average MAE 0.065 # 1
RGB-D Salient Object Detection NJU2K UCNet-ABP S-Measure 90.0 # 12
Average MAE 0.039 # 5
RGB-D Salient Object Detection NJU2K UCNet-CVAE S-Measure 90.2 # 10
Average MAE 0.039 # 5
RGB-D Salient Object Detection NLPR UCNet-CAVE S-Measure 91.7 # 9
Average MAE 0.025 # 8
RGB-D Salient Object Detection NLPR UCNet-ABP S-Measure 91.9 # 8
Average MAE 0.024 # 6
RGB-D Salient Object Detection SIP UCNet-ABP S-Measure 87.6 # 7
Average MAE 0.049 # 5
RGB-D Salient Object Detection SIP UCNet-CVAE S-Measure 88.3 # 3
Average MAE 0.045 # 3
RGB Salient Object Detection SOC UCNet-APB S-Measure 0.842 # 2
mean E-Measure 0.868 # 2
Average MAE 0.091 # 2
RGB Salient Object Detection SOC UCNet-CVAE S-Measure 0.849 # 1
mean E-Measure 0.872 # 1
Average MAE 0.089 # 1
RGB-D Salient Object Detection STERE UCNet-ABP S-Measure 90.4 # 6
Average MAE 0.037 # 1
RGB-D Salient Object Detection STERE UCNet-CVAE S-Measure 89.8 # 10
Average MAE 0.039 # 4

Methods


No methods listed for this paper. Add relevant methods here