Stacked Conditional Generative Adversarial Networks for Jointly Learning Shadow Detection and Shadow Removal

CVPR 2018  ·  Jifeng Wang, Xiang Li, Le Hui, Jian Yang ·

Understanding shadows from a single image spontaneously derives into two types of task in previous studies, containing shadow detection and shadow removal. In this paper, we present a multi-task perspective, which is not embraced by any existing work, to jointly learn both detection and removal in an end-to-end fashion that aims at enjoying the mutually improved benefits from each other. Our framework is based on a novel STacked Conditional Generative Adversarial Network (ST-CGAN), which is composed of two stacked CGANs, each with a generator and a discriminator. Specifically, a shadow image is fed into the first generator which produces a shadow detection mask. That shadow image, concatenated with its predicted mask, goes through the second generator in order to recover its shadow-free image consequently. In addition, the two corresponding discriminators are very likely to model higher level relationships and global scene characteristics for the detected shadow region and reconstruction via removing shadows, respectively. More importantly, for multi-task learning, our design of stacked paradigm provides a novel view which is notably different from the commonly used one as the multi-branch version. To fully evaluate the performance of our proposed framework, we construct the first large-scale benchmark with 1870 image triplets (shadow image, shadow mask image, and shadow-free image) under 135 scenes. Extensive experimental results consistently show the advantages of ST-CGAN over several representative state-of-the-art methods on two large-scale publicly available datasets and our newly released one.

PDF Abstract CVPR 2018 PDF CVPR 2018 Abstract

Datasets


Introduced in the Paper:

ISTD

Used in the Paper:

SBU SRD
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
RGB Salient Object Detection ISTD JDR Balanced Error Rate 7.35 # 3
Shadow Removal ISTD ST-CGAN RMSE 7.47 # 7
Shadow Detection SBU ST-CGAN BER 8.14 # 6
RGB Salient Object Detection SBU JDR Balanced Error Rate 8.14 # 6
RGB Salient Object Detection UCF JDR Balanced Error Rate 11.23 # 6

Methods


No methods listed for this paper. Add relevant methods here