Repetitive Reprediction Deep Decipher for Semi-Supervised Learning

9 Aug 2019  ·  Guo-Hua Wang, Jianxin Wu ·

Most recent semi-supervised deep learning (deep SSL) methods used a similar paradigm: use network predictions to update pseudo-labels and use pseudo-labels to update network parameters iteratively. However, they lack theoretical support and cannot explain why predictions are good candidates for pseudo-labels. In this paper, we propose a principled end-to-end framework named deep decipher (D2) for SSL. Within the D2 framework, we prove that pseudo-labels are related to network predictions by an exponential link function, which gives a theoretical support for using predictions as pseudo-labels. Furthermore, we demonstrate that updating pseudo-labels by network predictions will make them uncertain. To mitigate this problem, we propose a training strategy called repetitive reprediction (R2). Finally, the proposed R2-D2 method is tested on the large-scale ImageNet dataset and outperforms state-of-the-art methods by 5 percentage points.

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Semi-Supervised Image Classification cifar-100, 10000 Labels R2-D2 (CNN-13) Percentage error 32.87 # 24
Semi-Supervised Image Classification CIFAR-10, 4000 Labels R2-D2 (Shake-Shake) Percentage error 5.72 # 22
Semi-Supervised Image Classification ImageNet - 10% labeled data R2-D2 (ResNet-18) Top 5 Accuracy 90.48% # 23
Semi-Supervised Image Classification SVHN, 1000 labels R2-D2 (CNN-13) Accuracy 96.36 # 10

Methods


No methods listed for this paper. Add relevant methods here