Adaptive Dimension Reduction and Variational Inference for Transductive Few-Shot Classification

18 Sep 2022  ·  Yuqing Hu, Stéphane Pateux, Vincent Gripon ·

Transductive Few-Shot learning has gained increased attention nowadays considering the cost of data annotations along with the increased accuracy provided by unlabelled samples in the domain of few shot. Especially in Few-Shot Classification (FSC), recent works explore the feature distributions aiming at maximizing likelihoods or posteriors with respect to the unknown parameters. Following this vein, and considering the parallel between FSC and clustering, we seek for better taking into account the uncertainty in estimation due to lack of data, as well as better statistical properties of the clusters associated with each class. Therefore in this paper we propose a new clustering method based on Variational Bayesian inference, further improved by Adaptive Dimension Reduction based on Probabilistic Linear Discriminant Analysis. Our proposed method significantly improves accuracy in the realistic unbalanced transductive setting on various Few-Shot benchmarks when applied to features used in previous studies, with a gain of up to $6\%$ in accuracy. In addition, when applied to balanced setting, we obtain very competitive results without making use of the class-balance artefact which is disputable for practical use cases. We also provide the performance of our method on a high performing pretrained backbone, with the reported results further surpassing the current state-of-the-art accuracy, suggesting the genericity of the proposed method.

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Uses Extra
Training Data
Result Benchmark
Few-Shot Image Classification CIFAR-FS 5-way (1-shot) BAVARDAGE Accuracy 87.35 # 6
Few-Shot Image Classification CIFAR-FS 5-way (5-shot) BAVARDAGE Accuracy 90.63 # 8
Few-Shot Image Classification CUB 200 5-way 1-shot BAVARDAGE Accuracy 90.42 # 10
Few-Shot Image Classification CUB 200 5-way 5-shot BAVARDAGE Accuracy 93.50 # 7
Few-Shot Image Classification Dirichlet CUB-200 (5-way, 1-shot) BAVARDAGE 1:1 Accuracy 82.0 # 1
Few-Shot Image Classification Dirichlet CUB-200 (5-way, 5-shot) BAVARDAGE 1:1 Accuracy 90.7 # 1
Few-Shot Image Classification Dirichlet Mini-Imagenet (5-way, 1-shot) BAVARDAGE 1:1 Accuracy 71.0 # 1
Few-Shot Image Classification Dirichlet Mini-Imagenet (5-way, 5-shot) BAVARDAGE 1:1 Accuracy 83.6 # 1
Few-Shot Image Classification Dirichlet Tiered-Imagenet (5-way, 1-shot) BAVARDAGE 1:1 Accuracy 76.6 # 1
Few-Shot Image Classification Dirichlet Tiered-Imagenet (5-way, 5-shot) BAVARDAGE 1:1 Accuracy 86.5 # 2
Few-Shot Image Classification FC100 5-way (1-shot) BAVARDAGE Accuracy 57.27 # 1
Few-Shot Image Classification FC100 5-way (5-shot) BAVARDAGE Accuracy 70.60 # 1
Few-Shot Image Classification Mini-Imagenet 5-way (1-shot) BAVARDAGE Accuracy 84.80 # 7
Few-Shot Image Classification Mini-Imagenet 5-way (5-shot) BAVARDAGE Accuracy 91.65 # 4
Few-Shot Image Classification Tiered ImageNet 5-way (1-shot) BAVARDAGE Accuracy 85.20 # 5
Few-Shot Image Classification Tiered ImageNet 5-way (5-shot) BAVARDAGE Accuracy 90.41 # 6

Methods


No methods listed for this paper. Add relevant methods here