Synthetic Sample Selection for Generalized Zero-Shot Learning

6 Apr 2023  ·  Shreyank N Gowda ·

Generalized Zero-Shot Learning (GZSL) has emerged as a pivotal research domain in computer vision, owing to its capability to recognize objects that have not been seen during training. Despite the significant progress achieved by generative techniques in converting traditional GZSL to fully supervised learning, they tend to generate a large number of synthetic features that are often redundant, thereby increasing training time and decreasing accuracy. To address this issue, this paper proposes a novel approach for synthetic feature selection using reinforcement learning. In particular, we propose a transformer-based selector that is trained through proximal policy optimization (PPO) to select synthetic features based on the validation classification accuracy of the seen classes, which serves as a reward. The proposed method is model-agnostic and data-agnostic, making it applicable to both images and videos and versatile for diverse applications. Our experimental results demonstrate the superiority of our approach over existing feature-generating methods, yielding improved overall performance on multiple benchmarks.

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Zero-Shot Learning CUB-200-2011 SPOT average top-1 classification accuracy 62.9 # 6
Generalized Zero-Shot Learning CUB-200-2011 SPOT (DAA) Harmonic mean 67.0 # 1
Zero-Shot Action Recognition HMDB51 SPOT Top-1 Accuracy 35.9 # 16
Zero-Shot Action Recognition Olympics SPOT Top-1 Accuracy 68.7 # 1
Generalized Zero-Shot Learning Oxford 102 Flower SPOT (FREE) Harmonic mean 75.9 # 1
Zero-Shot Learning Oxford 102 Flower SPOT average top-1 classification accuracy 71.9 # 1
Generalized Zero-Shot Learning SUN Attribute SPOT (CMC-GAN) Harmonic mean 46.4 # 1
Zero-Shot Learning SUN Attribute SPOT (VAEGAN) average top-1 classification accuracy 66.04 # 1
Zero-Shot Action Recognition UCF101 SPOT Top-1 Accuracy 40.9 # 19

Methods