TAFE-Net: Task-Aware Feature Embeddings for Low Shot Learning

Learning good feature embeddings for images often requires substantial training data. As a consequence, in settings where training data is limited (e.g., few-shot and zero-shot learning), we are typically forced to use a generic feature embedding across various tasks. Ideally, we want to construct feature embeddings that are tuned for the given task. In this work, we propose Task-Aware Feature Embedding Networks (TAFE-Nets) to learn how to adapt the image representation to a new task in a meta learning fashion. Our network is composed of a meta learner and a prediction network. Based on a task input, the meta learner generates parameters for the feature layers in the prediction network so that the feature embedding can be accurately adjusted for that task. We show that TAFE-Net is highly effective in generalizing to new tasks or concepts and evaluate the TAFE-Net on a range of benchmarks in zero-shot and few-shot learning. Our model matches or exceeds the state-of-the-art on all tasks. In particular, our approach improves the prediction accuracy of unseen attribute-object pairs by 4 to 15 points on the challenging visual attribute-object composition task.

PDF Abstract CVPR 2019 PDF CVPR 2019 Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Few-Shot Image Classification aPY - 0-Shot TAFE-Net Accuracy 42.2 # 1
Few-Shot Image Classification AWA1 - 0-Shot TAFE-Net Accuracy 70.8 # 1
Few-Shot Image Classification AWA2 - 0-Shot TAFE-Net Accuracy 69.3 # 1
Few-Shot Image Classification CUB-200 - 0-Shot Learning TAFE-Net Accuracy 56.9% # 1
Few-Shot Image Classification SUN - 0-Shot TAFE-Net Accuracy 60.9% # 2


No methods listed for this paper. Add relevant methods here