Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
We propose an algorithm for meta-learning that is model-agnostic, in the sense that it is compatible with any model trained with gradient descent and applicable to a variety of different learning problems, including classification, regression, and reinforcement learning. The goal of meta-learning is to train a model on a variety of learning tasks, such that it can solve new learning tasks using only a small number of training samples. In our approach, the parameters of the model are explicitly trained such that a small number of gradient steps with a small amount of training data from a new task will produce good generalization performance on that task. In effect, our method trains the model to be easy to fine-tune. We demonstrate that this approach leads to state-of-the-art performance on two few-shot image classification benchmarks, produces good results on few-shot regression, and accelerates fine-tuning for policy gradient reinforcement learning with neural network policies.
PDF Abstract ICML 2017 PDF ICML 2017 AbstractCode
Results from the Paper
Task | Dataset | Model | Metric Name | Metric Value | Global Rank | Benchmark |
---|---|---|---|---|---|---|
Few-Shot Image Classification | Dirichlet Mini-Imagenet (5-way, 1-shot) | MAML | 1:1 Accuracy | 47.6 | # 12 | |
Few-Shot Image Classification | Dirichlet Mini-Imagenet (5-way, 5-shot) | MAML | 1:1 Accuracy | 64.5 | # 11 | |
Few-Shot Image Classification | Meta-Dataset | fo-MAML | Accuracy | 57.024 | # 17 | |
Few-Shot Image Classification | Meta-Dataset Rank | fo-MAML | Mean Rank | 10.25 | # 10 | |
Few-Shot Image Classification | Mini-Imagenet 10-way (1-shot) | MAML | Accuracy | 31.3 | # 13 | |
Few-Shot Image Classification | Mini-Imagenet 10-way (1-shot) | MAML + Transduction | Accuracy | 31.8 | # 12 | |
Few-Shot Image Classification | Mini-Imagenet 10-way (5-shot) | MAML | Accuracy | 46.9 | # 13 | |
Few-Shot Image Classification | Mini-Imagenet 10-way (5-shot) | MAML + Transduction | Accuracy | 48.2 | # 10 | |
Few-Shot Image Classification | OMNIGLOT - 1-Shot, 5-way | MAML | Accuracy | 98.7 | # 10 | |
Few-Shot Image Classification | OMNIGLOT - 5-Shot, 5-way | MAML | Accuracy | 99.9 | # 2 | |
Few-Shot Image Classification | Tiered ImageNet 10-way (1-shot) | MAML | Accuracy | 34.4 | # 12 | |
Few-Shot Image Classification | Tiered ImageNet 10-way (1-shot) | MAML + Transduction | Accuracy | 34.8 | # 11 | |
Few-Shot Image Classification | Tiered ImageNet 10-way (5-shot) | MAML | Accuracy | 53.3 | # 11 | |
Few-Shot Image Classification | Tiered ImageNet 10-way (5-shot) | MAML + Transduction | Accuracy | 54.7 | # 10 |