Meta-Curvature

NeurIPS 2019  ·  Eunbyung Park, Junier B. Oliva ·

We propose meta-curvature (MC), a framework to learn curvature information for better generalization and fast model adaptation. MC expands on the model-agnostic meta-learner (MAML) by learning to transform the gradients in the inner optimization such that the transformed gradients achieve better generalization performance to a new task... For training large scale neural networks, we decompose the curvature matrix into smaller matrices in a novel scheme where we capture the dependencies of the model's parameters with a series of tensor products. We demonstrate the effects of our proposed method on several few-shot learning tasks and datasets. Without any task specific techniques and architectures, the proposed method achieves substantial improvement upon previous MAML variants and outperforms the recent state-of-the-art methods. Furthermore, we observe faster convergence rates of the meta-training process. Finally, we present an analysis that explains better generalization performance with the meta-trained curvature. read more

PDF Abstract NeurIPS 2019 PDF NeurIPS 2019 Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Few-Shot Image Classification Mini-Imagenet 5-way (1-shot) MC2+ Accuracy 55.73 # 50
Few-Shot Image Classification Mini-Imagenet 5-way (5-shot) MC2+ Accuracy 70.33 # 52
Few-Shot Image Classification OMNIGLOT - 1-Shot, 20-way MC2+ Accuracy 88% # 18
Few-Shot Image Classification OMNIGLOT - 1-Shot, 5-way MC2+ Accuracy 99.97 # 1
Few-Shot Image Classification OMNIGLOT - 5-Shot, 20-way MC2+ Accuracy 99.65% # 1
Few-Shot Image Classification OMNIGLOT - 5-Shot, 5-way MC2+ Accuracy 99.89 # 4

Methods