Maxout Networks

18 Feb 2013  ·  Ian J. Goodfellow, David Warde-Farley, Mehdi Mirza, Aaron Courville, Yoshua Bengio ·

We consider the problem of designing models to leverage a recently introduced approximate model averaging technique called dropout. We define a simple new model called maxout (so named because its output is the max of a set of inputs, and because it is a natural companion to dropout) designed to both facilitate optimization by dropout and improve the accuracy of dropout's fast approximate model averaging technique. We empirically verify that the model successfully accomplishes both of these tasks. We use maxout and dropout to demonstrate state of the art classification performance on four benchmark datasets: MNIST, CIFAR-10, CIFAR-100, and SVHN.

PDF Abstract

Results from the Paper

Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Image Classification CIFAR-10 Maxout Network (k=2) Percentage correct 90.65 # 192
Image Classification CIFAR-100 Maxout Network (k=2) Percentage correct 61.43 # 185
Image Classification MNIST Maxout Networks Percentage error 0.5 # 29
Image Classification SVHN Maxout Percentage error 2.5 # 35