All you need is a good init

ICLR 2015 Dmytro MishkinJiri Matas

Layer-sequential unit-variance (LSUV) initialization - a simple method for weight initialization for deep net learning - is proposed. The method consists of the two steps... (read more)

PDF Abstract

Results from the Paper


TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK RESULT BENCHMARK
Image Classification CIFAR-10 Fitnet4-LSUV Percentage correct 94.2 # 43
Image Classification CIFAR-100 Fitnet4-LSUV Percentage correct 72.3 # 46
Image Classification MNIST Fitnet-LSUV-SVM Percentage error 0.4 # 12

Methods used in the Paper