HyperNetworks

27 Sep 2016 David Ha Andrew Dai Quoc V. Le

This work explores hypernetworks: an approach of using a one network, also known as a hypernetwork, to generate the weights for another network. Hypernetworks provide an abstraction that is similar to what is found in nature: the relationship between a genotype - the hypernetwork - and a phenotype - the main network... (read more)

PDF Abstract
TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK BENCHMARK
Language Modelling Penn Treebank (Character Level) 2-layer Norm HyperLSTM Bit per Character (BPC) 1.219 # 11
Number of params 14.4M # 4

Results from Other Papers


TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK SOURCE PAPER COMPARE
Language Modelling enwik8 Hypernetworks Bit per Character (BPC) 1.34 # 26
Number of params 27M # 21

Methods used in the Paper