Block Neural Autoregressive Flow

9 Apr 2019  ·  Nicola De Cao, Ivan Titov, Wilker Aziz ·

Normalising flows (NFS) map two density functions via a differentiable bijection whose Jacobian determinant can be computed efficiently. Recently, as an alternative to hand-crafted bijections, Huang et al. (2018) proposed neural autoregressive flow (NAF) which is a universal approximator for density functions. Their flow is a neural network (NN) whose parameters are predicted by another NN. The latter grows quadratically with the size of the former and thus an efficient technique for parametrization is needed. We propose block neural autoregressive flow (B-NAF), a much more compact universal approximator of density functions, where we model a bijection directly using a single feed-forward network. Invertibility is ensured by carefully designing each affine transformation with block matrices that make the flow autoregressive and (strictly) monotone. We compare B-NAF to NAF and other established flows on density estimation and approximate inference for latent variable models. Our proposed flow is competitive across datasets while using orders of magnitude fewer parameters.

PDF Abstract

Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Density Estimation BSDS300 B-NAF Log-likelihood 157.36 # 1
Density Estimation Caltech-101 B-NAF Negative ELBO 94.91 # 2
NLL 105.42 # 1
Density Estimation Freyfaces B-NAF Negative ELBO 4.33 # 2
NLL 4.42 # 1
Density Estimation MNIST B-NAF NLL 83.59 # 3
Negative ELBO 80.71 # 2
Density Estimation OMNIGLOT B-NAF Negative ELBO 94.83 # 2
NLL 100.08 # 1
Density Estimation UCI GAS B-NAF Log-likelihood 12.06 # 1
Density Estimation UCI HEPMASS B-NAF Log-likelihood -14.71 # 1
Density Estimation UCI MINIBOONE B-NAF Log-likelihood -8.95 # 1
Density Estimation UCI POWER B-NAF Log-likelihood 0.61 # 3

Methods


No methods listed for this paper. Add relevant methods here