Maximal Causes for Exponential Family Observables

4 Mar 2020  ·  S. Hamid Mousavi, Jakob Drefs, Florian Hirschberger, Jörg Lücke ·

Latent variable models represent observed variables as parameterized functions of a set of latent variables. Examples are factor analysis or probabilistic sparse coding which assume weighted linear summations to determine the mean of Gaussian distribution for the observables. However, in many cases observables do not follow a normal distribution, and a linear summation of latents is often at odds with non-Gaussian observables (e.g., means of the Bernoulli distribution have to lie in the unit interval). Furthermore, the assumption of a linear summation model may (for many types of data) not be closely aligned with the true data generation process even for Gaussian observables. Alternative superposition models (i.e., alternative links between latents and observables) have therefore been investigated repeatedly. Here we show that using the maximization instead of summation to link latents to observables allows for the derivation of a very general and concise set of parameter update equations. Concretely, we derive a set of update equations that has the same functional form for all distributions of the exponential family. Our results consequently provide directly applicable learning equations for commonly as well as for unusually distributed data. We numerically verify our analytical results assuming standard Gaussian, Gamma, Poisson, Bernoulli and Exponential distributions. We point to some potential applications by providing different experiments on the learning of variance structure, noise type estimation, and denoising.

PDF Abstract
No code implementations yet. Submit your code now


  Add Datasets introduced or used in this paper

Results from the Paper

  Submit results from this paper to get state-of-the-art GitHub badges and help the community compare results to other papers.


No methods listed for this paper. Add relevant methods here