no code implementations • 20 Jul 2023 • Nihat Ay, Jesse van Oostrum
This article studies the Fisher-Rao gradient, also referred to as the natural gradient, of the evidence lower bound, the ELBO, which plays a crucial role within the theory of the Variational Autonecoder, the Helmholtz Machine and the Free Energy Principle.
no code implementations • 20 Dec 2022 • Jesse van Oostrum, Peter van Hintum, Nihat Ay
Variational autoencoders and Helmholtz machines use a recognition network (encoder) to approximate the posterior distribution of a generative model (decoder).
no code implementations • 30 Jun 2022 • Jesse van Oostrum, Johannes Müller, Nihat Ay
The natural gradient field is a vector field that lives on a model equipped with a distinguished Riemannian metric, e. g. the Fisher-Rao metric, and represents the direction of steepest ascent of an objective function on the model with respect to this metric.
