Search Results for author:
Found 19 papers, 5 papers with code
Learning Sparse Codes with Entropy-Based ELBOs
Our main contributions are theoretical, however, and they are twofold: (1) for non-trivial posterior approximations, we provide the (to the knowledge of the authors) first analytical ELBO objective for standard probabilistic sparse coding; and (2) we provide the first demonstration on how a recently shown convergence of the ELBO to entropy sums can be used for learning.
On the Convergence of the ELBO to Entropy Sums
In this purely theoretical contribution, we show that (for a very large class of generative models) the variational lower bound is at all stationary points of learning equal to a sum of entropies.
Evolutionary Variational Optimization of Generative Models
In general, our investigations highlight the importance of research on optimization methods for generative models to achieve performance improvements.
Direct Evolutionary Optimization of Variational Autoencoders With Binary Latents
Discrete latent variables are considered important for real world data, which has motivated research on Variational Autoencoders (VAEs) with discrete latents.
The ELBO of Variational Autoencoders Converges to a Sum of Three Entropies
Here we show that for standard (i. e., Gaussian) VAEs the ELBO converges to a value given by the sum of three entropies: the (negative) entropy of the prior distribution, the expected (negative) entropy of the observable distribution, and the average entropy of the variational distributions (the latter is already part of the ELBO).
Generic Unsupervised Optimization for a Latent Variable Model With Exponential Family Observables
Here, we consider LVMs that are defined for a range of different distributions, i. e., observables can follow any (regular) distribution of the exponential family.
ProSper -- A Python Library for Probabilistic Sparse Coding with Non-Standard Priors and Superpositions
The library widens the scope of dictionary learning approaches beyond implementations of standard approaches such as ICA, NMF or standard L1 sparse coding.
Large Scale Clustering with Variational EM for Gaussian Mixture Models
The aim of the project (which resulted in this arXiv version and the later TPAMI paper) is the exploration of the current efficiency and large-scale limits in fitting a parametric model for clustering to data distributions.
Truncated Variational Sampling for "Black Box" Optimization of Generative Models
We investigate the optimization of two probabilistic generative models with binary latent variables using a novel variational EM approach.
Can clustering scale sublinearly with its clusters? A variational EM acceleration of GMMs and $k$-means
The basic idea is to use a partial variational E-step which reduces the linear complexity of $\mathcal{O}(NCD)$ required for a full E-step to a sublinear complexity.
$k$-means as a variational EM approximation of Gaussian mixture models
We show that $k$-means (Lloyd's algorithm) is obtained as a special case when truncated variational EM approximations are applied to Gaussian Mixture Models (GMM) with isotropic Gaussians.
Truncated Variational EM for Semi-Supervised Neural Simpletrons
Inference and learning for probabilistic generative networks is often very challenging and typically prevents scalability to as large networks as used for deep discriminative approaches.
Select-and-Sample for Spike-and-Slab Sparse Coding
As example model we use spike-and-slab sparse coding for V1 processing, and combine latent subspace selection with Gibbs sampling (select-and-sample).
Neurons Equipped with Intrinsic Plasticity Learn Stimulus Intensity Statistics
We introduce a novel generative mixture model that accounts for the class-specific statistics of stimulus intensities, and we derive a neural circuit that learns the input classes and their intensities.
Truncated Variational Expectation Maximization
The specific structure of truncated distributions allows for deriving novel and mathematically grounded results, which in turn can be used to formulate novel efficient algorithms to optimize the parameters of probabilistic generative models.
Neural Simpletrons - Minimalistic Directed Generative Networks for Learning with Few Labels
This results in powerful though very complex models that are hard to train and that demand additional labels for optimal parameter tuning, which are often not given when labeled data is very sparse.
What Are the Invariant Occlusive Components of Image Patches? A Probabilistic Generative Approach
By far most approaches to unsupervised learning learning of visual features, such as sparse coding or ICA, account for translations by representing the same features at different positions.
A Truncated EM Approach for Spike-and-Slab Sparse Coding
We investigate two approaches to optimize the parameters of spike-and-slab sparse coding: a novel truncated EM approach and, for comparison, an approach based on standard factored variational distributions.
Occlusive Components Analysis
We show that the object parameters can be learnt from an unlabelled set of images in which objects occlude one another.
