Search Results for author:
Found 19 papers, 11 papers with code
Disentangled Representations via Synergy Minimization
Scientists often seek simplified representations of complex systems to facilitate prediction and understanding.
Auto-Encoding Total Correlation Explanation
Advances in unsupervised learning enable reconstruction and generation of samples from complex distributions, but this success is marred by the inscrutability of the representations learned.
Invariant Representations without Adversarial Training
Representations of data that are invariant to changes in specified factors are useful for a wide range of problems: removing potential biases in prediction problems, controlling the effects of covariates, and disentangling meaningful factors of variation.
Exact Rate-Distortion in Autoencoders via Echo Noise
The noise is constructed in a data-driven fashion that does not require restrictive distributional assumptions.
Discovery and Separation of Features for Invariant Representation Learning
Supervised machine learning models often associate irrelevant nuisance factors with the prediction target, which hurts generalization.
All in the Exponential Family: Bregman Duality in Thermodynamic Variational Inference
We propose to choose intermediate distributions using equal spacing in the moment parameters of our exponential family, which matches grid search performance and allows the schedule to adaptively update over the course of training.
Gaussian Process Bandit Optimization of the Thermodynamic Variational Objective
Achieving the full promise of the Thermodynamic Variational Objective (TVO), a recently proposed variational lower bound on the log evidence involving a one-dimensional Riemann integral approximation, requires choosing a "schedule" of sorted discretization points.
Annealed Importance Sampling with q-Paths
Annealed importance sampling (AIS) is the gold standard for estimating partition functions or marginal likelihoods, corresponding to importance sampling over a path of distributions between a tractable base and an unnormalized target.
Likelihood Ratio Exponential Families
The exponential family is well known in machine learning and statistical physics as the maximum entropy distribution subject to a set of observed constraints, while the geometric mixture path is common in MCMC methods such as annealed importance sampling.
Stochastic Approximation of Gaussian Free Energy for Risk-Sensitive Reinforcement Learning
Since the Gaussian free energy is known to be a certainty-equivalent sensitive to the mean and the variance, the learning rule has applications in risk-sensitive decision-making.
q-Paths: Generalizing the Geometric Annealing Path using Power Means
Many common machine learning methods involve the geometric annealing path, a sequence of intermediate densities between two distributions of interest constructed using the geometric average.
Model-Free Risk-Sensitive Reinforcement Learning
Since the Gaussian free energy is known to be a certainty-equivalent sensitive to the mean and the variance, the learning rule has applications in risk-sensitive decision-making.
Your Policy Regularizer is Secretly an Adversary
Policy regularization methods such as maximum entropy regularization are widely used in reinforcement learning to improve the robustness of a learned policy.
Variational Representations of Annealing Paths: Bregman Information under Monotonic Embedding
Markov Chain Monte Carlo methods for sampling from complex distributions and estimating normalization constants often simulate samples from a sequence of intermediate distributions along an annealing path, which bridges between a tractable initial distribution and a target density of interest.
Action Matching: Learning Stochastic Dynamics from Samples
Learning the continuous dynamics of a system from snapshots of its temporal marginals is a problem which appears throughout natural sciences and machine learning, including in quantum systems, single-cell biological data, and generative modeling.
Information-Theoretic Diffusion
Denoising diffusion models have spurred significant gains in density modeling and image generation, precipitating an industrial revolution in text-guided AI art generation.
Improving Mutual Information Estimation with Annealed and Energy-Based Bounds
Since accurate estimation of MI without density information requires a sample size exponential in the true MI, we assume either a single marginal or the full joint density information is known.
A Computational Framework for Solving Wasserstein Lagrangian Flows
The dynamical formulation of the optimal transport can be extended through various choices of the underlying geometry ($\textit{kinetic energy}$), and the regularization of density paths ($\textit{potential energy}$).
All in the (Exponential) Family: Information Geometry and Thermodynamic Variational Inference
While the Evidence Lower Bound (ELBO) has become a ubiquitous objective for variational inference, the recently proposed Thermodynamic Variational Objective (TVO) leverages thermodynamic integration to provide a tighter and more general family of bounds.
