Search Results for author:
Found 37 papers, 6 papers with code
Sparse Covariance Modeling in High Dimensions with Gaussian Processes
This paper studies statistical relationships among components of high-dimensional observations varying across non-random covariates.
Simulation-based stacking
Simulation-based inference has been popular for amortized Bayesian computation.
Provable convergence guarantees for black-box variational inference
Black-box variational inference is widely used in situations where there is no proof that its stochastic optimization succeeds.
Discriminative calibration: Check Bayesian computation from simulations and flexible classifier
To check the accuracy of Bayesian computations, it is common to use rank-based simulation-based calibration (SBC).
Sample Average Approximation for Black-Box VI
We present a novel approach for black-box VI that bypasses the difficulties of stochastic gradient ascent, including the task of selecting step-sizes.
U-Statistics for Importance-Weighted Variational Inference
We propose the use of U-statistics to reduce variance for gradient estimation in importance-weighted variational inference.
Joint control variate for faster black-box variational inference
Black-box variational inference performance is sometimes hindered by the use of gradient estimators with high variance.
Langevin Diffusion Variational Inference
In fact, using our formulation we propose a new method that combines the strengths of previously existing algorithms; it uses underdamped Langevin transitions and powerful augmentations parameterized by a score network.
Variational Inference with Locally Enhanced Bounds for Hierarchical Models
Hierarchical models represent a challenging setting for inference algorithms.
Amortized Variational Inference for Simple Hierarchical Models
It is difficult to use subsampling with variational inference in hierarchical models since the number of local latent variables scales with the dataset.
Variational Marginal Particle Filters
We reveal that the marginal particle filter is obtained from sequential Monte Carlo by applying Rao-Blackwellization operations, which sacrifices the trajectory information for reduced variance and differentiability.
MCMC Variational Inference via Uncorrected Hamiltonian Annealing
Given an unnormalized target distribution we want to obtain approximate samples from it and a tight lower bound on its (log) normalization constant log Z. Annealed Importance Sampling (AIS) with Hamiltonian MCMC is a powerful method that can be used to do this.
Empirical Evaluation of Biased Methods for Alpha Divergence Minimization
In this paper we empirically evaluate biased methods for alpha-divergence minimization.
An Easy to Interpret Diagnostic for Approximate Inference: Symmetric Divergence Over Simulations
It is important to estimate the errors of probabilistic inference algorithms.
On the Difficulty of Unbiased Alpha Divergence Minimization
In this work we study unbiased methods for alpha-divergence minimization through the Signal-to-Noise Ratio (SNR) of the gradient estimator.
Approximation Based Variance Reduction for Reparameterization Gradients
Flexible variational distributions improve variational inference but are harder to optimize.
Advances in Black-Box VI: Normalizing Flows, Importance Weighting, and Optimization
The combination of these algorithmic components significantly advances the state-of-the-art "out of the box" variational inference.
Moment-Matching Conditions for Exponential Families with Conditioning or Hidden Data
Maximum likelihood learning with exponential families leads to moment-matching of the sufficient statistics, a classic result.
Thompson Sampling and Approximate Inference
We study the effects of approximate inference on the performance of Thompson sampling in the $k$-armed bandit problems.
A Rule for Gradient Estimator Selection, with an Application to Variational Inference
Inspired by this principle, we propose a technique to automatically select an estimator when a finite pool of estimators is given.
Automatically Trading off Time and Variance when Selecting Gradient Estimators
Inspired by this principle, we propose a technique to automatically select an estimator when a finite pool of estimators is given.
Thompson Sampling with Approximate Inference
We study the effects of approximate inference on the performance of Thompson sampling in the $k$-armed bandit problems.
Divide and Couple: Using Monte Carlo Variational Objectives for Posterior Approximation
Recent work in variational inference (VI) uses ideas from Monte Carlo estimation to tighten the lower bounds on the log-likelihood that are used as objectives.
Provable Gradient Variance Guarantees for Black-Box Variational Inference
Recent variational inference methods use stochastic gradient estimators whose variance is not well understood.
Provable Smoothness Guarantees for Black-Box Variational Inference
Black-box variational inference tries to approximate a complex target distribution though a gradient-based optimization of the parameters of a simpler distribution.
Using Large Ensembles of Control Variates for Variational Inference
Variational inference is increasingly being addressed with stochastic optimization.
Importance Weighting and Variational Inference
Recent work used importance sampling ideas for better variational bounds on likelihoods.
Conditional Inference in Pre-trained Variational Autoencoders via Cross-coding
Variational Autoencoders (VAEs) are a popular generative model, but one in which conditional inference can be challenging.
A Divergence Bound for Hybrids of MCMC and Variational Inference and an Application to Langevin Dynamics and SGVI
Two popular classes of methods for approximate inference are Markov chain Monte Carlo (MCMC) and variational inference.
Reflection, Refraction, and Hamiltonian Monte Carlo
Hamiltonian Monte Carlo (HMC) is a successful approach for sampling from continuous densities.
Clamping Improves TRW and Mean Field Approximations
We examine the effect of clamping variables for approximate inference in undirected graphical models with pairwise relationships and discrete variables.
Maximum Likelihood Learning With Arbitrary Treewidth via Fast-Mixing Parameter Sets
This paper proves that for any exponential family with bounded sufficient statistics, (not just graphical models) when parameters are constrained to a fast-mixing set, gradient descent with gradients approximated by sampling will approximate the maximum likelihood solution inside the set with high-probability.
Projecting Markov Random Field Parameters for Fast Mixing
Markov chain Monte Carlo (MCMC) algorithms are simple and extremely powerful techniques to sample from almost arbitrary distributions.
Finito: A Faster, Permutable Incremental Gradient Method for Big Data Problems
Recent advances in optimization theory have shown that smooth strongly convex finite sums can be minimized faster than by treating them as a black box "batch" problem.
Structured Learning via Logistic Regression
A successful approach to structured learning is to write the learning objective as a joint function of linear parameters and inference messages, and iterate between updates to each.
Projecting Ising Model Parameters for Fast Mixing
Inference in general Ising models is difficult, due to high treewidth making tree-based algorithms intractable.
Implicit Differentiation by Perturbation
This paper proposes a simple and efficient finite difference method for implicit differentiation of marginal inference results in discrete graphical models.
