Search Results for author:
Found 55 papers, 36 papers with code
Robust, Automated, and Accurate Black-box Variational Inference
RAABBVI adaptively decreases the learning rate by detecting convergence of the fixed--learning-rate iterates, then estimates the symmetrized Kullback--Leiber (KL) divergence between the current variational approximation and the optimal one.
Pathfinder: Parallel quasi-Newton variational inference
Pathfinder returns draws from the approximation with the lowest estimated Kullback-Leibler (KL) divergence to the true posterior.
Challenges for BBVI with Normalizing Flows
Current black-box variational inference (BBVI) methods require the user to make numerous design choices---such as the selection of variational objective and approximating family---yet there is little principled guidance on how to do so.
Challenges and Opportunities in High Dimensional Variational Inference
Our framework and supporting experiments help to distinguish between the behavior of BBVI methods for approximating low-dimensional versus moderate-to-high-dimensional posteriors.
Challenges and Opportunities in High-dimensional Variational Inference
Our framework and supporting experiments help to distinguish between the behavior of BBVI methods for approximating low-dimensional versus moderate-to-high-dimensional posteriors.
Bayesian hierarchical stacking: Some models are (somewhere) useful
We show that stacking is most effective when model predictive performance is heterogeneous in inputs, and we can further improve the stacked mixture with a hierarchical model.
Good practices for Bayesian Optimization of high dimensional structured spaces
The increasing availability of structured but high dimensional data has opened new opportunities for optimization.
What are the most important statistical ideas of the past 50 years?
We review the most important statistical ideas of the past half century, which we categorize as: counterfactual causal inference, bootstrapping and simulation-based inference, overparameterized models and regularization, Bayesian multilevel models, generic computation algorithms, adaptive decision analysis, robust inference, and exploratory data analysis.
Causal Inference
Methodology
Projection Predictive Inference for Generalized Linear and Additive Multilevel Models
Projection predictive inference is a decision theoretic Bayesian approach that decouples model estimation from decision making.
Methodology Computation
Adaptive Path Sampling in Metastable Posterior Distributions
The normalizing constant plays an important role in Bayesian computation, and there is a large literature on methods for computing or approximating normalizing constants that cannot be evaluated in closed form.
Computation Methodology
Robust, Accurate Stochastic Optimization for Variational Inference
We consider the problem of fitting variational posterior approximations using stochastic optimization methods.
Unbiased estimator for the variance of the leave-one-out cross-validation estimator for a Bayesian normal model with fixed variance
We show that it is possible to construct an unbiased estimator considering a specific predictive performance measure and model.
Methodology
Uncertainty in Bayesian Leave-One-Out Cross-Validation Based Model Comparison
We show that it is possible that the problematic skewness of the error distribution, which occurs when the models make similar predictions, does not fade away when the data size grows to infinity in certain situations.
Methodology
Stacking for Non-mixing Bayesian Computations: The Curse and Blessing of Multimodal Posteriors
When working with multimodal Bayesian posterior distributions, Markov chain Monte Carlo (MCMC) algorithms have difficulty moving between modes, and default variational or mode-based approximate inferences will understate posterior uncertainty.
Group Heterogeneity Assessment for Multilevel Models
Many data sets contain an inherent multilevel structure, for example, because of repeated measurements of the same observational units.
Preferential Batch Bayesian Optimization
This direction has been mainly driven by the use of BO in machine learning hyper-parameter configuration problems.
lgpr: An interpretable nonparametric method for inferring covariate effects from longitudinal data
The lgpr tool is implemented as a comprehensive and user-friendly R-package.
A Decision-Theoretic Approach for Model Interpretability in Bayesian Framework
Through experiments on real-word data sets, using decision trees as interpretable models and Bayesian additive regression models as reference models, we show that for the same level of interpretability, our approach generates more accurate models than the alternative of restricting the prior.
Uncertainty-aware Sensitivity Analysis Using Rényi Divergences
For nonlinear supervised learning models, assessing the importance of predictor variables or their interactions is not straightforward because it can vary in the domain of the variables.
Batch simulations and uncertainty quantification in Gaussian process surrogate-based approximate Bayesian computation
Surrogate models can be used to accelerate approximate Bayesian computation (ABC).
Batch simulations and uncertainty quantification in Gaussian process surrogate approximate Bayesian computation
We propose a numerical method to fully quantify the uncertainty in, for example, ABC posterior moments.
Implicitly Adaptive Importance Sampling
Adaptive importance sampling is a class of techniques for finding good proposal distributions for importance sampling.
Selecting the Metric in Hamiltonian Monte Carlo
We present a selection criterion for the Euclidean metric adapted during warmup in a Hamiltonian Monte Carlo sampler that makes it possible for a sampler to automatically pick the metric based on the model and the availability of warmup draws.
Computation Methodology
Parallel Gaussian process surrogate Bayesian inference with noisy likelihood evaluations
We consider Bayesian inference when only a limited number of noisy log-likelihood evaluations can be obtained.
Bayesian leave-one-out cross-validation for large data
Model inference, such as model comparison, model checking, and model selection, is an important part of model development.
Active Learning for Decision-Making from Imbalanced Observational Data
Machine learning can help personalized decision support by learning models to predict individual treatment effects (ITE).
Rank-normalization, folding, and localization: An improved $\widehat{R}$ for assessing convergence of MCMC
In this paper we show that the convergence diagnostic $\widehat{R}$ of Gelman and Rubin (1992) has serious flaws.
Computation Methodology
Approximate leave-future-out cross-validation for Bayesian time series models
One of the common goals of time series analysis is to use the observed series to inform predictions for future observations.
Methodology
Projective Inference in High-dimensional Problems: Prediction and Feature Selection
This paper discusses predictive inference and feature selection for generalized linear models with scarce but high-dimensional data.
Validating Bayesian Inference Algorithms with Simulation-Based Calibration
Verifying the correctness of Bayesian computation is challenging.
Methodology
Yes, but Did It Work?: Evaluating Variational Inference
While it's always possible to compute a variational approximation to a posterior distribution, it can be difficult to discover problems with this approximation.
Bayesian Estimation of Gaussian Graphical Models with Projection Predictive Selection
Here we introduce a Bayesian method for estimating sparse matrices, in which conditional relationships are determined with projection predictive selection.
Applications Methodology
Variable selection for Gaussian processes via sensitivity analysis of the posterior predictive distribution
Variable selection for Gaussian process models is often done using automatic relevance determination, which uses the inverse length-scale parameter of each input variable as a proxy for variable relevance.
Iterative Supervised Principal Components
In high-dimensional prediction problems, where the number of features may greatly exceed the number of training instances, fully Bayesian approach with a sparsifying prior is known to produce good results but is computationally challenging.
Methodology
User Modelling for Avoiding Overfitting in Interactive Knowledge Elicitation for Prediction
In human-in-the-loop machine learning, the user provides information beyond that in the training data.
Visualization in Bayesian workflow
Bayesian data analysis is about more than just computing a posterior distribution, and Bayesian visualization is about more than trace plots of Markov chains.
Methodology Applications
ELFI: Engine for Likelihood-Free Inference
The stand-alone ELFI graph can be used with any of the available inference methods without modifications.
Nudged elastic band calculations accelerated with Gaussian process regression
A Gaussian process model also provides an uncertainty estimate for the approximate energy surface, and this can be used to focus the calculations on the lesser-known part of the path, thereby reducing the number of needed energy and force evaluations to a half in the present calculations.
Using stacking to average Bayesian predictive distributions
The widely recommended procedure of Bayesian model averaging is flawed in the M-open setting in which the true data-generating process is not one of the candidate models being fit.
Methodology Computation
Correcting boundary over-exploration deficiencies in Bayesian optimization with virtual derivative sign observations
Bayesian optimization (BO) is a global optimization strategy designed to find the minimum of an expensive black-box function, typically defined on a compact subset of $\mathcal{R}^d$, by using a Gaussian process (GP) as a surrogate model for the objective.
Efficient acquisition rules for model-based approximate Bayesian computation
We propose to compute the uncertainty in the ABC posterior density, which is due to a lack of simulations to estimate this quantity accurately, and define a loss function that measures this uncertainty.
Minimum energy path calculations with Gaussian process regression
The calculation of minimum energy paths for transitions such as atomic and/or spin re-arrangements is an important task in many contexts and can often be used to determine the mechanism and rate of transitions.
Gaussian process modeling in approximate Bayesian computation to estimate horizontal gene transfer in bacteria
Approximate Bayesian computation (ABC) can be used for model fitting when the likelihood function is intractable but simulating from the model is feasible.
Chained Gaussian Processes
Gaussian process models are flexible, Bayesian non-parametric approaches to regression.
Bayesian inference for spatio-temporal spike-and-slab priors
In this work, we address the problem of solving a series of underdetermined linear inverse problems subject to a sparsity constraint.
Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC
Leave-one-out cross-validation (LOO) and the widely applicable information criterion (WAIC) are methods for estimating pointwise out-of-sample prediction accuracy from a fitted Bayesian model using the log-likelihood evaluated at the posterior simulations of the parameter values.
Computation Methodology
Pareto Smoothed Importance Sampling
Importance weighting is a general way to adjust Monte Carlo integration to account for draws from the wrong distribution, but the resulting estimate can be highly variable when the importance ratios have a heavy right tail.
Comparison of Bayesian predictive methods for model selection
From a predictive viewpoint, best results are obtained by accounting for model uncertainty by forming the full encompassing model, such as the Bayesian model averaging solution over the candidate models.
Bayesian leave-one-out cross-validation approximations for Gaussian latent variable models
The future predictive performance of a Bayesian model can be estimated using Bayesian cross-validation.
Expectation propagation as a way of life: A framework for Bayesian inference on partitioned data
A common divide-and-conquer approach for Bayesian computation with big data is to partition the data, perform local inference for each piece separately, and combine the results to obtain a global posterior approximation.
Approximate Inference for Nonstationary Heteroscedastic Gaussian process Regression
This paper presents a novel approach for approximate integration over the uncertainty of noise and signal variances in Gaussian process (GP) regression.
Expectation Propagation for Neural Networks with Sparsity-promoting Priors
Comparisons are made to two alternative models with ARD priors: a Gaussian process with a NN covariance function and marginal maximum a posteriori estimates of the relevance parameters, and a NN with Markov chain Monte Carlo integration over all the unknown model parameters.
Laplace approximation for logistic Gaussian process density estimation and regression
Logistic Gaussian process (LGP) priors provide a flexible alternative for modelling unknown densities.
Bayesian Modeling with Gaussian Processes using the GPstuff Toolbox
The prior over functions is defined implicitly by the mean and covariance function, which determine the smoothness and variability of the function.
Gaussian process regression with Student-t likelihood
In this work, we discuss the properties of a Gaussian process regression model with the Student-t likelihood and utilize the Laplace approximation for approximate inference.
