Search Results for author:
Found 42 papers, 8 papers with code
Black-Box Variational Inference as a Parametric Approximation to Langevin Dynamics
Variational inference (VI) and Markov chain Monte Carlo (MCMC) are approximate posterior inference algorithms that are often said to have complementary strengths, with VI being fast but biased and MCMC being slower but asymptotically unbiased.
Nearly Dimension-Independent Convergence of Mean-Field Black-Box Variational Inference
We prove that, given a mean-field location-scale variational family, black-box variational inference (BBVI) with the reparametrization gradient converges at an almost dimension-independent rate.
Purifying Approximate Differential Privacy with Randomized Post-processing
We propose a framework to convert $(\varepsilon, \delta)$-approximate Differential Privacy (DP) mechanisms into $(\varepsilon, 0)$-pure DP mechanisms, a process we call ``purification''.
Learning to Steer Learners in Games
We consider the problem of learning to exploit learning algorithms through repeated interactions in games.
Discovering Latent Structural Causal Models from Spatio-Temporal Data
Many important phenomena in scientific fields such as climate, neuroscience, and epidemiology are naturally represented as spatiotemporal gridded data with complex interactions.
Functional-level Uncertainty Quantification for Calibrated Fine-tuning on LLMs
To address these limitations, we propose Functional-Level Uncertainty Quantification for Calibrated Fine-Tuning (UQ4CT), which captures and calibrates epistemic uncertainty over the space of functions that map input prompts to outputs.
A Skewness-Based Criterion for Addressing Heteroscedastic Noise in Causal Discovery
Real-world data often violates the equal-variance assumption (homoscedasticity), making it essential to account for heteroscedastic noise in causal discovery.
Log-concave Sampling from a Convex Body with a Barrier: a Robust and Unified Dikin Walk
We design a walk that mixes in $\widetilde O((nd+dL^2R^2)\log(w/\delta))$ steps with a per iteration cost of $\widetilde O(n^\omega+n^2d^{3\omega-5})$.
Diffusion-BBO: Diffusion-Based Inverse Modeling for Online Black-Box Optimization
Recently, inverse modeling approaches that map the objective space to the design space with conditional diffusion models have demonstrated impressive capability in learning the data manifold.
Demystifying SGD with Doubly Stochastic Gradients
For these problems, a popular strategy is to employ SGD with doubly stochastic gradients (doubly SGD): the expectations are estimated using the gradient estimator of each component, while the sum is estimated by subsampling over these estimators.
Faster Sampling via Stochastic Gradient Proximal Sampler
Stochastic gradients have been widely integrated into Langevin-based methods to improve their scalability and efficiency in solving large-scale sampling problems.
Reverse Transition Kernel: A Flexible Framework to Accelerate Diffusion Inference
To generate data from trained diffusion models, most inference algorithms, such as DDPM, DDIM, and other variants, rely on discretizing the reverse SDEs or their equivalent ODEs.
Multi-Fidelity Residual Neural Processes for Scalable Surrogate Modeling
To address these limitations, we propose Multi-fidelity Residual Neural Processes (MFRNP), a novel multi-fidelity surrogate modeling framework.
Diffusion Models as Constrained Samplers for Optimization with Unknown Constraints
To constrain the optimization process to the data manifold, we reformulate the original optimization problem as a sampling problem from the product of the Boltzmann distribution defined by the objective function and the data distribution learned by the diffusion model.
Learning Granger Causality from Instance-wise Self-attentive Hawkes Processes
In particular, we are interested in discovering instance-level causal structures in an unsupervised manner.
Tractable MCMC for Private Learning with Pure and Gaussian Differential Privacy
Posterior sampling, i. e., exponential mechanism to sample from the posterior distribution, provides $\varepsilon$-pure differential privacy (DP) guarantees and does not suffer from potentially unbounded privacy breach introduced by $(\varepsilon,\delta)$-approximate DP.
Discovering Mixtures of Structural Causal Models from Time Series Data
In this work, we relax this assumption and perform causal discovery from time series data originating from a mixture of causal models.
Optimization on Pareto sets: On a theory of multi-objective optimization
But as the set of Pareto optimal vectors can be very large, we further consider a more practically significant Pareto-constrained optimization problem, where the goal is to optimize a preference function constrained to the Pareto set.
Langevin Thompson Sampling with Logarithmic Communication: Bandits and Reinforcement Learning
Thompson sampling (TS) is widely used in sequential decision making due to its ease of use and appealing empirical performance.
On the Convergence of Black-Box Variational Inference
We provide the first convergence guarantee for full black-box variational inference (BBVI), also known as Monte Carlo variational inference.
Statistical and Computational Trade-offs in Variational Inference: A Case Study in Inferential Model Selection
Focusing on Gaussian inferential models (or variational approximating families) with diagonal plus low-rank precision matrices, we initiate a theoretical study of the trade-offs in two aspects, Bayesian posterior inference error and frequentist uncertainty quantification error.
Multi-fidelity Hierarchical Neural Processes
MF-HNP is flexible enough to handle non-nested high dimensional data at different fidelity levels with varying input and output dimensions.
Dimension Independent Generalization of DP-SGD for Overparameterized Smooth Convex Optimization
This paper considers the generalization performance of differentially private convex learning.
On Optimal Early Stopping: Over-informative versus Under-informative Parametrization
We demonstrate experimentally that our theoretical results on optimal early stopping time corresponds to the training process of deep neural networks.
On Convergence of Federated Averaging Langevin Dynamics
We develop theoretical guarantees for FA-LD for strongly log-concave distributions with non-i. i. d data and study how the injected noise and the stochastic-gradient noise, the heterogeneity of data, and the varying learning rates affect the convergence.
When is the Convergence Time of Langevin Algorithms Dimension Independent? A Composite Optimization Viewpoint
There has been a surge of works bridging MCMC sampling and optimization, with a specific focus on translating non-asymptotic convergence guarantees for optimization problems into the analysis of Langevin algorithms in MCMC sampling.
Deep Bayesian Active Learning for Accelerating Stochastic Simulation
We propose Interactive Neural Process (INP), a deep Bayesian active learning framework for learning deep surrogate models to accelerate stochastic simulations.
Quantifying Uncertainty in Deep Spatiotemporal Forecasting
Deep learning is gaining increasing popularity for spatiotemporal forecasting.
DeepGLEAM: A hybrid mechanistic and deep learning model for COVID-19 forecasting
We introduce DeepGLEAM, a hybrid model for COVID-19 forecasting.
Efficient and Scalable Bayesian Neural Nets with Rank-1 Factors
Bayesian neural networks (BNNs) demonstrate promising success in improving the robustness and uncertainty quantification of modern deep learning.
On Thompson Sampling with Langevin Algorithms
The resulting approximate Thompson sampling algorithm has logarithmic regret and its computational complexity does not scale with the time horizon of the algorithm.
High-Order Langevin Diffusion Yields an Accelerated MCMC Algorithm
We propose a Markov chain Monte Carlo (MCMC) algorithm based on third-order Langevin dynamics for sampling from distributions with log-concave and smooth densities.
Bayesian Robustness: A Nonasymptotic Viewpoint
We study the problem of robustly estimating the posterior distribution for the setting where observed data can be contaminated with potentially adversarial outliers.
Is There an Analog of Nesterov Acceleration for MCMC?
We formulate gradient-based Markov chain Monte Carlo (MCMC) sampling as optimization on the space of probability measures, with Kullback-Leibler (KL) divergence as the objective functional.
Sampling Can Be Faster Than Optimization
Optimization algorithms and Monte Carlo sampling algorithms have provided the computational foundations for the rapid growth in applications of statistical machine learning in recent years.
Stochastic Gradient MCMC for State Space Models
However, inference in SSMs is often computationally prohibitive for long time series.
Deep Mixture of Experts via Shallow Embedding
Larger networks generally have greater representational power at the cost of increased computational complexity.
On the Theory of Variance Reduction for Stochastic Gradient Monte Carlo
We provide convergence guarantees in Wasserstein distance for a variety of variance-reduction methods: SAGA Langevin diffusion, SVRG Langevin diffusion and control-variate underdamped Langevin diffusion.
Estimate exponential memory decay in Hidden Markov Model and its applications
Inference in hidden Markov model has been challenging in terms of scalability due to dependencies in the observation data.
Stochastic Gradient MCMC Methods for Hidden Markov Models
Stochastic gradient MCMC (SG-MCMC) algorithms have proven useful in scaling Bayesian inference to large datasets under an assumption of i. i. d data.
A Complete Recipe for Stochastic Gradient MCMC
That is, any continuous Markov process that provides samples from the target distribution can be written in our framework.
