Search Results for author: Yi-An Ma

Found 34 papers, 5 papers with code

Black-Box Variational Inference as a Parametric Approximation to Langevin Dynamics

no code implementations ICML 2020 Matthew Hoffman, Yi-An Ma

Variational inference (VI) and Markov chain Monte Carlo (MCMC) are approximate posterior inference algorithms that are often said to have complementary strengths, with VI being fast but biased and MCMC being slower but asymptotically unbiased.

Variational Inference

Demystifying SGD with Doubly Stochastic Gradients

no code implementations3 Jun 2024 Kyurae Kim, Joohwan Ko, Yi-An Ma, Jacob R. Gardner

For these problems, a popular strategy is to employ SGD with doubly stochastic gradients (doubly SGD): the expectations are estimated using the gradient estimator of each component, while the sum is estimated by subsampling over these estimators.

Faster Sampling via Stochastic Gradient Proximal Sampler

no code implementations27 May 2024 Xunpeng Huang, Difan Zou, Yi-An Ma, Hanze Dong, Tong Zhang

Stochastic gradients have been widely integrated into Langevin-based methods to improve their scalability and efficiency in solving large-scale sampling problems.

Reverse Transition Kernel: A Flexible Framework to Accelerate Diffusion Inference

no code implementations26 May 2024 Xunpeng Huang, Difan Zou, Hanze Dong, Yi Zhang, Yi-An Ma, Tong Zhang

To generate data from trained diffusion models, most inference algorithms, such as DDPM, DDIM, and other variants, rely on discretizing the reverse SDEs or their equivalent ODEs.


Multi-Fidelity Residual Neural Processes for Scalable Surrogate Modeling

no code implementations29 Feb 2024 Ruijia Niu, Dongxia Wu, Kai Kim, Yi-An Ma, Duncan Watson-Parris, Rose Yu

Multi-fidelity surrogate modeling aims to learn an accurate surrogate at the highest fidelity level by combining data from multiple sources.

Decoder Gaussian Processes

Diffusion Models as Constrained Samplers for Optimization with Unknown Constraints

no code implementations28 Feb 2024 Lingkai Kong, Yuanqi Du, Wenhao Mu, Kirill Neklyudov, Valentin De Bortoli, Haorui Wang, Dongxia Wu, Aaron Ferber, Yi-An Ma, Carla P. Gomes, Chao Zhang

To constrain the optimization process to the data manifold, we reformulate the original optimization problem as a sampling problem from the product of the Boltzmann distribution defined by the objective function and the data distribution learned by the diffusion model.

Tractable MCMC for Private Learning with Pure and Gaussian Differential Privacy

no code implementations23 Oct 2023 Yingyu Lin, Yi-An Ma, Yu-Xiang Wang, Rachel Redberg, Zhiqi Bu

Posterior sampling, i. e., exponential mechanism to sample from the posterior distribution, provides $\varepsilon$-pure differential privacy (DP) guarantees and does not suffer from potentially unbounded privacy breach introduced by $(\varepsilon,\delta)$-approximate DP.

Discovering Mixtures of Structural Causal Models from Time Series Data

no code implementations10 Oct 2023 Sumanth Varambally, Yi-An Ma, Rose Yu

In this work, we relax this assumption and perform causal discovery from time series data originating from a mixture of causal models.

Causal Discovery Time Series +1

Optimization on Pareto sets: On a theory of multi-objective optimization

no code implementations4 Aug 2023 Abhishek Roy, Geelon So, Yi-An Ma

But as the set of Pareto optimal vectors can be very large, we further consider a more practically significant Pareto-constrained optimization problem, where the goal is to optimize a preference function constrained to the Pareto set.

Reverse Diffusion Monte Carlo

no code implementations5 Jul 2023 Xunpeng Huang, Hanze Dong, Yifan Hao, Yi-An Ma, Tong Zhang

We propose a Monte Carlo sampler from the reverse diffusion process.

Langevin Thompson Sampling with Logarithmic Communication: Bandits and Reinforcement Learning

no code implementations15 Jun 2023 Amin Karbasi, Nikki Lijing Kuang, Yi-An Ma, Siddharth Mitra

Thompson sampling (TS) is widely used in sequential decision making due to its ease of use and appealing empirical performance.

Decision Making Multi-Armed Bandits +3

On the Convergence of Black-Box Variational Inference

no code implementations NeurIPS 2023 Kyurae Kim, Jisu Oh, Kaiwen Wu, Yi-An Ma, Jacob R. Gardner

We provide the first convergence guarantee for full black-box variational inference (BBVI), also known as Monte Carlo variational inference.

Bayesian Inference Variational Inference

Statistical and Computational Trade-offs in Variational Inference: A Case Study in Inferential Model Selection

no code implementations22 Jul 2022 Kush Bhatia, Nikki Lijing Kuang, Yi-An Ma, Yixin Wang

Focusing on Gaussian inferential models (or variational approximating families) with diagonal plus low-rank precision matrices, we initiate a theoretical study of the trade-offs in two aspects, Bayesian posterior inference error and frequentist uncertainty quantification error.

Bayesian Inference Computational Efficiency +4

Multi-fidelity Hierarchical Neural Processes

1 code implementation10 Jun 2022 Dongxia Wu, Matteo Chinazzi, Alessandro Vespignani, Yi-An Ma, Rose Yu

MF-HNP is flexible enough to handle non-nested high dimensional data at different fidelity levels with varying input and output dimensions.

Epidemiology Gaussian Processes

On Optimal Early Stopping: Over-informative versus Under-informative Parametrization

no code implementations20 Feb 2022 Ruoqi Shen, Liyao Gao, Yi-An Ma

We demonstrate experimentally that our theoretical results on optimal early stopping time corresponds to the training process of deep neural networks.

On Convergence of Federated Averaging Langevin Dynamics

no code implementations9 Dec 2021 Wei Deng, Qian Zhang, Yi-An Ma, Zhao Song, Guang Lin

We develop theoretical guarantees for FA-LD for strongly log-concave distributions with non-i. i. d data and study how the injected noise and the stochastic-gradient noise, the heterogeneity of data, and the varying learning rates affect the convergence.

Uncertainty Quantification

When is the Convergence Time of Langevin Algorithms Dimension Independent? A Composite Optimization Viewpoint

no code implementations5 Oct 2021 Yoav Freund, Yi-An Ma, Tong Zhang

There has been a surge of works bridging MCMC sampling and optimization, with a specific focus on translating non-asymptotic convergence guarantees for optimization problems into the analysis of Langevin algorithms in MCMC sampling.

Deep Bayesian Active Learning for Accelerating Stochastic Simulation

1 code implementation5 Jun 2021 Dongxia Wu, Ruijia Niu, Matteo Chinazzi, Alessandro Vespignani, Yi-An Ma, Rose Yu

We propose Interactive Neural Process (INP), a deep Bayesian active learning framework for learning deep surrogate models to accelerate stochastic simulations.

Active Learning

Efficient and Scalable Bayesian Neural Nets with Rank-1 Factors

1 code implementation ICML 2020 Michael W. Dusenberry, Ghassen Jerfel, Yeming Wen, Yi-An Ma, Jasper Snoek, Katherine Heller, Balaji Lakshminarayanan, Dustin Tran

Bayesian neural networks (BNNs) demonstrate promising success in improving the robustness and uncertainty quantification of modern deep learning.

Uncertainty Quantification

On Thompson Sampling with Langevin Algorithms

no code implementations ICML 2020 Eric Mazumdar, Aldo Pacchiano, Yi-An Ma, Peter L. Bartlett, Michael. I. Jordan

The resulting approximate Thompson sampling algorithm has logarithmic regret and its computational complexity does not scale with the time horizon of the algorithm.

Thompson Sampling

High-Order Langevin Diffusion Yields an Accelerated MCMC Algorithm

no code implementations28 Aug 2019 Wenlong Mou, Yi-An Ma, Martin J. Wainwright, Peter L. Bartlett, Michael. I. Jordan

We propose a Markov chain Monte Carlo (MCMC) algorithm based on third-order Langevin dynamics for sampling from distributions with log-concave and smooth densities.

Vocal Bursts Intensity Prediction

Bayesian Robustness: A Nonasymptotic Viewpoint

no code implementations27 Jul 2019 Kush Bhatia, Yi-An Ma, Anca D. Dragan, Peter L. Bartlett, Michael. I. Jordan

We study the problem of robustly estimating the posterior distribution for the setting where observed data can be contaminated with potentially adversarial outliers.

Binary Classification regression

Is There an Analog of Nesterov Acceleration for MCMC?

no code implementations4 Feb 2019 Yi-An Ma, Niladri Chatterji, Xiang Cheng, Nicolas Flammarion, Peter Bartlett, Michael. I. Jordan

We formulate gradient-based Markov chain Monte Carlo (MCMC) sampling as optimization on the space of probability measures, with Kullback-Leibler (KL) divergence as the objective functional.

Sampling Can Be Faster Than Optimization

no code implementations20 Nov 2018 Yi-An Ma, Yuansi Chen, Chi Jin, Nicolas Flammarion, Michael. I. Jordan

Optimization algorithms and Monte Carlo sampling algorithms have provided the computational foundations for the rapid growth in applications of statistical machine learning in recent years.

Deep Mixture of Experts via Shallow Embedding

no code implementations5 Jun 2018 Xin Wang, Fisher Yu, Lisa Dunlap, Yi-An Ma, Ruth Wang, Azalia Mirhoseini, Trevor Darrell, Joseph E. Gonzalez

Larger networks generally have greater representational power at the cost of increased computational complexity.

Few-Shot Learning Zero-Shot Learning

On the Theory of Variance Reduction for Stochastic Gradient Monte Carlo

no code implementations ICML 2018 Niladri S. Chatterji, Nicolas Flammarion, Yi-An Ma, Peter L. Bartlett, Michael. I. Jordan

We provide convergence guarantees in Wasserstein distance for a variety of variance-reduction methods: SAGA Langevin diffusion, SVRG Langevin diffusion and control-variate underdamped Langevin diffusion.

Estimate exponential memory decay in Hidden Markov Model and its applications

no code implementations17 Oct 2017 Felix X. -F. Ye, Yi-An Ma, Hong Qian

Inference in hidden Markov model has been challenging in terms of scalability due to dependencies in the observation data.

Stochastic Gradient MCMC Methods for Hidden Markov Models

no code implementations ICML 2017 Yi-An Ma, Nicholas J. Foti, Emily B. Fox

Stochastic gradient MCMC (SG-MCMC) algorithms have proven useful in scaling Bayesian inference to large datasets under an assumption of i. i. d data.

Bayesian Inference

A Complete Recipe for Stochastic Gradient MCMC

no code implementations NeurIPS 2015 Yi-An Ma, Tianqi Chen, Emily B. Fox

That is, any continuous Markov process that provides samples from the target distribution can be written in our framework.

Physical Intuition

Cannot find the paper you are looking for? You can Submit a new open access paper.