Search Results for author:
Found 47 papers, 15 papers with code
Sliced-Wasserstein Flows: Nonparametric Generative Modeling via Optimal Transport and Diffusions
To the best of our knowledge, the proposed algorithm is the first nonparametric IGM algorithm with explicit theoretical guarantees.
Monte Carlo Variational Auto-Encoders
Variational auto-encoders (VAE) are popular deep latent variable models which are trained by maximizing an Evidence Lower Bound (ELBO).
Approximate Bayesian Computation with the Sliced-Wasserstein Distance
Approximate Bayesian Computation (ABC) is a popular method for approximate inference in generative models with intractable but easy-to-sample likelihood.
Local-Global MCMC kernels: the best of both worlds
Recent works leveraging learning to enhance sampling have shown promising results, in particular by designing effective non-local moves and global proposals.
Fast Approximation of the Sliced-Wasserstein Distance Using Concentration of Random Projections
The Sliced-Wasserstein distance (SW) is being increasingly used in machine learning applications as an alternative to the Wasserstein distance and offers significant computational and statistical benefits.
On Sampling with Approximate Transport Maps
Transport maps can ease the sampling of distributions with non-trivial geometries by transforming them into distributions that are easier to handle.
Tree-Based Diffusion Schrödinger Bridge with Applications to Wasserstein Barycenters
In this paper, we consider an entropic version of mOT with a tree-structured quadratic cost, i. e., a function that can be written as a sum of pairwise cost functions between the nodes of a tree.
Asymptotic Guarantees for Learning Generative Models with the Sliced-Wasserstein Distance
Minimum expected distance estimation (MEDE) algorithms have been widely used for probabilistic models with intractable likelihood functions and they have become increasingly popular due to their use in implicit generative modeling (e. g. Wasserstein generative adversarial networks, Wasserstein autoencoders).
Copula-like Variational Inference
This family is based on new copula-like densities on the hypercube with non-uniform marginals which can be sampled efficiently, i. e. with a complexity linear in the dimension of state space.
Maximum likelihood estimation of regularisation parameters in high-dimensional inverse problems: an empirical Bayesian approach. Part I: Methodology and Experiments
In this work, we propose a general empirical Bayesian method for setting regularisation parameters in imaging problems that are convex w. r. t.
Methodology Computation 62C12, 65C40, 68U10, 62F15, 65J20, 65C60, 65J22
Variational Inference of overparameterized Bayesian Neural Networks: a theoretical and empirical study
This paper studies the Variational Inference (VI) used for training Bayesian Neural Networks (BNN) in the overparameterized regime, i. e., when the number of neurons tends to infinity.
Statistical and Topological Properties of Sliced Probability Divergences
The idea of slicing divergences has been proven to be successful when comparing two probability measures in various machine learning applications including generative modeling, and consists in computing the expected value of a `base divergence' between one-dimensional random projections of the two measures.
Bridging the Gap between Constant Step Size Stochastic Gradient Descent and Markov Chains
We consider the minimization of an objective function given access to unbiased estimates of its gradient through stochastic gradient descent (SGD) with constant step-size.
Analysis of Langevin Monte Carlo via convex optimization
In this paper, we provide new insights on the Unadjusted Langevin Algorithm.
High-dimensional Bayesian inference via the Unadjusted Langevin Algorithm
We consider in this paper the problem of sampling a high-dimensional probability distribution $\pi$ having a density with respect to the Lebesgue measure on $\mathbb{R}^d$, known up to a normalization constant $x \mapsto \pi(x)= \mathrm{e}^{-U(x)}/\int_{\mathbb{R}^d} \mathrm{e}^{-U(y)} \mathrm{d} y$.
The promises and pitfalls of Stochastic Gradient Langevin Dynamics
As $N$ becomes large, we show that the SGLD algorithm has an invariant probability measure which significantly departs from the target posterior and behaves like Stochastic Gradient Descent (SGD).
Stochastic Gradient Richardson-Romberg Markov Chain Monte Carlo
We illustrate our framework on the popular Stochastic Gradient Langevin Dynamics (SGLD) algorithm and propose a novel SG-MCMC algorithm referred to as Stochastic Gradient Richardson-Romberg Langevin Dynamics (SGRRLD).
Markov Decision Process for MOOC users behavioral inference
Studies on massive open online courses (MOOCs) users discuss the existence of typical profiles and their impact on the learning process of the students.
Maximum entropy methods for texture synthesis: theory and practice
Recent years have seen the rise of convolutional neural network techniques in exemplar-based image synthesis.
MetFlow: A New Efficient Method for Bridging the Gap between Markov Chain Monte Carlo and Variational Inference
In this contribution, we propose a new computationally efficient method to combine Variational Inference (VI) with Markov Chain Monte Carlo (MCMC).
Convergence rates and approximation results for SGD and its continuous-time counterpart
This paper proposes a thorough theoretical analysis of Stochastic Gradient Descent (SGD) with non-increasing step sizes.
Convergence Analysis of Riemannian Stochastic Approximation Schemes
This paper analyzes the convergence for a large class of Riemannian stochastic approximation (SA) schemes, which aim at tackling stochastic optimization problems.
Quantitative Propagation of Chaos for SGD in Wide Neural Networks
In comparison to previous works on the subject, we consider settings in which the sequence of stepsizes in SGD can potentially depend on the number of neurons and the iterations.
Nonreversible MCMC from conditional invertible transforms: a complete recipe with convergence guarantees
This paper fills the gap by developing general tools to ensure that a class of nonreversible Markov kernels, possibly relying on complex transforms, has the desired invariance property and leads to convergent algorithms.
On the Stability of Random Matrix Product with Markovian Noise: Application to Linear Stochastic Approximation and TD Learning
This paper studies the exponential stability of random matrix products driven by a general (possibly unbounded) state space Markov chain.
On Riemannian Stochastic Approximation Schemes with Fixed Step-Size
This result gives rise to a family of stationary distributions indexed by the step-size, which is further shown to converge to a Dirac measure, concentrated at the solution of the problem at hand, as the step-size goes to 0.
Bayesian imaging using Plug & Play priors: when Langevin meets Tweedie
The proposed algorithms are demonstrated on several canonical problems such as image deblurring, inpainting, and denoising, where they are used for point estimation as well as for uncertainty visualisation and quantification.
NEO: Non Equilibrium Sampling on the Orbit of a Deterministic Transform
Sampling from a complex distribution $\pi$ and approximating its intractable normalizing constant Z are challenging problems.
QLSD: Quantised Langevin stochastic dynamics for Bayesian federated learning
The objective of Federated Learning (FL) is to perform statistical inference for data which are decentralised and stored locally on networked clients.
Tight High Probability Bounds for Linear Stochastic Approximation with Fixed Stepsize
This family of methods arises in many machine learning tasks and is used to obtain approximate solutions of a linear system $\bar{A}\theta = \bar{b}$ for which $\bar{A}$ and $\bar{b}$ can only be accessed through random estimates $\{({\bf A}_n, {\bf b}_n): n \in \mathbb{N}^*\}$.
DG-LMC: A Turn-key and Scalable Synchronous Distributed MCMC Algorithm via Langevin Monte Carlo within Gibbs
Performing reliable Bayesian inference on a big data scale is becoming a keystone in the modern era of machine learning.
NEO: Non Equilibrium Sampling on the Orbits of a Deterministic Transform
Sampling from a complex distribution $\pi$ and approximating its intractable normalizing constant $\mathrm{Z}$ are challenging problems.
On Maximum-a-Posteriori estimation with Plug & Play priors and stochastic gradient descent
Bayesian methods to solve imaging inverse problems usually combine an explicit data likelihood function with a prior distribution that explicitly models expected properties of the solution.
FedPop: A Bayesian Approach for Personalised Federated Learning
We provide non-asymptotic convergence guarantees for the proposed algorithms and illustrate their performances on various personalised federated learning tasks.
Finite-time High-probability Bounds for Polyak-Ruppert Averaged Iterates of Linear Stochastic Approximation
Our finite-time instance-dependent bounds for the averaged LSA iterates are sharp in the sense that the leading term we obtain coincides with the local asymptotic minimax limit.
Federated Averaging Langevin Dynamics: Toward a unified theory and new algorithms
This paper focuses on Bayesian inference in a federated learning context (FL).
Rosenthal-type inequalities for linear statistics of Markov chains
In this paper, we establish novel deviation bounds for additive functionals of geometrically ergodic Markov chains similar to Rosenthal and Bernstein inequalities for sums of independent random variables.
Non-asymptotic convergence bounds for Sinkhorn iterates and their gradients: a coupling approach
Our approach is novel in that it is purely probabilistic and relies on coupling by reflection techniques for controlled diffusions on the torus.
On the convergence of dynamic implementations of Hamiltonian Monte Carlo and No U-Turn Samplers
Under conditions similar to the ones existing for HMC, we also show that NUTS is geometrically ergodic.
VITS : Variational Inference Thomson Sampling for contextual bandits
In this paper, we introduce and analyze a variant of the Thompson sampling (TS) algorithm for contextual bandits.
Score diffusion models without early stopping: finite Fisher information is all you need
Our study provides a rigorous analysis, yielding simple, improved and sharp convergence bounds in KL applicable to any data distribution with finite Fisher information with respect to the standard Gaussian distribution.
Implicit Bias in Noisy-SGD: With Applications to Differentially Private Training
We first show that the phenomenon extends to Noisy-SGD (DP-SGD without clipping), suggesting that the stochasticity (and not the clipping) is the cause of this implicit bias, even with additional isotropic Gaussian noise.
Unbiased constrained sampling with Self-Concordant Barrier Hamiltonian Monte Carlo
In this paper, we propose Barrier Hamiltonian Monte Carlo (BHMC), a version of the HMC algorithm which aims at sampling from a Gibbs distribution $\pi$ on a manifold $\mathrm{M}$, endowed with a Hessian metric $\mathfrak{g}$ derived from a self-concordant barrier.
Watermarking Makes Language Models Radioactive
This paper investigates the radioactivity of LLM-generated texts, i. e. whether it is possible to detect that such input was used as training data.
Differentially Private Representation Learning via Image Captioning
In this work, we show that effective DP representation learning can be done via image captioning and scaling up to internet-scale multimodal datasets.
Incentivized Learning in Principal-Agent Bandit Games
This work considers a repeated principal-agent bandit game, where the principal can only interact with her environment through the agent.
Divide-and-Conquer Posterior Sampling for Denoising Diffusion Priors
In this work, we take a different approach and utilize the specific structure of the DDM prior to define a set of intermediate and simpler posterior sampling problems, resulting in a lower approximation error compared to previous methods.
