Search Results for author: Valentin De Bortoli

Found 34 papers, 17 papers with code

Dynamical Regimes of Diffusion Models

no code implementations28 Feb 2024 Giulio Biroli, Tony Bonnaire, Valentin De Bortoli, Marc Mézard

Using statistical physics methods, we study generative diffusion models in the regime where the dimension of space and the number of data are large, and the score function has been trained optimally.

Target Score Matching

no code implementations13 Feb 2024 Valentin De Bortoli, Michael Hutchinson, Peter Wirnsberger, Arnaud Doucet

Denoising Score Matching estimates the score of a noised version of a target distribution by minimizing a regression loss and is widely used to train the popular class of Denoising Diffusion Models.

Denoising regression

Augmented Bridge Matching

no code implementations12 Nov 2023 Valentin De Bortoli, Guan-Horng Liu, Tianrong Chen, Evangelos A. Theodorou, Weilie Nie

In this paper, we highlight that while flow and bridge matching processes preserve the information of the marginal distributions, they do \emph{not} necessarily preserve the coupling information unless additional, stronger optimality conditions are met.

Particle Guidance: non-I.I.D. Diverse Sampling with Diffusion Models

1 code implementation19 Oct 2023 Gabriele Corso, Yilun Xu, Valentin De Bortoli, Regina Barzilay, Tommi Jaakkola

In light of the widespread success of generative models, a significant amount of research has gone into speeding up their sampling time.

Conditional Image Generation

Nearly $d$-Linear Convergence Bounds for Diffusion Models via Stochastic Localization

no code implementations7 Aug 2023 Joe Benton, Valentin De Bortoli, Arnaud Doucet, George Deligiannidis

We provide the first convergence bounds which are linear in the data dimension (up to logarithmic factors) assuming only finite second moments of the data distribution.

Denoising

Unbalanced Diffusion Schrödinger Bridge

1 code implementation15 Jun 2023 Matteo Pariset, Ya-Ping Hsieh, Charlotte Bunne, Andreas Krause, Valentin De Bortoli

Schr\"odinger bridges (SBs) provide an elegant framework for modeling the temporal evolution of populations in physical, chemical, or biological systems.

Tree-Based Diffusion Schrödinger Bridge with Applications to Wasserstein Barycenters

1 code implementation NeurIPS 2023 Maxence Noble, Valentin De Bortoli, Arnaud Doucet, Alain Durmus

In this paper, we consider an entropic version of mOT with a tree-structured quadratic cost, i. e., a function that can be written as a sum of pairwise cost functions between the nodes of a tree.

Diffusion Models for Constrained Domains

1 code implementation11 Apr 2023 Nic Fishman, Leo Klarner, Valentin De Bortoli, Emile Mathieu, Michael Hutchinson

Denoising diffusion models are a novel class of generative algorithms that achieve state-of-the-art performance across a range of domains, including image generation and text-to-image tasks.

Denoising Image Generation +2

Diffusion Schrödinger Bridge Matching

no code implementations NeurIPS 2023 Yuyang Shi, Valentin De Bortoli, Andrew Campbell, Arnaud Doucet

However, while it is desirable in many applications to approximate the deterministic dynamic Optimal Transport (OT) map which admits attractive properties, DDMs and FMMs are not guaranteed to provide transports close to the OT map.

Denoising

SE(3) diffusion model with application to protein backbone generation

1 code implementation5 Feb 2023 Jason Yim, Brian L. Trippe, Valentin De Bortoli, Emile Mathieu, Arnaud Doucet, Regina Barzilay, Tommi Jaakkola

The design of novel protein structures remains a challenge in protein engineering for applications across biomedicine and chemistry.

Protein Structure Prediction

From Denoising Diffusions to Denoising Markov Models

1 code implementation7 Nov 2022 Joe Benton, Yuyang Shi, Valentin De Bortoli, George Deligiannidis, Arnaud Doucet

We propose a unifying framework generalising this approach to a wide class of spaces and leading to an original extension of score matching.

Denoising

Unbiased constrained sampling with Self-Concordant Barrier Hamiltonian Monte Carlo

1 code implementation NeurIPS 2023 Maxence Noble, Valentin De Bortoli, Alain Durmus

In this paper, we propose Barrier Hamiltonian Monte Carlo (BHMC), a version of the HMC algorithm which aims at sampling from a Gibbs distribution $\pi$ on a manifold $\mathrm{M}$, endowed with a Hessian metric $\mathfrak{g}$ derived from a self-concordant barrier.

Spectral Diffusion Processes

no code implementations28 Sep 2022 Angus Phillips, Thomas Seror, Michael Hutchinson, Valentin De Bortoli, Arnaud Doucet, Emile Mathieu

Score-based generative modelling (SGM) has proven to be a very effective method for modelling densities on finite-dimensional spaces.

Dimensionality Reduction

Convergence of denoising diffusion models under the manifold hypothesis

no code implementations10 Aug 2022 Valentin De Bortoli

This does not cover settings where the target distribution is supported on a lower-dimensional manifold or is given by some empirical distribution.

Audio Synthesis Denoising

Wavelet Score-Based Generative Modeling

no code implementations9 Aug 2022 Florentin Guth, Simon Coste, Valentin De Bortoli, Stephane Mallat

This is because of ill-conditioning properties of the score that we analyze mathematically.

Riemannian Diffusion Schrödinger Bridge

no code implementations7 Jul 2022 James Thornton, Michael Hutchinson, Emile Mathieu, Valentin De Bortoli, Yee Whye Teh, Arnaud Doucet

Our proposed method generalizes Diffusion Schr\"odinger Bridge introduced in \cite{debortoli2021neurips} to the non-Euclidean setting and extends Riemannian score-based models beyond the first time reversal.

Density Estimation

Can Push-forward Generative Models Fit Multimodal Distributions?

1 code implementation29 Jun 2022 Antoine Salmona, Valentin De Bortoli, Julie Delon, Agnès Desolneux

More precisely, we show that the total variation distance and the Kullback-Leibler divergence between the generated and the data distribution are bounded from below by a constant depending on the mode separation and the Lipschitz constant.

A Continuous Time Framework for Discrete Denoising Models

1 code implementation30 May 2022 Andrew Campbell, Joe Benton, Valentin De Bortoli, Tom Rainforth, George Deligiannidis, Arnaud Doucet

We provide the first complete continuous time framework for denoising diffusion models of discrete data.

Denoising

Riemannian Score-Based Generative Modelling

2 code implementations6 Feb 2022 Valentin De Bortoli, Emile Mathieu, Michael Hutchinson, James Thornton, Yee Whye Teh, Arnaud Doucet

Score-based generative models (SGMs) are a powerful class of generative models that exhibit remarkable empirical performance.

Denoising

On Maximum-a-Posteriori estimation with Plug & Play priors and stochastic gradient descent

no code implementations16 Jan 2022 Rémi Laumont, Valentin De Bortoli, Andrés Almansa, Julie Delon, Alain Durmus, Marcelo Pereyra

Bayesian methods to solve imaging inverse problems usually combine an explicit data likelihood function with a prior distribution that explicitly models expected properties of the solution.

Image Denoising

Simulating Diffusion Bridges with Score Matching

1 code implementation14 Nov 2021 Jeremy Heng, Valentin De Bortoli, Arnaud Doucet, James Thornton

This is known to be a challenging problem that has received much attention in the last two decades.

Econometrics

On quantitative Laplace-type convergence results for some exponential probability measures, with two applications

no code implementations25 Oct 2021 Valentin De Bortoli, Agnès Desolneux

Classical results require the invertibility of the Hessian of $U$ in order to establish such asymptotics.

Diffusion Schrödinger Bridge with Applications to Score-Based Generative Modeling

2 code implementations NeurIPS 2021 Valentin De Bortoli, James Thornton, Jeremy Heng, Arnaud Doucet

In contrast, solving the Schr\"odinger Bridge problem (SB), i. e. an entropy-regularized optimal transport problem on path spaces, yields diffusions which generate samples from the data distribution in finite time.

Bayesian imaging using Plug & Play priors: when Langevin meets Tweedie

no code implementations8 Mar 2021 Rémi Laumont, Valentin De Bortoli, Andrés Almansa, Julie Delon, Alain Durmus, Marcelo Pereyra

The proposed algorithms are demonstrated on several canonical problems such as image deblurring, inpainting, and denoising, where they are used for point estimation as well as for uncertainty visualisation and quantification.

Bayesian Inference Deblurring +2

Quantitative Propagation of Chaos for SGD in Wide Neural Networks

no code implementations NeurIPS 2020 Valentin De Bortoli, Alain Durmus, Xavier Fontaine, Umut Simsekli

In comparison to previous works on the subject, we consider settings in which the sequence of stepsizes in SGD can potentially depend on the number of neurons and the iterations.

Convergence rates and approximation results for SGD and its continuous-time counterpart

no code implementations8 Apr 2020 Xavier Fontaine, Valentin De Bortoli, Alain Durmus

This paper proposes a thorough theoretical analysis of Stochastic Gradient Descent (SGD) with non-increasing step sizes.

Stochastic Optimization

Maximum likelihood estimation of regularisation parameters in high-dimensional inverse problems: an empirical Bayesian approach. Part I: Methodology and Experiments

1 code implementation26 Nov 2019 Ana F. Vidal, Valentin De Bortoli, Marcelo Pereyra, Alain Durmus

In this work, we propose a general empirical Bayesian method for setting regularisation parameters in imaging problems that are convex w. r. t.

Methodology Computation 62C12, 65C40, 68U10, 62F15, 65J20, 65C60, 65J22

Approximate Bayesian Computation with the Sliced-Wasserstein Distance

1 code implementation28 Oct 2019 Kimia Nadjahi, Valentin De Bortoli, Alain Durmus, Roland Badeau, Umut Şimşekli

Approximate Bayesian Computation (ABC) is a popular method for approximate inference in generative models with intractable but easy-to-sample likelihood.

Image Denoising

Cannot find the paper you are looking for? You can Submit a new open access paper.