1 code implementation • 27 May 2025 • Badr Moufad, Yazid Janati, Alain Durmus, Ahmed Ghorbel, Eric Moulines, Jimmy Olsson
In particular, contrary to common intuition, CFG does not yield samples from the target distribution associated with the limiting CFG score as the noise level approaches zero -- where the data distribution is tilted by a power $w \gt 1$ of the conditional distribution.
no code implementations • 19 May 2025 • Aymeric Capitaine, Maxime Haddouche, Eric Moulines, Michael I. Jordan, Etienne Boursier, Alain Durmus
Decision-focused learning (DFL) is an increasingly popular paradigm for training predictive models whose outputs are used in decision-making tasks.
1 code implementation • 10 Mar 2025 • Paul Mangold, Alain Durmus, Aymeric Dieuleveut, Eric Moulines
This paper proposes a novel analysis for the Scaffold algorithm, a popular method for dealing with data heterogeneity in federated learning.
no code implementations • 24 Feb 2025 • Tom Sander, Pierre Fernandez, Saeed Mahloujifar, Alain Durmus, Chuan Guo
Benchmark contamination poses a significant challenge to the reliability of Large Language Models (LLMs) evaluations, as it is difficult to assert whether a model has been trained on a test set.
no code implementations • 24 Feb 2025 • Andrea Bertazzi, Tim Johnston, Gareth O. Roberts, Alain Durmus
This paper aims to provide differential privacy (DP) guarantees for Markov chain Monte Carlo (MCMC) algorithms.
1 code implementation • 5 Feb 2025 • Yazid Janati, Badr Moufad, Mehdi Abou El Qassime, Alain Durmus, Eric Moulines, Jimmy Olsson
Recent approaches use pre-trained diffusion models as priors to solve a wide range of such problems, only leveraging inference-time compute and thereby eliminating the need to retrain task-specific models on the same dataset.
no code implementations • 2 Dec 2024 • Paul Mangold, Alain Durmus, Aymeric Dieuleveut, Sergey Samsonov, Eric Moulines
In this paper, we present a novel analysis of FedAvg with constant step size, relying on the Markov property of the underlying process.
1 code implementation • 11 Nov 2024 • Tom Sander, Pierre Fernandez, Alain Durmus, Teddy Furon, Matthijs Douze
Image watermarking methods are not tailored to handle small watermarked areas.
no code implementations • 22 Oct 2024 • Antoine Scheid, Etienne Boursier, Alain Durmus, Michael I. Jordan, Pierre Ménard, Eric Moulines, Michal Valko
To our knowledge, this is the first theoretical contribution in this area to provide an offline approach as well as worst-case guarantees.
1 code implementation • 13 Oct 2024 • Badr Moufad, Yazid Janati, Lisa Bedin, Alain Durmus, Randal Douc, Eric Moulines, Jimmy Olsson
To tackle this issue, state-of-the-art approaches formulate the problem as that of sampling from a surrogate diffusion model targeting the posterior and decompose its scores into two terms: the prior score and an intractable guidance term.
no code implementations • 7 Oct 2024 • Marina Sheshukova, Denis Belomestny, Alain Durmus, Eric Moulines, Alexey Naumov, Sergey Samsonov
We address the problem of solving strongly convex and smooth minimization problems using stochastic gradient descent (SGD) algorithm with a constant step size.
no code implementations • 3 Oct 2024 • Lorenzo Mancini, Safwan Labbi, Karim Abed Meraim, Fouzi Boukhalfa, Alain Durmus, Paul Mangold, Eric Moulines
In this paper, we study the problem of joint channel selection, where vehicles with different technologies choose one or more Access Points (APs) to transmit messages in a network.
no code implementations • 12 Sep 2024 • Marta Gentiloni Silveri, Giovanni Conforti, Alain Durmus
Flow Matching (FM) (also referred to as stochastic interpolants or rectified flows) stands out as a class of generative models that aims to bridge in finite time the target distribution $\nu^\star$ with an auxiliary distribution $\mu$, leveraging a fixed coupling $\pi$ and a bridge which can either be deterministic or stochastic.
no code implementations • 28 Jul 2024 • Andrea Bertazzi, Dario Shariatian, Umut Simsekli, Eric Moulines, Alain Durmus
We introduce a novel class of generative models based on piecewise deterministic Markov processes (PDMPs), a family of non-diffusive stochastic processes consisting of deterministic motion and random jumps at random times.
1 code implementation • 26 Jul 2024 • Dario Shariatian, Umut Simsekli, Alain Durmus
Investigating noise distribution beyond Gaussian in diffusion generative models is an open problem.
no code implementations • 28 Jun 2024 • Antoine Scheid, Aymeric Capitaine, Etienne Boursier, Eric Moulines, Michael I Jordan, Alain Durmus
This result shows that the optimal approach for maximizing the social welfare in the presence of externality is to establish property rights, i. e., enable transfers and bargaining between the players.
no code implementations • 6 Jun 2024 • Tom Huix, Anna Korba, Alain Durmus, Eric Moulines
In this view, VI over this specific family can be casted as the minimization of a Mollified relative entropy, i. e. the KL between the convolution (with respect to a Gaussian kernel) of an atomic measure supported on Diracs, and the target distribution.
1 code implementation • 18 Mar 2024 • Yazid Janati, Badr Moufad, Alain Durmus, Eric Moulines, Jimmy Olsson
We present an innovative framework, divide-and-conquer posterior sampling, which leverages the inherent structure of DDMs to construct a sequence of intermediate posteriors that guide the produced samples to the target posterior.
no code implementations • 6 Mar 2024 • Antoine Scheid, Daniil Tiapkin, Etienne Boursier, Aymeric Capitaine, El Mahdi El Mhamdi, Eric Moulines, Michael I. Jordan, Alain Durmus
This work considers a repeated principal-agent bandit game, where the principal can only interact with her environment through the agent.
1 code implementation • 4 Mar 2024 • Tom Sander, Yaodong Yu, Maziar Sanjabi, Alain Durmus, Yi Ma, Kamalika Chaudhuri, Chuan Guo
In this work, we show that effective DP representation learning can be done via image captioning and scaling up to internet-scale multimodal datasets.
1 code implementation • 22 Feb 2024 • Tom Sander, Pierre Fernandez, Alain Durmus, Matthijs Douze, Teddy Furon
We discover that, on the contrary, it is possible to reliably determine if a language model was trained on synthetic data if that data is output by a watermarked LLM.
no code implementations • 13 Feb 2024 • Tom Sander, Maxime Sylvestre, Alain Durmus
We first show that the phenomenon extends to Noisy-SGD (DP-SGD without clipping), suggesting that the stochasticity (and not the clipping) is the cause of this implicit bias, even with additional isotropic Gaussian noise.
no code implementations • 23 Aug 2023 • Giovanni Conforti, Alain Durmus, Marta Gentiloni Silveri
Our study provides a rigorous analysis, yielding simple, improved and sharp convergence bounds in KL applicable to any data distribution with finite Fisher information with respect to the standard Gaussian distribution.
1 code implementation • 19 Jul 2023 • Pierre Clavier, Tom Huix, Alain Durmus
In this paper, we introduce and analyze a variant of the Thompson sampling (TS) algorithm for contextual bandits.
no code implementations • 7 Jul 2023 • Alain Durmus, Samuel Gruffaz, Miika Kailas, Eero Saksman, Matti Vihola
Under conditions similar to the ones existing for HMC, we also show that NUTS is geometrically ergodic.
1 code implementation • NeurIPS 2023 • Maxence Noble, Valentin De Bortoli, Arnaud Doucet, Alain Durmus
In this paper, we consider an entropic version of mOT with a tree-structured quadratic cost, i. e., a function that can be written as a sum of pairwise cost functions between the nodes of a tree.
no code implementations • 13 Apr 2023 • Giacomo Greco, Maxence Noble, Giovanni Conforti, Alain Durmus
Our approach is novel in that it is purely probabilistic and relies on coupling by reflection techniques for controlled diffusions on the torus.
no code implementations • 10 Mar 2023 • Alain Durmus, Eric Moulines, Alexey Naumov, Sergey Samsonov, Marina Sheshukova
In this paper, we establish novel deviation bounds for additive functionals of geometrically ergodic Markov chains similar to Rosenthal and Bernstein inequalities for sums of independent random variables.
1 code implementation • 9 Feb 2023 • Louis Grenioux, Alain Durmus, Éric Moulines, Marylou Gabrié
Transport maps can ease the sampling of distributions with non-trivial geometries by transforming them into distributions that are easier to handle.
no code implementations • 31 Oct 2022 • Vincent Plassier, Alain Durmus, Eric Moulines
This paper focuses on Bayesian inference in a federated learning context (FL).
1 code implementation • NeurIPS 2023 • Maxence Noble, Valentin De Bortoli, Alain Durmus
In this paper, we propose Barrier Hamiltonian Monte Carlo (BHMC), a version of the HMC algorithm which aims at sampling from a Gibbs distribution $\pi$ on a manifold $\mathrm{M}$, endowed with a Hessian metric $\mathfrak{g}$ derived from a self-concordant barrier.
no code implementations • 10 Jul 2022 • Alain Durmus, Eric Moulines, Alexey Naumov, Sergey Samsonov
Our finite-time instance-dependent bounds for the averaged LSA iterates are sharp in the sense that the leading term we obtain coincides with the local asymptotic minimax limit.
1 code implementation • 8 Jul 2022 • Tom Huix, Szymon Majewski, Alain Durmus, Eric Moulines, Anna Korba
This paper studies the Variational Inference (VI) used for training Bayesian Neural Networks (BNN) in the overparameterized regime, i. e., when the number of neurons tends to infinity.
no code implementations • 7 Jun 2022 • Nikita Kotelevskii, Maxime Vono, Eric Moulines, Alain Durmus
We provide non-asymptotic convergence guarantees for the proposed algorithms and illustrate their performances on various personalised federated learning tasks.
no code implementations • 16 Jan 2022 • Rémi Laumont, Valentin De Bortoli, Andrés Almansa, Julie Delon, Alain Durmus, Marcelo Pereyra
Bayesian methods to solve imaging inverse problems usually combine an explicit data likelihood function with a prior distribution that explicitly models expected properties of the solution.
1 code implementation • NeurIPS 2021 • Achille Thin, Yazid Janati El Idrissi, Sylvain Le Corff, Charles Ollion, Eric Moulines, Arnaud Doucet, Alain Durmus, Christian Robert
Sampling from a complex distribution $\pi$ and approximating its intractable normalizing constant $\mathrm{Z}$ are challenging problems.
1 code implementation • 4 Nov 2021 • Sergey Samsonov, Evgeny Lagutin, Marylou Gabrié, Alain Durmus, Alexey Naumov, Eric Moulines
Recent works leveraging learning to enhance sampling have shown promising results, in particular by designing effective non-local moves and global proposals.
2 code implementations • 30 Jun 2021 • Achille Thin, Nikita Kotelevskii, Arnaud Doucet, Alain Durmus, Eric Moulines, Maxim Panov
Variational auto-encoders (VAE) are popular deep latent variable models which are trained by maximizing an Evidence Lower Bound (ELBO).
2 code implementations • NeurIPS 2021 • Kimia Nadjahi, Alain Durmus, Pierre E. Jacob, Roland Badeau, Umut Şimşekli
The Sliced-Wasserstein distance (SW) is being increasingly used in machine learning applications as an alternative to the Wasserstein distance and offers significant computational and statistical benefits.
no code implementations • 11 Jun 2021 • Vincent Plassier, Maxime Vono, Alain Durmus, Eric Moulines
Performing reliable Bayesian inference on a big data scale is becoming a keystone in the modern era of machine learning.
no code implementations • NeurIPS 2021 • Alain Durmus, Eric Moulines, Alexey Naumov, Sergey Samsonov, Kevin Scaman, Hoi-To Wai
This family of methods arises in many machine learning tasks and is used to obtain approximate solutions of a linear system $\bar{A}\theta = \bar{b}$ for which $\bar{A}$ and $\bar{b}$ can only be accessed through random estimates $\{({\bf A}_n, {\bf b}_n): n \in \mathbb{N}^*\}$.
no code implementations • 1 Jun 2021 • Maxime Vono, Vincent Plassier, Alain Durmus, Aymeric Dieuleveut, Eric Moulines
The objective of Federated Learning (FL) is to perform statistical inference for data which are decentralised and stored locally on networked clients.
1 code implementation • 17 Mar 2021 • Achille Thin, Yazid Janati, Sylvain Le Corff, Charles Ollion, Arnaud Doucet, Alain Durmus, Eric Moulines, Christian Robert
Sampling from a complex distribution $\pi$ and approximating its intractable normalizing constant Z are challenging problems.
no code implementations • 8 Mar 2021 • Rémi Laumont, Valentin De Bortoli, Andrés Almansa, Julie Delon, Alain Durmus, Marcelo Pereyra
The proposed algorithms are demonstrated on several canonical problems such as image deblurring, inpainting, and denoising, where they are used for point estimation as well as for uncertainty visualisation and quantification.
no code implementations • 15 Feb 2021 • Alain Durmus, Pablo Jiménez, Éric Moulines, Salem Said
This result gives rise to a family of stationary distributions indexed by the step-size, which is further shown to converge to a Dirac measure, concentrated at the solution of the problem at hand, as the step-size goes to 0.
no code implementations • 30 Jan 2021 • Alain Durmus, Eric Moulines, Alexey Naumov, Sergey Samsonov, Hoi-To Wai
This paper studies the exponential stability of random matrix products driven by a general (possibly unbounded) state space Markov chain.
no code implementations • 31 Dec 2020 • Achille Thin, Nikita Kotelevskii, Christophe Andrieu, Alain Durmus, Eric Moulines, Maxim Panov
This paper fills the gap by developing general tools to ensure that a class of nonreversible Markov kernels, possibly relying on complex transforms, has the desired invariance property and leads to convergent algorithms.
no code implementations • NeurIPS 2020 • Valentin De Bortoli, Alain Durmus, Xavier Fontaine, Umut Simsekli
In comparison to previous works on the subject, we consider settings in which the sequence of stepsizes in SGD can potentially depend on the number of neurons and the iterations.
no code implementations • 27 May 2020 • Alain Durmus, Pablo Jiménez, Éric Moulines, Salem Said, Hoi-To Wai
This paper analyzes the convergence for a large class of Riemannian stochastic approximation (SA) schemes, which aim at tackling stochastic optimization problems.
no code implementations • 8 Apr 2020 • Xavier Fontaine, Valentin De Bortoli, Alain Durmus
This paper proposes a thorough theoretical analysis of Stochastic Gradient Descent (SGD) with non-increasing step sizes.
1 code implementation • NeurIPS 2020 • Kimia Nadjahi, Alain Durmus, Lénaïc Chizat, Soheil Kolouri, Shahin Shahrampour, Umut Şimşekli
The idea of slicing divergences has been proven to be successful when comparing two probability measures in various machine learning applications including generative modeling, and consists in computing the expected value of a `base divergence' between one-dimensional random projections of the two measures.
no code implementations • 27 Feb 2020 • Achille Thin, Nikita Kotelevskii, Jean-Stanislas Denain, Leo Grinsztajn, Alain Durmus, Maxim Panov, Eric Moulines
In this contribution, we propose a new computationally efficient method to combine Variational Inference (VI) with Markov Chain Monte Carlo (MCMC).
no code implementations • 3 Dec 2019 • Valentin De Bortoli, Agnes Desolneux, Alain Durmus, Bruno Galerne, Arthur Leclaire
Recent years have seen the rise of convolutional neural network techniques in exemplar-based image synthesis.
1 code implementation • 26 Nov 2019 • Ana F. Vidal, Valentin De Bortoli, Marcelo Pereyra, Alain Durmus
In this work, we propose a general empirical Bayesian method for setting regularisation parameters in imaging problems that are convex w. r. t.
Methodology Computation 62C12, 65C40, 68U10, 62F15, 65J20, 65C60, 65J22
1 code implementation • 28 Oct 2019 • Kimia Nadjahi, Valentin De Bortoli, Alain Durmus, Roland Badeau, Umut Şimşekli
Approximate Bayesian Computation (ABC) is a popular method for approximate inference in generative models with intractable but easy-to-sample likelihood.
no code implementations • 10 Jul 2019 • Firas Jarboui, Célya Gruson-daniel, Pierre Chanial, Alain Durmus, Vincent Rocchisani, Sophie-helene Goulet Ebongue, Anneliese Depoux, Wilfried Kirschenmann, Vianney Perchet
Studies on massive open online courses (MOOCs) users discuss the existence of typical profiles and their impact on the learning process of the students.
1 code implementation • NeurIPS 2019 • Kimia Nadjahi, Alain Durmus, Umut Şimşekli, Roland Badeau
Minimum expected distance estimation (MEDE) algorithms have been widely used for probabilistic models with intractable likelihood functions and they have become increasingly popular due to their use in implicit generative modeling (e. g. Wasserstein generative adversarial networks, Wasserstein autoencoders).
1 code implementation • NeurIPS 2019 • Marcel Hirt, Petros Dellaportas, Alain Durmus
This family is based on new copula-like densities on the hypercube with non-uniform marginals which can be sampled efficiently, i. e. with a complexity linear in the dimension of state space.
no code implementations • NeurIPS 2018 • Nicolas Brosse, Alain Durmus, Eric Moulines
As $N$ becomes large, we show that the SGLD algorithm has an invariant probability measure which significantly departs from the target posterior and behaves like Stochastic Gradient Descent (SGD).
1 code implementation • 21 Jun 2018 • Antoine Liutkus, Umut Şimşekli, Szymon Majewski, Alain Durmus, Fabian-Robert Stöter
To the best of our knowledge, the proposed algorithm is the first nonparametric IGM algorithm with explicit theoretical guarantees.
no code implementations • 26 Feb 2018 • Alain Durmus, Szymon Majewski, Błażej Miasojedow
In this paper, we provide new insights on the Unadjusted Langevin Algorithm.
no code implementations • 20 Jul 2017 • Aymeric Dieuleveut, Alain Durmus, Francis Bach
We consider the minimization of an objective function given access to unbiased estimates of its gradient through stochastic gradient descent (SGD) with constant step-size.
no code implementations • NeurIPS 2016 • Alain Durmus, Umut Simsekli, Eric Moulines, Roland Badeau, Gaël Richard
We illustrate our framework on the popular Stochastic Gradient Langevin Dynamics (SGLD) algorithm and propose a novel SG-MCMC algorithm referred to as Stochastic Gradient Richardson-Romberg Langevin Dynamics (SGRRLD).
no code implementations • 5 May 2016 • Alain Durmus, Eric Moulines
We consider in this paper the problem of sampling a high-dimensional probability distribution $\pi$ having a density with respect to the Lebesgue measure on $\mathbb{R}^d$, known up to a normalization constant $x \mapsto \pi(x)= \mathrm{e}^{-U(x)}/\int_{\mathbb{R}^d} \mathrm{e}^{-U(y)} \mathrm{d} y$.