Search Results for author:
Found 28 papers, 13 papers with code
Flowing Datasets with Wasserstein over Wasserstein Gradient Flows
Many applications in machine learning involve data represented as probability distributions.
DDEQs: Distributional Deep Equilibrium Models through Wasserstein Gradient Flows
Deep Equilibrium Models (DEQs) are a class of implicit neural networks that solve for a fixed point of a neural network in their forward pass.
Density Ratio Estimation with Conditional Probability Paths
Density ratio estimation in high dimensions can be reframed as integrating a certain quantity, the time score, over probability paths which interpolate between the two densities.
Polynomial time sampling from log-smooth distributions in fixed dimension under semi-log-concavity of the forward diffusion with application to strongly dissipative distributions
Under the assumption that the density to sample from is $L$-log-smooth and that the forward process is semi-log-concave: $-\nabla^2 \log(p_t) \succeq -\beta I_d$ for some $\beta \geq 0$, we prove that our algorithm achieves an expected $\epsilon$ error in $\text{KL}$ divergence in $O(d^7(L+\beta)^2L^{d+2}\epsilon^{-2(d+3)}(d+m_2(\mu))^{2(d+1)})$ time with $m_2(\mu)$ the second order moment of $\mu$.
Constrained Sampling with Primal-Dual Langevin Monte Carlo
This work considers the problem of sampling from a probability distribution known up to a normalization constant while satisfying a set of statistical constraints specified by the expected values of general nonlinear functions.
Provable Convergence and Limitations of Geometric Tempering for Langevin Dynamics
Geometric tempering is a popular approach to sampling from challenging multi-modal probability distributions by instead sampling from a sequence of distributions which interpolate, using the geometric mean, between an easier proposal distribution and the target distribution.
(De)-regularized Maximum Mean Discrepancy Gradient Flow
We introduce a (de)-regularization of the Maximum Mean Discrepancy (DrMMD) and its Wasserstein gradient flow.
Statistical and Geometrical properties of regularized Kernel Kullback-Leibler divergence
In this paper, we study the statistical and geometrical properties of the Kullback-Leibler divergence with kernel covariance operators (KKL) introduced by Bach [2022].
A Practical Diffusion Path for Sampling
Diffusion models are state-of-the-art methods in generative modeling when samples from a target probability distribution are available, and can be efficiently sampled, using score matching to estimate score vectors guiding a Langevin process.
Mirror and Preconditioned Gradient Descent in Wasserstein Space
As the problem of minimizing functionals on the Wasserstein space encompasses many applications in machine learning, different optimization algorithms on $\mathbb{R}^d$ have received their counterpart analog on the Wasserstein space.
Theoretical Guarantees for Variational Inference with Fixed-Variance Mixture of Gaussians
In this view, VI over this specific family can be casted as the minimization of a Mollified relative entropy, i. e. the KL between the convolution (with respect to a Gaussian kernel) of an atomic measure supported on Diracs, and the target distribution.
Unified PAC-Bayesian Study of Pessimism for Offline Policy Learning with Regularized Importance Sampling
Off-policy learning (OPL) often involves minimizing a risk estimator based on importance weighting to correct bias from the logging policy used to collect data.
Bayesian Off-Policy Evaluation and Learning for Large Action Spaces
In this framework, we propose sDM, a generic Bayesian approach for OPE and OPL, grounded in both algorithmic and theoretical foundations.
Implicit Diffusion: Efficient Optimization through Stochastic Sampling
We present a new algorithm to optimize distributions defined implicitly by parameterized stochastic diffusions.
A connection between Tempering and Entropic Mirror Descent
We establish that tempering SMC corresponds to entropic mirror descent applied to the reverse Kullback-Leibler (KL) divergence and obtain convergence rates for the tempering iterates.
Exponential Smoothing for Off-Policy Learning
In particular, it is also valid for standard IPS without making the assumption that the importance weights are bounded.
Sampling with Mollified Interaction Energy Descent
These energies rely on mollifier functions -- smooth approximations of the Dirac delta originated from PDE theory.
Variational Inference of overparameterized Bayesian Neural Networks: a theoretical and empirical study
This paper studies the Variational Inference (VI) used for training Bayesian Neural Networks (BNN) in the overparameterized regime, i. e., when the number of neurons tends to infinity.
Mirror Descent with Relative Smoothness in Measure Spaces, with application to Sinkhorn and EM
We also show that Expectation Maximization (EM) can always formally be written as a mirror descent.
Adaptive Importance Sampling meets Mirror Descent: a Bias-variance tradeoff
Adaptive importance sampling is a widely spread Monte Carlo technique that uses a re-weighting strategy to iteratively estimate the so-called target distribution.
Kernel Stein Discrepancy Descent
We investigate the properties of its Wasserstein gradient flow to approximate a target probability distribution $\pi$ on $\mathbb{R}^d$, known up to a normalization constant.
Proximal Causal Learning with Kernels: Two-Stage Estimation and Moment Restriction
In particular, we provide a unifying view of two-stage and moment restriction approaches for solving this problem in a nonlinear setting.
A Non-Asymptotic Analysis for Stein Variational Gradient Descent
We study the Stein Variational Gradient Descent (SVGD) algorithm, which optimises a set of particles to approximate a target probability distribution $\pi\propto e^{-V}$ on $\mathbb{R}^d$.
The Wasserstein Proximal Gradient Algorithm
Using techniques from convex optimization and optimal transport, we analyze the FB scheme as a minimization algorithm on the Wasserstein space.
Maximum Mean Discrepancy Gradient Flow
We construct a Wasserstein gradient flow of the maximum mean discrepancy (MMD) and study its convergence properties.
Dimensionality Reduction and (Bucket) Ranking: a Mass Transportation Approach
Whereas most dimensionality reduction techniques (e. g. PCA, ICA, NMF) for multivariate data essentially rely on linear algebra to a certain extent, summarizing ranking data, viewed as realizations of a random permutation $\Sigma$ on a set of items indexed by $i\in \{1,\ldots,\; n\}$, is a great statistical challenge, due to the absence of vector space structure for the set of permutations $\mathfrak{S}_n$.
A Structured Prediction Approach for Label Ranking
We propose to solve a label ranking problem as a structured output regression task.
Ranking Median Regression: Learning to Order through Local Consensus
In the probabilistic formulation of the 'Learning to Order' problem we propose, which extends the framework for statistical Kemeny ranking aggregation developped in \citet{CKS17}, this boils down to recovering conditional Kemeny medians of $\Sigma$ given $X$ from i. i. d.
