Search Results for author:
Found 22 papers, 4 papers with code
Quantum State Generation with Structure-Preserving Diffusion Model
This article considers the generative modeling of the states of quantum systems, and an approach based on denoising diffusion model is proposed.
Convergence of Kinetic Langevin Monte Carlo on Lie groups
Explicit, momentum-based dynamics for optimizing functions defined on Lie groups was recently constructed, based on techniques such as variational optimization and left trivialization.
Zeroth-Order Sampling Methods for Non-Log-Concave Distributions: Alleviating Metastability by Denoising Diffusion
It first describes a framework, Diffusion Monte Carlo (DMC), based on the simulation of a denoising diffusion process with its score function approximated by a generic Monte Carlo estimator.
DFU: scale-robust diffusion model for zero-shot super-resolution image generation
Diffusion generative models have achieved remarkable success in generating images with a fixed resolution.
Good regularity creates large learning rate implicit biases: edge of stability, balancing, and catapult
This regularity, together with gradient descent using a large learning rate that favors flatter regions, results in these nontrivial dynamical behaviors.
Mirror Diffusion Models for Constrained and Watermarked Generation
In this work, we propose Mirror Diffusion Models (MDM), a new class of diffusion models that generate data on convex constrained sets without losing any tractability.
Extragradient Type Methods for Riemannian Variational Inequality Problems
In the context of Euclidean space, it is established that the last-iterates of both the extragradient (EG) and past extragradient (PEG) methods converge to the solution of monotone variational inequality problems at a rate of $O\left(\frac{1}{\sqrt{T}}\right)$ (Cai et al., 2022).
Data-driven discovery of non-Newtonian astronomy via learning non-Euclidean Hamiltonian
Incorporating the Hamiltonian structure of physical dynamics into deep learning models provides a powerful way to improve the interpretability and prediction accuracy.
gDDIM: Generalized denoising diffusion implicit models
In the CLD, a diffusion model by augmenting the diffusion process with velocity, our algorithm achieves an FID score of 2. 26, on CIFAR10, with only 50 number of score function evaluations~(NFEs) and an FID score of 2. 86 with only 27 NFEs.
Alternating Mirror Descent for Constrained Min-Max Games
In this paper we study two-player bilinear zero-sum games with constrained strategy spaces.
Momentum Stiefel Optimizer, with Applications to Suitably-Orthogonal Attention, and Optimal Transport
The problem of optimization on Stiefel manifold, i. e., minimizing functions of (not necessarily square) matrices that satisfy orthogonality constraints, has been extensively studied.
Large Learning Rate Tames Homogeneity: Convergence and Balancing Effect
Moreover, we rigorously establish an implicit bias of GD induced by such a large learning rate, termed 'balancing', meaning that magnitudes of $X$ and $Y$ at the limit of GD iterations will be close even if their initialization is significantly unbalanced.
The Mirror Langevin Algorithm Converges with Vanishing Bias
The Mirror Langevin Diffusion (MLD) is a sampling analogue of mirror flow in continuous time, and it has nice convergence properties under log-Sobolev or Poincare inequalities relative to the Hessian metric, as shown by Chewi et al. (2020).
Sqrt(d) Dimension Dependence of Langevin Monte Carlo
This article considers the popular MCMC method of unadjusted Langevin Monte Carlo (LMC) and provides a non-asymptotic analysis of its sampling error in 2-Wasserstein distance.
Mean-Square Analysis with An Application to Optimal Dimension Dependence of Langevin Monte Carlo
This bound improves the best previously known $\widetilde{\mathcal{O}}\left(\frac{d}{\epsilon}\right)$ result and is optimal in both dimension $d$ and accuracy tolerance $\epsilon$ for log-smooth and log-strongly-convex target measures.
Data-driven Prediction of General Hamiltonian Dynamics via Learning Exactly-Symplectic Maps
For this special case, both generic approaches based on learning the vector field of the latent ODE and specialized approaches based on learning the Hamiltonian that generates the vector field exist.
Why Do Deep Residual Networks Generalize Better than Deep Feedforward Networks? --- A Neural Tangent Kernel Perspective
We then compare the kernel of deep ResNets with that of deep FFNets and discover that the class of functions induced by the kernel of FFNets is asymptotically not learnable, as the depth goes to infinity.
Hessian-Free High-Resolution Nesterov Acceleration for Sampling
Nesterov's Accelerated Gradient (NAG) for optimization has better performance than its continuous time limit (noiseless kinetic Langevin) when a finite step-size is employed \citep{shi2021understanding}.
Improving Sampling Accuracy of Stochastic Gradient MCMC Methods via Non-uniform Subsampling of Gradients
In our practical implementation of EWSG, the non-uniform subsampling is performed efficiently via a Metropolis-Hastings chain on the data index, which is coupled to the MCMC algorithm.
Stochasticity of Deterministic Gradient Descent: Large Learning Rate for Multiscale Objective Function
This article suggests that deterministic Gradient Descent, which does not use any stochastic gradient approximation, can still exhibit stochastic behaviors.
Why Do Deep Residual Networks Generalize Better than Deep Feedforward Networks? -- A Neural Tangent Kernel Perspective
We then compare the kernel of deep ResNets with that of deep FFNets and discover that the class of functions induced by the kernel of FFNets is asymptotically not learnable, as the depth goes to infinity.
Variational Optimization on Lie Groups, with Examples of Leading (Generalized) Eigenvalue Problems
The article considers smooth optimization of functions on Lie groups.
