Search Results for author:
Found 37 papers, 4 papers with code
BRIDLE: Generalized Self-supervised Learning with Quantization
To address these limitations, we introduce BRIDLE (Bidirectional Residual Quantization Interleaved Discrete Learning Encoder), a self-supervised encoder pretraining framework that incorporates residual quantization (RQ) into the bidirectional training process, and is generalized for pretraining with audio, image, and video.
Algorithmic Stability of Stochastic Gradient Descent with Momentum under Heavy-Tailed Noise
In this work, we establish generalization bounds for SGD with momentum (SGDm) under heavy-tailed gradient noise.
Non-Reversible Langevin Algorithms for Constrained Sampling
Based on the discretization of SRNLD, we propose skew-reflected non-reversible Langevin Monte Carlo (SRNLMC), and obtain non-asymptotic discretization error from SRNLD, and convergence guarantees to the target distribution in 1-Wasserstein distance.
VIX options in the SABR model
As a remedy, we propose a capped volatility process by capping the drift and diffusion terms in the $v_{t}$ process such that it becomes non-explosive and well-behaved, and study the short-maturity asymptotics for the pricing of VIX options.
Generalized EXTRA stochastic gradient Langevin dynamics
Motivated by the EXTRA algorithm and its generalizations for decentralized optimization, we propose the generalized EXTRA stochastic gradient Langevin dynamics, which eliminates this bias in the full-batch setting.
Short-maturity options on realized variance in local-stochastic volatility models
We derive the short-maturity asymptotics for prices of options on realized variance in local-stochastic volatility models.
Short-maturity Asian options in local-stochastic volatility models
Using large deviations theory methods, the asymptotics for the OTM options are expressed as a rate function which is represented as a two-dimensional variational problem.
Short-maturity asymptotics for VIX and European options in local-stochastic volatility models
We give explicit results for two classes of local-stochastic volatility models relevant in practice, with Heston-type and SABR-type stochastic volatility.
Differential Privacy of Noisy (S)GD under Heavy-Tailed Perturbations
Injecting heavy-tailed noise to the iterates of stochastic gradient descent (SGD) has received increasing attention over the past few years.
Short-maturity asymptotics for option prices with interest rates effects
We derive the short-maturity asymptotics for option prices in the local volatility model in a new short-maturity limit $T\to 0$ at fixed $\rho = (r-q) T$, where $r$ is the interest rate and $q$ is the dividend yield.
Intriguing Differences Between Zero-Shot and Systematic Evaluations of Vision-Language Transformer Models
Transformer-based models have dominated natural language processing and other areas in the last few years due to their superior (zero-shot) performance on benchmark datasets.
Convergence Analysis for General Probability Flow ODEs of Diffusion Models in Wasserstein Distances
Score-based generative modeling with probability flow ordinary differential equations (ODEs) has achieved remarkable success in a variety of applications.
Wasserstein Convergence Guarantees for a General Class of Score-Based Generative Models
We find that the experimental results are in good agreement with our theoretical predictions on the iteration complexity, and the models with our newly proposed forward processes can outperform existing models.
Asymptotics for Short Maturity Asian Options in Jump-Diffusion models with Local Volatility
We present a study of the short maturity asymptotics for Asian options in a jump-diffusion model with a local volatility component, where the jumps are modeled as a compound Poisson process.
Asymptotics for the Laplace transform of the time integral of the geometric Brownian motion
We present an asymptotic result for the Laplace transform of the time integral of the geometric Brownian motion $F(\theta, T) = \mathbb{E}[e^{-\theta X_T}]$ with $X_T = \int_0^T e^{\sigma W_s + ( a - \frac12 \sigma^2)s} ds$, which is exact in the limit $\sigma^2 T \to 0$ at fixed $\sigma^2 \theta T^2$ and $aT$.
Cyclic and Randomized Stepsizes Invoke Heavier Tails in SGD than Constant Stepsize
Our results bring a new understanding of the benefits of cyclic and randomized stepsizes compared to constant stepsize in terms of the tail behavior.
Algorithmic Stability of Heavy-Tailed SGD with General Loss Functions
Very recently, new generalization bounds have been proven, indicating a non-monotonic relationship between the generalization error and heavy tails, which is more pertinent to the reported empirical observations.
Sensitivities of Asian options in the Black-Scholes model
We propose analytical approximations for the sensitivities (Greeks) of the Asian options in the Black-Scholes model, following from a small maturity/volatility approximation for the option prices which has the exact short maturity limit, obtained using large deviations theory.
A delayed dual risk model
In this paper, we study a dual risk model with delays in the spirit of Dassios-Zhao.
Penalized Overdamped and Underdamped Langevin Monte Carlo Algorithms for Constrained Sampling
When $f$ is smooth and gradients are available, we get $\tilde{\mathcal{O}}(d/\varepsilon^{10})$ iteration complexity for PLD to sample the target up to an $\varepsilon$-error where the error is measured in the TV distance and $\tilde{\mathcal{O}}(\cdot)$ hides logarithmic factors.
Algorithmic Stability of Heavy-Tailed Stochastic Gradient Descent on Least Squares
Recent studies have shown that heavy tails can emerge in stochastic optimization and that the heaviness of the tails have links to the generalization error.
Heavy-Tail Phenomenon in Decentralized SGD
To have a more explicit control on the tail exponent, we then consider the case where the loss at each node is a quadratic, and show that the tail-index can be estimated as a function of the step-size, batch-size, and the topological properties of the network of the computational nodes.
Fractal Structure and Generalization Properties of Stochastic Optimization Algorithms
As our main contribution, we prove that the generalization error of a stochastic optimization algorithm can be bounded based on the `complexity' of the fractal structure that underlies its invariant measure.
Convergence Rates of Stochastic Gradient Descent under Infinite Noise Variance
In this paper, we provide convergence guarantees for SGD under a state-dependent and heavy-tailed noise with a potentially infinite variance, for a class of strongly convex objectives.
Asymmetric Heavy Tails and Implicit Bias in Gaussian Noise Injections
In this paper we focus on the so-called `implicit effect' of GNIs, which is the effect of the injected noise on the dynamics of SGD.
Decentralized Stochastic Gradient Langevin Dynamics and Hamiltonian Monte Carlo
Stochastic gradient Langevin dynamics (SGLD) and stochastic gradient Hamiltonian Monte Carlo (SGHMC) are two popular Markov Chain Monte Carlo (MCMC) algorithms for Bayesian inference that can scale to large datasets, allowing to sample from the posterior distribution of the parameters of a statistical model given the input data and the prior distribution over the model parameters.
The Heavy-Tail Phenomenon in SGD
We claim that depending on the structure of the Hessian of the loss at the minimum, and the choices of the algorithm parameters $\eta$ and $b$, the SGD iterates will converge to a \emph{heavy-tailed} stationary distribution.
Non-Convex Optimization via Non-Reversible Stochastic Gradient Langevin Dynamics
In this paper, we study the non reversible Stochastic Gradient Langevin Dynamics (NSGLD) which is based on discretization of the non-reversible Langevin diffusion.
Fractional Underdamped Langevin Dynamics: Retargeting SGD with Momentum under Heavy-Tailed Gradient Noise
Stochastic gradient descent with momentum (SGDm) is one of the most popular optimization algorithms in deep learning.
Asymptotics of the time-discretized log-normal SABR model: The implied volatility surface
We derive an almost sure limit and a large deviations result for the log-asset price in the limit of large number of time steps.
Robust Distributed Accelerated Stochastic Gradient Methods for Multi-Agent Networks
When gradients do not contain noise, we also prove that distributed accelerated methods can \emph{achieve acceleration}, requiring $\mathcal{O}(\kappa \log(1/\varepsilon))$ gradient evaluations and $\mathcal{O}(\kappa \log(1/\varepsilon))$ communications to converge to the same fixed point with the non-accelerated variant where $\kappa$ is the condition number and $\varepsilon$ is the target accuracy.
Accelerated Linear Convergence of Stochastic Momentum Methods in Wasserstein Distances
In the special case of strongly convex quadratic objectives, we can show accelerated linear rates in the $p$-Wasserstein metric for any $p\geq 1$ with improved sensitivity to noise for both AG and HB through a non-asymptotic analysis under some additional assumptions on the noise structure.
Breaking Reversibility Accelerates Langevin Dynamics for Global Non-Convex Optimization
We study two variants that are based on non-reversible Langevin diffusions: the underdamped Langevin dynamics (ULD) and the Langevin dynamics with a non-symmetric drift (NLD).
Global Convergence of Stochastic Gradient Hamiltonian Monte Carlo for Non-Convex Stochastic Optimization: Non-Asymptotic Performance Bounds and Momentum-Based Acceleration
We provide finite-time performance bounds for the global convergence of both SGHMC variants for solving stochastic non-convex optimization problems with explicit constants.
Asymptotic Analysis for Optimal Dividends in a Dual Risk Model
The dual risk model is a popular model in finance and insurance, which is often used to model the wealth process of a venture capital or high tech company.
Optimal Investment in a Dual Risk Model
In this paper, we propose to study the optimal investment strategy on research and development for the dual risk models to minimize the ruin probability of the underlying company.
A State-Dependent Dual Risk Model
In a dual risk model, the premiums are considered as the costs and the claims are regarded as the profits.
