Search Results for author:
Found 51 papers, 2 papers with code
High-dimensional Joint Sparsity Random Effects Model for Multi-task Learning
The traditional convex formulation employs the group Lasso relaxation to achieve joint sparsity across tasks.
High-dimensional Non-Gaussian Single Index Models via Thresholded Score Function Estimation
We consider estimating the parametric component of single index models in high dimensions.
On the Optimality of Kernel-Embedding Based Goodness-of-Fit Tests
The reproducing kernel Hilbert space (RKHS) embedding of distributions offers a general and flexible framework for testing problems in arbitrary domains and has attracted considerable amount of attention in recent years.
On Stein's Identity and Near-Optimal Estimation in High-dimensional Index Models
We consider estimating the parametric components of semi-parametric multiple index models in a high-dimensional and non-Gaussian setting.
Estimating High-dimensional Non-Gaussian Multiple Index Models via Stein’s Lemma
We consider estimating the parametric components of semiparametric multi-index models in high dimensions.
Tensor Methods for Additive Index Models under Discordance and Heterogeneity
Motivated by the sampling problems and heterogeneity issues common in high- dimensional big datasets, we consider a class of discordant additive index models.
Zeroth-order Nonconvex Stochastic Optimization: Handling Constraints, High-Dimensionality and Saddle-Points
In this paper, we propose and analyze zeroth-order stochastic approximation algorithms for nonconvex and convex optimization, with a focus on addressing constrained optimization, high-dimensional setting and saddle-point avoiding.
Zeroth-order (Non)-Convex Stochastic Optimization via Conditional Gradient and Gradient Updates
In this paper, we propose and analyze zeroth-order stochastic approximation algorithms for nonconvex and convex optimization.
Stochastic Zeroth-order Discretizations of Langevin Diffusions for Bayesian Inference
Our theoretical contributions extend the practical applicability of sampling algorithms to the noisy black-box and high-dimensional settings.
Normal Approximation for Stochastic Gradient Descent via Non-Asymptotic Rates of Martingale CLT
A crucial intermediate step is proving a non-asymptotic martingale central limit theorem (CLT), i. e., establishing the rates of convergence of a multivariate martingale difference sequence to a normal random vector, which might be of independent interest.
Multi-Point Bandit Algorithms for Nonstationary Online Nonconvex Optimization
In this paper, motivated by online reinforcement learning problems, we propose and analyze bandit algorithms for both general and structured nonconvex problems with nonstationary (or dynamic) regret as the performance measure, in both stochastic and non-stochastic settings.
Online and Bandit Algorithms for Nonstationary Stochastic Saddle-Point Optimization
We establish sub-linear regret bounds on the proposed notions of regret in both the online and bandit setting.
Zeroth-Order Algorithms for Nonconvex Minimax Problems with Improved Complexities
We establish that under the SGC assumption, the complexities of the stochastic algorithms match that of deterministic algorithms.
Stochastic Zeroth-order Riemannian Derivative Estimation and Optimization
We consider stochastic zeroth-order optimization over Riemannian submanifolds embedded in Euclidean space, where the task is to solve Riemannian optimization problem with only noisy objective function evaluations.
An Analysis of Constant Step Size SGD in the Non-convex Regime: Asymptotic Normality and Bias
Structured non-convex learning problems, for which critical points have favorable statistical properties, arise frequently in statistical machine learning.
Improved Complexities for Stochastic Conditional Gradient Methods under Interpolation-like Conditions
We analyze stochastic conditional gradient methods for constrained optimization problems arising in over-parametrized machine learning.
Fractal Gaussian Networks: A sparse random graph model based on Gaussian Multiplicative Chaos
We propose a novel stochastic network model, called Fractal Gaussian Network (FGN), that embodies well-defined and analytically tractable fractal structures.
Stochastic Multi-level Composition Optimization Algorithms with Level-Independent Convergence Rates
We show that the first algorithm, which is a generalization of \cite{GhaRuswan20} to the $T$ level case, can achieve a sample complexity of $\mathcal{O}(1/\epsilon^6)$ by using mini-batches of samples in each iteration.
Escaping Saddle-Points Faster under Interpolation-like Conditions
We next analyze Stochastic Cubic-Regularized Newton (SCRN) algorithm under interpolation-like conditions, and show that the oracle complexity to reach an $\epsilon$-local-minimizer under interpolation-like conditions, is $\tilde{\mathcal{O}}(1/\epsilon^{2. 5})$.
On the Ergodicity, Bias and Asymptotic Normality of Randomized Midpoint Sampling Method
The randomized midpoint method, proposed by [SL19], has emerged as an optimal discretization procedure for simulating the continuous time Langevin diffusions.
Escaping Saddle-Point Faster under Interpolation-like Conditions
We next analyze Stochastic Cubic-Regularized Newton (SCRN) algorithm under interpolation-like conditions, and show that the oracle complexity to reach an $\epsilon$-local-minimizer under interpolation-like conditions, is $O(1/\epsilon^{2. 5})$.
Statistical Inference for Polyak-Ruppert Averaged Zeroth-order Stochastic Gradient Algorithm
Statistical machine learning models trained with stochastic gradient algorithms are increasingly being deployed in critical scientific applications.
Nonparametric Modeling of Higher-Order Interactions via Hypergraphons
We study statistical and algorithmic aspects of using hypergraphons, that are limits of large hypergraphs, for modeling higher-order interactions.
On Empirical Risk Minimization with Dependent and Heavy-Tailed Data
In this work, we establish risk bounds for the Empirical Risk Minimization (ERM) with both dependent and heavy-tailed data-generating processes.
Topologically penalized regression on manifolds
We study a regression problem on a compact manifold M. In order to take advantage of the underlying geometry and topology of the data, the regression task is performed on the basis of the first several eigenfunctions of the Laplace-Beltrami operator of the manifold, that are regularized with topological penalties.
Heavy-tailed Sampling via Transformed Unadjusted Langevin Algorithm
We analyze the oracle complexity of sampling from polynomially decaying heavy-tailed target densities based on running the Unadjusted Langevin Algorithm on certain transformed versions of the target density.
A Projection-free Algorithm for Constrained Stochastic Multi-level Composition Optimization
We propose a projection-free conditional gradient-type algorithm for smooth stochastic multi-level composition optimization, where the objective function is a nested composition of $T$ functions and the constraint set is a closed convex set.
Towards a Theory of Non-Log-Concave Sampling: First-Order Stationarity Guarantees for Langevin Monte Carlo
For the task of sampling from a density $\pi \propto \exp(-V)$ on $\mathbb{R}^d$, where $V$ is possibly non-convex but $L$-gradient Lipschitz, we prove that averaged Langevin Monte Carlo outputs a sample with $\varepsilon$-relative Fisher information after $O( L^2 d^2/\varepsilon^2)$ iterations.
Mirror Descent Strikes Again: Optimal Stochastic Convex Optimization under Infinite Noise Variance
We study stochastic convex optimization under infinite noise variance.
Constrained Stochastic Nonconvex Optimization with State-dependent Markov Data
We study stochastic optimization algorithms for constrained nonconvex stochastic optimization problems with Markovian data.
A Flexible Approach for Normal Approximation of Geometric and Topological Statistics
We derive normal approximation results for a class of stabilizing functionals of binomial or Poisson point process, that are not necessarily expressible as sums of certain score functions.
Decentralized Stochastic Bilevel Optimization with Improved per-Iteration Complexity
Bilevel optimization recently has received tremendous attention due to its great success in solving important machine learning problems like meta learning, reinforcement learning, and hyperparameter optimization.
Regularized Stein Variational Gradient Flow
In this work, we propose the Regularized Stein Variational Gradient Flow which interpolates between the Stein Variational Gradient Flow and the Wasserstein Gradient Flow.
Improved Discretization Analysis for Underdamped Langevin Monte Carlo
As a byproduct, we also obtain the first KL divergence guarantees for ULMC without Hessian smoothness under strong log-concavity, which is based on a new result on the log-Sobolev constant along the underdamped Langevin diffusion.
High-dimensional Central Limit Theorems for Linear Functionals of Online Least-Squares SGD
Stochastic gradient descent (SGD) has emerged as the quintessential method in a data scientist's toolbox.
A One-Sample Decentralized Proximal Algorithm for Non-Convex Stochastic Composite Optimization
We focus on decentralized stochastic non-convex optimization, where $n$ agents work together to optimize a composite objective function which is a sum of a smooth term and a non-smooth convex term.
Mean-Square Analysis of Discretized Itô Diffusions for Heavy-tailed Sampling
We analyze the complexity of sampling from a class of heavy-tailed distributions by discretizing a natural class of It\^o diffusions associated with weighted Poincar\'e inequalities.
Towards a Complete Analysis of Langevin Monte Carlo: Beyond Poincaré Inequality
We do so by establishing upper and lower bounds for Langevin diffusions and LMC under weak Poincar\'e inequalities that are satisfied by a large class of densities including polynomially-decaying heavy-tailed densities (i. e., Cauchy-type).
High-dimensional scaling limits and fluctuations of online least-squares SGD with smooth covariance
We derive high-dimensional scaling limits and fluctuations for the online least-squares Stochastic Gradient Descent (SGD) algorithm by taking the properties of the data generating model explicitly into consideration.
Forward-backward Gaussian variational inference via JKO in the Bures-Wasserstein Space
Of key interest in statistics and machine learning is Gaussian VI, which approximates $\pi$ by minimizing the Kullback-Leibler (KL) divergence to $\pi$ over the space of Gaussians.
Optimal Algorithms for Stochastic Bilevel Optimization under Relaxed Smoothness Conditions
In this paper, we introduce a novel fully single-loop and Hessian-inversion-free algorithmic framework for stochastic bilevel optimization and present a tighter analysis under standard smoothness assumptions (first-order Lipschitzness of the UL function and second-order Lipschitzness of the LL function).
Gaussian random field approximation via Stein's method with applications to wide random neural networks
Specializing our general result, we obtain the first bounds on the Gaussian random field approximation of wide random neural networks of any depth and Lipschitz activation functions at the random field level.
Stochastic Nested Compositional Bi-level Optimization for Robust Feature Learning
We develop and analyze stochastic approximation algorithms for solving nested compositional bi-level optimization problems.
Online covariance estimation for stochastic gradient descent under Markovian sampling
We investigate the online overlapping batch-means covariance estimator for Stochastic Gradient Descent (SGD) under Markovian sampling.
Zeroth-order Riemannian Averaging Stochastic Approximation Algorithms
We present Zeroth-order Riemannian Averaging Stochastic Approximation (\texttt{Zo-RASA}) algorithms for stochastic optimization on Riemannian manifolds.
From Stability to Chaos: Analyzing Gradient Descent Dynamics in Quadratic Regression
We conduct a comprehensive investigation into the dynamics of gradient descent using large-order constant step-sizes in the context of quadratic regression models.
Adaptive and non-adaptive minimax rates for weighted Laplacian-eigenmap based nonparametric regression
We show both adaptive and non-adaptive minimax rates of convergence for a family of weighted Laplacian-Eigenmap based nonparametric regression methods, when the true regression function belongs to a Sobolev space and the sampling density is bounded from above and below.
Nonsmooth Nonparametric Regression via Fractional Laplacian Eigenmaps
More specifically, our approach is using the fractional Laplacian and is designed to handle the case when the true regression function lies in an $L_2$-fractional Sobolev space with order $s\in (0, 1)$.
Multivariate Gaussian Approximation for Random Forest via Region-based Stabilization
We derive Gaussian approximation bounds for random forest predictions based on a set of training points given by a Poisson process, under fairly mild regularity assumptions on the data generating process.
Meta-Learning with Generalized Ridge Regression: High-dimensional Asymptotics, Optimality and Hyper-covariance Estimation
Finally, we propose and analyze an estimator of the inverse covariance matrix of random regression coefficients based on data from the training tasks.
Minimax Optimal Goodness-of-Fit Testing with Kernel Stein Discrepancy
We explore the minimax optimality of goodness-of-fit tests on general domains using the kernelized Stein discrepancy (KSD).
