Search Results for author:
Found 39 papers, 4 papers with code
Optimizing ADMM and Over-Relaxed ADMM Parameters for Linear Quadratic Problems
The Alternating Direction Method of Multipliers (ADMM) has gained significant attention across a broad spectrum of machine learning applications.
Stability and Generalization for Minibatch SGD and Local SGD
We show both minibatch and local SGD achieve a linear speedup to attain the optimal risk bounds.
Generalization Guarantees of Gradient Descent for Multi-Layer Neural Networks
This sheds light on sufficient or necessary conditions for under-parameterized and over-parameterized NNs trained by GD to attain the desired risk rate of $O(1/\sqrt{n})$.
Generalization Analysis for Contrastive Representation Learning
For self-bounding Lipschitz loss functions, we further improve our results by developing optimistic bounds which imply fast rates in a low noise condition.
Generalization Bounds for Inductive Matrix Completion in Low-noise Settings
We study inductive matrix completion (matrix completion with side information) under an i. i. d.
On Stability and Generalization of Bilevel Optimization Problem
In this paper, we conduct a thorough analysis on the generalization of first-order (gradient-based) methods for the bilevel optimization problem.
Stability and Generalization Analysis of Gradient Methods for Shallow Neural Networks
While significant theoretical progress has been achieved, unveiling the generalization mystery of overparameterized neural networks still remains largely elusive.
Stability and Generalization for Markov Chain Stochastic Gradient Methods
To the best of our knowledge, this is the first generalization analysis of SGMs when the gradients are sampled from a Markov process.
Differentially Private Stochastic Gradient Descent with Low-Noise
In this paper, we focus on the privacy and utility (measured by excess risk bounds) performances of differentially private stochastic gradient descent (SGD) algorithms in the setting of stochastic convex optimization.
Stability and Generalization of Stochastic Optimization with Nonconvex and Nonsmooth Problems
In this paper, we initialize a systematic stability and generalization analysis of stochastic optimization on nonconvex and nonsmooth problems.
Differentially Private SGDA for Minimax Problems
We further provide its utility analysis in the nonconvex-strongly-concave setting which is the first-ever-known result in terms of the primal population risk.
Fine-grained Generalization Analysis of Inductive Matrix Completion
In this paper, we bridge the gap between the state-of-the-art theoretical results for matrix completion with the nuclear norm and their equivalent in \textit{inductive matrix completion}: (1) In the distribution-free setting, we prove bounds improving the previously best scaling of $O(rd^2)$ to $\widetilde{O}(d^{3/2}\sqrt{r})$, where $d$ is the dimension of the side information and $r$ is the rank.
Generalization Guarantee of SGD for Pairwise Learning
We develop a novel high-probability generalization bound for uniformly-stable algorithms to incorporate the variance information for better generalization, based on which we establish the first nonsmooth learning algorithm to achieve almost optimal high-probability and dimension-independent generalization bounds in linear time.
Simple Stochastic and Online Gradient Descent Algorithms for Pairwise Learning
A popular approach to handle streaming data in pairwise learning is an online gradient descent (OGD) algorithm, where one needs to pair the current instance with a buffering set of previous instances with a sufficiently large size and therefore suffers from a scalability issue.
Simple Stochastic and Online Gradient DescentAlgorithms for Pairwise Learning
A popular approach to handle streaming data in pairwise learning is an online gradient descent (OGD) algorithm, where one needs to pair the current instance with a buffering set of previous instances with a sufficiently large size and therefore suffers from a scalability issue.
Learning Interpretable Concept Groups in CNNs
We propose a novel training methodology -- Concept Group Learning (CGL) -- that encourages training of interpretable CNN filters by partitioning filters in each layer into concept groups, each of which is trained to learn a single visual concept.
Stability and Generalization for Randomized Coordinate Descent
Randomized coordinate descent (RCD) is a popular optimization algorithm with wide applications in solving various machine learning problems, which motivates a lot of theoretical analysis on its convergence behavior.
Fine-grained Generalization Analysis of Structured Output Prediction
Existing generalization analysis implies generalization bounds with at least a square-root dependency on the cardinality $d$ of the label set, which can be vacuous in practice.
Stability and Generalization of Stochastic Gradient Methods for Minimax Problems
In this paper, we provide a comprehensive generalization analysis of stochastic gradient methods for minimax problems under both convex-concave and nonconvex-nonconcave cases through the lens of algorithmic stability.
Fine-grained Generalization Analysis of Vector-valued Learning
In this paper, we initiate the generalization analysis of regularized vector-valued learning algorithms by presenting bounds with a mild dependency on the output dimension and a fast rate on the sample size.
Extreme Multi-Label Classification
General Classification
+2
Differentially Private SGD with Non-Smooth Losses
We significantly relax these restrictive assumptions and establish privacy and generalization (utility) guarantees for private SGD algorithms using output and gradient perturbations associated with non-smooth convex losses.
Sharper Generalization Bounds for Learning with Gradient-dominated Objective Functions
As a comparison, there is far less work to study the generalization behavior especially in a non-convex learning setting.
Sharper Generalization Bounds for Pairwise Learning
Pairwise learning refers to learning tasks with loss functions depending on a pair of training examples, which includes ranking and metric learning as specific examples.
Stochastic Hard Thresholding Algorithms for AUC Maximization
In this paper, we aim to develop stochastic hard thresholding algorithms for the important problem of AUC maximization in imbalanced classification.
Fine-Grained Analysis of Stability and Generalization for Stochastic Gradient Descent
In this paper, we provide a fine-grained analysis of stability and generalization for SGD by substantially relaxing these assumptions.
Optimal Stochastic and Online Learning with Individual Iterates
In this paper, we propose a theoretically sound strategy to select an individual iterate of the vanilla SCMD, which is able to achieve optimal rates for both convex and strongly convex problems in a non-smooth learning setting.
On Performance Estimation in Automatic Algorithm Configuration
Over the last decade, research on automated parameter tuning, often referred to as automatic algorithm configuration (AAC), has made significant progress.
Stochastic Proximal AUC Maximization
In this paper we consider the problem of maximizing the Area under the ROC curve (AUC) which is a widely used performance metric in imbalanced classification and anomaly detection.
Norm-based generalisation bounds for multi-class convolutional neural networks
This holds even when formulating the bounds in terms of the $L^2$-norm of the weight matrices, where previous bounds exhibit at least a square-root dependence on the number of classes.
A Generalization Error Bound for Multi-class Domain Generalization
Domain generalization is the problem of assigning labels to an unlabeled data set, given several similar data sets for which labels have been provided.
Stochastic Gradient Descent for Nonconvex Learning without Bounded Gradient Assumptions
While the behavior of SGD is well understood in the convex learning setting, the existing theoretical results for SGD applied to nonconvex objective functions are far from mature.
Stochastic Composite Mirror Descent: Optimal Bounds with High Probabilities
We apply the derived computational error bounds to study the generalization performance of multi-pass stochastic gradient descent (SGD) in a non-parametric setting.
Convergence of Online Mirror Descent
The condition is $\lim_{t\to\infty}\eta_t=0, \sum_{t=1}^{\infty}\eta_t=\infty$ in the case of positive variances.
Convergence of Unregularized Online Learning Algorithms
In this paper we study the convergence of online gradient descent algorithms in reproducing kernel Hilbert spaces (RKHSs) without regularization.
Data-dependent Generalization Bounds for Multi-class Classification
In this paper, we study data-dependent generalization error bounds exhibiting a mild dependency on the number of classes, making them suitable for multi-class learning with a large number of label classes.
Local Rademacher Complexity-based Learning Guarantees for Multi-Task Learning
We show a Talagrand-type concentration inequality for Multi-Task Learning (MTL), using which we establish sharp excess risk bounds for MTL in terms of distribution- and data-dependent versions of the Local Rademacher Complexity (LRC).
Local Rademacher Complexity Bounds based on Covering Numbers
This paper provides a general result on controlling local Rademacher complexities, which captures in an elegant form to relate the complexities with constraint on the expected norm to the corresponding ones with constraint on the empirical norm.
Localized Multiple Kernel Learning---A Convex Approach
We propose a localized approach to multiple kernel learning that can be formulated as a convex optimization problem over a given cluster structure.
Multi-class SVMs: From Tighter Data-Dependent Generalization Bounds to Novel Algorithms
This paper studies the generalization performance of multi-class classification algorithms, for which we obtain, for the first time, a data-dependent generalization error bound with a logarithmic dependence on the class size, substantially improving the state-of-the-art linear dependence in the existing data-dependent generalization analysis.
