Search Results for author:
Found 41 papers, 13 papers with code
Differentially Private Non-convex Learning for Multi-layer Neural Networks
This paper focuses on the problem of Differentially Private Stochastic Optimization for (multi-layer) fully connected neural networks with a single output node.
Outlier Robust Adversarial Training
Theoretically, we show that the learning objective of ORAT satisfies the $\mathcal{H}$-consistency in binary classification, which establishes it as a proper surrogate to adversarial 0/1 loss.
Stability and Generalization of Stochastic Compositional Gradient Descent Algorithms
Then, we establish the compositional uniform stability results for two popular stochastic compositional gradient descent algorithms, namely SCGD and SCSC.
Generalization Guarantees of Gradient Descent for Multi-Layer Neural Networks
This sheds light on sufficient or necessary conditions for under-parameterized and over-parameterized NNs trained by GD to attain the desired risk rate of $O(1/\sqrt{n})$.
Fairness-aware Differentially Private Collaborative Filtering
Recently, there has been an increasing adoption of differential privacy guided algorithms for privacy-preserving machine learning tasks.
Generalization Analysis for Contrastive Representation Learning
For self-bounding Lipschitz loss functions, we further improve our results by developing optimistic bounds which imply fast rates in a low noise condition.
Stability and Generalization Analysis of Gradient Methods for Shallow Neural Networks
While significant theoretical progress has been achieved, unveiling the generalization mystery of overparameterized neural networks still remains largely elusive.
Stability and Generalization for Markov Chain Stochastic Gradient Methods
To the best of our knowledge, this is the first generalization analysis of SGMs when the gradients are sampled from a Markov process.
Differentially Private Stochastic Gradient Descent with Low-Noise
In this paper, we focus on the privacy and utility (measured by excess risk bounds) performances of differentially private stochastic gradient descent (SGD) algorithms in the setting of stochastic convex optimization.
Minimax AUC Fairness: Efficient Algorithm with Provable Convergence
We conduct numerical experiments on both synthetic and real-world datasets to validate the effectiveness of the minimax framework and the proposed optimization algorithm.
AUC Maximization in the Era of Big Data and AI: A Survey
We also identify and discuss remaining and emerging issues for deep AUC maximization, and provide suggestions on topics for future work.
Differentially Private SGDA for Minimax Problems
We further provide its utility analysis in the nonconvex-strongly-concave setting which is the first-ever-known result in terms of the primal population risk.
Label Distributionally Robust Losses for Multi-class Classification: Consistency, Robustness and Adaptivity
We study a family of loss functions named label-distributionally robust (LDR) losses for multi-class classification that are formulated from distributionally robust optimization (DRO) perspective, where the uncertainty in the given label information are modeled and captured by taking the worse case of distributional weights.
Simple Stochastic and Online Gradient Descent Algorithms for Pairwise Learning
A popular approach to handle streaming data in pairwise learning is an online gradient descent (OGD) algorithm, where one needs to pair the current instance with a buffering set of previous instances with a sufficiently large size and therefore suffers from a scalability issue.
Generalization Guarantee of SGD for Pairwise Learning
We develop a novel high-probability generalization bound for uniformly-stable algorithms to incorporate the variance information for better generalization, based on which we establish the first nonsmooth learning algorithm to achieve almost optimal high-probability and dimension-independent generalization bounds in linear time.
Simple Stochastic and Online Gradient DescentAlgorithms for Pairwise Learning
A popular approach to handle streaming data in pairwise learning is an online gradient descent (OGD) algorithm, where one needs to pair the current instance with a buffering set of previous instances with a sufficiently large size and therefore suffers from a scalability issue.
Memory-Based Optimization Methods for Model-Agnostic Meta-Learning and Personalized Federated Learning
The proposed algorithms require sampling a constant number of tasks and data samples per iteration, making them suitable for the continual learning scenario.
Sum of Ranked Range Loss for Supervised Learning
A combination loss of AoRR and TKML is proposed as a new learning objective for improving the robustness of multi-label learning in the face of outliers in sample and labels alike.
Stability and Generalization of Stochastic Gradient Methods for Minimax Problems
In this paper, we provide a comprehensive generalization analysis of stochastic gradient methods for minimax problems under both convex-concave and nonconvex-nonconcave cases through the lens of algorithmic stability.
Federated Deep AUC Maximization for Heterogeneous Data with a Constant Communication Complexity
Deep AUC (area under the ROC curve) Maximization (DAM) has attracted much attention recently due to its great potential for imbalanced data classification.
Differentially Private SGD with Non-Smooth Losses
We significantly relax these restrictive assumptions and establish privacy and generalization (utility) guarantees for private SGD algorithms using output and gradient perturbations associated with non-smooth convex losses.
Sharper Generalization Bounds for Learning with Gradient-dominated Objective Functions
As a comparison, there is far less work to study the generalization behavior especially in a non-convex learning setting.
Stochastic Hard Thresholding Algorithms for AUC Maximization
In this paper, we aim to develop stochastic hard thresholding algorithms for the important problem of AUC maximization in imbalanced classification.
Learning by Minimizing the Sum of Ranked Range
In forming learning objectives, one oftentimes needs to aggregate a set of individual values to a single output.
Online AUC Optimization for Sparse High-Dimensional Datasets
The Area Under the ROC Curve (AUC) is a widely used performance measure for imbalanced classification arising from many application domains where high-dimensional sparse data is abundant.
Fine-Grained Analysis of Stability and Generalization for Stochastic Gradient Descent
In this paper, we provide a fine-grained analysis of stability and generalization for SGD by substantially relaxing these assumptions.
Stochastic AUC Maximization with Deep Neural Networks
In this paper, we consider stochastic AUC maximization problem with a deep neural network as the predictive model.
Stochastic Proximal AUC Maximization
In this paper we consider the problem of maximizing the Area under the ROC curve (AUC) which is a widely used performance metric in imbalanced classification and anomaly detection.
Dual Averaging Method for Online Graph-structured Sparsity
Online learning algorithms update models via one sample per iteration, thus efficient to process large-scale datasets and useful to detect malicious events for social benefits, such as disease outbreak and traffic congestion on the fly.
Stochastic Iterative Hard Thresholding for Graph-structured Sparsity Optimization
Stochastic optimization algorithms update models with cheap per-iteration costs sequentially, which makes them amenable for large-scale data analysis.
Stability and Optimization Error of Stochastic Gradient Descent for Pairwise Learning
From this fundamental trade-off, we obtain lower bounds for the optimization error of SGD algorithms and the excess expected risk over a class of pairwise losses.
Stochastic Proximal Algorithms for AUC Maximization
Stochastic optimization algorithms such as SGDs update the model sequentially with cheap per-iteration costs, making them amenable for large-scale data analysis.
A Univariate Bound of Area Under ROC
In this work, we describe a new surrogate loss based on a reformulation of the AUC risk, which does not require pairwise comparison but rankings of the predictions.
Learning with Correntropy-induced Losses for Regression with Mixture of Symmetric Stable Noise
Motivated by the practical way of generating non-Gaussian noise or outliers, we introduce mixture of symmetric stable noise, which include Gaussian noise, Cauchy noise, and their mixture as special cases, to model non-Gaussian noise or outliers.
Learning with Average Top-k Loss
We further give a learning theory analysis of \matk learning on the classification calibration of the \atk loss and the error bounds of \atk-SVM.
Stochastic Online AUC Maximization
From this saddle representation, a stochastic online algorithm (SOLAM) is proposed which has time and space complexity of one datum.
Unregularized Online Learning Algorithms with General Loss Functions
Firstly, we derive explicit convergence rates of the unregularized online learning algorithms for classification associated with a general gamma-activating loss (see Definition 1 in the paper).
Online Pairwise Learning Algorithms with Kernels
In this paper, we study an online algorithm for pairwise learning with a least-square loss function in an unconstrained setting of a reproducing kernel Hilbert space (RKHS), which we refer to as the Online Pairwise lEaRning Algorithm (OPERA).
Guaranteed Classification via Regularized Similarity Learning
In this paper, we propose a regularized similarity learning formulation associated with general matrix-norms, and establish their generalization bounds.
Analysis of SVM with Indefinite Kernels
The recent introduction of indefinite SVM by Luss and dAspremont [15] has effectively demonstrated SVM classification with a non-positive semi-definite kernel (indefinite kernel).
Sparse Metric Learning via Smooth Optimization
From this saddle representation, we develop an efficient smooth optimization approach for sparse metric learning although the learning model is based on a non-differential loss function.
