Search Results for author:
Found 10 papers, 1 papers with code
On the Optimization and Generalization of Multi-head Attention
Finally, we demonstrate that these conditions are satisfied for a simple tokenized-mixture model.
Fast Convergence in Learning Two-Layer Neural Networks with Separable Data
Normalized gradient descent has shown substantial success in speeding up the convergence of exponentially-tailed loss functions (which includes exponential and logistic losses) on linear classifiers with separable data.
Generalization and Stability of Interpolating Neural Networks with Minimal Width
Specifically, in a realizable scenario where model weights can achieve arbitrarily small training error $\epsilon$ and their distance from initialization is $g(\epsilon)$, we demonstrate that gradient descent with $n$ training data achieves training error $O(g(1/T)^2 /T)$ and generalization error $O(g(1/T)^2 /n)$ at iteration $T$, provided there are at least $m=\Omega(g(1/T)^4)$ hidden neurons.
On Generalization of Decentralized Learning with Separable Data
Motivated by overparameterized learning settings, in which models are trained to zero training loss, we study algorithmic and generalization properties of decentralized learning with gradient descent on separable data.
Asymptotic Behavior of Adversarial Training in Binary Classification
It has been consistently reported that many machine learning models are susceptible to adversarial attacks i. e., small additive adversarial perturbations applied to data points can cause misclassification.
Fundamental Limits of Ridge-Regularized Empirical Risk Minimization in High Dimensions
For a stylized setting with Gaussian features and problem dimensions that grow large at a proportional rate, we start with sharp performance characterizations and then derive tight lower bounds on the estimation and prediction error that hold over a wide class of loss functions and for any value of the regularization parameter.
Quantized Decentralized Stochastic Learning over Directed Graphs
We consider a decentralized stochastic learning problem where data points are distributed among computing nodes communicating over a directed graph.
Sharp Asymptotics and Optimal Performance for Inference in Binary Models
We study convex empirical risk minimization for high-dimensional inference in binary models.
Sharp Guarantees for Solving Random Equations with One-Bit Information
We study the performance of a wide class of convex optimization-based estimators for recovering a signal from corrupted one-bit measurements in high-dimensions.
Robust and Communication-Efficient Collaborative Learning
We consider a decentralized learning problem, where a set of computing nodes aim at solving a non-convex optimization problem collaboratively.
