Search Results for author:
Found 11 papers, 4 papers with code
Beyond Lipschitz: Sharp Generalization and Excess Risk Bounds for Full-Batch GD
For nonconvex smooth losses, we prove that full-batch GD efficiently generalizes close to any stationary point at termination, and recovers the generalization error guarantees of stochastic algorithms with fewer assumptions.
Learning Distributionally Robust Models at Scale via Composite Optimization
To train machine learning models that are robust to distribution shifts in the data, distributionally robust optimization (DRO) has been proven very effective.
Black-Box Generalization: Stability of Zeroth-Order Learning
These bounds coincide with those for SGD, and rather surprisingly are independent of $d$, $K$ and the batch size $m$, under appropriate choices of a slightly decreased learning rate.
FedSKETCH: Communication-Efficient and Private Federated Learning via Sketching
Communication complexity and privacy are the two key challenges in Federated Learning where the goal is to perform a distributed learning through a large volume of devices.
Federated Learning with Compression: Unified Analysis and Sharp Guarantees
In federated learning, communication cost is often a critical bottleneck to scale up distributed optimization algorithms to collaboratively learn a model from millions of devices with potentially unreliable or limited communication and heterogeneous data distributions.
Efficient Fair Principal Component Analysis
It has been shown that dimension reduction methods such as PCA may be inherently prone to unfairness and treat data from different sensitive groups such as race, color, sex, etc., unfairly.
On the Convergence of Local Descent Methods in Federated Learning
To bridge this gap, we demonstrate that by properly analyzing the effect of unbiased gradients and sampling schema in federated setting, under mild assumptions, the implicit variance reduction feature of local distributed methods generalize to heterogeneous data shards and exhibits the best known convergence rates of homogeneous setting both in general nonconvex and under {\pl}~ condition (generalization of strong-convexity).
Local SGD with Periodic Averaging: Tighter Analysis and Adaptive Synchronization
Specifically, we show that for loss functions that satisfy the Polyak-{\L}ojasiewicz condition, $O((pT)^{1/3})$ rounds of communication suffice to achieve a linear speed up, that is, an error of $O(1/pT)$, where $T$ is the total number of model updates at each worker.
Trading Redundancy for Communication: Speeding up Distributed SGD for Non-convex Optimization
Communication overhead is one of the key challenges that hinder the scalability of distributed optimization algorithms to train large neural networks.
On the Optimal Recovery Threshold of Coded Matrix Multiplication
We provide novel coded computation strategies for distributed matrix-matrix products that outperform the recent "Polynomial code" constructions in recovery threshold, i. e., the required number of successful workers.
Information Theory Distributed, Parallel, and Cluster Computing Information Theory
Low-Complexity Stochastic Generalized Belief Propagation
However, this reduction can be larger than only one order of magnitude in alphabet size.
