Search Results for author:
Found 11 papers, 3 papers with code
Auto-Train-Once: Controller Network Guided Automatic Network Pruning from Scratch
Current techniques for deep neural network (DNN) pruning often involve intricate multi-step processes that require domain-specific expertise, making their widespread adoption challenging.
On the Role of Server Momentum in Federated Learning
To our best knowledge, this is the first work that thoroughly analyzes the performances of server momentum with a hyperparameter scheduler and system heterogeneity.
Leveraging Foundation Models to Improve Lightweight Clients in Federated Learning
Federated Learning (FL) is a distributed training paradigm that enables clients scattered across the world to cooperatively learn a global model without divulging confidential data.
Solving a Class of Non-Convex Minimax Optimization in Federated Learning
We propose FL algorithms (FedSGDA+ and FedSGDA-M) and reduce existing complexity results for the most common minimax problems.
Serverless Federated AUPRC Optimization for Multi-Party Collaborative Imbalanced Data Mining
To address the above challenge, we study the serverless multi-party collaborative AUPRC maximization problem since serverless multi-party collaborative training can cut down the communications cost by avoiding the server node bottleneck, and reformulate it as a conditional stochastic optimization problem in a serverless multi-party collaborative learning setting and propose a new ServerLess biAsed sTochastic gradiEnt (SLATE) algorithm to directly optimize the AUPRC.
Performance and Energy Consumption of Parallel Machine Learning Algorithms
Parallelization of training algorithms is a common strategy to speed up the process of training.
Decentralized Riemannian Algorithm for Nonconvex Minimax Problems
The minimax optimization over Riemannian manifolds (possibly nonconvex constraints) has been actively applied to solve many problems, such as robust dimensionality reduction and deep neural networks with orthogonal weights (Stiefel manifold).
Faster Adaptive Federated Learning
Federated learning has attracted increasing attention with the emergence of distributed data.
Distributed Dynamic Safe Screening Algorithms for Sparse Regularization
To the best of our knowledge, this is the first work of distributed safe dynamic screening method.
Efficient Mirror Descent Ascent Methods for Nonsmooth Minimax Problems
For our stochastic algorithms, we first prove that the mini-batch stochastic mirror descent ascent (SMDA) method obtains a sample complexity of $O(\kappa^3\epsilon^{-4})$ for finding an $\epsilon$-stationary point, where $\kappa$ denotes the condition number.
AdaGDA: Faster Adaptive Gradient Descent Ascent Methods for Minimax Optimization
Specifically, we propose a fast Adaptive Gradient Descent Ascent (AdaGDA) method based on the basic momentum technique, which reaches a lower gradient complexity of $\tilde{O}(\kappa^4\epsilon^{-4})$ for finding an $\epsilon$-stationary point without large batches, which improves the existing results of the adaptive GDA methods by a factor of $O(\sqrt{\kappa})$.
