Search Results for author:
Found 18 papers, 5 papers with code
From Local SGD to Local Fixed Point Methods for Federated Learning
Most algorithms for solving optimization problems or finding saddle points of convex-concave functions are fixed point algorithms.
FedComLoc: Communication-Efficient Distributed Training of Sparse and Quantized Models
Federated Learning (FL) has garnered increasing attention due to its unique characteristic of allowing heterogeneous clients to process their private data locally and interact with a central server, while being respectful of privacy.
LoCoDL: Communication-Efficient Distributed Learning with Local Training and Compression
In Distributed optimization and Learning, and even more in the modern framework of federated learning, communication, which is slow and costly, is critical.
Revisiting Decentralized ProxSkip: Achieving Linear Speedup
We demonstrate that the leading communication complexity of ProxSkip is $\mathcal{O}\left(\frac{p\sigma^2}{n\epsilon^2}\right)$ for non-convex and convex settings, and $\mathcal{O}\left(\frac{p\sigma^2}{n\epsilon}\right)$ for the strongly convex setting, where $n$ represents the number of nodes, $p$ denotes the probability of communication, $\sigma^2$ signifies the level of stochastic noise, and $\epsilon$ denotes the desired accuracy level.
Near-Linear Time Projection onto the $\ell_{1,\infty}$ Ball; Application to Sparse Autoencoders
In this paper, we introduce a new projection algorithm for the $\ell_{1,\infty}$ norm ball.
Explicit Personalization and Local Training: Double Communication Acceleration in Federated Learning
Federated Learning is an evolving machine learning paradigm, in which multiple clients perform computations based on their individual private data, interspersed by communication with a remote server.
TAMUNA: Doubly Accelerated Federated Learning with Local Training, Compression, and Partial Participation
In federated learning, a large number of users collaborate to learn a global model.
Provably Doubly Accelerated Federated Learning: The First Theoretically Successful Combination of Local Training and Communication Compression
In federated learning, a large number of users are involved in a global learning task, in a collaborative way.
Joint demosaicing and fusion of multiresolution coded acquisitions: A unified image formation and reconstruction method
Novel optical imaging devices allow for hybrid acquisition modalities such as compressed acquisitions with locally different spatial and spectral resolutions captured by a single focal plane array.
EF-BV: A Unified Theory of Error Feedback and Variance Reduction Mechanisms for Biased and Unbiased Compression in Distributed Optimization
Our general approach works with a new, larger class of compressors, which has two parameters, the bias and the variance, and includes unbiased and biased compressors as particular cases.
Tikhonov Regularization of Circle-Valued Signals
It is common to have to process signals or images whose values are cyclic and can be represented as points on the complex circle, like wrapped phases, angles, orientations, or color hues.
MURANA: A Generic Framework for Stochastic Variance-Reduced Optimization
We propose a generic variance-reduced algorithm, which we call MUltiple RANdomized Algorithm (MURANA), for minimizing a sum of several smooth functions plus a regularizer, in a sequential or distributed manner.
An Optimal Algorithm for Strongly Convex Minimization under Affine Constraints
Optimization problems under affine constraints appear in various areas of machine learning.
Optimization and Control
Optimal Gradient Compression for Distributed and Federated Learning
In the average-case analysis, we design a simple compression operator, Spherical Compression, which naturally achieves the lower bound.
Distributed Proximal Splitting Algorithms with Rates and Acceleration
We analyze several generic proximal splitting algorithms well suited for large-scale convex nonsmooth optimization.
From Local SGD to Local Fixed-Point Methods for Federated Learning
Most algorithms for solving optimization problems or finding saddle points of convex-concave functions are fixed-point algorithms.
Dualize, Split, Randomize: Toward Fast Nonsmooth Optimization Algorithms
We consider minimizing the sum of three convex functions, where the first one F is smooth, the second one is nonsmooth and proximable and the third one is the composition of a nonsmooth proximable function with a linear operator L. This template problem has many applications, for instance, in image processing and machine learning.
On-the-fly Approximation of Multivariate Total Variation Minimization
In the context of change-point detection, addressed by Total Variation minimization strategies, an efficient on-the-fly algorithm has been designed leading to exact solutions for univariate data.
