Search Results for author:
Found 19 papers, 3 papers with code
Personalized Federated Learning with Multiple Known Clusters
We consider the problem of personalized federated learning when there are known cluster structures within users.
A Field Guide to Federated Optimization
Federated learning and analytics are a distributed approach for collaboratively learning models (or statistics) from decentralized data, motivated by and designed for privacy protection.
Personalized Federated Learning: A Unified Framework and Universal Optimization Techniques
We investigate the optimization aspects of personalized Federated Learning (FL).
Smoothness Matrices Beat Smoothness Constants: Better Communication Compression Techniques for Distributed Optimization
In order to further alleviate the communication burden inherent in distributed optimization, we propose a novel communication sparsification strategy that can take full advantage of the smoothness matrices associated with local losses.
Local SGD: Unified Theory and New Efficient Methods
We present a unified framework for analyzing local SGD methods in the convex and strongly convex regimes for distributed/federated training of supervised machine learning models.
Lower Bounds and Optimal Algorithms for Personalized Federated Learning
Our first contribution is establishing the first lower bounds for this formulation, for both the communication complexity and the local oracle complexity.
Optimization for Supervised Machine Learning: Randomized Algorithms for Data and Parameters
With the increase of the volume of data and the size and complexity of the statistical models used to formulate these often ill-conditioned optimization tasks, there is a need for new efficient algorithms able to cope with these challenges.
Stochastic Subspace Cubic Newton Method
In this paper, we propose a new randomized second-order optimization algorithm---Stochastic Subspace Cubic Newton (SSCN)---for minimizing a high dimensional convex function $f$.
Variance Reduced Coordinate Descent with Acceleration: New Method With a Surprising Application to Finite-Sum Problems
We propose an accelerated version of stochastic variance reduced coordinate descent -- ASVRCD.
Federated Learning of a Mixture of Global and Local Models
We propose a new optimization formulation for training federated learning models.
Learning to Optimize via Dual space Preconditioning
Preconditioning an minimization algorithm improve its convergence and can lead to a minimizer in one iteration in some extreme cases.
A Unified Theory of SGD: Variance Reduction, Sampling, Quantization and Coordinate Descent
In this paper we introduce a unified analysis of a large family of variants of proximal stochastic gradient descent ({\tt SGD}) which so far have required different intuitions, convergence analyses, have different applications, and which have been developed separately in various communities.
One Method to Rule Them All: Variance Reduction for Data, Parameters and Many New Methods
We propose a remarkably general variance-reduced method suitable for solving regularized empirical risk minimization problems with either a large number of training examples, or a large model dimension, or both.
Best Pair Formulation & Accelerated Scheme for Non-convex Principal Component Pursuit
The best pair problem aims to find a pair of points that minimize the distance between two disjoint sets.
99% of Distributed Optimization is a Waste of Time: The Issue and How to Fix it
We propose a fix based on a new update-sparsification method we develop in this work, which we suggest be used on top of existing methods.
A Privacy Preserving Randomized Gossip Algorithm via Controlled Noise Insertion
In this work we present a randomized gossip algorithm for solving the average consensus problem while at the same time protecting the information about the initial private values stored at the nodes.
SEGA: Variance Reduction via Gradient Sketching
In each iteration, SEGA updates the current estimate of the gradient through a sketch-and-project operation using the information provided by the latest sketch, and this is subsequently used to compute an unbiased estimate of the true gradient through a random relaxation procedure.
A Nonconvex Projection Method for Robust PCA
Robust principal component analysis (RPCA) is a well-studied problem with the goal of decomposing a matrix into the sum of low-rank and sparse components.
Privacy Preserving Randomized Gossip Algorithms
In this work we present three different randomized gossip algorithms for solving the average consensus problem while at the same time protecting the information about the initial private values stored at the nodes.
Optimization and Control
