Search Results for author:
Found 28 papers, 2 papers with code
Differentially Private Online Federated Learning with Correlated Noise
We propose a novel differentially private algorithm for online federated learning that employs temporally correlated noise to improve the utility while ensuring the privacy of the continuously released models.
Asynchronous Distributed Optimization with Delay-free Parameters
In contrast to alternatives, our algorithms can converge to the fixed point set of their synchronous counterparts using step-sizes that are independent of the delays.
Composite federated learning with heterogeneous data
We propose a novel algorithm for solving the composite Federated Learning (FL) problem.
Dynamic Privacy Allocation for Locally Differentially Private Federated Learning with Composite Objectives
This paper proposes a locally differentially private federated learning algorithm for strongly convex but possibly nonsmooth problems that protects the gradients of each worker against an honest but curious server.
Delay-adaptive step-sizes for asynchronous learning
In this paper, we show that it is possible to use learning rates that depend on the actual time-varying delays in the system.
On Uniform Boundedness Properties of SGD and its Momentum Variants
In this note, we investigate uniform boundedness properties of iterates and function values along the trajectories of the stochastic gradient descent algorithm and its important momentum variant.
A Fast and Accurate Splitting Method for Optimal Transport: Analysis and Implementation
In addition, we establish a linear convergence rate for our formulation of the OT problem.
Asynchronous Iterations in Optimization: New Sequence Results and Sharper Algorithmic Guarantees
We introduce novel convergence results for asynchronous iterations that appear in the analysis of parallel and distributed optimization algorithms.
Bandwidth-based Step-Sizes for Non-Convex Stochastic Optimization
Many popular learning-rate schedules for deep neural networks combine a decaying trend with local perturbations that attempt to escape saddle points and bad local minima.
On the Convergence of Step Decay Step-Size for Stochastic Optimization
The convergence of stochastic gradient descent is highly dependent on the step-size, especially on non-convex problems such as neural network training.
Stability and Convergence of Stochastic Gradient Clipping: Beyond Lipschitz Continuity and Smoothness
We also study the convergence of a clipped method with momentum, which includes clipped SGD as a special case, for weakly convex problems under standard assumptions.
Advances in Asynchronous Parallel and Distributed Optimization
Motivated by large-scale optimization problems arising in the context of machine learning, there have been several advances in the study of asynchronous parallel and distributed optimization methods during the past decade.
A flexible framework for communication-efficient machine learning: from HPC to IoT
With the increasing scale of machine learning tasks, it has become essential to reduce the communication between computing nodes.
Convergence of a Stochastic Gradient Method with Momentum for Non-Smooth Non-Convex Optimization
This paper establishes the convergence rate of a stochastic subgradient method with a momentum term of Polyak type for a broad class of non-smooth, non-convex, and constrained optimization problems.
Anderson Acceleration of Proximal Gradient Methods
We therefore propose a simple scheme for stabilization that combines the global worst-case guarantees of proximal gradient methods with the local adaptation and practical speed-up of Anderson acceleration.
Efficient Stochastic Programming in Julia
These structure-exploiting solvers are based on variations of the classical L-shaped and progressive-hedging algorithms.
Optimization and Control Mathematical Software 90C15, 90C06, 90C90
Compressed Gradient Methods with Hessian-Aided Error Compensation
Several gradient compression techniques have been proposed to reduce the communication load at the price of a loss in solution accuracy.
Noisy Accelerated Power Method for Eigenproblems with Applications
This paper introduces an efficient algorithm for finding the dominant generalized eigenvectors of a pair of symmetric matrices.
Curvature-Exploiting Acceleration of Elastic Net Computations
This paper introduces an efficient second-order method for solving the elastic net problem.
Harnessing the Power of Serverless Runtimes for Large-Scale Optimization
The event-driven and elastic nature of serverless runtimes makes them a very efficient and cost-effective alternative for scaling up computations.
POLO: a POLicy-based Optimization library
We present POLO --- a C++ library for large-scale parallel optimization research that emphasizes ease-of-use, flexibility and efficiency in algorithm design.
The Convergence of Sparsified Gradient Methods
Distributed training of massive machine learning models, in particular deep neural networks, via Stochastic Gradient Descent (SGD) is becoming commonplace.
Distributed learning with compressed gradients
Asynchronous computation and gradient compression have emerged as two key techniques for achieving scalability in distributed optimization for large-scale machine learning.
Continuous-time Value Function Approximation in Reproducing Kernel Hilbert Spaces
Motivated by the success of reinforcement learning (RL) for discrete-time tasks such as AlphaGo and Atari games, there has been a recent surge of interest in using RL for continuous-time control of physical systems (cf.
Analysis and Implementation of an Asynchronous Optimization Algorithm for the Parameter Server
This paper presents an asynchronous incremental aggregated gradient algorithm and its implementation in a parameter server framework for solving regularized optimization problems.
Optimal Radio Frequency Energy Harvesting with Limited Energy Arrival Knowledge
While the idea of RF-EH is appealing, it is not always beneficial to attempt to harvest energy; in environments where the ambient energy is low, nodes could consume more energy being awake with their harvesting circuits turned on than what they can extract from the ambient radio signals; it is then better to enter a sleep mode until the ambient RF energy increases.
An Asynchronous Mini-Batch Algorithm for Regularized Stochastic Optimization
Mini-batch optimization has proven to be a powerful paradigm for large-scale learning.
Stochastic Online Shortest Path Routing: The Value of Feedback
The parameters, and hence the optimal path, can only be estimated by routing packets through the network and observing the realized delays.
