Search Results for author:
Found 9 papers, 3 papers with code
MAST: Model-Agnostic Sparsified Training
We introduce a novel optimization problem formulation that departs from the conventional way of minimizing machine learning model loss as a black-box function.
Towards a Better Theoretical Understanding of Independent Subnetwork Training
We identify fundamental differences between IST and alternative approaches, such as distributed methods with compressed communication, and provide a precise analysis of its optimization performance on a quadratic model.
Shifted Compression Framework: Generalizations and Improvements
Communication is one of the key bottlenecks in the distributed training of large-scale machine learning models, and lossy compression of exchanged information, such as stochastic gradients or models, is one of the most effective instruments to alleviate this issue.
Certified Robustness in Federated Learning
However, as in the single-node supervised learning setup, models trained in federated learning suffer from vulnerability to imperceptible input transformations known as adversarial attacks, questioning their deployment in security-related applications.
ADOM: Accelerated Decentralized Optimization Method for Time-Varying Networks
We propose ADOM - an accelerated method for smooth and strongly convex decentralized optimization over time-varying networks.
Uncertainty Principle for Communication Compression in Distributed and Federated Learning and the Search for an Optimal Compressor
In designing a compression method, one aims to communicate as few bits as possible, which minimizes the cost per communication round, while at the same time attempting to impart as little distortion (variance) to the communicated messages as possible, which minimizes the adverse effect of the compression on the overall number of communication rounds.
Adaptive Catalyst for Smooth Convex Optimization
In this paper, we present a generic framework that allows accelerating almost arbitrary non-accelerated deterministic and randomized algorithms for smooth convex optimization problems.
Optimization and Control
Revisiting Stochastic Extragradient
We fix a fundamental issue in the stochastic extragradient method by providing a new sampling strategy that is motivated by approximating implicit updates.
SGD: General Analysis and Improved Rates
By specializing our theorem to different mini-batching strategies, such as sampling with replacement and independent sampling, we derive exact expressions for the stepsize as a function of the mini-batch size.
