Search Results for author:
Found 16 papers, 4 papers with code
ZO-AdaMM: Zeroth-Order Adaptive Momentum Method for Black-Box Optimization
In this paper, we propose a zeroth-order AdaMM (ZO-AdaMM) algorithm, that generalizes AdaMM to the gradient-free regime.
Face Aging via Diffusion-based Editing
We propose FADING, a novel approach to address Face Aging via DIffusion-based editiNG.
Min-Max Optimization without Gradients: Convergence and Applications to Adversarial ML
In this paper, we study the problem of constrained robust (min-max) optimization ina black-box setting, where the desired optimizer cannot access the gradients of the objective function but may query its values.
Distributed Adversarial Training to Robustify Deep Neural Networks at Scale
Spurred by that, we propose distributed adversarial training (DAT), a large-batch adversarial training framework implemented over multiple machines.
On the Convergence of A Class of Adam-Type Algorithms for Non-Convex Optimization
We prove that under our derived conditions, these methods can achieve the convergence rate of order $O(\log{T}/\sqrt{T})$ for nonconvex stochastic optimization.
signSGD via Zeroth-Order Oracle
Our study shows that ZO signSGD requires $\sqrt{d}$ times more iterations than signSGD, leading to a convergence rate of $O(\sqrt{d}/\sqrt{T})$ under mild conditions, where $d$ is the number of optimization variables, and $T$ is the number of iterations.
Understand the dynamics of GANs via Primal-Dual Optimization
By developing new primal-dual optimization tools, we show that, with a proper stepsize choice, the widely used first-order iterative algorithm in training GANs would in fact converge to a stationary solution with a sublinear rate.
Distributed Training with Heterogeneous Data: Bridging Median- and Mean-Based Algorithms
We show that these algorithms are non-convergent whenever there is some disparity between the expected median and mean over the local gradients.
Private Stochastic Non-Convex Optimization: Adaptive Algorithms and Tighter Generalization Bounds
We obtain this rate by providing the first analyses on a collection of private gradient-based methods, including adaptive algorithms DP RMSProp and DP Adam.
Understanding Gradient Clipping in Private SGD: A Geometric Perspective
Deep learning models are increasingly popular in many machine learning applications where the training data may contain sensitive information.
Min-Max Optimization without Gradients: Convergence and Applications to Black-Box Evasion and Poisoning Attacks
In this paper, we study the problem of constrained min-max optimization in a black-box setting, where the desired optimizer cannot access the gradients of the objective function but may query its values.
Convergent Adaptive Gradient Methods in Decentralized Optimization
Specifically, we propose a general algorithmic framework that can convert existing adaptive gradient methods to their decentralized counterparts.
Understanding Clipping for Federated Learning: Convergence and Client-Level Differential Privacy
Recently, there has been a line of work on incorporating the formal privacy notion of differential privacy with FL.
On the Convergence of Decentralized Adaptive Gradient Methods
Adaptive gradient methods including Adam, AdaGrad, and their variants have been very successful for training deep learning models, such as neural networks.
Toward Communication Efficient Adaptive Gradient Method
While adaptive gradient methods have been proven effective for training neural nets, the study of adaptive gradient methods in federated learning is scarce.
Dynamic Differential-Privacy Preserving SGD
The equivalent privacy costs controlled by maintaining the same gradient clipping thresholds and noise powers in each step result in unstable updates and a lower model accuracy when compared to the non-DP counterpart.
