Search Results for author:
Found 6 papers, 2 papers with code
A Penalty-Based Method for Communication-Efficient Decentralized Bilevel Programming
A key feature of the proposed algorithm is to estimate the hyper-gradient of the penalty function via decentralized computation of matrix-vector products and few vector communications, which is then integrated within an alternating algorithm to obtain finite-time convergence analysis under different convexity assumptions.
Dynamic Regret of Adaptive Gradient Methods for Strongly Convex Problems
Adaptive gradient algorithms such as ADAGRAD and its variants have gained popularity in the training of deep neural networks.
Online Bilevel Optimization: Regret Analysis of Online Alternating Gradient Methods
This paper introduces \textit{online bilevel optimization} in which a sequence of time-varying bilevel problems is revealed one after the other.
Dynamic Regret Analysis for Online Meta-Learning
The online meta-learning framework has arisen as a powerful tool for the continual lifelong learning setting.
Adaptive First-and Zeroth-order Methods for Weakly Convex Stochastic Optimization Problems
In this paper, we design and analyze a new family of adaptive subgradient methods for solving an important class of weakly convex (possibly nonsmooth) stochastic optimization problems.
DADAM: A Consensus-based Distributed Adaptive Gradient Method for Online Optimization
Adaptive gradient-based optimization methods such as \textsc{Adagrad}, \textsc{Rmsprop}, and \textsc{Adam} are widely used in solving large-scale machine learning problems including deep learning.
