no code implementations • 15 Apr 2024 • Nachuan Xiao, Kuangyu Ding, Xiaoyin Hu, Kim-Chuan Toh
Preliminary numerical experiments on deep learning tasks illustrate that our proposed framework yields efficient variants of Lagrangian-based methods with convergence guarantees for nonconvex nonsmooth constrained optimization problems.
no code implementations • 13 Oct 2023 • Kuangyu Ding, Nachuan Xiao, Kim-Chuan Toh
As a practical application of our proposed framework, we propose a novel Adam-family method named Adam with Decoupled Weight Decay (AdamD), and establish its convergence properties under mild conditions.
no code implementations • 26 Jun 2023 • Kuangyu Ding, Jingyang Li, Kim-Chuan Toh
Experimental results on representative benchmarks demonstrate the effectiveness and robustness of MSBPG in training neural networks.
