Inverse-Free Fast Natural Gradient Descent Method for Deep Learning

no code implementations • 6 Mar 2024 • Xinwei Ou, Ce Zhu, Xiaolin Huang, Yipeng Liu

Second-order optimization techniques have the potential to achieve faster convergence rates compared to first-order methods through the incorporation of second-order derivatives or statistics.

Image Classification Machine Translation +1

Low Rank Optimization for Efficient Deep Learning: Making A Balance between Compact Architecture and Fast Training

no code implementations • 22 Mar 2023 • Xinwei Ou, Zhangxin Chen, Ce Zhu, Yipeng Liu

However, the high computational complexity and storage cost makes deep learning hard to be used on resource-constrained devices, and it is not environmental-friendly with much power cost.

Model Compression Quantization