Efficient All-reduce for Distributed DNN Training in Optical Interconnect System

no code implementations • 22 Jul 2022 • Fei Dai, Yawen Chen, Zhiyi Huang, Haibo Zhang, Fangfang Zhang

Our results also show that WRHT can reduce the communication time of all-reduce operation by 92. 42% and 91. 31% compared to two existing all-reduce algorithms running in the electrical interconnect system.

Accelerating Fully Connected Neural Network on Optical Network-on-Chip (ONoC)

no code implementations • 30 Sep 2021 • Fei Dai, Yawen Chen, Haibo Zhang, Zhiyi Huang

Compared with ENoC, simulation results show that under batch sizes of 64 and 128, on average ONoC can achieve 21. 02% and 12. 95% on reducing training time with 47. 85% and 39. 27% on saving energy, respectively.

Efficient Second-Order Optimization for Deep Learning with Kernel Machines

no code implementations • 29 Sep 2021 • Yawen Chen, Zeyi Wen, Yile Chen, Jian Chen, Jin Huang

However, the recomputation of the Hessian matrix in the second-order optimization posts much extra computation and memory burden in the training.

Improving Efficiency of SVM k-fold Cross-validation by Alpha Seeding

no code implementations • 23 Nov 2016 • Zeyi Wen, Bin Li, Rao Kotagiri, Jian Chen, Yawen Chen, Rui Zhang

The k-fold cross-validation is commonly used to evaluate the effectiveness of SVMs with the selected hyper-parameters.