EmbRace: Accelerating Sparse Communication for Distributed Training of NLP Neural Networks

no code implementations • 18 Oct 2021 • Shengwei Li, Zhiquan Lai, Dongsheng Li, Yiming Zhang, Xiangyu Ye, Yabo Duan

EmbRace introduces Sparsity-aware Hybrid Communication, which integrates AlltoAll and model parallelism into data-parallel training, so as to reduce the communication overhead of highly sparse parameters.

Image Classification Scheduling

Beating the Standard Quantum Limit under Ambient Conditions with Solid-State Spins

no code implementations • 28 Jan 2021 • Tianyu Xie, Zhiyuan Zhao, Xi Kong, Wenchao Ma, Mengqi Wang, Xiangyu Ye, Pei Yu, Zhiping Yang, Shaoyi Xu, Pengfei Wang, Ya Wang, Fazhan Shi, Jiangfeng Du

However, it has not been realized in solid-state spin systems at ambient conditions, owing to its intrinsic complexity for the preparation and survival of pure and entangled quantum states.

Quantum Physics