BigDL 2.0: Seamless Scaling of AI Pipelines from Laptops to Distributed Cluster

1 code implementation • CVPR 2022 • Jason Dai, Ding Ding, Dongjie Shi, Shengsheng Huang, Jiao Wang, Xin Qiu, Kai Huang, Guoqiong Song, Yang Wang, Qiyuan Gong, Jiaming Song, Shan Yu, Le Zheng, Yina Chen, Junwei Deng, Ge Song

To address this challenge, we have open sourced BigDL 2. 0 at https://github. com/intel-analytics/BigDL/ under Apache 2. 0 license (combining the original BigDL and Analytics Zoo projects); using BigDL 2. 0, users can simply build conventional Python notebooks on their laptops (with possible AutoML support), which can then be transparently accelerated on a single node (with up-to 9. 6x speedup in our experiments), and seamlessly scaled out to a large cluster (across several hundreds servers in real-world use cases).

AutoML Distributed Computing +1