Search Results for author: Aohan Zeng

Found 3 papers, 3 papers with code

GLM-130B: An Open Bilingual Pre-trained Model

2 code implementations • 5 Oct 2022 • Aohan Zeng, Xiao Liu, Zhengxiao Du, Zihan Wang, Hanyu Lai, Ming Ding, Zhuoyi Yang, Yifan Xu, Wendi Zheng, Xiao Xia, Weng Lam Tam, Zixuan Ma, Yufei Xue, Jidong Zhai, WenGuang Chen, Peng Zhang, Yuxiao Dong, Jie Tang

We introduce GLM-130B, a bilingual (English and Chinese) pre-trained language model with 130 billion parameters.

Ranked #1 on Language Modelling on CLUE (CMRC2018)

Language Modelling Multi-task Language Understanding +1

1,688

Paper
Code

FastMoE: A Fast Mixture-of-Expert Training System

3 code implementations • 24 Mar 2021 • Jiaao He, Jiezhong Qiu, Aohan Zeng, Zhilin Yang, Jidong Zhai, Jie Tang

However, training trillion-scale MoE requires algorithm and system co-design for a well-tuned high performance distributed training system.

Language Modelling

876

Paper
Code

CogDL: A Toolkit for Deep Learning on Graphs

1 code implementation • 1 Mar 2021 • Yukuo Cen, Zhenyu Hou, Yan Wang, Qibin Chen, Yizhen Luo, Zhongming Yu, Hengrui Zhang, Xingcheng Yao, Aohan Zeng, Shiguang Guo, Yuxiao Dong, Yang Yang, Peng Zhang, Guohao Dai, Yu Wang, Chang Zhou, Hongxia Yang, Jie Tang

Deep learning on graphs has attracted tremendous attention from the graph learning community in recent years.

Graph Classification Graph Embedding +5

1,356

Paper
Code

Cannot find the paper you are looking for? You can Submit a new open access paper.