Parameter-Efficient Tuning Makes a Good Classification Head

1 code implementation • 30 Oct 2022 • Zhuoyi Yang, Ming Ding, Yanhui Guo, Qingsong Lv, Jie Tang

In this paper, we find that parameter-efficient tuning makes a good classification head, with which we can simply replace the randomly initialized heads for a stable performance gain.

Classification Natural Language Understanding

GLM-130B: An Open Bilingual Pre-trained Model

2 code implementations • 5 Oct 2022 • Aohan Zeng, Xiao Liu, Zhengxiao Du, Zihan Wang, Hanyu Lai, Ming Ding, Zhuoyi Yang, Yifan Xu, Wendi Zheng, Xiao Xia, Weng Lam Tam, Zixuan Ma, Yufei Xue, Jidong Zhai, WenGuang Chen, Peng Zhang, Yuxiao Dong, Jie Tang

We introduce GLM-130B, a bilingual (English and Chinese) pre-trained language model with 130 billion parameters.

Language Modelling Multi-task Language Understanding +1

CogView: Mastering Text-to-Image Generation via Transformers

3 code implementations • NeurIPS 2021 • Ming Ding, Zhuoyi Yang, Wenyi Hong, Wendi Zheng, Chang Zhou, Da Yin, Junyang Lin, Xu Zou, Zhou Shao, Hongxia Yang, Jie Tang

Text-to-Image generation in the general domain has long been an open problem, which requires both a powerful generative model and cross-modal understanding.

Super-Resolution Text to image generation +1

Distributed High-dimensional Regression Under a Quantile Loss Function

no code implementations • 13 Jun 2019 • Xi Chen, Weidong Liu, Xiaojun Mao, Zhuoyi Yang

This paper studies distributed estimation and support recovery for high-dimensional linear regression model with heavy-tailed noise.

regression

Distributed Inference for Linear Support Vector Machine

no code implementations • 29 Nov 2018 • Xiaozhou Wang, Zhuoyi Yang, Xi Chen, Weidong Liu

In this paper, we propose a multi-round distributed linear-type (MDL) estimator for conducting inference for linear SVM.