TeacherLM: Teaching to Fish Rather Than Giving the Fish, Language Modeling Likewise

no code implementations • 29 Oct 2023 • Nan He, Hanyu Lai, Chenyang Zhao, Zirui Cheng, Junting Pan, Ruoyu Qin, Ruofan Lu, Rui Lu, Yunchen Zhang, Gangming Zhao, Zhaohui Hou, Zhiyuan Huang, Shaoqing Lu, Ding Liang, Mingjie Zhan

Based on TeacherLM-7. 1B, we augmented 58 NLP datasets and taught various student models with different parameters from OPT and BLOOM series in a multi-task setting.

Data Augmentation Language Modelling

Solving Challenging Math Word Problems Using GPT-4 Code Interpreter with Code-based Self-Verification

1 code implementation • 15 Aug 2023 • Aojun Zhou, Ke Wang, Zimu Lu, Weikang Shi, Sichun Luo, Zipeng Qin, Shaoqing Lu, Anya Jia, Linqi Song, Mingjie Zhan, Hongsheng Li

We found that its success can be largely attributed to its powerful skills in generating and executing code, evaluating the output of code execution, and rectifying its solution when receiving unreasonable outputs.

Arithmetic Reasoning Math +1

Learning Locality and Isotropy in Dialogue Modeling

1 code implementation • 29 May 2022 • Han Wu, Haochen Tan, Mingjie Zhan, Gangming Zhao, Shaoqing Lu, Ding Liang, Linqi Song

Existing dialogue modeling methods have achieved promising performance on various dialogue tasks with the aid of Transformer and the large-scale pre-trained language models.

MixMix: All You Need for Data-Free Compression Are Feature and Data Mixing

no code implementations • ICCV 2021 • Yuhang Li, Feng Zhu, Ruihao Gong, Mingzhu Shen, Xin Dong, Fengwei Yu, Shaoqing Lu, Shi Gu

However, the inversion process only utilizes biased feature statistics stored in one model and is from low-dimension to high-dimension.

Knowledge Distillation Model Compression +1