SiBert: Enhanced Chinese Pre-trained Language Model with Sentence Insertion

1 code implementation • LREC 2020 • Jiahao Chen, Chenjie Cao, Xiuyan Jiang

However, some studies show that customized self-supervised tasks for a particular type of downstream task can effectively help the pre-trained model to capture more corresponding knowledge and semantic information.

Cloze Test Language Modelling +3