LICHEE: Improving Language Model Pre-training with Multi-grained Tokenization

1 code implementation • Findings (ACL) 2021 • Weidong Guo, Mingjun Zhao, Lusheng Zhang, Di Niu, Jinwen Luo, Zhenhua Liu, Zhenyang Li, Jianbo Tang

Language model pre-training based on large corpora has achieved tremendous success in terms of constructing enriched contextual representations and has led to significant performance gains on a diverse range of Natural Language Understanding (NLU) tasks.

Language Modelling Natural Language Understanding