First, they simply mix additionally-constructed training instances and original ones to train models, which fails to help models be explicitly aware of the procedure of gradual corrections.
However, there exists a gap between the learned knowledge of PLMs and the goal of CSC task.
Math Word Problem (MWP) solving needs to discover the quantitative relationships over natural language narratives.
Chinese Spell Checking (CSC) aims to detect and correct erroneous characters for user-generated text in the Chinese language.
Ranked #2 on Chinese Spell Checking on SIGHAN 2015
Pre-trained Transformer-based neural language models, such as BERT, have achieved remarkable results on varieties of NLP tasks.
Our approach outperforms previous unsupervised approaches by a large margin and is competitive with early supervised models.
Ranked #185 on Question Answering on SQuAD1.1