Stronger Baselines for Grammatical Error Correction Using a Pretrained Encoder-Decoder Model

Asian Chapter of the Association for Computational Linguistics 2020 · Satoru Katsumata, Mamoru Komachi ·

Studies on grammatical error correction (GEC) have reported on the effectiveness of pretraining a Seq2Seq model with a large amount of pseudodata. However, this approach requires time-consuming pretraining of GEC because of the size of the pseudodata. In this study, we explored the utility of bidirectional and auto-regressive transformers (BART) as a generic pretrained encoder-decoder model for GEC. With the use of this generic pretrained model for GEC, the time-consuming pretraining can be eliminated. We find that monolingual and multilingual BART models achieve high performance in GEC, with one of the results being comparable to the current strong results in English GEC.

PDF Abstract

Code

Add Remove Mark official

No code implementations yet. Submit your code now

Tasks

Add Remove

Grammatical Error Correction

Datasets

AKCES-GEC

Results from the Paper

Add Remove

Submit results from this paper to get state-of-the-art GitHub badges and help the community compare results to other papers.

Methods

Add Remove

Adam • BART • BPE • Dense Connections • Dropout • GELU • Layer Normalization • Linear Layer • LSTM • Multi-Head Attention • Residual Connection • Scaled Dot-Product Attention • Seq2Seq • Sigmoid Activation • Softmax • Tanh Activation

Edit Social Preview

Stronger Baselines for Grammatical Error Correction Using a Pretrained Encoder-Decoder Model

Code Edit Add Remove Mark official

Tasks Edit Add Remove

Datasets Edit

Results from the Paper Edit Add Remove

Methods Edit Add Remove

Code

Add Remove Mark official

Tasks

Add Remove

Datasets

Results from the Paper

Add Remove

Methods

Add Remove