Sentence ordering task deals with finding the correct order of sentences given a randomly ordered paragraph.
Modeling the structure of coherent texts is a key NLP problem.
In this paper, we propose a novel deep coherence model (DCM) using a convolutional neural network architecture to capture the text coherence.
BERT set many state-of-the-art results over varied NLU benchmarks by pre-training over two tasks: masked language modelling (MLM) and next sentence prediction (NSP), the latter of which has been highly criticized.
Our experiments on five benchmark datasets show that our method outperforms all the existing baselines significantly, achieving a new state-of-the-art performance.