Short Text Clustering with Transformers

31 Jan 2021  ·  Leonid Pugachev, Mikhail Burtsev ·

Recent techniques for the task of short text clustering often rely on word embeddings as a transfer learning component. This paper shows that sentence vector representations from Transformers in conjunction with different clustering methods can be successfully applied to address the task. Furthermore, we demonstrate that the algorithm of enhancement of clustering via iterative classification can further improve initial clustering performance with different classifiers, including those based on pre-trained Transformer language models.

PDF Abstract


  Add Datasets introduced or used in this paper
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Short Text Clustering Stackoverflow Deep ECIC Accuracy Deep ECIC # 1
Acc Deep ECIC # 1