KDSTM: Neural Semi-supervised Topic Modeling with Knowledge Distillation

no code implementations • 4 Jul 2023 • Weijie Xu, Xiaoyu Jiang, Jay Desai, Bin Han, Fuqin Yan, Francis Iannacci

In text classification tasks, fine tuning pretrained language models like BERT and GPT-3 yields competitive accuracy; however, both methods require pretraining on large text datasets.

Knowledge Distillation text-classification +1