ThaiLMCut: Unsupervised Pretraining for Thai Word Segmentation

LREC 2020 • Suteera Seeha • Ivan Bilan • Liliana Mamani Sanchez • Johannes Huber • Michael Matuschek • Hinrich Sch{\"u}tze

We propose ThaiLMCut, a semi-supervised approach for Thai word segmentation which utilizes a bi-directional character language model (LM) as a way to leverage useful linguistic knowledge from unlabeled data. After the language model is trained on substantial unlabeled corpora, the weights of its embedding and recurrent layers are transferred to a supervised word segmentation model which continues fine-tuning them on a word segmentation task... (read more)

PDF Abstract

Tasks
Add Remove

Datasets

  Add Datasets introduced or used in this paper

Results from the Paper
Add Remove

Ranked #2 on Thai Word Segmentation on BEST-2010

     Get a GitHub badge
TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK BENCHMARK
Thai Word Segmentation BEST-2010 ThaiLMCut F1-Score 0.9878 # 2
Compare

Methods used in the Paper
Add Remove

METHOD TYPE
🤖 No Methods Found Help the community by adding them if they're not listed; e.g. Deep Residual Learning for Image Recognition uses ResNet
Contact us on: hello@paperswithcode.com. Papers With Code is a free resource with all data licensed under CC-BY-SA.
Terms Privacy Cookies policy