Towards better substitution-based word sense induction

29 May 2019  ·  Asaf Amrami, Yoav Goldberg ·

Word sense induction (WSI) is the task of unsupervised clustering of word usages within a sentence to distinguish senses. Recent work obtain strong results by clustering lexical substitutes derived from pre-trained RNN language models (ELMo). Adapting the method to BERT improves the scores even further. We extend the previous method to support a dynamic rather than a fixed number of clusters as supported by other prominent methods, and propose a method for interpreting the resulting clusters by associating them with their most informative substitutes. We then perform extensive error analysis revealing the remaining sources of errors in the WSI task. Our code is available at

PDF Abstract


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Word Sense Induction SemEval 2010 WSI BERT+DP F-Score 71.3 # 1
V-Measure 40.4 # 1
AVG 53.6 # 1
Word Sense Induction SemEval 2013 BERT+DP F-BC 64.0 # 1
F_NMI 21.4 # 1
AVG 37.0 # 1