Multilingual Dataset for Training and Evaluating Diacritics Restoration Systems

Introduced by N{\'a}plava et al. in Diacritics Restoration Using Neural Networks

The dataset contains training and evaluation data for 12 languages: - Vietnamese - Romanian - Latvian - Czech - Polish - Slovak - Irish - Hungarian - French - Turkish - Spanish - Croatian

For each language, one training, one development and one testing set acquired from Wikipedia articles is provided. Moreover, each language dataset contains (substantially larger) training set collected from (general) Web texts. All sets, except for Wikipedia and Web training sets that can contain similar sentences, are disjoint. Data are segmented into sentences which are further word tokenized.

Homepage

Benchmarks

Add a new result Link an existing benchmark

Task	Dataset Variant	Best Model
Czech Text Diacritization	Multilingual Dataset for Training and Evaluating Diacritics Restoration Systems	BERT
Vietnamese Text Diacritization	Multilingual Dataset for Training and Evaluating Diacritics Restoration Systems	BERT
Latvian Text Diacritization	Multilingual Dataset for Training and Evaluating Diacritics Restoration Systems	BERT
Polish Text Diacritization	Multilingual Dataset for Training and Evaluating Diacritics Restoration Systems	BERT
Slovak Text Diacritization	Multilingual Dataset for Training and Evaluating Diacritics Restoration Systems	BERT
French Text Diacritization	Multilingual Dataset for Training and Evaluating Diacritics Restoration Systems	BERT
Irish Text Diacritization	Multilingual Dataset for Training and Evaluating Diacritics Restoration Systems	BERT
Spanish Text Diacritization	Multilingual Dataset for Training and Evaluating Diacritics Restoration Systems	BERT
Croatian Text Diacritization	Multilingual Dataset for Training and Evaluating Diacritics Restoration Systems	BERT
Hungarian Text Diacritization	Multilingual Dataset for Training and Evaluating Diacritics Restoration Systems	BERT
Turkish Text Diacritization	Multilingual Dataset for Training and Evaluating Diacritics Restoration Systems	BERT
Romanian Text Diacritization	Multilingual Dataset for Training and Evaluating Diacritics Restoration Systems	BERT