All that is English may be Hindi: Enhancing language identification through automatic ranking of the likeliness of word borrowing in social media

no code implementations • EMNLP 2017 • Jasabanta Patro, Bidisha Samanta, Saurabh Singh, Abhipsa Basu, Prithwish Mukherjee, Monojit Choudhury, Animesh Mukherjee

Based on this likeliness estimate we asked annotators to re-annotate the language tags of foreign words in predominantly native contexts.

Language Identification TAG

All that is English may be Hindi: Enhancing language identification through automatic ranking of likeliness of word borrowing in social media

no code implementations • 25 Jul 2017 • Jasabanta Patro, Bidisha Samanta, Saurabh Singh, Abhipsa Basu, Prithwish Mukherjee, Monojit Choudhury, Animesh Mukherjee

Based on this likeliness estimate we asked annotators to re-annotate the language tags of foreign words in predominantly native contexts.

Language Identification TAG

Is this word borrowed? An automatic approach to quantify the likeliness of borrowing in social media

no code implementations • 15 Mar 2017 • Jasabanta Patro, Bidisha Samanta, Saurabh Singh, Prithwish Mukherjee, Monojit Choudhury, Animesh Mukherjee

We first propose context based clustering method to sample a set of candidate words from the social media data. Next, we propose three novel and similar metrics based on the usage of these words by the users in different tweets; these metrics were used to score and rank the candidate words indicating their borrowed likeliness.

Clustering