To do so, we sample balanced subsets of data that are labeled by demographically distinct annotators.
In the scope of this study, we want to investigate annotator bias — a form of bias that annotators cause due to different knowledge in regards to the task and their subjective perception.
A prevalent form of bias in hate speech and abusive language datasets is annotator bias caused by the annotator’s subjective perception and the complexity of the annotation task.
As hate speech spreads on social media and online communities, research continues to work on its automatic detection.
Sentiment analysis is often a crowdsourcing task prone to subjective labels given by many annotators.
Since traditional social media platforms continue to ban actors spreading hate speech or other forms of abusive languages (a process known as deplatforming), these actors migrate to alternative platforms that do not moderate users content.