Crowdsourced Text Aggregation
1 papers with code • 2 benchmarks • 1 datasets
One of the most important parts of processing responses from crowd workers is aggregation: given several conflicting opinions, a method should extract the truth. This problem is also known as truth-inference in crowdsourcing. Text aggregation problem is dedicated to extracting the correct information from crowd workers' responses for a crowdsourcing task where the output is a text: audio transcription, translation, character recognition, etc.
The main obstacle towards designing aggregation methods for more advanced applications is the absence of training data, and in this work, we focus on bridging this gap in speech recognition.