Crowdsourced Text Aggregation

1 papers with code • 2 benchmarks • 1 datasets

One of the most important parts of processing responses from crowd workers is aggregation: given several conflicting opinions, a method should extract the truth. This problem is also known as truth-inference in crowdsourcing. Text aggregation problem is dedicated to extracting the correct information from crowd workers' responses for a crowdsourcing task where the output is a text: audio transcription, translation, character recognition, etc.


Most implemented papers

CrowdSpeech and VoxDIY: Benchmark Datasets for Crowdsourced Audio Transcription

Toloka/CrowdSpeech 2 Jul 2021

The main obstacle towards designing aggregation methods for more advanced applications is the absence of training data, and in this work, we focus on bridging this gap in speech recognition.