Crowdsourced Text Aggregation
1 papers with code • 2 benchmarks • 1 datasets
One of the most important parts of processing responses from crowd workers is aggregation: given several conflicting opinions, a method should extract the truth. This problem is also known as truth-inference in crowdsourcing. Text aggregation problem is dedicated to extracting the correct information from crowd workers' responses for a crowdsourcing task where the output is a text: audio transcription, translation, character recognition, etc.
Benchmarks
These leaderboards are used to track progress in Crowdsourced Text Aggregation
| Trend | Dataset | Best Model | Paper | Code | Compare |
|---|---|---|---|---|---|
|
|||||
|
CrowdSpeech
Most implemented papers
CrowdSpeech and VoxDIY: Benchmark Datasets for Crowdsourced Audio Transcription
The main obstacle towards designing aggregation methods for more advanced applications is the absence of training data, and in this work, we focus on bridging this gap in speech recognition.
