ObjectLab: Automated Diagnosis of Mislabeled Images in Object Detection Data

1 code implementation • 2 Sep 2023 • Ulyana Tkachenko, Aditya Thyagarajan, Jonas Mueller

Despite powering sensitive systems like autonomous vehicles, object detection remains fairly brittle in part due to annotation errors that plague most real-world training datasets.

Autonomous Vehicles Object +2

CROWDLAB: Supervised learning to infer consensus labels and quality scores for data with multiple annotators

2 code implementations • 13 Oct 2022 • Hui Wen Goh, Ulyana Tkachenko, Jonas Mueller

For analyzing such data, we introduce CROWDLAB, a straightforward approach to utilize any trained classifier to estimate: (1) A consensus label for each example that aggregates the available annotations; (2) A confidence score for how likely each consensus label is correct; (3) A rating for each annotator quantifying the overall correctness of their labels.