no code implementations • NeurIPS 2009 • Jacob Whitehill, Ting-Fan Wu, Jacob Bergsma, Javier R. Movellan, Paul L. Ruvolo
However, using these services to label large databases brings with it new theoretical and practical challenges: (1) The labelers may have wide ranging levels of expertise which are unknown a priori, and in some cases may be adversarial; (2) images may vary in their level of difficulty; and (3) multiple labels for the same image must be combined to provide an estimate of the actual label of the image.