A common step in developing an understanding of a vertical domain, e. g. shopping, dining, movies, medicine, etc., is curating a taxonomy of categories specific to the domain.
In this paper we present the first steps towards hardening the science of measuring AI systems, by adopting metrology, the science of measurement and its application, and applying it to human (crowd) powered evaluations.
The potential for learned models to amplify existing societal biases has been broadly recognized.
Distant supervision is a popular method for performing relation extraction from text that is known to produce noisy labels.
However, in many domains, there is ambiguity in the data, as well as a multitude of perspectives of the information examples.
Human-Computer Interaction Social and Information Networks
Distant supervision (DS) is a well-established method for relation extraction from text, based on the assumption that when a knowledge-base contains a relation between a term pair, then sentences that contain that pair are likely to express the relation.