Search Results for author:
Found 3 papers, 1 papers with code
The price of debiasing automatic metrics in natural language evalaution
For evaluating generation systems, automatic metrics such as BLEU cost nothing to run but have been shown to correlate poorly with human judgment, leading to systematic bias against certain model improvements.
Importance sampling for unbiased on-demand evaluation of knowledge base population
Our first contribution is a new importance-sampling based evaluation which corrects for this bias by annotating a new system{'}s predictions on-demand via crowdsourcing.
On-the-Job Learning with Bayesian Decision Theory
Our goal is to deploy a high-accuracy system starting with zero training examples.
Cannot find the paper you are looking for? You can
Submit a new open access paper.
