Search Results for author:
Found 7 papers, 2 papers with code
ZHAW-InIT - Social Media Geolocation at VarDial 2020
We describe our approaches for the Social Media Geolocation (SMG) task at the VarDial Evaluation Campaign 2020.
A Measure of the System Dependence of Automated Metrics
Automated metrics for Machine Translation have made significant progress, with the goal of replacing expensive and time-consuming human evaluations.
Favi-Score: A Measure for Favoritism in Automated Preference Ratings for Generative AI Evaluation
Thus, we propose that preference-based metrics ought to be evaluated on both sign accuracy scores and favoritism.
Correction of Errors in Preference Ratings from Automated Metrics for Text Generation
A major challenge in the field of Text Generation is evaluation: Human evaluations are cost-intensive, and automated metrics often display considerable disagreement with human judgments.
On the Effectiveness of Automated Metrics for Text Generation Systems
A major challenge in the field of Text Generation is evaluation because we lack a sound theory that can be leveraged to extract guidelines for evaluation campaigns.
Probing the Robustness of Trained Metrics for Conversational Dialogue Systems
This paper introduces an adversarial method to stress-test trained metrics to evaluate conversational dialogue systems.
Spot The Bot: A Robust and Efficient Framework for the Evaluation of Conversational Dialogue Systems
In this work, we introduce \emph{Spot The Bot}, a cost-efficient and robust evaluation framework that replaces human-bot conversations with conversations between bots.
