Search Results for author:
Found 25 papers, 5 papers with code
Are We Summarizing the Right Way? A Survey of Dialogue Summarization Data Sets
Dialogue summarization is a long-standing task in the field of NLP, and several data sets with dialogues and associated human-written summaries of different styles exist.
A Measure of the System Dependence of Automated Metrics
Automated metrics for Machine Translation have made significant progress, with the goal of replacing expensive and time-consuming human evaluations.
Error-preserving Automatic Speech Recognition of Young English Learners' Language
In this work, we build an ASR system that satisfies these requirements: it works on spontaneous speech by young language learners and preserves their errors.
Favi-Score: A Measure for Favoritism in Automated Preference Ratings for Generative AI Evaluation
Thus, we propose that preference-based metrics ought to be evaluated on both sign accuracy scores and favoritism.
Dialect Transfer for Swiss German Speech Translation
This paper investigates the challenges in building Swiss German speech translation systems, specifically focusing on the impact of dialect diversity and differences between Swiss German and Standard German.
ScienceBenchmark: A Complex Real-World Benchmark for Evaluating Natural Language to SQL Systems
Thus, the challenge is many-fold: creating NL-to-SQL systems for highly complex domains with a small amount of hand-made training data augmented with synthetic data.
Correction of Errors in Preference Ratings from Automated Metrics for Text Generation
A major challenge in the field of Text Generation is evaluation: Human evaluations are cost-intensive, and automated metrics often display considerable disagreement with human judgments.
Text-to-Speech Pipeline for Swiss German -- A comparison
In this work, we studied the synthesis of Swiss German speech using different Text-to-Speech (TTS) models.
STT4SG-350: A Speech Corpus for All Swiss German Dialect Regions
We train an ASR model on the training set and achieve an average BLEU score of 74. 7 on the test set.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+5
On the Effectiveness of Automated Metrics for Text Generation Systems
A major challenge in the field of Text Generation is evaluation because we lack a sound theory that can be leveraged to extract guidelines for evaluation campaigns.
SDS-200: A Swiss German Speech to Standard German Text Corpus
We present SDS-200, a corpus of Swiss German dialectal speech with Standard German text translations, annotated with dialect, age, and gender information of the speakers.
Report from the NSF Future Directions Workshop on Automatic Evaluation of Dialog: Research Directions and Challenges
This is a report on the NSF Future Directions Workshop on Automatic Evaluation of Dialog.
Probing the Robustness of Trained Metrics for Conversational Dialogue Systems
This paper introduces an adversarial method to stress-test trained metrics to evaluate conversational dialogue systems.
Spot The Bot: A Robust and Efficient Framework for the Evaluation of Conversational Dialogue Systems
In this work, we introduce \emph{Spot The Bot}, a cost-efficient and robust evaluation framework that replaces human-bot conversations with conversations between bots.
DoQA - Accessing Domain-Specific FAQs via Conversational QA
We present DoQA, a dataset with 2, 437 dialogues and 10, 917 QA pairs.
DoQA -- Accessing Domain-Specific FAQs via Conversational QA
We present DoQA, a dataset with 2, 437 dialogues and 10, 917 QA pairs.
A Methodology for Creating Question Answering Corpora Using Inverse Data Annotation
For this, we introduce an intermediate representation that is based on the logical query plan in a database called Operation Trees (OT).
Towards a Metric for Automated Conversational Dialogue System Evaluation and Improvement
We present "AutoJudge", an automated evaluation method for conversational dialogue systems.
Survey on Evaluation Methods for Dialogue Systems
We cover each class by introducing the main technologies developed for the dialogue systems and then by presenting the evaluation methods regarding this class.
Twist Bytes - German Dialect Identification with Data Mining Optimization
We describe our approaches used in the German Dialect Identification (GDI) task at the VarDial Evaluation Campaign 2018.
TopicThunder at SemEval-2017 Task 4: Sentiment Classification Using a Convolutional Neural Network with Distant Supervision
In this paper, we propose a classifier for predicting topic-specific sentiments of English Twitter messages.
Leveraging Large Amounts of Weakly Supervised Data for Multi-Language Sentiment Classification
This paper presents a novel approach for multi-lingual sentiment classification in short texts.
A Directional Diffusion Algorithm for Inpainting
The problem of inpainting involves reconstructing the missing areas of an image.
