Search Results for author:
Found 15 papers, 8 papers with code
Automatic Evaluation and Moderation of Open-domain Dialogue Systems
The development of Open-Domain Dialogue Systems (ODS)is a trending topic due to the large number of research challenges, large societal and business impact, and advances in the underlying technology.
FineD-Eval: Fine-grained Automatic Dialogue-Level Evaluation
Recent model-based reference-free metrics for open-domain dialogue evaluation exhibit promising correlations with human judgment.
DynaEval: Unifying Turn and Dialogue Level Evaluation
Effective evaluation metrics should reflect the dynamics of such interaction.
Overview of Robust and Multilingual Automatic Evaluation Metrics for Open-Domain Dialogue Systems at DSTC 11 Track 4
The advent and fast development of neural networks have revolutionized the research on dialogue systems and subsequently have triggered various challenges regarding their automatic evaluation.
Truly Multi-modal YouTube-8M Video Classification with Video, Audio, and Text
Beyond that, we extend the original competition by including text information in the classification, making this a truly multi-modal approach with vision, audio and text.
MDD-Eval: Self-Training on Augmented Data for Multi-Domain Dialogue Evaluation
Chatbots are designed to carry out human-like conversations across different domains, such as general chit-chat, knowledge exchange, and persona-grounded conversations.
Ranked #1 on
Dialogue Evaluation
on USR-TopicalChat
xDial-Eval: A Multilingual Open-Domain Dialogue Evaluation Benchmark
The English dialogue data are extended to nine other languages with commercial machine translation systems.
A Comprehensive Analysis of the Effectiveness of Large Language Models as Automatic Dialogue Evaluators
Yet, existing works on utilizing LLMs for automatic dialogue evaluation are limited in their scope in terms of the number of meta-evaluation datasets, mode of evaluation, coverage of LLMs, etc.
End-to-End Video Classification with Knowledge Graphs
In particular, we unify traditional "knowledgeless" machine learning models and knowledge graphs in a novel end-to-end framework.
Dialog System Technology Challenge 7
This paper introduces the Seventh Dialog System Technology Challenges (DSTC), which use shared datasets to explore the problem of building dialog systems.
Joint Learning of Word and Label Embeddings for Sequence Labelling in Spoken Language Understanding
We propose an architecture to jointly learn word and label embeddings for slot filling in spoken language understanding.
Overview of the Ninth Dialog System Technology Challenge: DSTC9
Interactive evaluation of dialog, and 4.
Investigating the Impact of Pre-trained Language Models on Dialog Evaluation
Yet, the impact of different Pr-LMs on the performance of automatic metrics is not well-understood.
Report from the NSF Future Directions Workshop on Automatic Evaluation of Dialog: Research Directions and Challenges
This is a report on the NSF Future Directions Workshop on Automatic Evaluation of Dialog.
PoE: a Panel of Experts for Generalized Automatic Dialogue Assessment
To tackle the multi-domain dialogue evaluation task, we propose a Panel of Experts (PoE), a multitask network that consists of a shared transformer encoder and a collection of lightweight adapters.
