Search Results for author:
Found 18 papers, 10 papers with code
Beyond Single-Audio: Advancing Multi-Audio Processing in Audio Large Language Models
To this end, we propose a novel multi-audio-LLM (MALLM) to capture audio context among multiple similar audios using discriminative learning on our proposed synthetic data.
CVQA: Culturally-diverse Multilingual Visual Question Answering Benchmark
Visual Question Answering (VQA) is an important task in multimodal AI, and it is often used to test the ability of vision-language models to understand and reason on knowledge present in both visual and textual data.
Unveiling the Achilles' Heel of NLG Evaluators: A Unified Adversarial Framework Driven by Large Language Models
Specifically, inspired by the recent success of large language models (LLMs) in text generation and evaluation, we adopt strong LLMs as both the data generator and gold evaluator.
A Comprehensive Analysis of the Effectiveness of Large Language Models as Automatic Dialogue Evaluators
Yet, existing works on utilizing LLMs for automatic dialogue evaluation are limited in their scope in terms of the number of meta-evaluation datasets, mode of evaluation, coverage of LLMs, etc.
xDial-Eval: A Multilingual Open-Domain Dialogue Evaluation Benchmark
The English dialogue data are extended to nine other languages with commercial machine translation systems.
Overview of Robust and Multilingual Automatic Evaluation Metrics for Open-Domain Dialogue Systems at DSTC 11 Track 4
The advent and fast development of neural networks have revolutionized the research on dialogue systems and subsequently have triggered various challenges regarding their automatic evaluation.
PoE: a Panel of Experts for Generalized Automatic Dialogue Assessment
To tackle the multi-domain dialogue evaluation task, we propose a Panel of Experts (PoE), a multitask network that consists of a shared transformer encoder and a collection of lightweight adapters.
FineD-Eval: Fine-grained Automatic Dialogue-Level Evaluation
Recent model-based reference-free metrics for open-domain dialogue evaluation exhibit promising correlations with human judgment.
Report from the NSF Future Directions Workshop on Automatic Evaluation of Dialog: Research Directions and Challenges
This is a report on the NSF Future Directions Workshop on Automatic Evaluation of Dialog.
MDD-Eval: Self-Training on Augmented Data for Multi-Domain Dialogue Evaluation
Chatbots are designed to carry out human-like conversations across different domains, such as general chit-chat, knowledge exchange, and persona-grounded conversations.
Ranked #1 on
Dialogue Evaluation
on USR-TopicalChat
Automatic Evaluation and Moderation of Open-domain Dialogue Systems
The development of Open-Domain Dialogue Systems (ODS)is a trending topic due to the large number of research challenges, large societal and business impact, and advances in the underlying technology.
Investigating the Impact of Pre-trained Language Models on Dialog Evaluation
Yet, the impact of different Pr-LMs on the performance of automatic metrics is not well-understood.
DynaEval: Unifying Turn and Dialogue Level Evaluation
Effective evaluation metrics should reflect the dynamics of such interaction.
Overview of the Ninth Dialog System Technology Challenge: DSTC9
Interactive evaluation of dialog, and 4.
Joint Learning of Word and Label Embeddings for Sequence Labelling in Spoken Language Understanding
We propose an architecture to jointly learn word and label embeddings for slot filling in spoken language understanding.
Dialog System Technology Challenge 7
This paper introduces the Seventh Dialog System Technology Challenges (DSTC), which use shared datasets to explore the problem of building dialog systems.
End-to-End Video Classification with Knowledge Graphs
In particular, we unify traditional "knowledgeless" machine learning models and knowledge graphs in a novel end-to-end framework.
Truly Multi-modal YouTube-8M Video Classification with Video, Audio, and Text
Beyond that, we extend the original competition by including text information in the classification, making this a truly multi-modal approach with vision, audio and text.
