Search Results for author: John Mendonça

Found 7 papers, 4 papers with code

Soda-Eval: Open-Domain Dialogue Evaluation in the age of LLMs

no code implementations20 Aug 2024 John Mendonça, Isabel Trancoso, Alon Lavie

Although human evaluation remains the gold standard for open-domain dialogue evaluation, the growing popularity of automated evaluation using Large Language Models (LLMs) has also extended to dialogue.

Dialogue Evaluation

ECoh: Turn-level Coherence Evaluation for Multilingual Dialogues

1 code implementation16 Jul 2024 John Mendonça, Isabel Trancoso, Alon Lavie

Motivated by the need for lightweight, open source, and multilingual dialogue evaluators, this paper introduces GenResCoh (Generated Responses targeting Coherence).

Coherence Evaluation Dialogue Evaluation

On the Benchmarking of LLMs for Open-Domain Dialogue Evaluation

no code implementations4 Jul 2024 John Mendonça, Alon Lavie, Isabel Trancoso

Large Language Models (LLMs) have showcased remarkable capabilities in various Natural Language Processing tasks.

Benchmarking Chatbot +1

Simple LLM Prompting is State-of-the-Art for Robust and Multilingual Dialogue Evaluation

1 code implementation31 Aug 2023 John Mendonça, Patrícia Pereira, Helena Moniz, João Paulo Carvalho, Alon Lavie, Isabel Trancoso

Despite significant research effort in the development of automatic dialogue evaluation metrics, little thought is given to evaluating dialogues other than in English.

Dialogue Evaluation

Towards Multilingual Automatic Dialogue Evaluation

1 code implementation31 Aug 2023 John Mendonça, Alon Lavie, Isabel Trancoso

The main limiting factor in the development of robust multilingual dialogue evaluation metrics is the lack of multilingual data and the limited availability of open sourced multilingual dialogue systems.

Dialogue Evaluation Machine Translation +1

Using Self-Supervised Feature Extractors with Attention for Automatic COVID-19 Detection from Speech

no code implementations30 Jun 2021 John Mendonça, Rubén Solera-Ureña, Alberto Abad, Isabel Trancoso

Experimental results demonstrate that models trained on features extracted from self-supervised models perform similarly or outperform fully-supervised models and models based on handcrafted features.

Cannot find the paper you are looking for? You can Submit a new open access paper.