Search Results for author:
Found 20 papers, 12 papers with code
DialSummEval: Revisiting Summarization Evaluation for Dialogues
Dialogue summarization is receiving increasing attention from researchers due to its extraordinary difficulty and unique application value.
Re-evaluating Automatic LLM System Ranking for Alignment with Human Preference
Due to the high cost and time-consuming nature of human evaluations, an automatic LLM bencher (i. e., an automatic evaluation framework that aims to rank LLMs based on their alignment with human preferences) is indispensable.
Analyzing and Evaluating Correlation Measures in NLG Meta-Evaluation
The correlation between NLG automatic evaluation metrics and human evaluation is often regarded as a critical criterion for assessing the capability of an evaluation metric.
Themis: A Reference-free NLG Evaluation Language Model with Flexibility and Interpretability
The evaluation of natural language generation (NLG) tasks is a significant and longstanding research area.
PVUW 2024 Challenge on Complex Video Understanding: Methods and Results
Moreover, we provide a new motion expression guided video segmentation dataset MeViS to study the natural language-guided video understanding in complex environments.
Better than Random: Reliable NLG Human Evaluation with Constrained Active Sampling
Human evaluation is viewed as a reliable evaluation method for NLG which is expensive and time-consuming.
1st Place Solution for MeViS Track in CVPR 2024 PVUW Workshop: Motion Expression guided Video Segmentation
Motion Expression guided Video Segmentation (MeViS), as an emerging task, poses many new challenges to the field of referring video object segmentation (RVOS).
Place Anything into Any Video
This paper introduces a novel and efficient system named Place-Anything, which facilitates the insertion of any object into any video solely based on a picture or text description of the target object or element.
Are LLM-based Evaluators Confusing NLG Quality Criteria?
Some prior work has shown that LLMs perform well in NLG evaluation for different tasks.
LLM-based NLG Evaluation: Current Status and Challenges
Evaluating natural language generation (NLG) is a vital but challenging problem in artificial intelligence.
Learning Cross-Modal Affinity for Referring Video Object Segmentation Targeting Limited Samples
Referring video object segmentation (RVOS), as a supervised learning task, relies on sufficient annotated data for a given scene.
Referring Video Object Segmentation
Semantic Segmentation
+1
Reference Matters: Benchmarking Factual Error Correction for Dialogue Summarization with Fine-grained Evaluation Framework
To address this problem, we are the first to manually annotate a FEC dataset for dialogue summarization containing 4000 items and propose FERRANTI, a fine-grained evaluation framework based on reference correction that automatically evaluates the performance of FEC models on different error categories.
Is Summary Useful or Not? An Extrinsic Human Evaluation of Text Summaries on Downstream Tasks
The results show that summaries generated by fine-tuned models lead to higher consistency in usefulness across all three tasks, as rankings of fine-tuned summarization systems are close across downstream tasks according to the proposed extrinsic metrics.
Caption Anything: Interactive Image Description with Diverse Multimodal Controls
Controllable image captioning is an emerging multimodal topic that aims to describe the image with natural language following human purpose, $\textit{e. g.}$, looking at the specified regions or telling in a particular text style.
Missing Information, Unresponsive Authors, Experimental Flaws: The Impossibility of Assessing the Reproducibility of Previous Human Evaluations in NLP
We report our efforts in identifying a set of previous human evaluations in NLP that would be suitable for a coordinated study examining what makes human evaluations in NLP more/less reproducible.
Track Anything: Segment Anything Meets Videos
Therefore, in this report, we propose Track Anything Model (TAM), which achieves high-performance interactive tracking and segmentation in videos.
Human-like Summarization Evaluation with ChatGPT
Evaluating text summarization is a challenging problem, and existing evaluation metrics are far from satisfactory.
Social Biases in Automatic Evaluation Metrics for NLG
Many studies have revealed that word embeddings, language models, and models for specific downstream tasks in NLP are prone to social biases, especially gender bias.
Multi-scale Location-aware Kernel Representation for Object Detection
Our MLKP can be efficiently computed on a modified multi-scale feature map using a low-dimensional polynomial kernel approximation. Moreover, different from existing orderless global representations based on high-order statistics, our proposed MLKP is location retentive and sensitive so that it can be flexibly adopted to object detection.
