Search Results for author:
Found 10 papers, 4 papers with code
ViLMA: A Zero-Shot Benchmark for Linguistic and Temporal Grounding in Video-Language Models
With the ever-increasing popularity of pretrained Video-Language Models (VidLMs), there is a pressing need to develop robust evaluation methodologies that delve deeper into their visio-linguistic capabilities.
Interpreting Vision and Language Generative Models with Semantic Visual Priors
When applied to Image-to-text models, interpretability methods often provide token-by-token explanations namely, they compute a visual explanation for each token of the generated sequence.
HL Dataset: Visually-grounded Description of Scenes, Actions and Rationales
We present the High-Level Dataset a dataset extending 14997 images from the COCO dataset, aligned with a new set of 134, 973 human-annotated (high-level) captions collected along three axes: scenes, actions, and rationales.
Understanding Cross-modal Interactions in V&L Models that Generate Scene Descriptions
Image captioning models tend to describe images in an object-centric way, emphasising visible objects.
VALSE: A Task-Independent Benchmark for Vision and Language Models Centered on Linguistic Phenomena
We propose VALSE (Vision And Language Structured Evaluation), a novel benchmark designed for testing general-purpose pretrained vision and language (V&L) models for their visio-linguistic grounding capabilities on specific linguistic phenomena.
What Vision-Language Models `See' when they See Scenes
Images can be described in terms of the objects they contain, or in terms of the types of scene or place that they instantiate.
On the interaction of automatic evaluation and task framing in headline style transfer
An ongoing debate in the NLG community concerns the best way to evaluate systems, with human evaluation often being considered the most reliable method, compared to corpus-based metrics.
Norm It! Lexical Normalization for Italian and Its Downstream Effects for Dependency Parsing
However, for Italian, there is no benchmark available for lexical normalization, despite the presence of many benchmarks for other tasks involving social media data.
Invisible to People but not to Machines: Evaluation of Style-aware HeadlineGeneration in Absence of Reliable Human Judgment
We automatically generate headlines that are expected to comply with the specific styles of two different Italian newspapers.
GePpeTto Carves Italian into a Language Model
We provide a thorough analysis of GePpeTto's quality by means of both an automatic and a human-based evaluation.
