Search Results for author:
Found 9 papers, 5 papers with code
Grounding Plural Phrases: Countering Evaluation Biases by Individuation
Phrase grounding (PG) is a multimodal task that grounds language in images.
On Measuring Faithfulness or Self-consistency of Natural Language Explanations
In this work we argue that these faithfulness tests do not measure faithfulness to the models' inner workings -- but rather their self-consistency at output level.
ViLMA: A Zero-Shot Benchmark for Linguistic and Temporal Grounding in Video-Language Models
With the ever-increasing popularity of pretrained Video-Language Models (VidLMs), there is a pressing need to develop robust evaluation methodologies that delve deeper into their visio-linguistic capabilities.
MM-SHAP: A Performance-agnostic Metric for Measuring Multimodal Contributions in Vision and Language Models & Tasks
We apply MM-SHAP in two ways: (1) to compare models for their average degree of multimodality, and (2) to measure for individual models the contribution of individual modalities for different tasks and datasets.
VALSE: A Task-Independent Benchmark for Vision and Language Models Centered on Linguistic Phenomena
We propose VALSE (Vision And Language Structured Evaluation), a novel benchmark designed for testing general-purpose pretrained vision and language (V&L) models for their visio-linguistic grounding capabilities on specific linguistic phenomena.
MAGMA -- Multimodal Augmentation of Generative Models through Adapter-based Finetuning
Large-scale pretraining is fast becoming the norm in Vision-Language (VL) modeling.
What is Multimodality?
The last years have shown rapid developments in the field of multimodal machine learning, combining e. g., vision, text or speech.
Seeing past words: Testing the cross-modal capabilities of pretrained V&L models on counting tasks
We investigate the reasoning ability of pretrained vision and language (V&L) models in two tasks that require multimodal integration: (1) discriminating a correct image-sentence pair from an incorrect one, and (2) counting entities in an image.
AMR Similarity Metrics from Principles
Different metrics have been proposed to compare Abstract Meaning Representation (AMR) graphs.
Ranked #4 on
Graph Matching
on RARE
