Search Results for author:
Found 7 papers, 4 papers with code
Grounding Plural Phrases: Countering Evaluation Biases by Individuation
Phrase grounding (PG) is a multimodal task that grounds language in images.
MM-SHAP: A Performance-agnostic Metric for Measuring Multimodal Contributions in Vision and Language Models & Tasks
But how to quantify the amount of unimodal collapse reliably, at dataset and instance-level, to diagnose and combat unimodal collapse in a targeted way?
VALSE: A Task-Independent Benchmark for Vision and Language Models Centered on Linguistic Phenomena
We propose VALSE (Vision And Language Structured Evaluation), a novel benchmark designed for testing general-purpose pretrained vision and language (V&L) models for their visio-linguistic grounding capabilities on specific linguistic phenomena.
Ranked #1 on
image-sentence alignment
on VALSE
MAGMA -- Multimodal Augmentation of Generative Models through Adapter-based Finetuning
Large-scale pretraining is fast becoming the norm in Vision-Language (VL) modeling.
What is Multimodality?
The last years have shown rapid developments in the field of multimodal machine learning, combining e. g., vision, text or speech.
Seeing past words: Testing the cross-modal capabilities of pretrained V&L models on counting tasks
We investigate the reasoning ability of pretrained vision and language (V&L) models in two tasks that require multimodal integration: (1) discriminating a correct image-sentence pair from an incorrect one, and (2) counting entities in an image.
AMR Similarity Metrics from Principles
Different metrics have been proposed to compare Abstract Meaning Representation (AMR) graphs.
