Search Results for author:
Found 11 papers, 10 papers with code
Beyond Task Performance: Evaluating and Reducing the Flaws of Large Multimodal Models with In-Context Learning
Based on our ICL study, (3) we push ICL further and propose new multimodal ICL variants such as; Multitask-ICL, Chain-of-Hindsight-ICL, and Self-Correcting-ICL.
UnIVAL: Unified Model for Image, Video, Audio and Language Tasks
Our model is efficiently pretrained on many tasks, based on task balancing and multimodal curriculum learning.
Improving Selective Visual Question Answering by Learning from Your Peers
In this work, we explore Selective VQA in both in-distribution (ID) and OOD scenarios, where models are presented with mixtures of ID and OOD data.
eP-ALM: Efficient Perceptual Augmentation of Language Models
In this work, we propose to rather direct effort to efficient adaptations of existing models, and propose to augment Language Models with perception.
Dynamic Query Selection for Fast Visual Perceiver
Transformers have been matching deep convolutional networks for vision architectures in recent works.
Fishr: Invariant Gradient Variances for Out-of-Distribution Generalization
In this paper, we introduce a new regularization - named Fishr - that enforces domain invariance in the space of the gradients of the loss: specifically, the domain-level variances of gradients are matched across training domains.
Ranked #22 on
Domain Generalization
on TerraIncognita
Beyond Question-Based Biases: Assessing Multimodal Shortcut Learning in Visual Question Answering
We use this new evaluation in a large-scale study of existing approaches for VQA.
Ranked #1 on
Visual Question Answering (VQA)
on VQA-CE
Overcoming Statistical Shortcuts for Open-ended Visual Counting
First, we propose the Modifying Count Distribution (MCD) protocol, which penalizes models that over-rely on statistical shortcuts.
RUBi: Reducing Unimodal Biases for Visual Question Answering
We propose RUBi, a new learning strategy to reduce biases in any VQA model.
RUBi: Reducing Unimodal Biases in Visual Question Answering
We propose RUBi, a new learning strategy to reduce biases in any VQA model.
Ranked #7 on
Visual Question Answering (VQA)
on VQA-CP
Sampling strategies in Siamese Networks for unsupervised speech representation learning
We apply these results to pairs of words discovered using an unsupervised algorithm and show an improvement on state-of-the-art in unsupervised representation learning using siamese networks.
