Search Results for author:
Found 25 papers, 12 papers with code
Revisiting Multi-Modal LLM Evaluation
With the advent of multi-modal large language models (MLLMs), datasets used for visual question answering (VQA) and referring expression comprehension have seen a resurgence.
They're All Doctors: Synthesizing Diverse Counterfactuals to Mitigate Associative Bias
Vision Language Models (VLMs) such as CLIP are powerful models; however they can exhibit unwanted biases, making them less safe when deployed directly in applications such as text-to-image, text-to-video retrievals, reverse search, or classification tasks.
FairDeDup: Detecting and Mitigating Vision-Language Fairness Disparities in Semantic Dataset Deduplication
Recent dataset deduplication techniques have demonstrated that content-aware dataset pruning can dramatically reduce the cost of training Vision-Language Pretrained (VLP) models without significant performance losses compared to training on the original dataset.
FINEMATCH: Aspect-based Fine-grained Image and Text Mismatch Detection and Correction
To address this, we propose FineMatch, a new aspect-based fine-grained text and image matching benchmark, focusing on text and image mismatch detection and correction.
Building Vision-Language Models on Solid Foundations with Masked Distillation
Recent advancements in Vision-Language Models (VLMs) have marked a significant leap in bridging the gap between computer vision and natural language processing.
SCoRD: Subject-Conditional Relation Detection with Text-Augmented Data
To solve this problem, we propose an auto-regressive model that given a subject, it predicts its relations, objects, and object locations by casting this output as a sequence of tokens.
Improving Visual Grounding by Encouraging Consistent Gradient-based Explanations
We propose a margin-based loss for tuning joint vision-language models so that their gradient-based explanations are consistent with region-level annotations provided by humans for relatively smaller grounding datasets.
OccamNets: Mitigating Dataset Bias by Favoring Simpler Hypotheses
We propose a new direction: modifying the network architecture to impose inductive biases that make the network robust to dataset bias.
Ranked #3 on
Action Recognition
on BAR
Beyond Pixels: A Sample Based Method for understanding the decisions of Neural Networks
This can manifest itself in the use of spurious correlations in the data to make decisions.
Learning to Predict Visual Attributes in the Wild
In this paper, we introduce a large-scale in-the-wild visual attribute prediction dataset consisting of over 927K attribute annotations for over 260K object instances.
Are Bias Mitigation Techniques for Deep Learning Effective?
We introduce a new dataset called Biased MNIST that enables assessment of robustness to multiple bias sources.
AESOP: Abstract Encoding of Stories, Objects, and Pictures
Visual storytelling and story comprehension are uniquely human skills that play a central role in how we learn about and experience the world.
On the Value of Out-of-Distribution Testing: An Example of Goodhart's Law
Out-of-distribution (OOD) testing is increasingly popular for evaluating a machine learning system's ability to generalize beyond the biases of a training set.
Do We Need Fully Connected Output Layers in Convolutional Networks?
Traditionally, deep convolutional neural networks consist of a series of convolutional and pooling layers followed by one or more fully connected (FC) layers to perform the final classification.
Visual Grounding Methods for VQA are Working for the Wrong Reasons!
Existing Visual Question Answering (VQA) methods tend to exploit dataset biases and spurious statistical correlations, instead of producing right answers for the right reasons.
REMIND Your Neural Network to Prevent Catastrophic Forgetting
While there is neuroscientific evidence that the brain replays compressed memories, existing methods for convolutional networks replay raw images.
Answering Questions about Data Visualizations using Efficient Bimodal Fusion
Chart question answering (CQA) is a newly proposed visual question answering (VQA) task where an algorithm must answer questions about data visualizations, e. g. bar charts, pie charts, and line graphs.
Challenges and Prospects in Vision and Language Research
Language grounded image understanding tasks have often been proposed as a method for evaluating progress in artificial intelligence.
Answer Them All! Toward Universal Visual Question Answering Models
Visual Question Answering (VQA) research is split into two camps: the first focuses on VQA datasets that require natural image understanding and the second focuses on synthetic datasets that test reasoning.
TallyQA: Answering Complex Counting Questions
Most counting questions in visual question answering (VQA) datasets are simple and require no more than object detection.
Ranked #3 on
Object Counting
on HowMany-QA
DVQA: Understanding Data Visualizations via Question Answering
Bar charts are an effective way to convey numeric information, but today's algorithms cannot parse them.
Data Augmentation for Visual Question Answering
Data augmentation is widely used to train deep neural networks for image classification tasks.
An Analysis of Visual Question Answering Algorithms
As a result, evaluation scores are inflated and predominantly determined by answering easier questions, making it difficult to compare different methods.
Visual Question Answering: Datasets, Algorithms, and Future Challenges
We then exhaustively review existing algorithms for VQA.
Answer-Type Prediction for Visual Question Answering
Recently, algorithms for object recognition and related tasks have become sufficiently proficient that new vision tasks can now be pursued.
