Visual Reasoning
211 papers with code • 12 benchmarks • 41 datasets
Ability to understand actions and reasoning associated with any visual images
Benchmarks
These leaderboards are used to track progress in Visual Reasoning
| Trend | Dataset | Best Model | Paper | Code | Compare |
|---|---|---|---|---|---|
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
Libraries
Use these libraries to find Visual Reasoning models and implementations
RefCOCO
SHAPES
SNLI-VE
RAVEN
NLVR
AbstractReasoning
PGM
CATER
Most implemented papers
FiLM: Visual Reasoning with a General Conditioning Layer
ethanjperez/film
•
•
We introduce a general-purpose conditioning method for neural networks called FiLM: Feature-wise Linear Modulation.
Learning to Compose Dynamic Tree Structures for Visual Contexts
KaihuaTang/Scene-Graph-Benchmark.pytorch
•
•
CVPR 2019
We propose to compose dynamic tree structures that place the objects in an image into a visual context, helping visual reasoning tasks such as scene graph generation and visual Q&A.
BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation
salesforce/lavis
•
•
Furthermore, performance improvement has been largely achieved by scaling up the dataset with noisy image-text pairs collected from the web, which is a suboptimal source of supervision.
CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning
ethanjperez/film
•
•
CVPR 2017
When building artificial intelligence systems that can reason and answer questions about visual data, we need diagnostic tests to analyze our progress and discover shortcomings.
Inferring and Executing Programs for Visual Reasoning
facebookresearch/clevr-iep
•
•
ICCV 2017
Existing methods for visual reasoning attempt to directly map inputs to outputs using black-box architectures without explicitly modeling the underlying reasoning processes.
GQA: A New Dataset for Real-World Visual Reasoning and Compositional Question Answering
stanfordnlp/mac-network
•
•
CVPR 2019
We introduce GQA, a new dataset for real-world visual reasoning and compositional question answering, seeking to address key shortcomings of previous VQA datasets.
ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision
dandelin/vilt
•
•
Vision-and-Language Pre-training (VLP) has improved performance on various joint vision-and-language downstream tasks.
Align before Fuse: Vision and Language Representation Learning with Momentum Distillation
salesforce/lavis
•
•
NeurIPS 2021
Most existing methods employ a transformer-based multimodal encoder to jointly model visual tokens (region-based image features) and word tokens.
CoCa: Contrastive Captioners are Image-Text Foundation Models
We apply a contrastive loss between unimodal image and text embeddings, in addition to a captioning loss on the multimodal decoder outputs which predicts text tokens autoregressively.
MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models
Our work, for the first time, uncovers that properly aligning the visual features with an advanced large language model can possess numerous advanced multi-modal abilities demonstrated by GPT-4, such as detailed image description generation and website creation from hand-drawn drafts.
