Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding

Modeling textual or visual information with vector representations trained from large language or visual datasets has been successfully explored in recent years. However, tasks such as visual question answering require combining these vector representations with each other... (read more)

PDF Abstract EMNLP 2016 PDF EMNLP 2016 Abstract
TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK RESULT BENCHMARK
Visual Question Answering COCO Visual Question Answering (VQA) real images 1.0 multiple choice MCB 7 att. Percentage correct 70.1 # 1
Visual Question Answering COCO Visual Question Answering (VQA) real images 1.0 open ended MCB 7 att. Percentage correct 66.5 # 1
Phrase Grounding Flickr30k Entities Test MCB Accuracy 48.69 # 1
Phrase Grounding ReferIt MCB Accuracy 28.91 # 1
Visual Question Answering Visual7W MCB+Att. Percentage correct 62.2 # 3
Visual Question Answering VQA v1 test-dev MCB (ResNet) Accuracy 64.2 # 3
Visual Question Answering VQA v2 test-dev MCB Accuracy 64.7 # 20

Methods used in the Paper


METHOD TYPE
Average Pooling
Pooling Operations
Residual Connection
Skip Connections
ReLU
Activation Functions
1x1 Convolution
Convolutions
Batch Normalization
Normalization
Bottleneck Residual Block
Skip Connection Blocks
Global Average Pooling
Pooling Operations
Residual Block
Skip Connection Blocks
Kaiming Initialization
Initialization
Max Pooling
Pooling Operations
Convolution
Convolutions
ResNet
Convolutional Neural Networks