Benchmarks
Greatest papers with code
Commands 4 Autonomous Vehicles (C4AV) Workshop Summary
In this work, we deviate from recent, popular task settings and consider the problem under an autonomous vehicle scenario.
Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding
EMNLP 2016
• Cadene/vqa.pytorch
• 
Approaches to multimodal pooling include element-wise product or sum, as well as concatenation of the visual and textual representations.
ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks
NeurIPS 2019
• facebookresearch/vilbert-multi-task
•
We present ViLBERT (short for Vision-and-Language BERT), a model for learning task-agnostic joint representations of image content and natural language.
IMAGE RETRIEVAL QUESTION ANSWERING VISUAL COMMONSENSE REASONING VISUAL GROUNDING VISUAL QUESTION ANSWERING
MDETR -- Modulated Detection for End-to-End Multi-Modal Understanding
• ashkamath/mdetr
•
We also investigate the utility of our model as an object detector on a given label set when fine-tuned in a few-shot setting.
Ranked #1 on
Visual Question Answering
on CLEVR-Humans
PHRASE GROUNDING QUESTION ANSWERING REFERRING EXPRESSION COMPREHENSION REFERRING EXPRESSION SEGMENTATION VISUAL GROUNDING VISUAL QUESTION ANSWERING
Grounding of Textual Phrases in Images by Reconstruction
We propose a novel approach which learns grounding by reconstructing a given phrase using an attention mechanism, which can be either latent or optimized directly.
Ranked #4 on
Phrase Grounding
on Flickr30k Entities Test
LANGUAGE MODELLING NATURAL LANGUAGE VISUAL GROUNDING PHRASE GROUNDING VISUAL GROUNDING
A Fast and Accurate One-Stage Approach to Visual Grounding
ICCV 2019
• zyang-ur/onestage_grounding
•
We propose a simple, fast, and accurate one-stage approach to visual grounding, inspired by the following insight.
Finding beans in burgers: Deep semantic-visual embedding with localization
CVPR 2018
• technicolor-research/dsve-loc
•
Several works have proposed to learn a two-path neural network that maps images and texts, respectively, to a same shared Euclidean space where geometry captures useful semantic relationships.
Visual Coreference Resolution in Visual Dialog using Neural Module Networks
ECCV 2018
• facebookresearch/corefnmn
• 
Visual dialog entails answering a series of questions grounded in an image, using dialog history as context.
Ranked #1 on
Common Sense Reasoning
on Visual Dialog v0.9
COMMON SENSE REASONING COREFERENCE RESOLUTION VISUAL DIALOG VISUAL GROUNDING VISUAL QUESTION ANSWERING
Visual Relation Grounding in Videos
In this paper, we explore a novel task named visual Relation Grounding in Videos (vRGV).
QUESTION ANSWERING VIDEO QUESTION ANSWERING VISUAL GROUNDING
Connecting Vision and Language with Localized Narratives
We ask annotators to describe an image with their voice while simultaneously hovering their mouse over the region they are describing.
Ranked #1 on
Image Captioning
on Localized Narratives