Referring Expression Comprehension
45 papers with code • 7 benchmarks • 6 datasets
Benchmarks
These leaderboards are used to track progress in Referring Expression Comprehension
| Trend | Dataset | Best Model | Paper | Code | Compare |
|---|---|---|---|---|---|
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
Libraries
Use these libraries to find Referring Expression Comprehension models and implementations
RefCOCO
Google Refexp
Talk2Car
GRIT
A Game Of Sorts
Most implemented papers
ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks
facebookresearch/vilbert-multi-task
•
•
NeurIPS 2019
We present ViLBERT (short for Vision-and-Language BERT), a model for learning task-agnostic joint representations of image content and natural language.
Compositional Attention Networks for Machine Reasoning
stanfordnlp/mac-network
•
•
ICLR 2018
We present the MAC network, a novel fully differentiable neural network architecture, designed to facilitate explicit and expressive reasoning.
UNITER: UNiversal Image-TExt Representation Learning
ChenRocks/UNITER
•
•
ECCV 2020
Different from previous work that applies joint random masking to both modalities, we use conditional masking on pre-training tasks (i. e., masked language/region modeling is conditioned on full observation of image/text).
CLEVR-Ref+: Diagnosing Visual Reasoning with Referring Expressions
ruotianluo/iep-ref
•
•
CVPR 2019
Yet there has been evidence that current benchmark datasets suffer from bias, and current state-of-the-art models cannot be easily evaluated on their intermediate reasoning process.
VL-BERT: Pre-training of Generic Visual-Linguistic Representations
jackroos/VL-BERT
•
•
ICLR 2020
We introduce a new pre-trainable generic representation for visual-linguistic tasks, called Visual-Linguistic BERT (VL-BERT for short).
A Joint Speaker-Listener-Reinforcer Model for Referring Expressions
lichengunc/speaker_listener_reinforcer
•
•
CVPR 2017
The speaker generates referring expressions, the listener comprehends referring expressions, and the reinforcer introduces a reward function to guide sampling of more discriminative expressions.
A Fast and Accurate One-Stage Approach to Visual Grounding
zyang-ur/onestage_grounding
•
•
ICCV 2019
We propose a simple, fast, and accurate one-stage approach to visual grounding, inspired by the following insight.
Large-Scale Adversarial Training for Vision-and-Language Representation Learning
zhegan27/VILLA
•
•
NeurIPS 2020
We present VILLA, the first known effort on large-scale adversarial training for vision-and-language (V+L) representation learning.
TransVG: End-to-End Visual Grounding with Transformers
djiajunustc/TransVG
•
•
ICCV 2021
In this paper, we present a neat yet effective transformer-based framework for visual grounding, namely TransVG, to address the task of grounding a language query to the corresponding region onto an image.
MDETR -- Modulated Detection for End-to-End Multi-Modal Understanding
ashkamath/mdetr
•
•
We also investigate the utility of our model as an object detector on a given label set when fine-tuned in a few-shot setting.
