Visual Dialog
Visual Dialog requires an AI agent to hold a meaningful dialog with humans in natural, conversational language about visual content. Specifically, given an image, a dialog history, and a follow-up question about the image, the task is to answer the question.
Leaderboards
Greatest papers with code
Visual Dialog
CVPR 2017
• facebookresearch/ParlAI
• 
We introduce the task of Visual Dialog, which requires an AI agent to hold a meaningful dialog with humans in natural, conversational language about visual content.
Learning Cooperative Visual Dialog Agents with Deep Reinforcement Learning
ICCV 2017
• batra-mlp-lab/visdial-rl
•
Specifically, we pose a cooperative 'image guessing' game between two agents -- Qbot and Abot -- who communicate in natural language dialog so that Qbot can select an unseen image from a lineup of images.
Best of Both Worlds: Transferring Knowledge from Discriminative Learning to a Generative Visual Dialog Model
NeurIPS 2017
• jiasenlu/visDial.pytorch
•
In contrast, discriminative dialog models (D) that are trained to rank a list of candidate human responses outperform their generative counterparts; in terms of automatic metrics, diversity, and informativeness of the responses.
Recursive Visual Attention in Visual Dialog
Visual dialog is a challenging vision-language task, which requires the agent to answer multi-round questions about an image.
Visual Coreference Resolution in Visual Dialog using Neural Module Networks
ECCV 2018
• facebookresearch/corefnmn
• 
Visual dialog entails answering a series of questions grounded in an image, using dialog history as context.
COREFERENCE RESOLUTION VISUAL DIALOG VISUAL QUESTION ANSWERING
DualVD: An Adaptive Dual Encoding Model for Deep Visual Understanding in Visual Dialogue
• JXZe/DualVD
•
More importantly, we can tell which modality (visual or semantic) has more contribution in answering the current question by visualizing the gate values.
FEATURE SELECTION QUESTION ANSWERING VISUAL DIALOG VISUAL QUESTION ANSWERING
Large-Scale Answerer in Questioner's Mind for Visual Dialog Question Generation
Answerer in Questioner's Mind (AQM) is an information-theoretic framework that has been recently proposed for task-oriented dialog systems.
Answerer in Questioner's Mind: Information Theoretic Approach to Goal-Oriented Visual Dialog
NeurIPS 2018
• naver/aqm-plus
•
Goal-oriented dialogue tasks occur when a questioner asks an action-oriented question and an answerer responds with the intent of letting the questioner know a correct action to take.
Large-scale Pretraining for Visual Dialog: A Simple State-of-the-Art Baseline
Next, we find that additional finetuning using "dense" annotations in VisDial leads to even higher NDCG -- more than 10% over our base model -- but hurts MRR -- more than 17% below our base model!
Reasoning Visual Dialogs with Structural and Partial Observations
CVPR 2019
• zilongzheng/visdial-gnn
•
The answer to a given question is represented by a node with missing value.