Image-Question-Answer Synergistic Network for Visual Dialog

CVPR 2019  ·  Dalu Guo, Chang Xu, DaCheng Tao ·

The image, question (combined with the history for de-referencing), and the corresponding answer are three vital components of visual dialog. Classical visual dialog systems integrate the image, question, and history to search for or generate the best matched answer, and so, this approach significantly ignores the role of the answer. In this paper, we devise a novel image-question-answer synergistic network to value the role of the answer for precise visual dialog. We extend the traditional one-stage solution to a two-stage solution. In the first stage, candidate answers are coarsely scored according to their relevance to the image and question pair. Afterward, in the second stage, answers with high probability of being correct are re-ranked by synergizing with image and question. On the Visual Dialog v1.0 dataset, the proposed synergistic network boosts the discriminative visual dialog model to achieve a new state-of-the-art of 57.88\% normalized discounted cumulative gain. A generative visual dialog model equipped with the proposed technique also shows promising improvements.

PDF Abstract CVPR 2019 PDF CVPR 2019 Abstract
No code implementations yet. Submit your code now
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Visual Dialog Visual Dialog v1.0 test-std Synergistic NDCG (x 100) 57.32 # 52
MRR (x 100) 62.20 # 36
R@1 47.90 # 35
R@5 80.43 # 28
Mean 4.17 # 44

Methods


No methods listed for this paper. Add relevant methods here