Learning to Count Objects in Natural Images for Visual Question Answering
Visual Question Answering (VQA) models have struggled with counting objects in natural images so far. We identify a fundamental problem due to soft attention in these models as a cause. To circumvent this problem, we propose a neural network component that allows robust counting from object proposals. Experiments on a toy task show the effectiveness of this component and we obtain state-of-the-art accuracy on the number category of the VQA v2 dataset without negatively affecting other categories, even outperforming ensemble models with our single model. On a difficult balanced pair metric, the component gives a substantial improvement in counting over a strong baseline by 6.6%.
PDF Abstract ICLR 2018 PDF ICLR 2018 AbstractCode
Visual Question Answering
CLEVR
Visual Question Answering v2.0
Results from the Paper
Methods
No methods listed for this paper. Add
relevant methods here
