Differentiable End-to-End Program Executor for Sample and Computationally Efficient VQA

1 Jan 2021 · Karan Samel, Zelin Zhao, Kuan Wang, Robin Luo, Binghong Chen, Le Song ·

We present a differentiable end-to-end program executor (DePe), which addresses Visual Question Answering (VQA) in a sample and computationally efficient manner. DePe parses the question into probabilistic programs and softly executes them to acquire the final answer. These functional programs adopt soft-logic functions to enable approximate probabilistic logic reasoning. In addition to the language, DePe also jointly learns visual object-centric representations in an end-to-end manner. We demonstrate through extensive experiments that DePe is more sample and computationally efficient than other VQA methodologies while retaining state-of-the-art performance.

PDF Abstract