We present the MAC network, a novel fully differentiable neural network architecture, designed to facilitate explicit and expressive reasoning.
In LXMERT, we build a large-scale Transformer model that consists of three encoders: an object relationship encoder, a language encoder, and a cross-modality encoder.
SOTA for Visual Reasoning on NLVR
LANGUAGE MODELLING QUESTION ANSWERING REGRESSION VISUAL QUESTION ANSWERING VISUAL REASONING
Human activity recognition is typically addressed by detecting key concepts like global and local motion, features related to object classes present in the scene, as well as features related to the global context.
HUMAN ACTIVITY RECOGNITION OBJECT DETECTION VISUAL REASONING
We introduce a general-purpose conditioning method for neural networks called FiLM: Feature-wise Linear Modulation.
Existing methods for visual reasoning attempt to directly map inputs to outputs using black-box architectures without explicitly modeling the underlying reasoning processes.
When building artificial intelligence systems that can reason and answer questions about visual data, we need diagnostic tests to analyze our progress and discover shortcomings.
QUESTION ANSWERING VISUAL QUESTION ANSWERING VISUAL REASONING
We propose VisualBERT, a simple and flexible framework for modeling a broad range of vision-and-language tasks.
SOTA for Visual Reasoning on NLVR
LANGUAGE MODELLING VISUAL QUESTION ANSWERING VISUAL REASONING
We aim to dismantle the prevalent black-box neural architectures used in complex visual reasoning tasks, into the proposed eXplainable and eXplicit Neural Modules (XNMs), which advance beyond existing neural module networks towards using scene graphs --- objects as nodes and the pairwise relationships as edges --- for explainable and explicit reasoning with structured knowledge.
COG is much simpler than the general problem of video analysis, yet it addresses many of the problems relating to visual and logical reasoning and memory -- problems that remain challenging for modern deep learning architectures.
To resolve, such questions often require reference to multiple plot elements and synthesis of information distributed spatially throughout a figure.