Search Results for author:
Found 29 papers, 9 papers with code
Vision-Language Pretraining: Current Trends and the Future
In addition to the larger pretraining datasets, the transformer architecture (Vaswani et al., 2017) and in particular self-attention applied to two modalities are responsible for the impressive performance of the recent pretrained models on downstream tasks (Hendricks et al., 2021).
SelecMix: Debiased Learning by Contradicting-pair Sampling
Experiments on standard benchmarks demonstrate the effectiveness of the method, in particular when label noise complicates the identification of bias-conflicting examples.
ID and OOD Performance Are Sometimes Inversely Correlated on Real-world Datasets
This paper shows that inverse correlations between ID and OOD performance do happen in real-world benchmarks.
Predicting is not Understanding: Recognizing and Addressing Underspecification in Machine Learning
Underspecification refers to the existence of multiple models that are indistinguishable in their in-domain accuracy, even though they differ in other desirable properties such as out-of-distribution (OOD) performance.
EBMs vs. CL: Exploring Self-Supervised Visual Pretraining for Visual Question Answering
In a simple setting similar to CLEVR, we find that CL representations also improve systematic generalization, and even match the performance of representations from a larger, supervised, ImageNet-pretrained model.
Active Learning by Feature Mixing
We identify unlabelled instances with sufficiently-distinct features by seeking inconsistencies in predictions resulting from interventions on their representations.
Image Retrieval on Real-life Images with Pre-trained Vision-and-Language Models
We demonstrate that with a relatively simple architecture, CIRPLANT outperforms existing methods on open-domain images, while matching state-of-the-art accuracy on the existing narrow datasets, such as fashion.
Ranked #3 on
Image Retrieval
on CIRR
Evading the Simplicity Bias: Training a Diverse Set of Models Discovers Solutions with Superior OOD Generalization
The method - the first to evade the simplicity bias - highlights the need for a better understanding and control of inductive biases in deep learning.
Beyond Question-Based Biases: Assessing Multimodal Shortcut Learning in Visual Question Answering
We use this new evaluation in a large-scale study of existing approaches for VQA.
Ranked #1 on
Visual Question Answering (VQA)
on VQA-CE
Reasoning over Vision and Language: Exploring the Benefits of Supplemental Knowledge
The technique brings clear benefits to knowledge-demanding question answering tasks (OK-VQA, FVQA) by capturing semantic and relational knowledge absent from existing models.
Unshuffling Data for Improved Generalization in Visual Question Answering
subsets treated as multiple training environments can guide the learning of models with better out-of-distribution generalization.
Counterfactual Vision-and-Language Navigation: Unravelling the Unseen
The task of vision-and-language navigation (VLN) requires an agent to follow text instructions to find its way through simulated household environments.
On the Value of Out-of-Distribution Testing: An Example of Goodhart's Law
Out-of-distribution (OOD) testing is increasingly popular for evaluating a machine learning system's ability to generalize beyond the biases of a training set.
Visual Question Answering with Prior Class Semantics
We present a novel mechanism to embed prior knowledge in a model for visual question answering.
Learning What Makes a Difference from Counterfactual Examples and Gradient Supervision
One of the primary challenges limiting the applicability of deep learning is its susceptibility to learning spurious correlations rather than the underlying mechanisms of the task of interest.
Multi-Label Image Classification
Natural Language Inference
+3
Unshuffling Data for Improved Generalization
subsets treated as multiple training environments can guide the learning of models with better out-of-distribution generalization.
On Incorporating Semantic Prior Knowledge in Deep Learning Through Embedding-Space Constraints
We also show that incorporating this type of prior knowledge with our method brings consistent improvements, independently from the amount of supervised data used.
On Incorporating Semantic Prior Knowlegde in Deep Learning Through Embedding-Space Constraints
We also show that incorporating this type of prior knowledge with our method brings consistent improvements, independently from the amount of supervised data used.
V-PROM: A Benchmark for Visual Reasoning Using Visual Progressive Matrices
One of the primary challenges faced by deep learning is the degree to which current methods exploit superficial statistics and dataset bias, rather than learning to generalise over the specific representations they have experienced.
Actively Seeking and Learning from Live Data
One of the key limitations of traditional machine learning methods is their requirement for training data that exemplifies all the information to be learned.
Visual Question Answering as a Meta Learning Task
At test time, the method is provided with a support set of example questions/answers, over which it reasons to resolve the given question.
Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environments
This is significant because a robot interpreting a natural-language navigation instruction on the basis of what it sees is carrying out a vision and language process that is similar to Visual Question Answering.
Ranked #3 on
Visual Navigation
on R2R
Tips and Tricks for Visual Question Answering: Learnings from the 2017 Challenge
This paper presents a state-of-the-art model for visual question answering (VQA), which won the first place in the 2017 VQA Challenge.
Ranked #32 on
Visual Question Answering (VQA)
on VQA v2 test-dev
Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering
Top-down visual attention mechanisms have been used extensively in image captioning and visual question answering (VQA) to enable deeper image understanding through fine-grained analysis and even multiple steps of reasoning.
Ranked #57 on
Visual Question Answering (VQA)
on VQA v2 test-std
Zero-Shot Visual Question Answering
Answering general questions about images requires methods capable of Zero-Shot VQA, that is, methods able to answer questions beyond the scope of the training questions.
Graph-Structured Representations for Visual Question Answering
This paper proposes to improve visual question answering (VQA) with structured representations of both scene contents and questions.
Visual Question Answering: A Survey of Methods and Datasets
Visual Question Answering (VQA) is a challenging task that has received increasing attention from both the computer vision and the natural language processing communities.
Learning to Extract Motion from Videos in Convolutional Neural Networks
Our contributions on network design and rotation invariance offer insights nonspecific to motion estimation.
Learning Similarity Metrics for Dynamic Scene Segmentation
This paper addresses the segmentation of videos with arbitrary motion, including dynamic textures, using novel motion features and a supervised learning approach.
