Search Results for author:
Found 14 papers, 9 papers with code
TAG: Boosting Text-VQA via Text-aware Visual Question-answer Generation
To address this deficiency, we develop a new method to generate high-quality and diverse QA pairs by explicitly utilizing the existing rich text available in the scene context of each image.
Can domain adaptation make object recognition work for everyone?
Despite the rapid progress in deep visual recognition, modern computer vision datasets significantly overrepresent the developed world and models trained on such datasets underperform on images from unseen geographies.
CLIP-Lite: Information Efficient Visual Representation Learning with Language Supervision
We propose CLIP-Lite, an information efficient method for visual representation learning by feature alignment with textual annotations.
PreViTS: Contrastive Pretraining with Video Tracking Supervision
In this work, we propose PreViTS, an SSL framework that utilizes an unsupervised tracking signal for selecting clips containing the same object, which helps better utilize temporal transformations of objects.
Align before Fuse: Vision and Language Representation Learning with Momentum Distillation
Most existing methods employ a transformer-based multimodal encoder to jointly model visual tokens (region-based image features) and word tokens.
Ranked #5 on
Open Vocabulary Attribute Detection
on OVAD-Box benchmark
(using extra training data)
CASTing Your Model: Learning to Localize Improves Self-Supervised Representations
Recent advances in self-supervised learning (SSL) have largely closed the gap with supervised ImageNet pretraining.
SOrT-ing VQA Models : Contrastive Gradient Learning for Improved Consistency
Recent research in Visual Question Answering (VQA) has revealed state-of-the-art models to be inconsistent in their understanding of the world -- they answer seemingly difficult questions requiring reasoning correctly but get simpler associated sub-questions wrong.
SQuINTing at VQA Models: Introspecting VQA Models with Sub-Questions
We quantify the extent to which this phenomenon occurs by creating a new Reasoning split of the VQA dataset and collecting VQA-introspect, a new dataset1 which consists of 238K new perception questions which serve as sub questions corresponding to the set of perceptual tasks needed to effectively answer the complex reasoning questions in the Reasoning split.
Trick or TReAT: Thematic Reinforcement for Artistic Typography
An approach to make text visually appealing and memorable is semantic reinforcement - the use of visual cues alluding to the context or theme in which the word is being used to reinforce the message (e. g., Google Doodles).
Taking a HINT: Leveraging Explanations to Make Vision and Language Models More Grounded
Many vision and language models suffer from poor visual grounding - often falling back on easy-to-learn language priors rather than basing their decisions on visual concepts in the image.
Choose Your Neuron: Incorporating Domain Knowledge through Neuron-Importance
Our approach, which we call Neuron Importance-AwareWeight Transfer (NIWT), learns to map domain knowledge about novel "unseen" classes onto this dictionary of learned concepts and then optimizes for network parameters that can effectively combine these concepts - essentially learning classifiers by discovering and composing learned semantic concepts in deep networks.
Grad-CAM: Why did you say that?
We propose a technique for making Convolutional Neural Network (CNN)-based models more transparent by visualizing input regions that are 'important' for predictions -- or visual explanations.
Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization
For captioning and VQA, we show that even non-attention based models can localize inputs.
Counting Everyday Objects in Everyday Scenes
In this work, we build dedicated models for counting designed to tackle the large variance in counts, appearances, and scales of objects found in natural scenes.
Ranked #1 on
Object Counting
on COCO count-test
