Search Results for author:
Found 19 papers, 10 papers with code
DesCo: Learning Object Recognition with Rich Language Descriptions
In fact, our experiments show that GLIP, the state-of-the-art vision-language model for object detection, often disregards contextual information in the language descriptions and instead relies heavily on detecting objects solely by their names.
Symbolic Chain-of-Thought Distillation: Small Models Can Also "Think" Step-by-Step
We release our corpus of chain-of-thought samples and code.
MetaVL: Transferring In-Context Learning Ability From Language Models to Vision-Language Models
In this paper, we study an interesting hypothesis: can we transfer the in-context learning ability from the language domain to VL domain?
GLIPv2: Unifying Localization and Vision-Language Understanding
We present GLIPv2, a grounded VL understanding model, that serves both localization tasks (e. g., object detection, instance segmentation) and Vision-Language (VL) understanding tasks (e. g., VQA, image captioning).
Ranked #1 on
Phrase Grounding
on Flickr30k Entities Test
(using extra training data)
DisinfoMeme: A Multimodal Dataset for Detecting Meme Intentionally Spreading Out Disinformation
Disinformation has become a serious problem on social media.
GeoMLAMA: Geo-Diverse Commonsense Probing on Multilingual Pre-Trained Language Models
In this paper, we introduce a benchmark dataset, Geo-Diverse Commonsense Multilingual Language Models Analysis (GeoMLAMA), for probing the diversity of the relational knowledge in multilingual PLMs.
On the Paradox of Learning to Reason from Data
Logical reasoning is needed in a wide range of NLP tasks.
ELEVATER: A Benchmark and Toolkit for Evaluating Language-Augmented Visual Models
In general, these language-augmented visual models demonstrate strong transferability to a variety of datasets and tasks.
Ranked #1 on
Zero-Shot Image Classification
on ODinW
RegionCLIP: Region-based Language-Image Pretraining
However, we show that directly applying such models to recognize image regions for object detection leads to poor performance due to a domain shift: CLIP was trained to match an image as a whole to a text description, without capturing the fine-grained alignment between image regions and text spans.
Ranked #5 on
Open Vocabulary Object Detection
on MSCOCO
(using extra training data)
SGEITL: Scene Graph Enhanced Image-Text Learning for Visual Commonsense Reasoning
As for pre-training, a scene-graph-aware pre-training method is proposed to leverage structure knowledge extracted in the visual scene graph.
Grounded Language-Image Pre-training
The unification brings two benefits: 1) it allows GLIP to learn from both detection and grounding data to improve both tasks and bootstrap a good grounding model; 2) GLIP can leverage massive image-text pairs by generating grounding boxes in a self-training fashion, making the learned representation semantic-rich.
Ranked #1 on
2D Object Detection
on RF100
Broaden the Vision: Geo-Diverse Visual Commonsense Reasoning
Commonsense is defined as the knowledge that is shared by everyone.
Ranked #1 on
Visual Commonsense Reasoning
on GD-VCR
Cultural Vocal Bursts Intensity Prediction
Visual Commonsense Reasoning
BERTHop: An Effective Vision-and-Language Model for Chest X-ray Disease Diagnosis
Vision-and-language(V&L) models take image and text as input and learn to capture the associations between them.
How Much Can CLIP Benefit Vision-and-Language Tasks?
Most existing Vision-and-Language (V&L) models rely on pre-trained visual encoders, using a relatively small set of manually-annotated data (as compared to web-crawled data), to perceive the visual world.
Ranked #4 on
Vision and Language Navigation
on RxR
(using extra training data)
Unsupervised Vision-and-Language Pre-training Without Parallel Images and Captions
Pre-trained contextual vision-and-language (V&L) models have achieved impressive performance on various benchmarks.
What Does BERT with Vision Look At?
Pre-trained visually grounded language models such as ViLBERT, LXMERT, and UNITER have achieved significant performance improvement on vision-and-language tasks but what they learn during pre-training remains unclear.
VisualBERT: A Simple and Performant Baseline for Vision and Language
We propose VisualBERT, a simple and flexible framework for modeling a broad range of vision-and-language tasks.
Ranked #1 on
Visual Reasoning
on NLVR
Efficient Contextual Representation Learning With Continuous Outputs
Contextual representation models have achieved great success in improving various downstream natural language processing tasks.
Efficient Contextual Representation Learning Without Softmax Layer
Our framework reduces the time spent on the output layer to a negligible level, eliminates almost all the trainable parameters of the softmax layer and performs language modeling without truncating the vocabulary.
