Search Results for author:
Found 16 papers, 11 papers with code
Efficient Multilingual Multi-modal Pre-training through Triple Contrastive Loss
In this multilingual approach, a typical setup is to use pairs of (image and English-text) and translation pairs.
CheX-GPT: Harnessing Large Language Models for Enhanced Chest X-ray Report Labeling
Moreover, models using expert-annotated data are limited by data scarcity and pre-defined classes, impacting their performance, flexibility and scalability.
Honeybee: Locality-enhanced Projector for Multimodal LLM
In Multimodal Large Language Models (MLLMs), a visual projector plays a crucial role in bridging pre-trained vision encoders with LLMs, enabling profound visual understanding while harnessing the LLMs' robust capabilities.
Ranked #1 on
Science Question Answering
on ScienceQA
(using extra training data)
Learning Pseudo-Labeler beyond Noun Concepts for Open-Vocabulary Object Detection
Specifically, the proposed method aims to learn arbitrary image-to-text mapping for pseudo-labeling of arbitrary concepts, named Pseudo-Labeling for Arbitrary Concepts (PLAC).
Large Language Models are Temporal and Causal Reasoners for Video Question Answering
We observe that the LLMs provide effective priors in exploiting $\textit{linguistic shortcuts}$ for temporal and causal reasoning in Video Question Answering (VideoQA).
Ranked #1 on
Video Question Answering
on TVQA
CXR-CLIP: Toward Large Scale Chest X-ray Language-Image Pre-training
In this paper, we tackle the lack of image-text data in chest X-ray by expanding image-label pair as image-text pair via general prompt and utilizing multiple images and multiple sections in a radiologic report.
NICE: CVPR 2023 Challenge on Zero-shot Image Captioning
In this report, we introduce NICE (New frontiers for zero-shot Image Captioning Evaluation) project and share the results and outcomes of 2023 challenge.
Open-Vocabulary Object Detection using Pseudo Caption Labels
Recent open-vocabulary detection methods aim to detect novel objects by distilling knowledge from vision-language models (VLMs) trained on a vast amount of image-text pairs.
MELTR: Meta Loss Transformer for Learning to Fine-tune Video Foundation Models
Therefore, we propose MEta Loss TRansformer (MELTR), a plug-in module that automatically and non-linearly combines various loss functions to aid learning the target task via auxiliary learning.
Ranked #2 on
Video Captioning
on YouCook2
Noise-aware Learning from Web-crawled Image-Text Data for Image Captioning
Image captioning is one of the straightforward tasks that can take advantage of large-scale web-crawled data which provides rich knowledge about the visual world for a captioning model.
Learning to Generate Text-grounded Mask for Open-world Semantic Segmentation from Only Image-Text Pairs
Existing open-world segmentation methods have shown impressive advances by employing contrastive learning (CL) to learn diverse visual concepts and transferring the learned image-level understanding to the segmentation task.
Ranked #2 on
Semantic Segmentation
on CC3M-TagMask
Contrastive Learning
Open Vocabulary Semantic Segmentation
+4
Sparse DETR: Efficient End-to-End Object Detection with Learnable Sparsity
Deformable DETR uses the multiscale feature to ameliorate performance, however, the number of encoder tokens increases by 20x compared to DETR, and the computation cost of the encoder attention remains a bottleneck.
Spatially Consistent Representation Learning
While these contrastive methods mainly focus on generating invariant global representations at the image-level under semantic-preserving transformations, they are prone to overlook spatial consistency of local representations and therefore have a limitation in pretraining for localization tasks such as object detection and instance segmentation.
BABO: Background Activation Black-Out for Efficient Object Detection
Recent advances in deep learning have enabled complex real-world use cases comprised of multiple vision tasks and detection tasks are being shifted to the edge side as a pre-processing step of the entire workload.
PVANet: Lightweight Deep Neural Networks for Real-time Object Detection
In object detection, reducing computational cost is as important as improving accuracy for most practical usages.
PVANET: Deep but Lightweight Neural Networks for Real-time Object Detection
This paper presents how we can achieve the state-of-the-art accuracy in multi-category object detection task while minimizing the computational cost by adapting and combining recent technical innovations.
