Search Results for author:
Found 18 papers, 8 papers with code
General Item Representation Learning for Cold-start Content Recommendations
Cold-start item recommendation is a long-standing challenge in recommendation systems.
Honeybee: Locality-enhanced Projector for Multimodal LLM
In Multimodal Large Language Models (MLLMs), a visual projector plays a crucial role in bridging pre-trained vision encoders with LLMs, enabling profound visual understanding while harnessing the LLMs' robust capabilities.
Ranked #2 on
Science Question Answering
on ScienceQA
(using extra training data)
Learning Pseudo-Labeler beyond Noun Concepts for Open-Vocabulary Object Detection
Specifically, the proposed method aims to learn arbitrary image-to-text mapping for pseudo-labeling of arbitrary concepts, named Pseudo-Labeling for Arbitrary Concepts (PLAC).
NICE: CVPR 2023 Challenge on Zero-shot Image Captioning
In this report, we introduce NICE (New frontiers for zero-shot Image Captioning Evaluation) project and share the results and outcomes of 2023 challenge.
Noise-aware Learning from Web-crawled Image-Text Data for Image Captioning
Image captioning is one of the straightforward tasks that can take advantage of large-scale web-crawled data which provides rich knowledge about the visual world for a captioning model.
Learning to Generate Text-grounded Mask for Open-world Semantic Segmentation from Only Image-Text Pairs
Existing open-world segmentation methods have shown impressive advances by employing contrastive learning (CL) to learn diverse visual concepts and transferring the learned image-level understanding to the segmentation task.
Ranked #2 on
Semantic Segmentation
on CC3M-TagMask
Contrastive Learning
Open Vocabulary Semantic Segmentation
+4
MSTR: Multi-Scale Transformer for End-to-End Human-Object Interaction Detection
Human-Object Interaction (HOI) detection is the task of identifying a set of <human, object, interaction> triplets from an image.
Boundary-aware Self-supervised Learning for Video Scene Segmentation
Inspired from this, we tackle video scene segmentation, which is a task of temporally localizing scene boundaries in a video, with a self-supervised learning framework where we mainly focus on designing effective pretext tasks.
Winning the ICCV'2021 VALUE Challenge: Task-aware Ensemble and Transfer Learning with Visual Concepts
The VALUE (Video-And-Language Understanding Evaluation) benchmark is newly introduced to evaluate and analyze multi-modal representation learning algorithms on three video-and-language tasks: Retrieval, QA, and Captioning.
Boundary-aware Pre-training for Video Scene Segmentation
Inspired from this, we tackle video scene segmentation, which is a task of temporally localizing scene boundaries in a video, with a self-supervised learning framework where we mainly focus on designing effective pretext tasks.
Local-Global Video-Text Interactions for Temporal Grounding
This paper addresses the problem of text-to-video temporal grounding, which aims to identify the time interval in a video semantically relevant to a text query.
Towards Oracle Knowledge Distillation with Neural Architecture Search
We present a novel framework of knowledge distillation that is capable of learning powerful and efficient student models from ensemble teacher networks.
Streamlined Dense Video Captioning
Dense video captioning is an extremely challenging task since accurate and coherent description of events in a video requires holistic understanding of video contents as well as contextual reasoning of individual events.
Learning to Specialize with Knowledge Distillation for Visual Question Answering
The proposed framework is model-agnostic and applicable to any tasks other than VQA, e. g., image classification with a large number of labels but few per-class examples, which is known to be difficult under existing MCL schemes.
Transfer Learning via Unsupervised Task Discovery for Visual Question Answering
Specifically, we employ linguistic knowledge sources such as structured lexical database (e. g. WordNet) and visual descriptions for unsupervised task discovery, and transfer a learned task conditional visual classifier as an answering unit in a visual question answering model.
Regularizing Deep Neural Networks by Noise: Its Interpretation and Optimization
Overfitting is one of the most critical challenges in deep neural networks, and there are various types of regularization methods to improve generalization performance.
Text-guided Attention Model for Image Captioning
Visual attention plays an important role to understand images and demonstrates its effectiveness in generating natural language descriptions of images.
MarioQA: Answering Questions by Watching Gameplay Videos
To address this objective, we automatically generate a customized synthetic VideoQA dataset using {\em Super Mario Bros.} gameplay videos so that it contains events with different levels of reasoning complexity.
