Search Results for author:
Found 18 papers, 16 papers with code
Cheap and Quick: Efficient Vision-Language Instruction Tuning for Large Language Models
To validate MMA, we apply it to a recent LLM called LLaMA and term this formed large vision-language instructed model as LaVIN.
Improving Image Captioning by Leveraging Intra- and Inter-layer Global Representation in Transformer Network
The latter contains a Global Adaptive Controller that can adaptively fuse the global information into the decoder to guide the caption generation.
Towards Efficient Visual Adaption via Structural Re-parameterization
Experimental results show the superior performance and efficiency of RepAdapter than the state-of-the-art PETL methods.
Feast Your Eyes: Mixture-of-Resolution Adaptation for Multimodal Large Language Models
Contrary to previous works, we study this problem from the perspective of image resolution, and reveal that a combination of low- and high-resolution visual features can effectively mitigate this shortcoming.
Ranked #57 on
Visual Question Answering
on MM-Vet
Multi-task Collaborative Network for Joint Referring Expression Comprehension and Segmentation
In addition, we address a key challenge in this multi-task setup, i. e., the prediction conflict, with two innovative designs namely, Consistency Energy Maximization (CEM) and Adaptive Soft Non-Located Suppression (ASNLS).
Generalized Referring Expression Comprehension
Referring Expression
+2
SeqTR: A Simple yet Universal Network for Visual Grounding
In this paper, we propose a simple yet universal network termed SeqTR for visual grounding tasks, e. g., phrase localization, referring expression comprehension (REC) and segmentation (RES).
Towards End-to-end Semi-supervised Learning for One-stage Object Detection
In addition to this challenge, we also reveal two key issues in one-stage SSOD, which are low-quality pseudo-labeling and multi-task optimization conflict, respectively.
Active Teacher for Semi-Supervised Object Detection
In this paper, we study teacher-student learning from the perspective of data initialization and propose a novel algorithm called Active Teacher(Source code are available at: \url{https://github. com/HunterJ-Lin/ActiveTeacher}) for semi-supervised object detection (SSOD).
A Survivor in the Era of Large-Scale Pretraining: An Empirical Study of One-Stage Referring Expression Comprehension
But the most encouraging finding is that with much less training overhead and parameters, SimREC can still achieve better performance than a set of large-scale pre-trained models, e. g., UNITER and VILLA, portraying the special role of REC in existing V&L research.
3D-STMN: Dependency-Driven Superpoint-Text Matching Network for End-to-End 3D Referring Expression Segmentation
In 3D Referring Expression Segmentation (3D-RES), the earlier approach adopts a two-stage paradigm, extracting segmentation proposals and then matching them with referring expressions.
Towards Lightweight Transformer via Group-wise Transformation for Vision-and-Language Tasks
Despite the exciting performance, Transformer is criticized for its excessive parameters and computation cost.
A Real-time Global Inference Network for One-stage Referring Expression Comprehension
Referring Expression Comprehension (REC) is an emerging research spot in computer vision, which refers to detecting the target region in an image given an text description.
Deep Instruction Tuning for Segment Anything Model
Segment Anything Model (SAM) exhibits powerful yet versatile capabilities on (un) conditional image segmentation tasks recently.
Fast Text-to-3D-Aware Face Generation and Manipulation via Direct Cross-modal Mapping and Geometric Regularization
Text-to-3D-aware face (T3D Face) generation and manipulation is an emerging research hot spot in machine learning, which still suffers from low efficiency and poor quality.
Towards Omni-supervised Referring Expression Segmentation
To address this issue, we propose a new learning task for RES called Omni-supervised Referring Expression Segmentation (Omni-RES), which aims to make full use of unlabeled, fully labeled and weakly labeled data, e. g., referring points or grounding boxes, for efficient RES training.
Towards Language-guided Visual Recognition via Dynamic Convolutions
Based on the LaConv module, we further build the first fully language-driven convolution network, termed as LaConvNet, which can unify the visual recognition and multi-modal reasoning in one forward structure.
RefCLIP: A Universal Teacher for Weakly Supervised Referring Expression Comprehension
Based on RefCLIP, we further propose the first model-agnostic weakly supervised training scheme for existing REC models, where RefCLIP acts as a mature teacher to generate pseudo-labels for teaching common REC models.
RefTeacher: A Strong Baseline for Semi-Supervised Referring Expression Comprehension
In this paper, we present the first attempt of semi-supervised learning for REC and propose a strong baseline method called RefTeacher.
