Search Results for author:
Found 15 papers, 7 papers with code
MATHGLANCE: Multimodal Large Language Models Do Not Know Where to Look in Mathematical Diagrams
Diagrams serve as a fundamental form of visual language, representing complex concepts and their inter-relationships through structured symbols, shapes, and spatial arrangements.
Visual Position Prompt for MLLM based Visual Grounding
We also introduce a VPP-SFT dataset with 0. 6M samples, consolidating high-quality visual grounding data into a compact format for efficient model training.
Open Eyes, Then Reason: Fine-grained Visual Mathematical Understanding in MLLMs
Current multimodal large language models (MLLMs) often underperform on mathematical problem-solving tasks that require fine-grained visual understanding.
Descriptive Caption Enhancement with Visual Specialists for Multimodal Perception
We propose to leverage off-the-shelf visual specialists, which were trained from annotated images initially not for image captioning, for enhancing the image caption.
Continual SFT Matches Multimodal RLHF with Negative Supervision
Our nSFT disentangles this negative supervision in RLHF paradigm, and continually aligns VLMs with a simple SFT loss.
Improving Multi-modal Large Language Model through Boosting Vision Capabilities
QLadder employs a learnable ``\textit{ladder}'' structure to deeply aggregates the intermediate representations from the frozen pretrained visual encoder (e. g., CLIP image encoder).
Ranked #165 on
Visual Question Answering
on MM-Vet
FullAnno: A Data Engine for Enhancing Image Comprehension of MLLMs
To this end, we devised the FullAnno system, which is a data engine that can generate large-scale, high-quality, and fine-grained image annotations consisting of the category and position of objects, region descriptions, text information, as well as image dense captions.
CSGO: Content-Style Composition in Text-to-Image Generation
Based on this pipeline, we construct a dataset IMAGStyle, the first large-scale style transfer dataset containing 210k image triplets, available for the community to explore and research.
VRP-SAM: SAM with Visual Reference Prompt
In this paper, we propose a novel Visual Reference Prompt (VRP) encoder that empowers the Segment Anything Model (SAM) to utilize annotated reference images as prompts for segmentation, creating the VRP-SAM model.
Exploring Effective Factors for Improving Visual In-Context Learning
By doing this, the model can leverage the diverse knowledge stored in different parts of the model to improve its performance on new tasks.
s-Adaptive Decoupled Prototype for Few-Shot Object Detection
To provide precise information for the query image, the prototype is decoupled into task-specific ones, which provide tailored guidance for 'where to look' and 'what to look for', respectively.
Self-Supervised Guided Segmentation Framework for Unsupervised Anomaly Detection
In this paper, a novel Self-Supervised Guided Segmentation Framework (SGSF) is proposed by jointly exploring effective generation method of forged anomalous samples and the normal sample features as the guidance information of segmentation for anomaly detection.
Singular Value Fine-tuning: Few-shot Segmentation requires Few-parameters Fine-tuning
In this paper, we rethink the paradigm and explore a new regime: {\em fine-tuning a small part of parameters in the backbone}.
Ranked #15 on
Few-Shot Semantic Segmentation
on COCO-20i (1-shot)
SSA: Semantic Structure Aware Inference for Weakly Pixel-Wise Dense Predictions without Cost
The pixel-wise dense prediction tasks based on weakly supervisions currently use Class Attention Maps (CAM) to generate pseudo masks as ground-truth.
Weakly-Supervised Object Localization
Weakly supervised Semantic Segmentation
+1
CTNet: Context-based Tandem Network for Semantic Segmentation
Specifically, the Spatial Contextual Module (SCM) is leveraged to uncover the spatial contextual dependency between pixels by exploring the correlation between pixels and categories.
Ranked #76 on
Semantic Segmentation
on ADE20K val
