Search Results for author:
Found 47 papers, 35 papers with code
Zooming In on Fakes: A Novel Dataset for Localized AI-Generated Image Detection with Forgery Amplification Approach
In addition, we further propose \textbf{NFA-ViT}, a Noise-guided Forgery Amplification Vision Transformer that enhances the detection of localized forgeries by amplifying forgery-related features across the entire image.
An Efficient and Mixed Heterogeneous Model for Image Restoration
This hierarchical and adaptive design enables the model to leverage the strengths of CNNs in local feature extraction, Mamba in global context modeling, and attention mechanisms in dynamic feature refinement.
JavisDiT: Joint Audio-Video Diffusion Transformer with Hierarchical Spatio-Temporal Prior Synchronization
This paper introduces JavisDiT, a novel Joint Audio-Video Diffusion Transformer designed for synchronized audio-video generation (JAVG).
MLLM-Selector: Necessity and Diversity-driven High-Value Data Selection for Enhanced Visual Instruction Tuning
Visual instruction tuning (VIT) has emerged as a crucial technique for enabling multi-modal large language models (MLLMs) to follow user instructions adeptly.
ComfyGPT: A Self-Optimizing Multi-Agent System for Comprehensive ComfyUI Workflow Generation
Second, we proposed FlowAgent, a LLM-based workflow generation agent that uses both supervised fine-tuning (SFT) and reinforcement learning (RL) to improve workflow generation accuracy.
QuoTA: Query-oriented Token Assignment via CoT Query Decouple for Long Video Comprehension
Specifically, (i) QuoTA strategically allocates frame-level importance scores based on query relevance, enabling one-time visual token assignment before cross-modal interactions in decoder layers, (ii) we decouple the query through Chain-of-Thoughts reasoning to facilitate more precise LVLM-based frame importance scoring, and (iii) QuoTA offers a plug-and-play functionality that extends to existing LVLMs.
Towards General Visual-Linguistic Face Forgery Detection(V2)
Face manipulation techniques have achieved significant advances, presenting serious challenges to security and social trust.
IPDN: Image-enhanced Prompt Decoding Network for 3D Referring Expression Segmentation
To tackle intent ambiguity, we designed a Prompt-Aware Decoder (PAD) that guides the decoding process by deriving task-driven signals from the interaction between the expression and visual features.
RG-SAN: Rule-Guided Spatial Awareness Network for End-to-End 3D Referring Expression Segmentation
The RG-SAN consists of the Text-driven Localization Module (TLM) and the Rule-guided Weak Supervision (RWS) strategy.
Any-to-3D Generation via Hybrid Diffusion Supervision
To our knowledge, this is the first method to generate 3D objects from any modality prompts.
Video-RAG: Visually-aligned Retrieval-Augmented Long Video Comprehension
Existing large video-language models (LVLMs) struggle to comprehend long videos correctly due to limited context.
Synergistic Dual Spatial-aware Generation of Image-to-Text and Text-to-Image
In the visual spatial understanding (VSU) area, spatial image-to-text (SI2T) and spatial text-to-image (ST2I) are two fundamental tasks that appear in dual form.
$γ-$MoD: Exploring Mixture-of-Depth Adaptation for Multimodal Large Language Models
In $\gamma$-MoD, a novel metric is proposed to guide the deployment of MoDs in the MLLM, namely rank of attention maps (ARank).
Ranked #170 on
Visual Question Answering
on MM-Vet
I2EBench: A Comprehensive Benchmark for Instruction-based Image Editing
We will open-source I2EBench, including all instructions, input images, human annotations, edited images from all evaluated methods, and a simple script for evaluating the results from new IIE models.
TraDiffusion: Trajectory-Based Training-Free Image Generation
In this work, we propose a training-free, trajectory-based controllable T2I approach, termed TraDiffusion.
ControlMLLM: Training-Free Visual Prompt Learning for Multimodal Large Language Models
We observe that attention, as the core module of MLLMs, connects text prompt tokens and visual tokens, ultimately determining the final results.
3D-GRES: Generalized 3D Referring Expression Segmentation
3D Referring Expression Segmentation (3D-RES) is dedicated to segmenting a specific instance within a 3D space based on a natural language description.
INF-LLaVA: Dual-perspective Perception for High-Resolution Multimodal Large Language Model
However, the quadratic complexity of the vision encoder in MLLMs constrains the resolution of input images.
Ranked #166 on
Visual Question Answering
on MM-Vet
Exploring Phrase-Level Grounding with Text-to-Image Diffusion Model
Specifically, we introduce the DiffPNG framework, a straightforward yet effective approach that fully capitalizes on the diffusion's architecture for segmentation by decomposing the process into a sequence of localization, segmentation, and refinement steps.
Multi-branch Collaborative Learning Network for 3D Visual Grounding
3D referring expression comprehension (3DREC) and segmentation (3DRES) have overlapping objectives, indicating their potential for collaboration.
HRSAM: Efficient Interactive Segmentation in High-Resolution Images
Within FLA, we implement Flash Swin attention, achieving over a 35% speedup compared to traditional Swin attention, and propose a KV-only padding mechanism to enhance extrapolation.
Evaluating and Analyzing Relationship Hallucinations in Large Vision-Language Models
We identify three types of relationship co-occurrences that lead to hallucinations: relationship-relationship, subject-relationship, and relationship-object.
Beat: Bi-directional One-to-Many Embedding Alignment for Text-based Person Retrieval
To address this issue, we propose an effective bi-directional one-to-many embedding paradigm that offers a clear optimization direction for each sample, thus mitigating the optimization problem.
Towards Semantic Equivalence of Tokenization in Multimodal LLM
The resulting vision tokens effectively preserve semantic integrity and capture both low-frequency and high-frequency visual features.
Ranked #73 on
Visual Question Answering
on MM-Vet
SAM as the Guide: Mastering Pseudo-Label Refinement in Semi-Supervised Referring Expression Segmentation
These strategies are designed to extract the most accurate masks from SAM's output, thus guiding the training of the student model with enhanced precision.
Image Captioning via Dynamic Path Customization
This paper explores a novel dynamic network for vision and language tasks, where the inferring structure is customized on the fly for different inputs.
X-Oscar: A Progressive Framework for High-quality Text-guided 3D Animatable Avatar Generation
Recent advancements in automatic 3D avatar generation guided by text have made significant progress.
Toward Open-Set Human Object Interaction Detection
To address this challenge, we introduce a simple Disentangled HOI Detection (DHD) model for detecting novel relationships by integrating an open-set object detector with a Visual Language Model (VLM).
Rotated Multi-Scale Interaction Network for Referring Remote Sensing Image Segmentation
Referring Remote Sensing Image Segmentation (RRSIS) is a new challenge that combines computer vision and natural language processing, delineating specific regions in aerial images as described by textual queries.
X-Dreamer: Creating High-quality 3D Content by Bridging the Domain Gap Between Text-to-2D and Text-to-3D Generation
Nevertheless, a substantial domain gap exists between 2D images and 3D assets, primarily attributed to variations in camera-related attributes and the exclusive presence of foreground objects.
Semi-Supervised Panoptic Narrative Grounding
Remarkably, our SS-PNG-NW+ outperforms fully-supervised models with only 30% and 50% supervision data, exceeding their performance by 0. 8% and 1. 1% respectively.
NICE: Improving Panoptic Narrative Detection and Segmentation with Cascading Collaborative Learning
To address this, we introduce two cascading modules based on the barycenter of the mask, which are Coordinate Guided Aggregation (CGA) and Barycenter Driven Localization (BDL), responsible for segmentation and detection, respectively.
JM3D & JM3D-LLM: Elevating 3D Understanding with Joint Multi-modal Cues
The rising importance of 3D understanding, pivotal in computer vision, autonomous driving, and robotics, is evident.
3D-STMN: Dependency-Driven Superpoint-Text Matching Network for End-to-End 3D Referring Expression Segmentation
In 3D Referring Expression Segmentation (3D-RES), the earlier approach adopts a two-stage paradigm, extracting segmentation proposals and then matching them with referring expressions.
MMAPS: End-to-End Multi-Grained Multi-Modal Attribute-Aware Product Summarization
Furthermore, we model product attributes based on both text and image modalities so that multi-modal product characteristics can be manifested in the generated summaries.
Beyond First Impressions: Integrating Joint Multi-modal Cues for Comprehensive 3D Representation
Insufficient synergy neglects the idea that a robust 3D representation should align with the joint vision-language space, rather than independently aligning with each modality.
X-Mesh: Towards Fast and Accurate Text-driven 3D Stylization via Dynamic Textual Guidance
Text-driven 3D stylization is a complex and crucial task in the fields of computer vision (CV) and computer graphics (CG), aimed at transforming a bare mesh to fit a target text.
Towards Local Visual Modeling for Image Captioning
In this paper, we study the local visual modeling with grid features for image captioning, which is critical for generating accurate and detailed captions.
Towards Real-Time Panoptic Narrative Grounding by an End-to-End Grounding Network
Extensive experiments on the PNG benchmark dataset demonstrate the effectiveness and efficiency of our method.
RSTNet: Captioning With Adaptive Attention on Visual and Non-Visual Words
Then, we build a BERTbased language model to extract language context and propose Adaptive-Attention (AA) module on top of a transformer decoder to adaptively measure the contribution of visual and language cues before making decisions for word prediction.
Variable selection with missing data in both covariates and outcomes: Imputation and machine learning
We investigate a general variable selection approach when both the covariates and outcomes can be missing at random and have general missing data patterns.
Dual-Level Collaborative Transformer for Image Captioning
Descriptive region features extracted by object detection networks have played an important role in the recent advancements of image captioning.
Improving Image Captioning by Leveraging Intra- and Inter-layer Global Representation in Transformer Network
The latter contains a Global Adaptive Controller that can adaptively fuse the global information into the decoder to guide the caption generation.
Estimating heterogeneous survival treatment effect in observational data using machine learning
Methods for estimating heterogeneous treatment effect in observational data have largely focused on continuous or binary outcomes, and have been relatively less vetted with survival outcomes.
Variational Structured Semantic Inference for Diverse Image Captioning
To model these two inherent diversities in image captioning, we propose a Variational Structured Semantic Inferring model (termed VSSI-cap) executed in a novel structured encoder-inferer-decoder schema.
Semantic-aware Image Deblurring
Specially, we propose a novel Structured-Spatial Semantic Embedding model for image deblurring (termed S3E-Deblur), which introduces a novel Structured-Spatial Semantic tree model (S3-tree) to bridge two basic tasks in computer vision: image deblurring (ImD) and image captioning (ImC).
