Search Results for author:
Found 15 papers, 6 papers with code
Evaluating Image Caption via Cycle-consistent Text-to-Image Generation
Recent research has revealed that the modality gap generally exists in the representation of contrastive learning-based multi-modal systems, undermining the reliability of cross-modality metrics like CLIPScore.
MDP3: A Training-free Approach for List-wise Frame Selection in Video-LLMs
Existing methods, such as uniform frame sampling and query-frame matching, do not capture all of these principles.
UNIC-Adapter: Unified Image-instruction Adapter with Multi-modal Transformer for Image Generation
Recently, text-to-image generation models have achieved remarkable advancements, particularly with diffusion models facilitating high-quality image synthesis from textual descriptions.
OmniEvalKit: A Modular, Lightweight Toolbox for Evaluating Large Language Model and its Omni-Extensions
The rapid advancements in Large Language Models (LLMs) have significantly expanded their applications, ranging from multilingual support to domain-specific tasks and multimodal integration.
PEMF-VVTO: Point-Enhanced Video Virtual Try-on via Mask-free Paradigm
Then, based on the pre-acquired sparse frame-cloth and frame-frame point alignments, we design the point-enhanced spatial attention (PSA) and point-enhanced temporal attention (PTA) to further improve the try-on accuracy and video coherence of the mask-free model.
Meissonic: Revitalizing Masked Generative Transformers for Efficient High-Resolution Text-to-Image Synthesis
We present Meissonic, which elevates non-autoregressive masked image modeling (MIM) text-to-image to a level comparable with state-of-the-art diffusion models like SDXL.
Advancing Tool-Augmented Large Language Models: Integrating Insights from Errors in Inference Trees
In this study, we propose an inference trajectory optimization framework based on the preference data extracted from decision trees to address this limitation.
Wings: Learning Multimodal LLMs without Text-only Forgetting
Initially, image and text inputs are aligned with visual learners operating alongside the main attention, balancing focus on visual elements.
Parrot: Multilingual Visual Instruction Tuning
In this paper, we introduce Parrot, a novel method that utilizes textual guidance to drive visual token alignment at the language level.
Ovis: Structural Embedding Alignment for Multimodal Large Language Model
However, the misalignment between two embedding strategies in MLLMs -- the structural textual embeddings based on an embedding look-up table and the continuous embeddings generated directly by the vision encoder -- makes challenges for a more seamless fusion of visual and textual information.
TAI++: Text as Image for Multi-Label Image Classification by Co-Learning Transferable Prompt
Then, a co-learning strategy with a dual-adapter module is designed to transfer visual knowledge from pseudo-visual prompt to text prompt, enhancing their visual representation abilities.
Sparse Attentive Memory Network for Click-through Rate Prediction with Long Sequences
SAM supports efficient training and real-time inference for user behavior sequences with lengths on the scale of thousands.
Iterative Memory Network for Long Sequential User Behavior Modeling in Recommender Systems
Extensive empirical studies show that our method outperforms various state-of-the-art sequential modeling methods on both public and industrial datasets for long sequential user behavior modeling.
Multi-label Zero-shot Classification by Learning to Transfer from External Knowledge
On the other hand, there is a large semantic gap between seen and unseen classes in the existing multi-label classification datasets.
Deep Time-Stream Framework for Click-Through Rate Prediction by Tracking Interest Evolution
Click-through rate (CTR) prediction is an essential task in industrial applications such as video recommendation.
