Search Results for author:
Found 22 papers, 17 papers with code
Identity-Preserving Text-to-Video Generation by Frequency Decomposition
We propose a hierarchical training strategy to leverage frequency information for identity preservation, transforming a vanilla pre-trained video generation model into an IPT2V model.
Video-RAG: Visually-aligned Retrieval-Augmented Long Video Comprehension
Existing large video-language models (LVLMs) struggle to comprehend long videos correctly due to limited context.
Autoregressive Models in Vision: A Survey
Autoregressive modeling has been a huge success in the field of natural language processing (NLP).
A Survey of Camouflaged Object Detection and Beyond
Camouflaged Object Detection (COD) refers to the task of identifying and segmenting objects that blend seamlessly into their surroundings, posing a significant challenge for computer vision systems.
MUSE: Mamba is Efficient Multi-scale Learner for Text-video Retrieval
Text-Video Retrieval (TVR) aims to align and associate relevant video content with corresponding natural language queries.
Evolver: Chain-of-Evolution Prompting to Boost Large Multimodal Models for Hateful Meme Detection
To this end, we propose Evolver, which incorporates LMMs via Chain-of-Evolution (CoE) Prompting, by integrating the evolution attribute and in-context information of memes.
ChronoMagic-Bench: A Benchmark for Metamorphic Evaluation of Text-to-Time-lapse Video Generation
We propose a novel text-to-video (T2V) generation benchmark, ChronoMagic-Bench, to evaluate the temporal and metamorphic capabilities of the T2V models (e. g. Sora and Lumiere) in time-lapse video generation.
LOOK-M: Look-Once Optimization in KV Cache for Efficient Multimodal Long-Context Inference
Long-context Multimodal Large Language Models (MLLMs) demand substantial computational resources for inference as the growth of their multimodal Key-Value (KV) cache, in response to increasing input lengths, challenges memory and time efficiency.
RAP: Efficient Text-Video Retrieval with Sparse-and-Correlated Adapter
Text-Video Retrieval (TVR) aims to align relevant video content with natural language queries.
MagicTime: Time-lapse Video Generation Models as Metamorphic Simulators
Recent advances in Text-to-Video generation (T2V) have achieved remarkable success in synthesizing high-quality general videos from textual descriptions.
LLMBind: A Unified Modality-Task Integration Framework
In the multi-modal domain, the dependence of various models on specific input formats leads to user confusion and hinders progress.
MoE-LLaVA: Mixture of Experts for Large Vision-Language Models
In this work, we propose a simple yet effective training strategy MoE-Tuning for LVLMs.
Ranked #117 on
Visual Question Answering
on MM-Vet
Continuous-Multiple Image Outpainting in One-Step via Positional Query and A Diffusion-based Approach
At inference, we generate images with arbitrary expansion multiples by inputting an anchor image and its corresponding positional embeddings.
GPT-4V(ision) as A Social Media Analysis Engine
Our investigation begins with a preliminary quantitative analysis for each task using existing benchmark datasets, followed by a careful review of the results and a selection of qualitative samples that illustrate GPT-4V's potential in understanding multimodal social media content.
A Survey of Large Language Models in Medicine: Progress, Application, and Challenge
Therefore, this review aims to provide a detailed overview of the development and deployment of LLMs in medicine, including the challenges and opportunities they face.
Improving Scene Graph Generation with Superpixel-Based Interaction Learning
(ii) We explore intra-entity and cross-entity interactions among the superpixels to enrich fine-grained interactions between entities at an earlier stage.
Text-Video Retrieval with Disentangled Conceptualization and Set-to-Set Alignment
In this paper, we propose the Disentangled Conceptualization and Set-to-set Alignment (DiCoSA) to simulate the conceptualizing and reasoning process of human beings.
Cross-Modality Time-Variant Relation Learning for Generating Dynamic Scene Graphs
In this paper, we propose a Time-variant Relation-aware TRansformer (TR$^2$), which aims to model the temporal change of relations in dynamic scene graphs.
Video-Text as Game Players: Hierarchical Banzhaf Interaction for Cross-Modal Representation Learning
Contrastive learning-based video-language representation learning approaches, e. g., CLIP, have achieved outstanding performance, which pursue semantic interaction upon pre-defined video-text pairs.
Ranked #8 on
Video Question Answering
on MSRVTT-QA
Expectation-Maximization Contrastive Learning for Compact Video-and-Language Representations
Most video-and-language representation learning approaches employ contrastive learning, e. g., CLIP, to project the video and text features into a common latent space according to the semantic similarities of text-video pairs.
Ranked #2 on
Video Retrieval
on LSMDC
(text-to-video Mean Rank metric)
Toward 3D Spatial Reasoning for Human-like Text-based Visual Question Answering
Under this setting, these 2D spatial reasoning approaches cannot distinguish the fine-grain spatial relations between visual objects and scene texts on the same image plane, thereby impairing the interpretability and performance of TextVQA models.
Guoym at SemEval-2020 Task 8: Ensemble-based Classification of Visuo-Lingual Metaphor in Memes
In our system, we utilize five types of representation of data as input of base classifiers to extract information from different aspects.
