Search Results for author:
Found 16 papers, 8 papers with code
VidComposition: Can MLLMs Analyze Compositions in Compiled Videos?
The advancement of Multimodal Large Language Models (MLLMs) has enabled significant progress in multimodal understanding, expanding their capacity to analyze video content.
Scaling Concept With Text-Guided Diffusion Models
Through an empirical study, we identify a trend where concepts can be decomposed in text-guided diffusion models.
MMCOMPOSITION: Revisiting the Compositionality of Pre-trained Vision-Language Models
With MMCOMPOSITION, we can quantify and explore the compositionality of the mainstream VLMs.
EAGLE: Egocentric AGgregated Language-video Engine
The rapid evolution of egocentric video analysis brings new insights into understanding human activities and intentions from a first-person perspective.
AIM 2024 Challenge on Video Saliency Prediction: Methods and Results
The goal of the participants was to develop a method for predicting accurate saliency maps for the provided set of video sequences.
CaRDiff: Video Salient Object Ranking Chain of Thought Reasoning for Saliency Prediction with Diffusion
In this paper, we propose CaRDiff (Caption, Rank, and generate with Diffusion), a framework that imitates the process by integrating a multimodal large language model (MLLM), a grounding module, and a diffusion model, to enhance video saliency prediction.
Do More Details Always Introduce More Hallucinations in LVLM-based Image Captioning?
This has sparked a debate on the question: Do more details always introduce more hallucinations in LVLM-based image captioning?
V2Xum-LLM: Cross-Modal Video Summarization with Temporal Prompt Instruction Tuning
Recent efforts have been made to expand from unimodal to multimodal video summarization, categorizing the task into three sub-tasks based on the summary's modality: video-to-video (V2V), video-to-text (V2T), and a combination of video and text summarization (V2VT).
DPStyler: Dynamic PromptStyler for Source-Free Domain Generalization
Moreover, since the Style Generation module, responsible for generating style word vectors using random sampling or style mixing, makes the model sensitive to input text prompts, we introduce a model ensemble method to mitigate this sensitivity.
Empowering LLMs with Pseudo-Untrimmed Videos for Audio-Visual Temporal Understanding
This deficiency hinders LLMs from learning the alignment between time, audio-visual events, and text tokens, thus impairing their ability to temporally localize audio-visual events in videos.
GaussianStyle: Gaussian Head Avatar via StyleGAN
Existing methods like Neural Radiation Fields (NeRF) and 3D Gaussian Splatting (3DGS) have made significant strides in facial attribute control such as facial animation and components editing, yet they struggle with fine-grained representation and scalability in dynamic head modeling.
Video Understanding with Large Language Models: A Survey
With the burgeoning growth of online video platforms and the escalating volume of video content, the demand for proficient video understanding tools has intensified markedly.
LaunchpadGPT: Language Model as Music Visualization Designer on Launchpad
To assist and inspire the design of the Launchpad light effect, and provide a more accessible approach for beginners to create music visualization with this instrument, we proposed the LaunchpadGPT model to generate music visualization designs on Launchpad automatically.
LLMVA-GEBC: Large Language Model with Video Adapter for Generic Event Boundary Captioning
This paper proposes an effective model LLMVA-GEBC (Large Language Model with Video Adapter for Generic Event Boundary Captioning): (1) We utilize a pretrained LLM for generating human-like captions with high quality.
Caption Anything: Interactive Image Description with Diverse Multimodal Controls
Controllable image captioning is an emerging multimodal topic that aims to describe the image with natural language following human purpose, $\textit{e. g.}$, looking at the specified regions or telling in a particular text style.
Multi-modal Segment Assemblage Network for Ad Video Editing with Importance-Coherence Reward
The existing method performs well at video segmentation stages but suffers from the problems of dependencies on extra cumbersome models and poor performance at the segment assemblage stage.
