Search Results for author:
Found 28 papers, 13 papers with code
ProFashion: Prototype-guided Fashion Video Generation with Multiple Reference Images
To address these issues, we propose ProFashion, a fashion video generation framework leveraging multiple reference images to achieve improved view consistency and temporal coherency.
Generative AI for Film Creation: A Survey of Recent Advances
Generative AI (GenAI) is transforming filmmaking, equipping artists with tools like text-to-image and image-to-video diffusion, neural radiance fields, avatar generation, and 3D synthesis.
Simulating the Real World: A Unified Survey of Multimodal Generative Models
Understanding and replicating the real world is a critical challenge in Artificial General Intelligence (AGI) research.
Light-A-Video: Training-free Video Relighting via Progressive Light Fusion
Second, leveraging the physical principle of light transport independence, we apply linear blending between the source video's appearance and the relighted appearance, using a Progressive Light Fusion (PLF) strategy to ensure smooth temporal transitions in illumination.
Keyframe-Guided Creative Video Inpainting
Video inpainting, which aims to fill missing regions with visually coherent content, has emerged as a crucial technique for creative applications such as editing.
ScriptViz: A Visualization Tool to Aid Scriptwriting based on a Large Movie Database
Scriptwriters usually rely on their mental visualization to create a vivid story by using their imagination to see, feel, and experience the scenes they are writing.
CinePreGen: Camera Controllable Video Previsualization via Engine-powered Diffusion
With advancements in video generative AI models (e. g., SORA), creators are increasingly using these techniques to enhance video previsualization.
Cinematic Behavior Transfer via NeRF-based Differentiable Filming
In the evolving landscape of digital media and video production, the precise manipulation and reproduction of visual elements like camera movements and character actions are highly desired.
SparseCtrl: Adding Sparse Controls to Text-to-Video Diffusion Models
The development of text-to-video (T2V), i. e., generating videos with a given text prompt, has been significantly advanced in recent years.
Automated Conversion of Music Videos into Lyric Videos
Musicians and fans often produce lyric videos, a form of music videos that showcase the song's lyrics, for their favorite songs.
Zero-shot Skeleton-based Action Recognition via Mutual Information Estimation and Maximization
Specifically, 1) we maximize the MI between visual and semantic space for distribution alignment; 2) we leverage the temporal information for estimating the MI by encouraging MI to increase as more frames are observed.
AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning
Once trained, the motion module can be inserted into a personalized T2I model to form a personalized animation generator.
HireVAE: An Online and Adaptive Factor Model Based on Hierarchical and Regime-Switch VAE
Factor model is a fundamental investment tool in quantitative investment, which can be empowered by deep learning to become more flexible and efficient in practical complicated investing situations.
CrossGET: Cross-Guided Ensemble of Tokens for Accelerating Vision-Language Transformers
To pursue more efficient vision-language Transformers, this paper introduces Cross-Guided Ensemble of Tokens (CrossGET), a general acceleration framework for vision-language Transformers.
Self-supervised Action Representation Learning from Partial Spatio-Temporal Skeleton Sequences
Specifically, we construct a negative-sample-free triplet steam structure that is composed of an anchor stream without any masking, a spatial masking stream with Central Spatial Masking (CSM), and a temporal masking stream with Motion Attention Temporal Masking (MATM).
Adding Conditional Control to Text-to-Image Diffusion Models
ControlNet locks the production-ready large diffusion models, and reuses their deep and robust encoding layers pretrained with billions of images as a strong backbone to learn a diverse set of conditional controls.
Dynamic Storyboard Generation in an Engine-based Virtual Environment for Video Production
Amateurs working on mini-films and short-form videos usually spend lots of time and effort on the multi-round complicated process of setting and adjusting scenes, plots, and cameras to deliver satisfying video shots.
Temporal and Contextual Transformer for Multi-Camera Editing of TV Shows
The ability to choose an appropriate camera view among multiple cameras plays a vital role in TV shows delivery.
A Molecular Multimodal Foundation Model Associating Molecule Graphs with Natural Language
Although artificial intelligence (AI) has made significant progress in understanding molecules in a wide range of fields, existing models generally acquire the single cognitive ability from the single molecular modality.
Ranked #13 on
Molecule Captioning
on ChEBI-20
AutoGPart: Intermediate Supervision Search for Generalizable 3D Part Segmentation
To solve the above issues, we propose AutoGPart, a generic method enabling training generalizable 3D part segmentation networks with the task prior considered.
BungeeNeRF: Progressive Neural Radiance Field for Extreme Multi-scale Scene Rendering
The wide span of viewing positions within these scenes yields multi-scale renderings with very different levels of detail, which poses great challenges to neural radiance field and biases it towards compromised results.
BlockPlanner: City Block Generation With Vectorized Graph Representation
City modeling is the foundation for computational urban planning, navigation, and entertainment.
A Unified Framework for Shot Type Classification Based on Subject Centric Lens
Shots are key narrative elements of various videos, e. g. movies, TV series, and user-generated videos that are thriving over the Internet.
Online Multi-modal Person Search in Videos
The task of searching certain people in videos has seen increasing potential in real-world applications, such as video organization and editing.
MovieNet: A Holistic Dataset for Movie Understanding
We believe that such a holistic dataset would promote the researches on story-based long video understanding and beyond.
A Local-to-Global Approach to Multi-modal Movie Scene Segmentation
Scene, as the crucial unit of storytelling in movies, contains complex activities of actors and their interactions in a physical environment.
Automatic Music Accompanist
The computer musician is able to produce musical accompaniment that relates musically to the human performance.
Sound Multimedia Audio and Speech Processing
HotFlip: White-Box Adversarial Examples for Text Classification
We propose an efficient method to generate white-box adversarial examples to trick a character-level neural classifier.
