Search Results for author:
Found 20 papers, 6 papers with code
VD3D: Taming Large Video Diffusion Transformers for 3D Camera Control
Recently, new methods demonstrate the ability to generate videos with controllable camera poses these techniques leverage pre-trained U-Net-based diffusion models that explicitly disentangle spatial and temporal generation.
VIMI: Grounding Video Generation through Multi-modal Instruction
In the first stage, we propose a multimodal conditional video generation framework for pretraining on these augmented datasets, establishing a foundational model for grounded video generation.
Taming Data and Transformers for Audio Generation
Generating ambient sounds and effects is a challenging problem due to data scarcity and often insufficient caption quality, making it difficult to employ large-scale generative models for the task.
Ranked #2 on
Audio captioning
on AudioCaps
(using extra training data)
Hierarchical Patch Diffusion Models for High-Resolution Video Generation
Diffusion models have demonstrated remarkable performance in image and video synthesis.
Ranked #3 on
Video Generation
on UCF-101
4Real: Towards Photorealistic 4D Scene Generation via Video Diffusion Models
We then learn the canonical 3D representation of the video using a freeze-time video, delicately generated from the reference video.
SF-V: Single Forward Video Generation Model
Diffusion-based video generation models have demonstrated remarkable success in obtaining high-fidelity videos through the iterative denoising process.
Harnessing Large Language Models for Training-free Video Anomaly Detection
Video anomaly detection (VAD) aims to temporally locate abnormal events in a video.
Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teachers
Next, we finetune a retrieval model on a small subset where the best caption of each video is manually selected and then employ the model in the whole dataset to select the best caption as the annotation.
Snap Video: Scaled Spatiotemporal Transformers for Text-to-Video Synthesis
Since video content is highly redundant, we argue that naively bringing advances of image models to the video generation domain reduces motion fidelity, visual quality and impairs scalability.
Ranked #1 on
Text-to-Video Generation
on MSR-VTT
Delving into CLIP latent space for Video Anomaly Recognition
We tackle the complex problem of detecting and recognising anomalies in surveillance videos at the frame level, utilising only video-level supervision.
Interactive Neural Painting
This paper advances the state of the art in this emerging research domain by proposing the first approach for Interactive NP.
Quantum Multi-Model Fitting
Geometric model fitting is a challenging but fundamental computer vision problem.
Promptable Game Models: Text-Guided Game Simulation via Masked Diffusion Models
Most captivatingly, our PGM unlocks the director's mode, where the game is played by specifying goals for the agents in the form of a prompt.
Unsupervised Volumetric Animation
We propose a novel approach for unsupervised 3D animation of non-rigid deformable objects.
InfiniCity: Infinite-Scale City Synthesis
Toward infinite-scale 3D city synthesis, we propose a novel framework, InfiniCity, which constructs and renders an unconstrainedly large and 3D-grounded environment from random noises.
Quantum Motion Segmentation
Motion segmentation is a challenging problem that seeks to identify independent motions in two or several input images.
Playable Environments: Video Manipulation in Space and Time
We present Playable Environments - a new representation for interactive video generation and manipulation in space and time.
Playable Video Generation
This paper introduces the unsupervised learning problem of playable video generation (PVG).
Learning to Cluster under Domain Shift
While unsupervised domain adaptation methods based on deep architectures have achieved remarkable success in many computer vision tasks, they rely on a strong assumption, i. e. labeled source data must be available.
