Search Results for author:
Found 46 papers, 23 papers with code
CameraCtrl II: Dynamic Scene Exploration via Camera-controlled Video Diffusion Models
This paper introduces CameraCtrl II, a framework that enables large-scale dynamic scene exploration through a camera-controlled video diffusion model.
Long Context Tuning for Video Generation
Recent advances in video generation can produce realistic, minute-long single-shot videos with scalable diffusion transformers.
Diffusion Adversarial Post-Training for One-Step Video Generation
The diffusion models are widely used for image and video generation, but their iterative generation process is slow and expansive.
SeedVR: Seeding Infinity in Diffusion Transformer Towards Generic Video Restoration
Video restoration poses non-trivial challenges in maintaining fidelity while recovering temporally consistent details from unknown degradations in the wild.
Edicho: Consistent Image Editing in the Wild
As a verified need, consistent editing across in-the-wild images remains a technical challenge arising from various unmanageable factors, like object poses, lighting conditions, and photography environments.
Scaling Laws For Diffusion Transformers
Diffusion transformers (DiT) have already achieved appealing synthesis and scaling properties in content recreation, e. g., image and video generation.
3DitScene: Editing Any Scene via Language-guided Disentangled Gaussian Splatting
This results in a lack of a unified approach to effectively control and manipulate scenes at the 3D level with different levels of granularity.
CameraCtrl: Enabling Camera Control for Text-to-Video Generation
Controllability plays a crucial role in video generation since it allows users to create desired content.
GRM: Large Gaussian Reconstruction Model for Efficient 3D Reconstruction and Generation
We introduce GRM, a large-scale reconstructor capable of recovering a 3D asset from sparse-view images in around 0. 1s.
Real-time 3D-aware Portrait Editing from a Single Image
Second, thanks to the powerful priors, our module could focus on the learning of editing-related variations, such that it manages to handle various types of editing simultaneously in the training phase and further supports fast adaptation to user-specified customized types of editing during inference (e. g., with ~5min fine-tuning per style).
Towards Text-guided 3D Scene Composition
We marry the locality of objects with globality of scenes by introducing a hybrid 3D representation - explicit for objects and implicit for scenes.
SceneWiz3D: Towards Text-guided 3D Scene Composition
We are witnessing significant breakthroughs in the technology for generating 3D objects from text.
GenDeF: Learning Generative Deformation Field for Video Generation
We offer a new perspective on approaching the task of video generation.
BerfScene: Bev-conditioned Equivariant Radiance Fields for Infinite 3D Scene Generation
Generating large-scale 3D scenes cannot simply apply existing 3D object synthesis technique since 3D scenes usually hold complex spatial configurations and consist of a number of objects at varying scales.
SMaRt: Improving GANs with Score Matching Regularity
In this work, we revisit the mathematical foundations of GANs, and theoretically reveal that the native adversarial loss for GAN training is insufficient to fix the problem of subsets with positive Lebesgue measure of the generated data manifold lying out of the real data manifold.
SparseCtrl: Adding Sparse Controls to Text-to-Video Diffusion Models
The development of text-to-video (T2V), i. e., generating videos with a given text prompt, has been significantly advanced in recent years.
LAVIE: High-Quality Video Generation with Cascaded Latent Diffusion Models
To this end, we propose LaVie, an integrated video generation framework that operates on cascaded video latent diffusion models, comprising a base T2V model, a temporal interpolation model, and a video super-resolution model.
Ranked #4 on
Text-to-Video Generation
on EvalCrafter Text-to-Video (ECTV) Dataset
(using extra training data)
Exploring Sparse MoE in GANs for Text-conditioned Image Synthesis
Due to the difficulty in scaling up, generative adversarial networks (GANs) seem to be falling from grace on the task of text-conditioned image synthesis.
AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning
Once trained, the motion module can be inserted into a personalized T2I model to form a personalized animation generator.
Spatial Steerability of GANs via Self-Supervision from Discriminator
In this work, we propose a self-supervised approach to improve the spatial steerability of GANs without searching for steerable directions in the latent space or requiring extra annotations.
GH-Feat: Learning Versatile Generative Hierarchical Features from GANs
In this work we investigate that such a generative feature learned from image synthesis exhibits great potentials in solving a wide range of computer vision tasks, including both generative ones and more importantly discriminative ones.
LinkGAN: Linking GAN Latents to Pixels for Controllable Image Synthesis
This work presents an easy-to-use regularizer for GAN training, which helps explicitly link some axes of the latent space to a set of pixels in the synthesized image.
DisCoScene: Spatially Disentangled Generative Radiance Fields for Controllable 3D-aware Scene Synthesis
Existing 3D-aware image synthesis approaches mainly focus on generating a single canonical object and show limited capacity in composing a complex scene containing a variety of objects.
Towards Smooth Video Composition
Video generation requires synthesizing consistent and persistent frames with dynamic content over time.
Ranked #1 on
Video Generation
on YouTube Driving
GLeaD: Improving GANs with A Generator-Leading Task
Generative adversarial network (GAN) is formulated as a two-player game between a generator (G) and a discriminator (D), where D is asked to differentiate whether an image comes from real data or is produced by G. Under such a formulation, D plays as the rule maker and hence tends to dominate the competition.
Improving GANs with A Dynamic Discriminator
Two capacity adjusting schemes are developed for training GANs under different data regimes: i) given a sufficient amount of training data, the discriminator benefits from a progressively increased learning capacity, and ii) when the training data is limited, gradually decreasing the layer width mitigates the over-fitting issue of the discriminator.
Prototypical Contrast Adaptation for Domain Adaptive Semantic Segmentation
Unsupervised Domain Adaptation (UDA) aims to adapt the model trained on the labeled source domain to an unlabeled target domain.
Ranked #15 on
Unsupervised Domain Adaptation
on SYNTHIA-to-Cityscapes
Accelerating Diffusion Models via Early Stop of the Diffusion Process
By modeling the reverse process of gradually diffusing the data distribution into a Gaussian distribution, generating a sample in DDPMs can be regarded as iteratively denoising a randomly sampled Gaussian noise.
3D-aware Image Synthesis via Learning Structural and Textural Representations
The feature field is further accumulated into a 2D feature map as the textural representation, followed by a neural renderer for appearance synthesis.
Cross-Model Pseudo-Labeling for Semi-Supervised Action Recognition
Typically in recent work, the pseudo-labels are obtained by training a model on the labeled data, and then using confident predictions from the model to teach itself.
Improving GAN Equilibrium by Raising Spatial Awareness
We further propose to align the spatial awareness of G with the attention map induced from D. Through this way we effectively lessen the information gap between D and G. Extensive results show that our method pushes the two-player game in GANs closer to the equilibrium, leading to a better synthesis performance.
Improving Out-of-Distribution Robustness of Classifiers Through Interpolated Generative Models
In this paper, we employ interpolated generative models to generate OoD samples at training time via data augmentation.
Data-Efficient Instance Generation from Instance Discrimination
Meanwhile, the learned instance discrimination capability from the discriminator is in turn exploited to encourage the generator for diverse generation.
Ranked #11 on
Image Generation
on FFHQ 256 x 256
(FD metric)
Instance Localization for Self-supervised Detection Pretraining
The pretext task is to predict the instance category given the composited images as well as the foreground bounding boxes.
Generative Hierarchical Features from Synthesizing Images
Generative Adversarial Networks (GANs) have recently advanced image synthesis by learning the underlying distribution of the observed data.
Unsupervised Landmark Learning from Unpaired Data
Recent attempts for unsupervised landmark learning leverage synthesized image pairs that are similar in appearance but different in poses.
Video Representation Learning with Visual Tempo Consistency
Visual tempo, which describes how fast an action goes, has shown its potential in supervised action recognition.
InterFaceGAN: Interpreting the Disentangled Face Representation Learned by GANs
In this work, we propose a framework called InterFaceGAN to interpret the disentangled face representation learned by the state-of-the-art GAN models and study the properties of the facial semantics encoded in the latent space.
Temporal Pyramid Network for Action Recognition
Previous works often capture the visual tempo through sampling raw videos at multiple rates and constructing an input-level frame pyramid, which usually requires a costly multi-branch network to handle.
Ranked #105 on
Action Recognition
on Something-Something V2
Semantic Hierarchy Emerges in Deep Generative Representations for Scene Synthesis
Despite the success of Generative Adversarial Networks (GANs) in image synthesis, there lacks enough understanding on what generative models have learned inside the deep generative representations and how photo-realistic images are able to be composed of the layer-wise stochasticity introduced in recent GANs.
Learning Where to Focus for Efficient Video Object Detection
Transferring existing image-based detectors to the video is non-trivial since the quality of frames is always deteriorated by part occlusion, rare pose, and motion blur.
Ranked #24 on
Video Object Detection
on ImageNet VID
Semantic Hierarchy Emerges in the Deep Generative Representations for Scene Synthesis
Despite the success of Generative Adversarial Networks (GANs) in image synthesis, there lacks enough understanding on what networks have learned inside the deep generative representations and how photo-realistic images are able to be composed from random noises.
Penalizing Top Performers: Conservative Loss for Semantic Segmentation Adaptation
Due to the expensive and time-consuming annotations (e. g., segmentation) for real-world images, recent works in computer vision resort to synthetic data.
Pose Guided Human Video Generation
Human pose, on the other hand, can represent motion patterns intrinsically and interpretably, and impose the geometric constraints regardless of appearance.
