Search Results for author:
Found 45 papers, 19 papers with code
CameraCtrl: Enabling Camera Control for Text-to-Video Generation
Controllability plays a crucial role in video generation since it allows users to create desired content.
GRM: Large Gaussian Reconstruction Model for Efficient 3D Reconstruction and Generation
We introduce GRM, a large-scale reconstructor capable of recovering a 3D asset from sparse-view images in around 0. 1s.
Real-time 3D-aware Portrait Editing from a Single Image
This work presents 3DPE, a practical method that can efficiently edit a face image following given prompts, like reference images or text descriptions, in a 3D-aware manner.
SceneWiz3D: Towards Text-guided 3D Scene Composition
We are witnessing significant breakthroughs in the technology for generating 3D objects from text.
Learning Naturally Aggregated Appearance for Efficient 3D Editing
Neural radiance fields, which represent a 3D scene as a color field and a density field, have demonstrated great progress in novel view synthesis yet are unfavorable for editing due to the implicitness.
BerfScene: Bev-conditioned Equivariant Radiance Fields for Infinite 3D Scene Generation
Generating large-scale 3D scenes cannot simply apply existing 3D object synthesis technique since 3D scenes usually hold complex spatial configurations and consist of a number of objects at varying scales.
Gaussian Shell Maps for Efficient 3D Human Generation
Instead of rasterizing the shells directly, we sample 3D Gaussians on the shells whose attributes are encoded in the texture features.
PF-LRM: Pose-Free Large Reconstruction Model for Joint Pose and Shape Prediction
We propose a Pose-Free Large Reconstruction Model (PF-LRM) for reconstructing a 3D object from a few unposed images even with little visual overlap, while simultaneously estimating the relative camera poses in ~1. 3 seconds on a single A100 GPU.
DMV3D: Denoising Multi-View Diffusion using 3D Large Reconstruction Model
We propose \textbf{DMV3D}, a novel 3D generation approach that uses a transformer-based 3D large reconstruction model to denoise multi-view diffusion.
Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Model
Text-to-3D with diffusion models has achieved remarkable progress in recent years.
In-Domain GAN Inversion for Faithful Reconstruction and Editability
This work fills in this gap by proposing in-domain GAN inversion, which consists of a domain-guided encoder and a domain-regularized optimizer, to regularize the inverted code in the native latent space of the pre-trained GAN model.
Exploring Sparse MoE in GANs for Text-conditioned Image Synthesis
Due to the difficulty in scaling up, generative adversarial networks (GANs) seem to be falling from grace on the task of text-conditioned image synthesis.
Efficient 3D Articulated Human Generation with Layered Surface Volumes
These layers are rendered using alpha compositing with fast differentiable rasterization, and they can be interpreted as a volumetric representation that allocates its capacity to a manifold of finite thickness around the template.
3D generation on ImageNet
Existing 3D-from-2D generators are typically designed for well-curated single-category datasets, where all the objects have (approximately) the same scale, 3D location, and orientation, and the camera always points to the center of the scene.
Spatial Steerability of GANs via Self-Supervision from Discriminator
In this work, we propose a self-supervised approach to improve the spatial steerability of GANs without searching for steerable directions in the latent space or requiring extra annotations.
Learning 3D-aware Image Synthesis with Unknown Pose Distribution
Existing methods for 3D-aware image synthesis largely depend on the 3D pose distribution pre-estimated on the training set.
GH-Feat: Learning Versatile Generative Hierarchical Features from GANs
In this work we investigate that such a generative feature learned from image synthesis exhibits great potentials in solving a wide range of computer vision tasks, including both generative ones and more importantly discriminative ones.
DisCoScene: Spatially Disentangled Generative Radiance Fields for Controllable 3D-aware Scene Synthesis
Existing 3D-aware image synthesis approaches mainly focus on generating a single canonical object and show limited capacity in composing a complex scene containing a variety of objects.
Towards Smooth Video Composition
Video generation requires synthesizing consistent and persistent frames with dynamic content over time.
Ranked #1 on
Video Generation
on YouTube Driving
GLeaD: Improving GANs with A Generator-Leading Task
Generative adversarial network (GAN) is formulated as a two-player game between a generator (G) and a discriminator (D), where D is asked to differentiate whether an image comes from real data or is produced by G. Under such a formulation, D plays as the rule maker and hence tends to dominate the competition.
Deep Generative Models on 3D Representations: A Survey
In this survey, we thoroughly review the ongoing developments of 3D generative models, including methods that employ 2D and 3D supervision.
Improving 3D-aware Image Synthesis with A Geometry-aware Discriminator
We argue that, considering the two-player game in the formulation of GANs, only making the generator 3D-aware is not enough.
Improving GANs with A Dynamic Discriminator
Two capacity adjusting schemes are developed for training GANs under different data regimes: i) given a sufficient amount of training data, the discriminator benefits from a progressively increased learning capacity, and ii) when the training data is limited, gradually decreasing the layer width mitigates the over-fitting issue of the discriminator.
Learning Hierarchical Cross-Modal Association for Co-Speech Gesture Generation
To enhance the quality of synthesized gestures, we develop a contrastive learning strategy based on audio-text alignment for better audio representations.
Ranked #3 on
Gesture Generation
on TED Gesture Dataset
High-fidelity GAN Inversion with Padding Space
In this work, we propose to involve the padding space of the generator to complement the latent space with spatial information.
Region-Based Semantic Factorization in GANs
Despite the rapid advancement of semantic discovery in the latent space of Generative Adversarial Networks (GANs), existing approaches either are limited to finding global attributes or rely on a number of segmentation masks to identify local attributes.
Semantic-Aware Implicit Neural Audio-Driven Video Portrait Generation
Moreover, to enable portrait rendering in one unified neural radiance field, a Torso Deformation module is designed to stabilize the large-scale non-rigid torso motions.
3D-aware Image Synthesis via Learning Structural and Textural Representations
The feature field is further accumulated into a 2D feature map as the textural representation, followed by a neural renderer for appearance synthesis.
Cross-Model Pseudo-Labeling for Semi-Supervised Action Recognition
Typically in recent work, the pseudo-labels are obtained by training a model on the labeled data, and then using confident predictions from the model to teach itself.
Improving GAN Equilibrium by Raising Spatial Awareness
We further propose to align the spatial awareness of G with the attention map induced from D. Through this way we effectively lessen the information gap between D and G. Extensive results show that our method pushes the two-player game in GANs closer to the equilibrium, leading to a better synthesis performance.
One-Shot Generative Domain Adaptation
We then equip the well-learned discriminator backbone with an attribute classifier to ensure that the generator captures the appropriate characters from the reference.
Improving Out-of-Distribution Robustness of Classifiers Through Interpolated Generative Models
In this paper, we employ interpolated generative models to generate OoD samples at training time via data augmentation.
Learning Object-Compositional Neural Radiance Field for Editable Scene Rendering
In this paper, we present a novel neural scene rendering system, which learns an object-compositional neural radiance field and produces realistic rendering with editing capability for a clustered and real-world scene.
CompConv: A Compact Convolution Module for Efficient Feature Learning
Convolutional Neural Networks (CNNs) have achieved remarkable success in various computer vision tasks but rely on tremendous computational cost.
Data-Efficient Instance Generation from Instance Discrimination
Meanwhile, the learned instance discrimination capability from the discriminator is in turn exploited to encourage the generator for diverse generation.
Ranked #6 on
Image Generation
on FFHQ 256 x 256
(FD metric)
Decorating Your Own Bedroom: Locally Controlling Image Generation with Generative Adversarial Networks
Generative Adversarial Networks (GANs) have made great success in synthesizing high-quality images.
Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humans
To this end, we propose Neural Body, a new human body representation which assumes that the learned neural representations at different frames share the same set of latent codes anchored to a deformable mesh, so that the observations across frames can be naturally integrated.
Generative Hierarchical Features from Synthesizing Images
Generative Adversarial Networks (GANs) have recently advanced image synthesis by learning the underlying distribution of the observed data.
Unsupervised Landmark Learning from Unpaired Data
Recent attempts for unsupervised landmark learning leverage synthesized image pairs that are similar in appearance but different in poses.
Video Representation Learning with Visual Tempo Consistency
Visual tempo, which describes how fast an action goes, has shown its potential in supervised action recognition.
Temporal Pyramid Network for Action Recognition
Previous works often capture the visual tempo through sampling raw videos at multiple rates and constructing an input-level frame pyramid, which usually requires a costly multi-branch network to handle.
Ranked #105 on
Action Recognition
on Something-Something V2
Dense RepPoints: Representing Visual Objects with Dense Point Sets
We present a new object representation, called Dense RepPoints, that utilizes a large set of points to describe an object at multiple levels, including both box level and pixel level.
A Main/Subsidiary Network Framework for Simplifying Binary Neural Networks
To reduce memory footprint and run-time latency, techniques such as neural net-work pruning and binarization have been explored separately.
A Main/Subsidiary Network Framework for Simplifying Binary Neural Network
To reduce memory footprint and run-time latency, techniques such as neural network pruning and binarization have been explored separately.
