Search Results for author:
Found 42 papers, 22 papers with code
MagicEraser: Erasing Any Objects via Semantics-Aware Control
However, the object erasure task, which is in increasing demand, aims to erase objects and generate harmonious background.
TexGen: Text-Guided 3D Texture Generation with Multi-view Sampling and Resampling
For view consistent sampling, first of all we maintain a texture map in RGB space that is parameterized by the denoising step and updated after each sampling step of the diffusion model to progressively reduce the view discrepancy.
EmoTalk3D: High-Fidelity Free-View Synthesis of Emotional 3D Talking Head
We present a novel approach for synthesizing 3D talking heads with controllable emotion, featuring enhanced lip synchronization and rendering quality.
GSD: View-Guided Gaussian Splatting Diffusion for 3D Reconstruction
We present GSD, a diffusion model approach based on Gaussian Splatting (GS) representation for 3D object reconstruction from a single view.
AutoTVG: A New Vision-language Pre-training Paradigm for Temporal Video Grounding
Temporal Video Grounding (TVG) aims to localize a moment from an untrimmed video given the language description.
MirrorGaussian: Reflecting 3D Gaussians for Reconstructing Mirror Reflections
To tackle this problem, we present MirrorGaussian, the first method for mirror scene reconstruction with real-time rendering based on 3D Gaussian Splatting.
The Ninth NTIRE 2024 Efficient Super-Resolution Challenge Report
In sub-track 1, the practical runtime performance of the submissions was evaluated, and the corresponding score was used to determine the ranking.
Co-Speech Gesture Video Generation via Motion-Decoupled Diffusion Model
While previous works mostly generate structural human skeletons, resulting in the omission of appearance information, we focus on the direct generation of audio-driven co-speech gesture videos in this work.
Self-Adaptive Reality-Guided Diffusion for Artifact-Free Super-Resolution
Following this, the Reality Guidance Refinement (RGR) process refines artifacts by integrating this mask with realistic latent representations, improving alignment with the original image.
LayerDiff: Exploring Text-guided Multi-layered Composable Image Synthesis via Layer-Collaborative Diffusion Model
Specifically, an inter-layer attention module is designed to encourage information exchange and learning between layers, while a text-guided intra-layer attention module incorporates layer-specific prompts to direct the specific-content generation for each layer.
VastGaussian: Vast 3D Gaussians for Large Scene Reconstruction
Existing NeRF-based methods for large scene reconstruction often have limitations in visual quality and rendering speed.
PanGu-Draw: Advancing Resource-Efficient Text-to-Image Synthesis with Time-Decoupled Training and Reusable Coop-Diffusion
Current large-scale diffusion models represent a giant leap forward in conditional image synthesis, capable of interpreting diverse cues like text, human poses, and edges.
DreamControl: Control-Based Text-to-3D Generation with 3D Self-Prior
The priors are then regarded as input conditions to maintain reasonable geometries, in which conditional LoRA and weighted score are further proposed to optimize detailed textures.
DreamVideo: High-Fidelity Image-to-Video Generation with Image Retention and Text Guidance
Especially for fidelity, our model has a powerful image retention ability and delivers the best results in UCF101 compared to other image-to-video models to our best knowledge.
BIVDiff: A Training-Free Framework for General-Purpose Video Synthesis via Bridging Image and Video Diffusion Models
Now text-to-image foundation models are widely applied to various downstream image synthesis tasks, such as controllable image generation and image editing, while downstream video synthesis tasks are less explored for several reasons.
Semantics-aware Motion Retargeting with Vision-Language Models
Capturing and preserving motion semantics is essential to motion retargeting between animation characters.
Learning Unorthogonalized Matrices for Rotation Estimation
Estimating 3D rotations is a common procedure for 3D computer vision.
Ranked #16 on
3D Human Pose Estimation
on 3DPW
Fuse Your Latents: Video Editing with Multi-source Latent Diffusion Models
This paper is the first to reveal that T2I and T2V LDMs can complement each other in terms of structure and temporal consistency, ultimately generating high-quality videos.
TextField3D: Towards Enhancing Open-Vocabulary 3D Generation with Noisy Text Fields
To this end, an NTFGen module is proposed to model general text latent code in noisy fields.
UnifiedGesture: A Unified Gesture Synthesis Model for Multiple Skeletons
The automatic co-speech gesture generation draws much attention in computer animation.
Reuse and Diffuse: Iterative Denoising for Text-to-Video Generation
Conditioned on an initial video clip with a small number of frames, additional frames are iteratively generated by reusing the original latent features and following the previous diffusion process.
Any-Size-Diffusion: Toward Efficient Text-Driven Synthesis for Any-Size HD Images
Stable diffusion, a generative model used in text-to-image synthesis, frequently encounters resolution-induced composition problems when generating images of varying sizes.
Towards High-Fidelity Text-Guided 3D Face Generation and Manipulation Using only Images
In this paper, we propose a text-guided 3D faces generation method, refer as TG-3DFace, for generating realistic 3D faces using text guidance.
The DiffuseStyleGesture+ entry to the GENEA Challenge 2023
In this paper, we introduce the DiffuseStyleGesture+, our solution for the Generation and Evaluation of Non-verbal Behavior for Embodied Agents (GENEA) Challenge 2023, which aims to foster the development of realistic, automated systems for generating conversational gestures.
Low-Light Image Enhancement with Illumination-Aware Gamma Correction and Complete Image Modelling Network
We propose to integrate the effectiveness of gamma correction with the strong modelling capacities of deep networks, which enables the correction factor gamma to be learned in a coarse to elaborate manner via adaptively perceiving the deviated illumination.
Few-Shot Learning with Visual Distribution Calibration and Cross-Modal Distribution Alignment
However, with only a few training images, there exist two crucial problems: (1) the visual feature distributions are easily distracted by class-irrelevant information in images, and (2) the alignment between the visual and language feature distributions is difficult.
AttriCLIP: A Non-Incremental Learner for Incremental Knowledge Learning
Text consists of a category name and a fixed number of learnable parameters which are selected from our designed attribute word bank and serve as attributes.
CLIPPING: Distilling CLIP-Based Models With a Student Base for Video-Language Retrieval
Pre-training a vison-language model and then fine-tuning it on downstream tasks have become a popular paradigm.
Co-Speech Gesture Synthesis by Reinforcement Learning With Contrastive Pre-Trained Rewards
There is a growing demand of automatically synthesizing co-speech gestures for virtual characters.
HiVLP: Hierarchical Interactive Video-Language Pre-Training
However, compared to image-language pre-training, VLP has lagged far behind due to the lack of large amounts of video-text pairs.
PIDRo: Parallel Isomeric Attention with Dynamic Routing for Text-Video Retrieval
The parallel isomeric attention module is used as the video encoder, which consists of two parallel branches modeling the spatial-temporal information of videos from both patch and frame levels.
Ranked #3 on
Video Retrieval
on MSR-VTT-1kA
CLIFF: Carrying Location Information in Full Frames into Human Pose and Shape Estimation
Top-down methods dominate the field of 3D human pose and shape estimation, because they are decoupled from human detection and allow researchers to focus on the core problem.
Ranked #1 on
Unsupervised 3D Human Pose Estimation
on Human3.6M
(PA-MPJPE metric)
Instance Segmentation in 3D Scenes using Semantic Superpoint Tree Networks
State-of-the-art methods largely rely on a general pipeline that first learns point-wise features discriminative at semantic and instance levels, followed by a separate step of point grouping for proposing object instances.
Ranked #10 on
3D Instance Segmentation
on S3DIS
Multiple instance active learning for object detection
Despite the substantial progress of active learning for image recognition, there still lacks an instance-level active learning method specified for object detection.
Ranked #1 on
Active Object Detection
on MS COCO
Generating Diverse Structure for Image Inpainting With Hierarchical VQ-VAE
We propose a two-stage model for diverse inpainting, where the first stage generates multiple coarse results each of which has a different structure, and the second stage refines each coarse result separately by augmenting texture.
DualPoseNet: Category-level 6D Object Pose and Size Estimation Using Dual Pose Network with Refined Learning of Pose Consistency
DualPoseNet stacks two parallel pose decoders on top of a shared pose encoder, where the implicit decoder predicts object poses with a working mechanism different from that of the explicit one; they thus impose complementary supervision on the training of pose encoder.
Ranked #4 on
6D Pose Estimation using RGBD
on REAL275
Quality-Aware Network for Human Parsing
How to estimate the quality of the network output is an important issue, and currently there is no effective solution in the field of human parsing.
PcmNet: Position-Sensitive Context Modeling Network for Temporal Action Localization
In this paper, we propose a temporal-position-sensitive context modeling approach to incorporate both positional and semantic information for more precise action localization.
Renovating Parsing R-CNN for Accurate Multiple Human Parsing
Multiple human parsing aims to segment various human parts and associate each part with the corresponding instance simultaneously.
DiverseDepth: Affine-invariant Depth Prediction Using Diverse Data
Compared with previous learning objectives, i. e., learning metric depth or relative depth, we propose to learn the affine-invariant depth using our diverse dataset to ensure both generalization and high-quality geometric shapes of scenes.
Index Network
By viewing the indices as a function of the feature map, we introduce the concept of "learning to index", and present a novel index-guided encoder-decoder framework where indices are self-learned adaptively from data and are used to guide the downsampling and upsampling stages, without extra training supervision.
Ranked #2 on
Grayscale Image Denoising
on Set12 sigma30
Indices Matter: Learning to Index for Deep Image Matting
We show that existing upsampling operators can be unified with the notion of the index function.
