Search Results for author: Xingang Pan

Found 30 papers, 13 papers with code

MVIP-NeRF: Multi-view 3D Inpainting on NeRF Scenes via Diffusion Prior

no code implementations5 May 2024 Honghua Chen, Chen Change Loy, Xingang Pan

Despite the emergence of successful NeRF inpainting methods built upon explicit RGB and depth 2D inpainting supervisions, these methods are inherently constrained by the capabilities of their underlying 2D inpainters.

3D Inpainting

ComboVerse: Compositional 3D Assets Creation Using Spatially-Aware Diffusion Guidance

no code implementations19 Mar 2024 Yongwei Chen, Tengfei Wang, Tong Wu, Xingang Pan, Kui Jia, Ziwei Liu

Though promising results have been achieved in single object generation, these methods often struggle to model complex 3D assets that inherently contain multiple objects.

3D Generation Object

DiffMorpher: Unleashing the Capability of Diffusion Models for Image Morphing

no code implementations12 Dec 2023 Kaiwen Zhang, Yifan Zhou, Xudong Xu, Xingang Pan, Bo Dai

Our key idea is to capture the semantics of the two images by fitting two LoRAs to them respectively, and interpolate between both the LoRA parameters and the latent noises to ensure a smooth semantic transition, where correspondence automatically emerges without the need for annotation.

Image Generation Image Morphing

MATLABER: Material-Aware Text-to-3D via LAtent BRDF auto-EncodeR

no code implementations18 Aug 2023 Xudong Xu, Zhaoyang Lyu, Xingang Pan, Bo Dai

In this work, we propose Material-Aware Text-to-3D via LAtent BRDF auto-EncodeR (\textbf{MATLABER}) that leverages a novel latent BRDF auto-encoder for material generation.

3D Generation Text to 3D

AvatarStudio: Text-driven Editing of 3D Dynamic Human Head Avatars

no code implementations1 Jun 2023 Mohit Mendiratta, Xingang Pan, Mohamed Elgharib, Kartik Teotia, Mallikarjun B R, Ayush Tewari, Vladislav Golyanik, Adam Kortylewski, Christian Theobalt

Our method edits the full head in a canonical space, and then propagates these edits to remaining time steps via a pretrained deformation network.

Drag Your GAN: Interactive Point-based Manipulation on the Generative Image Manifold

5 code implementations18 May 2023 Xingang Pan, Ayush Tewari, Thomas Leimkühler, Lingjie Liu, Abhimitra Meka, Christian Theobalt

Synthesizing visual content that meets users' needs often requires flexible and precise controllability of the pose, shape, expression, and layout of the generated objects.

Image Manipulation Point Tracking +1

GVP: Generative Volumetric Primitives

no code implementations31 Mar 2023 Mallikarjun B R, Xingang Pan, Mohamed Elgharib, Christian Theobalt

Advances in 3D-aware generative models have pushed the boundary of image synthesis with explicit camera control.

Image Generation Knowledge Distillation

AssetField: Assets Mining and Reconfiguration in Ground Feature Plane Representation

no code implementations ICCV 2023 Yuanbo Xiangli, Linning Xu, Xingang Pan, Nanxuan Zhao, Bo Dai, Dahua Lin

Traditional modeling pipelines keep an asset library storing unique object templates, which is both versatile and memory efficient in practice.

Novel View Synthesis Object

Grid-guided Neural Radiance Fields for Large Urban Scenes

no code implementations CVPR 2023 Linning Xu, Yuanbo Xiangli, Sida Peng, Xingang Pan, Nanxuan Zhao, Christian Theobalt, Bo Dai, Dahua Lin

An alternative solution is to use a feature grid representation, which is computationally efficient and can naturally scale to a large scene with increased grid resolutions.

gCoRF: Generative Compositional Radiance Fields

no code implementations31 Oct 2022 Mallikarjun BR, Ayush Tewari, Xingang Pan, Mohamed Elgharib, Christian Theobalt

We start with a global generative model (GAN) and learn to decompose it into different semantic parts using supervision from 2D segmentation masks.

Image Generation

Voxurf: Voxel-based Efficient and Accurate Neural Surface Reconstruction

1 code implementation26 Aug 2022 Tong Wu, Jiaqi Wang, Xingang Pan, Xudong Xu, Christian Theobalt, Ziwei Liu, Dahua Lin

Previous methods based on neural volume rendering mostly train a fully implicit model with MLPs, which typically require hours of training for a single scene.

Surface Reconstruction

GAN2X: Non-Lambertian Inverse Rendering of Image GANs

no code implementations18 Jun 2022 Xingang Pan, Ayush Tewari, Lingjie Liu, Christian Theobalt

2D images are observations of the 3D physical world depicted with the geometry, material, and illumination components.

3D Face Reconstruction Inverse Rendering

Disentangled3D: Learning a 3D Generative Model with Disentangled Geometry and Appearance from Monocular Images

no code implementations CVPR 2022 Ayush Tewari, Mallikarjun B R, Xingang Pan, Ohad Fried, Maneesh Agrawala, Christian Theobalt

Our model can disentangle the geometry and appearance variations in the scene, i. e., we can independently sample from the geometry and appearance spaces of the generative model.


BungeeNeRF: Progressive Neural Radiance Field for Extreme Multi-scale Scene Rendering

no code implementations10 Dec 2021 Yuanbo Xiangli, Linning Xu, Xingang Pan, Nanxuan Zhao, Anyi Rao, Christian Theobalt, Bo Dai, Dahua Lin

The wide span of viewing positions within these scenes yields multi-scale renderings with very different levels of detail, which poses great challenges to neural radiance field and biases it towards compromised results.

Generative Occupancy Fields for 3D Surface-Aware Image Synthesis

1 code implementation NeurIPS 2021 Xudong Xu, Xingang Pan, Dahua Lin, Bo Dai

In this paper, we propose Generative Occupancy Fields (GOF), a novel model based on generative radiance fields that can learn compact object surfaces without impeding its training convergence.

3D-Aware Image Synthesis Object

A Shading-Guided Generative Implicit Model for Shape-Accurate 3D-Aware Image Synthesis

1 code implementation NeurIPS 2021 Xingang Pan, Xudong Xu, Chen Change Loy, Christian Theobalt, Bo Dai

Motivated by the observation that a 3D object should look realistic from multiple viewpoints, these methods introduce a multi-view constraint as regularization to learn valid 3D radiance fields from 2D images.

3D-Aware Image Synthesis 3D Shape Reconstruction +2

Talk-to-Edit: Fine-Grained Facial Editing via Dialog

1 code implementation ICCV 2021 Yuming Jiang, Ziqi Huang, Xingang Pan, Chen Change Loy, Ziwei Liu

In this work, we propose Talk-to-Edit, an interactive facial editing framework that performs fine-grained attribute manipulation through dialog between the user and the system.

Attribute Facial Editing +1

Do 2D GANs Know 3D Shape? Unsupervised 3D shape reconstruction from 2D Image GANs

1 code implementation ICLR 2021 Xingang Pan, Bo Dai, Ziwei Liu, Chen Change Loy, Ping Luo

Through our investigation, we found that such a pre-trained GAN indeed contains rich 3D knowledge and thus can be used to recover 3D shape from a single 2D image in an unsupervised manner.

3D Shape Reconstruction Object

Self-Supervised Scene De-occlusion

2 code implementations CVPR 2020 Xiaohang Zhan, Xingang Pan, Bo Dai, Ziwei Liu, Dahua Lin, Chen Change Loy

This is achieved via Partial Completion Network (PCNet)-mask (M) and -content (C), that learn to recover fractions of object masks and contents, respectively, in a self-supervised manner.

Image Manipulation Scene Understanding

Channel Equilibrium Networks for Learning Deep Representation

1 code implementation ICML 2020 Wenqi Shao, Shitao Tang, Xingang Pan, Ping Tan, Xiaogang Wang, Ping Luo

Unlike prior arts that simply removed the inhibited channels, we propose to "wake them up" during training by designing a novel neural building block, termed Channel Equilibrium (CE) block, which enables channels at the same layer to contribute equally to the learned representation.

Channel Equilibrium Networks

no code implementations25 Sep 2019 Wenqi Shao, Shitao Tang, Xingang Pan, Ping Tan, Xiaogang Wang, Ping Luo

However, over-sparse CNNs have many collapsed channels (i. e. many channels with undesired zero values), impeding their learning ability.

Open Compound Domain Adaptation

no code implementations CVPR 2020 Ziwei Liu, Zhongqi Miao, Xingang Pan, Xiaohang Zhan, Dahua Lin, Stella X. Yu, Boqing Gong

A typical domain adaptation approach is to adapt models trained on the annotated data in a source domain (e. g., sunny weather) for achieving high performance on the test data in a target domain (e. g., rainy weather).

Domain Adaptation Facial Expression Recognition +2

Switchable Whitening for Deep Representation Learning

1 code implementation ICCV 2019 Xingang Pan, Xiaohang Zhan, Jianping Shi, Xiaoou Tang, Ping Luo

Unlike existing works that design normalization techniques for specific tasks, we propose Switchable Whitening (SW), which provides a general form unifying different whitening methods as well as standardization methods.

Domain Adaptation Image Classification +4

Self-Supervised Learning via Conditional Motion Propagation

1 code implementation CVPR 2019 Xiaohang Zhan, Xingang Pan, Ziwei Liu, Dahua Lin, Chen Change Loy

Instead of explicitly modeling the motion probabilities, we design the pretext task as a conditional motion propagation problem.

Human Parsing Instance Segmentation +2

Two at Once: Enhancing Learning and Generalization Capacities via IBN-Net

24 code implementations ECCV 2018 Xingang Pan, Ping Luo, Jianping Shi, Xiaoou Tang

IBN-Net carefully integrates Instance Normalization (IN) and Batch Normalization (BN) as building blocks, and can be wrapped into many advanced deep networks to improve their performances.

All-day Semantic Segmentation Domain Generalization +2

Spatial As Deep: Spatial CNN for Traffic Scene Understanding

8 code implementations17 Dec 2017 Xingang Pan, Jianping Shi, Ping Luo, Xiaogang Wang, Xiaoou Tang

Although CNN has shown strong capability to extract semantics from raw pixels, its capacity to capture spatial relationships of pixels across rows and columns of an image is not fully explored.

Lane Detection Scene Understanding

Cannot find the paper you are looking for? You can Submit a new open access paper.