Search Results for author: Chuanxia Zheng

Found 30 papers, 18 papers with code

One-shot Human Motion Transfer via Occlusion-Robust Flow Prediction and Neural Texturing

no code implementations9 Dec 2024 Yuzhu Ji, Chuanxia Zheng, Tat-Jen Cham

To address the limitations and by both considering the importance of appearance and geometry for motion transfer, in this work, we proposed a unified framework that combines multi-scale feature warping and neural texture mapping to recover better 2D appearance and 2. 5D geometry, partly by exploiting the information from DensePose, yet adapting to its inherent limited accuracy.

Neural Rendering

MVSplat360: Feed-Forward 360 Scene Synthesis from Sparse Views

1 code implementation7 Nov 2024 Yuedong Chen, Chuanxia Zheng, Haofei Xu, Bohan Zhuang, Andrea Vedaldi, Tat-Jen Cham, Jianfei Cai

To evaluate MVSplat360's performance, we introduce a new benchmark using the challenging DL3DV-10K dataset, where MVSplat360 achieves superior visual quality compared to state-of-the-art methods on wide-sweeping or even 360{\deg} NVS tasks.

3DGS 3D Reconstruction +3

Splatt3R: Zero-shot Gaussian Splatting from Uncalibrated Image Pairs

no code implementations25 Aug 2024 Brandon Smart, Chuanxia Zheng, Iro Laina, Victor Adrian Prisacariu

In this paper, we introduce Splatt3R, a pose-free, feed-forward method for in-the-wild 3D reconstruction and novel view synthesis from stereo pairs.

3D geometry 3D Reconstruction +1

Puppet-Master: Scaling Interactive Video Generation as a Motion Prior for Part-Level Dynamics

no code implementations8 Aug 2024 Ruining Li, Chuanxia Zheng, Christian Rupprecht, Andrea Vedaldi

We present Puppet-Master, an interactive video generative model that can serve as a motion prior for part-level dynamics.

Video Generation

Flash3D: Feed-Forward Generalisable 3D Scene Reconstruction from a Single Image

1 code implementation6 Jun 2024 Stanislaw Szymanowicz, Eldar Insafutdinov, Chuanxia Zheng, Dylan Campbell, João F. Henriques, Christian Rupprecht, Andrea Vedaldi

In this paper, we propose Flash3D, a method for scene reconstruction and novel view synthesis from a single image which is both very generalisable and efficient.

3D Scene Reconstruction Monocular Depth Estimation +1

DragAPart: Learning a Part-Level Motion Prior for Articulated Objects

no code implementations22 Mar 2024 Ruining Li, Chuanxia Zheng, Christian Rupprecht, Andrea Vedaldi

We introduce DragAPart, a method that, given an image and a set of drags as input, generates a new image of the same object that responds to the action of the drags.

ClusteringSDF: Self-Organized Neural Implicit Surfaces for 3D Decomposition

no code implementations21 Mar 2024 Tianhao Wu, Chuanxia Zheng, Tat-Jen Cham, Qianyi Wu

3D decomposition/segmentation still remains a challenge as large-scale 3D annotated data is not readily available.

Segmentation

Amodal Ground Truth and Completion in the Wild

1 code implementation CVPR 2024 Guanqi Zhan, Chuanxia Zheng, Weidi Xie, Andrew Zisserman

In contrast, we use 3D data to establish an automatic pipeline to determine authentic ground truth amodal masks for partially occluded objects in real images.

Image Segmentation Segmentation +1

Free3D: Consistent Novel View Synthesis without 3D Representation

1 code implementation CVPR 2024 Chuanxia Zheng, Andrea Vedaldi

Similar to Zero-1-to-3, we start from a pre-trained 2D image generator for generalization, and fine-tune it for NVS.

3D Reconstruction Novel View Synthesis

One More Step: A Versatile Plug-and-Play Module for Rectifying Diffusion Schedule Flaws and Enhancing Low-Frequency Controls

no code implementations CVPR 2024 Minghui Hu, Jianbin Zheng, Chuanxia Zheng, Chaoyue Wang, DaCheng Tao, Tat-Jen Cham

By integrating a compact network and incorporating an additional simple yet effective step during inference, OMS elevates image fidelity and harmonizes the dichotomy between training and inference, while preserving original model parameters.

Denoising

A General Protocol to Probe Large Vision Models for 3D Physical Understanding

1 code implementation10 Oct 2023 Guanqi Zhan, Chuanxia Zheng, Weidi Xie, Andrew Zisserman

To this end, we make the following contributions: (i) We introduce a general and lightweight protocol to evaluate whether features of an off-the-shelf large vision model encode a number of physical 'properties' of the 3D scene, by training discriminative classifiers on the features for these properties.

Online Clustered Codebook

1 code implementation ICCV 2023 Chuanxia Zheng, Andrea Vedaldi

Vector Quantisation (VQ) is experiencing a comeback in machine learning, where it is increasingly used in representation learning.

Representation Learning

PanoDiffusion: 360-degree Panorama Outpainting via Diffusion

no code implementations6 Jul 2023 Tianhao Wu, Chuanxia Zheng, Tat-Jen Cham

Generating complete 360-degree panoramas from narrow field of view images is ongoing research as omnidirectional RGB data is not readily available.

Denoising

Cocktail: Mixing Multi-Modality Controls for Text-Conditional Image Generation

no code implementations1 Jun 2023 Minghui Hu, Jianbin Zheng, Daqing Liu, Chuanxia Zheng, Chaoyue Wang, DaCheng Tao, Tat-Jen Cham

In this work, we propose Cocktail, a pipeline to mix various modalities into one embedding, amalgamated with a generalized ControlNet (gControlNet), a controllable normalisation (ControlNorm), and a spatial guidance sampling method, to actualize multi-modal and spatially-refined control for text-conditional diffusion models.

Conditional Image Generation

Explicit Correspondence Matching for Generalizable Neural Radiance Fields

1 code implementation24 Apr 2023 Yuedong Chen, Haofei Xu, Qianyi Wu, Chuanxia Zheng, Tat-Jen Cham, Jianfei Cai

The key to our approach lies in the explicitly modeled correspondence matching information, so as to provide the geometry prior to the prediction of NeRF color and density for volume rendering.

NeRF Novel View Synthesis

Vector Quantized Wasserstein Auto-Encoder

no code implementations12 Feb 2023 Tung-Long Vuong, Trung Le, He Zhao, Chuanxia Zheng, Mehrtash Harandi, Jianfei Cai, Dinh Phung

Learning deep discrete latent presentations offers a promise of better symbolic and summarized abstractions that are more useful to subsequent downstream tasks.

Clustering Decoder +1

Unified Discrete Diffusion for Simultaneous Vision-Language Generation

1 code implementation27 Nov 2022 Minghui Hu, Chuanxia Zheng, Heliang Zheng, Tat-Jen Cham, Chaoyue Wang, Zuopeng Yang, DaCheng Tao, Ponnuthurai N. Suganthan

The recently developed discrete diffusion models perform extraordinarily well in the text-to-image task, showing significant promise for handling the multi-modality signals.

multimodal generation Text Generation +1

MoVQ: Modulating Quantized Vectors for High-Fidelity Image Generation

2 code implementations19 Sep 2022 Chuanxia Zheng, Long Tung Vuong, Jianfei Cai, Dinh Phung

Although two-stage Vector Quantized (VQ) generative models allow for synthesizing high-fidelity and high-resolution images, their quantization operator encodes similar patches within an image into the same index, resulting in a repeated artifact for similar adjacent regions using existing decoder architectures.

Decoder Image Generation +3

Object-Compositional Neural Implicit Surfaces

1 code implementation20 Jul 2022 Qianyi Wu, Xian Liu, Yuedong Chen, Kejie Li, Chuanxia Zheng, Jianfei Cai, Jianmin Zheng

This paper proposes a novel framework, ObjectSDF, to build an object-compositional neural implicit representation with high fidelity in 3D reconstruction and object representation.

3D Reconstruction Novel View Synthesis +1

High-Quality Pluralistic Image Completion via Code Shared VQGAN

no code implementations5 Apr 2022 Chuanxia Zheng, Guoxian Song, Tat-Jen Cham, Jianfei Cai, Dinh Phung, Linjie Luo

In this work, we present a novel framework for pluralistic image completion that can achieve both high quality and diversity at much faster inference speed.

Diversity Image Reconstruction +1

Sem2NeRF: Converting Single-View Semantic Masks to Neural Radiance Fields

1 code implementation21 Mar 2022 Yuedong Chen, Qianyi Wu, Chuanxia Zheng, Tat-Jen Cham, Jianfei Cai

In light of recent advances in NeRF-based 3D-aware generative models, we introduce a new task, Semantic-to-NeRF translation, that aims to reconstruct a 3D scene modelled by NeRF, conditioned on one single-view semantic mask as input.

3D-Aware Image Synthesis Decoder +2

Synthesizing Photorealistic Images with Deep Generative Learning

no code implementations23 Feb 2022 Chuanxia Zheng

The goal of this thesis is to present my research contributions towards solving various visual synthesis and generation tasks, comprising image translation, image completion, and completed scene decomposition.

Depth Estimation Image Generation +1

AgileGAN: stylizing portraits by inversion-consistent transfer learning

1 code implementation ACM Transactions on Graphics 2021 Guoxian Song, Linjie Luo, Jing Liu, Wan-Chun Ma, Chun-Pong Lai, Chuanxia Zheng, Tat-Jen Cham

While substantial progress has been made in automated stylization, generating high quality stylistic portraits is still a challenge, and even the recent popular Toonify suffers from several artifacts when used on real input images.

Attribute motion retargeting +1

Visiting the Invisible: Layer-by-Layer Completed Scene Decomposition

1 code implementation12 Apr 2021 Chuanxia Zheng, Duy-Son Dao, Guoxian Song, Tat-Jen Cham, Jianfei Cai

In this work, we propose a higher-level scene understanding system to tackle both visible and invisible parts of objects and backgrounds in a given scene.

Instance Segmentation Scene Understanding +1

The Spatially-Correlative Loss for Various Image Translation Tasks

2 code implementations CVPR 2021 Chuanxia Zheng, Tat-Jen Cham, Jianfei Cai

We propose a novel spatially-correlative loss that is simple, efficient and yet effective for preserving scene structure consistency while supporting large appearance changes during unpaired image-to-image (I2I) translation.

Self-Supervised Learning Translation

A Unified 3D Human Motion Synthesis Model via Conditional Variational Auto-Encoder

no code implementations ICCV 2021 Yujun Cai, Yiwei Wang, Yiheng Zhu, Tat-Jen Cham, Jianfei Cai, Junsong Yuan, Jun Liu, Chuanxia Zheng, Sijie Yan, Henghui Ding, Xiaohui Shen, Ding Liu, Nadia Magnenat Thalmann

Notably, by considering this problem as a conditional generation process, we estimate a parametric distribution of the missing regions based on the input conditions, from which to sample and synthesize the full motion series.

Decoder motion prediction +1

Pluralistic Image Completion

1 code implementation CVPR 2019 Chuanxia Zheng, Tat-Jen Cham, Jianfei Cai

In this paper, we present an approach for \textbf{pluralistic image completion} -- the task of generating multiple and diverse plausible solutions for image completion.

Decoder Diversity +1

Cannot find the paper you are looking for? You can Submit a new open access paper.