Search Results for author: Sida Peng

Found 45 papers, 17 papers with code

Generating Human Motion in 3D Scenes from Text Descriptions

no code implementations13 May 2024 Zhi Cen, Huaijin Pi, Sida Peng, Zehong Shen, Minghui Yang, Shuai Zhu, Hujun Bao, Xiaowei Zhou

For motion generation, we design an object-centric scene representation for the generative model to focus on the target object, thereby reducing the scene complexity and facilitating the modeling of the relationship between human motions and the object.


MaPa: Text-driven Photorealistic Material Painting for 3D Shapes

no code implementations26 Apr 2024 Shangzhan Zhang, Sida Peng, Tao Xu, Yuanbo Yang, Tianrun Chen, Nan Xue, Yujun Shen, Hujun Bao, Ruizhen Hu, Xiaowei Zhou

Instead of relying on extensive paired data, i. e., 3D meshes with material graphs and corresponding text descriptions, to train a material graph generative model, we propose to leverage the pre-trained 2D diffusion model as a bridge to connect the text and material graphs.

TELA: Text to Layer-wise 3D Clothed Human Generation

no code implementations25 Apr 2024 Junting Dong, Qi Fang, Zehuan Huang, Xudong Xu, Jingbo Wang, Sida Peng, Bo Dai

Previous works usually encode the human body and clothes as a holistic model and generate the whole model in a single-stage optimization, which makes them struggle for clothing editing and meanwhile lose fine-grained control over the whole generation process.

Disentanglement Virtual Try-on

SpatialTracker: Tracking Any 2D Pixels in 3D Space

no code implementations5 Apr 2024 Yuxi Xiao, Qianqian Wang, Shangzhan Zhang, Nan Xue, Sida Peng, Yujun Shen, Xiaowei Zhou

Recovering dense and long-range pixel motion in videos is a challenging problem.

GVGEN: Text-to-3D Generation with Volumetric Representation

no code implementations19 Mar 2024 Xianglong He, Junyi Chen, Sida Peng, Di Huang, Yangguang Li, Xiaoshui Huang, Chun Yuan, Wanli Ouyang, Tong He

To simplify the generation of GaussianVolume and empower the model to generate instances with detailed 3D geometry, we propose a coarse-to-fine pipeline.

3D Generation 3D Reconstruction +1

Efficient LoFTR: Semi-Dense Local Feature Matching with Sparse-Like Speed

1 code implementation7 Mar 2024 Yifan Wang, Xingyi He, Sida Peng, Dongli Tan, Xiaowei Zhou

Furthermore, we find spatial variance exists in LoFTR's fine correlation module, which is adverse to matching accuracy.

3D Reconstruction Image Retrieval

AI Revolution on Chat Bot: Evidence from a Randomized Controlled Experiment

no code implementations19 Jan 2024 Sida Peng, Wojciech Swiatek, Allen Gao, Paul Cullivan, Haoge Chang

In recent years, generative AI has undergone major advancements, demonstrating significant promise in augmenting human productivity.

Information Retrieval Retrieval

Street Gaussians for Modeling Dynamic Urban Scenes

no code implementations2 Jan 2024 Yunzhi Yan, Haotong Lin, Chenxu Zhou, Weijie Wang, Haiyang Sun, Kun Zhan, Xianpeng Lang, Xiaowei Zhou, Sida Peng

We introduce Street Gaussians, a new explicit scene representation that tackles all these limitations.

UniDream: Unifying Diffusion Priors for Relightable Text-to-3D Generation

no code implementations14 Dec 2023 Zexiang Liu, Yangguang Li, Youtian Lin, Xin Yu, Sida Peng, Yan-Pei Cao, Xiaojuan Qi, Xiaoshui Huang, Ding Liang, Wanli Ouyang

Recent advancements in text-to-3D generation technology have significantly advanced the conversion of textual descriptions into imaginative well-geometrical and finely textured 3D objects.

3D Generation Text to 3D

SAM-guided Graph Cut for 3D Instance Segmentation

no code implementations13 Dec 2023 Haoyu Guo, He Zhu, Sida Peng, Yuang Wang, Yujun Shen, Ruizhen Hu, Xiaowei Zhou

Experimental results on the ScanNet, ScanNet++ and KITTI-360 datasets demonstrate that our method achieves robust segmentation performance and can generalize across different types of scenes.

3D Instance Segmentation Segmentation +1

EasyVolcap: Accelerating Neural Volumetric Video Research

1 code implementation11 Dec 2023 Zhen Xu, Tao Xie, Sida Peng, Haotong Lin, Qing Shuai, Zhiyuan Yu, Guangzhao He, Jiaming Sun, Hujun Bao, Xiaowei Zhou

Volumetric video is a technology that digitally records dynamic events such as artistic performances, sporting events, and remote conversations.

4K4D: Real-Time 4D View Synthesis at 4K Resolution

no code implementations17 Oct 2023 Zhen Xu, Sida Peng, Haotong Lin, Guangzhao He, Jiaming Sun, Yujun Shen, Hujun Bao, Xiaowei Zhou

Experiments show that our representation can be rendered at over 400 FPS on the DNA-Rendering dataset at 1080p resolution and 80 FPS on the ENeRF-Outdoor dataset at 4K resolution using an RTX 4090 GPU, which is 30x faster than previous methods and achieves the state-of-the-art rendering quality.


Hierarchical Generation of Human-Object Interactions with Diffusion Probabilistic Models

no code implementations ICCV 2023 Huaijin Pi, Sida Peng, Minghui Yang, Xiaowei Zhou, Hujun Bao

This paper presents a novel approach to generating the 3D motion of a human interacting with a target object, with a focus on solving the challenge of synthesizing long-range and diverse motions, which could not be fulfilled by existing auto-regressive models or path planning-based methods.

Human-Object Interaction Detection

Relightable and Animatable Neural Avatar from Sparse-View Video

no code implementations15 Aug 2023 Zhen Xu, Sida Peng, Chen Geng, Linzhan Mou, Zihan Yan, Jiaming Sun, Hujun Bao, Xiaowei Zhou

Based on the HDQ algorithm, we leverage sphere tracing to efficiently estimate the surface intersection and light visibility.

Inverse Rendering

Dyn-E: Local Appearance Editing of Dynamic Neural Radiance Fields

no code implementations24 Jul 2023 Shangzhan Zhang, Sida Peng, Yinji ShenTu, Qing Shuai, Tianrun Chen, Kaicheng Yu, Hujun Bao, Xiaowei Zhou

We extensively evaluate our approach on various scenes and show that our approach achieves spatially and temporally consistent editing results.

Detector-Free Structure from Motion

1 code implementation27 Jun 2023 Xingyi He, Jiaming Sun, Yifan Wang, Sida Peng, QiXing Huang, Hujun Bao, Xiaowei Zhou

We propose a new detector-free SfM framework to draw benefits from the recent success of detector-free matchers to avoid the early determination of keypoints, while solving the multi-view inconsistency issue of detector-free matchers.

Keypoint Detection

Neural Scene Chronology

1 code implementation CVPR 2023 Haotong Lin, Qianqian Wang, Ruojin Cai, Sida Peng, Hadar Averbuch-Elor, Xiaowei Zhou, Noah Snavely

Specifically, we represent the scene as a space-time radiance field with a per-image illumination embedding, where temporally-varying scene changes are encoded using a set of learned step functions.

Learning Human Mesh Recovery in 3D Scenes

no code implementations CVPR 2023 Zehong Shen, Zhi Cen, Sida Peng, Qing Shuai, Hujun Bao, Xiaowei Zhou

We present a novel method for recovering the absolute pose and shape of a human in a pre-scanned scene given a single image.

Human Mesh Recovery

Representing Volumetric Videos as Dynamic MLP Maps

no code implementations CVPR 2023 Sida Peng, Yunzhi Yan, Qing Shuai, Hujun Bao, Xiaowei Zhou

This paper introduces a novel representation of volumetric videos for real-time view synthesis of dynamic scenes.


Grid-guided Neural Radiance Fields for Large Urban Scenes

no code implementations CVPR 2023 Linning Xu, Yuanbo Xiangli, Sida Peng, Xingang Pan, Nanxuan Zhao, Christian Theobalt, Bo Dai, Dahua Lin

An alternative solution is to use a feature grid representation, which is computationally efficient and can naturally scale to a large scene with increased grid resolutions.

Learning Neural Volumetric Representations of Dynamic Humans in Minutes

1 code implementation CVPR 2023 Chen Geng, Sida Peng, Zhen Xu, Hujun Bao, Xiaowei Zhou

In this paper, we propose a novel method for learning neural volumetric videos of dynamic humans from sparse view videos in minutes with competitive visual quality.

Painting 3D Nature in 2D: View Synthesis of Natural Scenes from a Single Semantic Mask

no code implementations CVPR 2023 Shangzhan Zhang, Sida Peng, Tianrun Chen, Linzhan Mou, Haotong Lin, Kaicheng Yu, Yiyi Liao, Xiaowei Zhou

We introduce a novel approach that takes a single semantic mask as input to synthesize multi-view consistent color images of natural scenes, trained with a collection of single images from the Internet.

3D-Aware Image Synthesis

Learning 3D-aware Image Synthesis with Unknown Pose Distribution

no code implementations CVPR 2023 Zifan Shi, Yujun Shen, Yinghao Xu, Sida Peng, Yiyi Liao, Sheng Guo, Qifeng Chen, Dit-yan Yeung

Existing methods for 3D-aware image synthesis largely depend on the 3D pose distribution pre-estimated on the training set.

3D-Aware Image Synthesis

iVS-Net: Learning Human View Synthesis from Internet Videos

no code implementations ICCV 2023 Junting Dong, Qi Fang, Tianshuo Yang, Qing Shuai, Chengyu Qiao, Sida Peng

However, these methods usually rely on limited multi-view images typically collected in the studio or commercial high-quality 3D scans for training, which heavily prohibits their generalization capability for in-the-wild images.

Ponder: Point Cloud Pre-training via Neural Rendering

no code implementations ICCV 2023 Di Huang, Sida Peng, Tong He, Honghui Yang, Xiaowei Zhou, Wanli Ouyang

We propose a novel approach to self-supervised learning of point cloud representations by differentiable neural rendering.

3D Reconstruction Image Generation +2

Deep Generative Models on 3D Representations: A Survey

1 code implementation27 Oct 2022 Zifan Shi, Sida Peng, Yinghao Xu, Andreas Geiger, Yiyi Liao, Yujun Shen

In this survey, we thoroughly review the ongoing developments of 3D generative models, including methods that employ 2D and 3D supervision.

3D-Aware Image Synthesis 3D Shape Generation

Neural 3D Scene Reconstruction with the Manhattan-world Assumption

1 code implementation CVPR 2022 Haoyu Guo, Sida Peng, Haotong Lin, Qianqian Wang, Guofeng Zhang, Hujun Bao, Xiaowei Zhou

Based on the Manhattan-world assumption, planar constraints are employed to regularize the geometry in floor and wall regions predicted by a 2D semantic segmentation network.

2D Semantic Segmentation 3D Reconstruction +2

MPS-NeRF: Generalizable 3D Human Rendering from Multiview Images

no code implementations31 Mar 2022 Xiangjun Gao, Jiaolong Yang, Jongyoo Kim, Sida Peng, Zicheng Liu, Xin Tong

For this task, we propose a simple yet effective method to train a generalizable NeRF with multiview images as conditional input.

Novel View Synthesis

Animatable Implicit Neural Representations for Creating Realistic Avatars from Videos

1 code implementation15 Mar 2022 Sida Peng, Zhen Xu, Junting Dong, Qianqian Wang, Shangzhan Zhang, Qing Shuai, Hujun Bao, Xiaowei Zhou

Some recent works have proposed to decompose a non-rigidly deforming scene into a canonical neural radiance field and a set of deformation fields that map observation-space points to the canonical space, thereby enabling them to learn the dynamic scene from images.

3D-aware Image Synthesis via Learning Structural and Textural Representations

1 code implementation CVPR 2022 Yinghao Xu, Sida Peng, Ceyuan Yang, Yujun Shen, Bolei Zhou

The feature field is further accumulated into a 2D feature map as the textural representation, followed by a neural renderer for appearance synthesis.

3D-Aware Image Synthesis Generative Adversarial Network

Efficient Neural Radiance Fields for Interactive Free-viewpoint Video

no code implementations2 Dec 2021 Haotong Lin, Sida Peng, Zhen Xu, Yunzhi Yan, Qing Shuai, Hujun Bao, Xiaowei Zhou

We propose a novel scene representation, called ENeRF, for the fast creation of interactive free-viewpoint videos.

Depth Estimation Depth Prediction +1

Neural Rays for Occlusion-aware Image-based Rendering

1 code implementation CVPR 2022 YuAn Liu, Sida Peng, Lingjie Liu, Qianqian Wang, Peng Wang, Christian Theobalt, Xiaowei Zhou, Wenping Wang

On such a 3D point, these generalization methods will include inconsistent image features from invisible views, which interfere with the radiance field construction.

Neural Rendering Novel View Synthesis +1

Animatable Neural Radiance Fields for Modeling Dynamic Human Bodies

1 code implementation ICCV 2021 Sida Peng, Junting Dong, Qianqian Wang, Shangzhan Zhang, Qing Shuai, Xiaowei Zhou, Hujun Bao

Moreover, the learned blend weight fields can be combined with input skeletal motions to generate new deformation fields to animate the human model.

Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humans

3 code implementations CVPR 2021 Sida Peng, Yuanqing Zhang, Yinghao Xu, Qianqian Wang, Qing Shuai, Hujun Bao, Xiaowei Zhou

To this end, we propose Neural Body, a new human body representation which assumes that the learned neural representations at different frames share the same set of latent codes anchored to a deformable mesh, so that the observations across frames can be naturally integrated.

Novel View Synthesis Representation Learning

Doubly Robust Semiparametric Difference-in-Differences Estimators with High-Dimensional Data

no code implementations7 Sep 2020 Yang Ning, Sida Peng, Jing Tao

This paper proposes a doubly robust two-stage semiparametric difference-in-difference estimator for estimating heterogeneous treatment effects with high-dimensional data.

valid Vocal Bursts Intensity Prediction

Deep Snake for Real-Time Instance Segmentation

1 code implementation CVPR 2020 Sida Peng, Wen Jiang, Huaijin Pi, Xiuli Li, Hujun Bao, Xiaowei Zhou

Based on deep snake, we develop a two-stage pipeline for instance segmentation: initial contour proposal and contour deformation, which can handle errors in object localization.

Object Object Localization +3

GIFT: Learning Transformation-Invariant Dense Visual Descriptors via Group CNNs

1 code implementation NeurIPS 2019 Yuan Liu, Zehong Shen, Zhixuan Lin, Sida Peng, Hujun Bao, Xiaowei Zhou

Instead of feature pooling, we use group convolutions to exploit underlying structures of the extracted features on the group, resulting in descriptors that are both discriminative and provably invariant to the group of transformations.

Pose Estimation

PVNet: Pixel-wise Voting Network for 6DoF Pose Estimation

4 code implementations CVPR 2019 Sida Peng, Yu-An Liu, Qi-Xing Huang, Hujun Bao, Xiaowei Zhou

We further create a Truncation LINEMOD dataset to validate the robustness of our approach against truncation.

Ranked #2 on 6D Pose Estimation using RGB on YCB-Video (Mean AUC metric)

6D Pose Estimation using RGB

Robust Estimation of Causal Effects via High-Dimensional Covariate Balancing Propensity Score

no code implementations20 Dec 2018 Yang Ning, Sida Peng, Kosuke Imai

We first use a class of penalized M-estimators for the propensity score and outcome models.


Cannot find the paper you are looking for? You can Submit a new open access paper.