Search Results for author: Zhaopeng Cui

Found 56 papers, 18 papers with code

GaussianPrediction: Dynamic 3D Gaussian Prediction for Motion Extrapolation and Free View Synthesis

no code implementations30 May 2024 Boming Zhao, Yuan Li, Ziyu Sun, Lin Zeng, Yujun Shen, Rui Ma, yinda zhang, Hujun Bao, Zhaopeng Cui

In this paper, we introduce GaussianPrediction, a novel framework that empowers 3D Gaussian representations with dynamic scene modeling and future scenario synthesis in dynamic environments.

Decision Making Novel View Synthesis +1

Sat2Scene: 3D Urban Scene Generation from Satellite Images with Diffusion

no code implementations CVPR 2024 Zuoyue Li, Zhenqiang Li, Zhaopeng Cui, Marc Pollefeys, Martin R. Oswald

Directly generating scenes from satellite imagery offers exciting possibilities for integration into applications like games and map services.

3D Generation Neural Rendering +2

PNeRFLoc: Visual Localization with Point-based Neural Radiance Fields

no code implementations17 Dec 2023 Boming Zhao, Luwei Yang, Mao Mao, Hujun Bao, Zhaopeng Cui

In this paper, we propose a novel visual localization framework, \ie, PNeRFLoc, based on a unified point-based representation.

Data Augmentation Neural Rendering +3

DreamSpace: Dreaming Your Room Space with Text-Driven Panoramic Texture Propagation

no code implementations19 Oct 2023 Bangbang Yang, Wenqi Dong, Lin Ma, WenBo Hu, Xiao Liu, Zhaopeng Cui, Yuewen Ma

To ensure meaningful and aligned textures to the scene, we develop a novel coarse-to-fine panoramic texture generation approach with dual texture alignment, which both considers the geometry and texture cues of the captured scenes.

Texture Synthesis

Multi-Modal Neural Radiance Field for Monocular Dense SLAM with a Light-Weight ToF Sensor

no code implementations ICCV 2023 Xinyang Liu, Yijin Li, Yanbin Teng, Hujun Bao, Guofeng Zhang, yinda zhang, Zhaopeng Cui

Specifically, we propose a multi-modal implicit scene representation that supports rendering both the signals from the RGB camera and light-weight ToF sensor which drives the optimization by comparing with the raw sensor inputs.

Pose Tracking

Graph-based Asynchronous Event Processing for Rapid Object Recognition

no code implementations ICCV 2021 Yijin Li, Han Zhou, Bangbang Yang, Ye Zhang, Zhaopeng Cui, Hujun Bao, Guofeng Zhang

Different from traditional video cameras, event cameras capture asynchronous events stream in which each event encodes pixel location, trigger time, and the polarity of the brightness changes.

graph construction Object Recognition

Novel-view Synthesis and Pose Estimation for Hand-Object Interaction from Sparse Views

no code implementations ICCV 2023 Wentian Qu, Zhaopeng Cui, yinda zhang, Chenyu Meng, Cuixia Ma, Xiaoming Deng, Hongan Wang

Hand-object interaction understanding and the barely addressed novel view synthesis are highly desired in the immersive communication, whereas it is challenging due to the high deformation of hand and heavy occlusions between hand and object.

Neural Rendering Novel View Synthesis +3

SINE: Semantic-driven Image-based NeRF Editing with Prior-guided Editing Field

1 code implementation CVPR 2023 Chong Bao, yinda zhang, Bangbang Yang, Tianxing Fan, Zesong Yang, Hujun Bao, Guofeng Zhang, Zhaopeng Cui

Despite the great success in 2D editing using user-friendly tools, such as Photoshop, semantic strokes, or even text prompts, similar capabilities in 3D areas are still limited, either relying on 3D modeling skills or allowing editing within only a few categories.

PATS: Patch Area Transportation with Subdivision for Local Feature Matching

no code implementations CVPR 2023 Junjie Ni, Yijin Li, Zhaoyang Huang, Hongsheng Li, Hujun Bao, Zhaopeng Cui, Guofeng Zhang

However, estimating scale differences between these patches is non-trivial since the scale differences are determined by both relative camera poses and scene structures, and thus spatially varying over image pairs.

Graph Matching Optical Flow Estimation +2

NICER-SLAM: Neural Implicit Scene Encoding for RGB SLAM

no code implementations7 Feb 2023 Zihan Zhu, Songyou Peng, Viktor Larsson, Zhaopeng Cui, Martin R. Oswald, Andreas Geiger, Marc Pollefeys

Neural implicit representations have recently become popular in simultaneous localization and mapping (SLAM), especially in dense visual SLAM.

3D Scene Reconstruction Novel View Synthesis +2

DPS-Net: Deep Polarimetric Stereo Depth Estimation

no code implementations ICCV 2023 Chaoran Tian, Weihong Pan, Zimo Wang, Mao Mao, Guofeng Zhang, Hujun Bao, Ping Tan, Zhaopeng Cui

Stereo depth estimation usually struggles to deal with textureless scenes for both traditional and learning-based methods due to the inherent dependence on image correspondence matching.

Stereo Depth Estimation

RAGO: Recurrent Graph Optimizer For Multiple Rotation Averaging

1 code implementation CVPR 2022 Heng Li, Zhaopeng Cui, Shuaicheng Liu, Ping Tan

Our graph optimizer iteratively refines the global camera rotations by minimizing each node's single rotation objective function.

Generative Category-Level Shape and Pose Estimation with Semantic Primitives

1 code implementation3 Oct 2022 Guanglin Li, Yifeng Li, Zhichao Ye, Qihang Zhang, Tao Kong, Zhaopeng Cui, Guofeng Zhang

Then, by using a SIM(3)-invariant shape descriptor, we gracefully decouple the shape and pose of an object, thus supporting latent shape optimization of target objects in arbitrary poses.

6D Pose Estimation using RGBD Object

IntrinsicNeRF: Learning Intrinsic Neural Radiance Fields for Editable Novel View Synthesis

1 code implementation ICCV 2023 Weicai Ye, Shuo Chen, Chong Bao, Hujun Bao, Marc Pollefeys, Zhaopeng Cui, Guofeng Zhang

Existing inverse rendering combined with neural rendering methods can only perform editable novel view synthesis on object-specific scenes, while we present intrinsic neural radiance fields, dubbed IntrinsicNeRF, which introduce intrinsic decomposition into the NeRF-based neural rendering method and can extend its application to room-scale scenes.

Clustering Inverse Rendering +2

DELTAR: Depth Estimation from a Light-weight ToF Sensor and RGB Image

no code implementations27 Sep 2022 Yijin Li, Xinyang Liu, Wenqi Dong, Han Zhou, Hujun Bao, Guofeng Zhang, yinda zhang, Zhaopeng Cui

Light-weight time-of-flight (ToF) depth sensors are small, cheap, low-energy and have been massively deployed on mobile devices for the purposes like autofocus, obstacle detection, etc.

3D Reconstruction Depth Completion +2

NeuMesh: Learning Disentangled Neural Mesh-based Implicit Field for Geometry and Texture Editing

no code implementations25 Jul 2022 Bangbang Yang, Chong Bao, Junyi Zeng, Hujun Bao, yinda zhang, Zhaopeng Cui, Guofeng Zhang

Very recently neural implicit rendering techniques have been rapidly evolved and shown great advantages in novel view synthesis and 3D scene reconstruction.

3D Scene Reconstruction Neural Rendering +1

CompNVS: Novel View Synthesis with Scene Completion

no code implementations23 Jul 2022 Zuoyue Li, Tianxing Fan, Zhenqiang Li, Zhaopeng Cui, Yoichi Sato, Marc Pollefeys, Martin R. Oswald

We introduce a scalable framework for novel view synthesis from RGB-D images with largely incomplete scene coverage.

Novel View Synthesis Scene Understanding

DeFlowSLAM: Self-Supervised Scene Motion Decomposition for Dynamic Dense SLAM

1 code implementation18 Jul 2022 Weicai Ye, Xingyuan Yu, Xinyue Lan, Yuhang Ming, Jinyu Li, Hujun Bao, Zhaopeng Cui, Guofeng Zhang

We present a novel dual-flow representation of scene motion that decomposes the optical flow into a static flow field caused by the camera motion and another dynamic flow field caused by the objects' movements in the scene.

Pose Estimation Simultaneous Localization and Mapping

Factorized and Controllable Neural Re-Rendering of Outdoor Scene for Photo Extrapolation

no code implementations14 Jul 2022 Boming Zhao, Bangbang Yang, Zhenyang Li, Zuoyue Li, Guofeng Zhang, Jiashu Zhao, Dawei Yin, Zhaopeng Cui, Hujun Bao

Expanding an existing tourist photo from a partially captured scene to a full scene is one of the desired experiences for photography applications.

PVO: Panoptic Visual Odometry

1 code implementation CVPR 2023 Weicai Ye, Xinyue Lan, Shuo Chen, Yuhang Ming, Xingyuan Yu, Hujun Bao, Zhaopeng Cui, Guofeng Zhang

We present PVO, a novel panoptic visual odometry framework to achieve more comprehensive modeling of the scene motion, geometry, and panoptic segmentation information.

Optical Flow Estimation Pose Estimation +3

TC-SfM: Robust Track-Community-Based Structure-from-Motion

no code implementations13 Jun 2022 Lei Wang, Linlin Ge, Shan Luo, Zihan Yan, Zhaopeng Cui, Jieqing Feng

Specifically, a novel structure is proposed, namely, {\textit{track-community}}, in which each community consists of a group of tracks and represents a local segment in the scene.

Community Detection

Neural Rendering in a Room: Amodal 3D Understanding and Free-Viewpoint Rendering for the Closed Scene Composed of Pre-Captured Objects

no code implementations5 May 2022 Bangbang Yang, yinda zhang, Yijin Li, Zhaopeng Cui, Sean Fanello, Hujun Bao, Guofeng Zhang

We, as human beings, can understand and picture a familiar scene from arbitrary viewpoints given a single image, whereas this is still a grand challenge for computers.

Data Augmentation Neural Rendering +1

FD-SLAM: 3-D Reconstruction Using Features and Dense Matching

no code implementations25 Mar 2022 Xingrui Yang, Yuhang Ming, Zhaopeng Cui, Andrew Calway

It is well known that visual SLAM systems based on dense matching are locally accurate but are also susceptible to long-term drift and map corruption.

Pose Estimation

Hybrid Tracker with Pixel and Instance for Video Panoptic Segmentation

no code implementations2 Mar 2022 Weicai Ye, Xinyue Lan, Ge Su, Hujun Bao, Zhaopeng Cui, Guofeng Zhang

HybridTracker performs pixel tracker and instance tracker in parallel to obtain the association matrices, which are fused into a matching matrix.

Optical Flow Estimation Segmentation +1

SceneSqueezer: Learning To Compress Scene for Camera Relocalization

no code implementations CVPR 2022 Luwei Yang, Rakesh Shrestha, Wenbo Li, Shuaicheng Liu, Guofeng Zhang, Zhaopeng Cui, Ping Tan

Standard visual localization methods build a priori 3D model of a scene which is used to establish correspondences against the 2D keypoints in a query image.

Camera Relocalization Image Registration +3

NICE-SLAM: Neural Implicit Scalable Encoding for SLAM

1 code implementation CVPR 2022 Zihan Zhu, Songyou Peng, Viktor Larsson, Weiwei Xu, Hujun Bao, Zhaopeng Cui, Martin R. Oswald, Marc Pollefeys

Neural implicit representations have recently shown encouraging results in various domains, including promising progress in simultaneous localization and mapping (SLAM).

Simultaneous Localization and Mapping

LatentHuman: Shape-and-Pose Disentangled Latent Representation for Human Bodies

no code implementations30 Nov 2021 Sandro Lombardi, Bangbang Yang, Tianxing Fan, Hujun Bao, Guofeng Zhang, Marc Pollefeys, Zhaopeng Cui

In this work, we propose a novel neural implicit representation for the human body, which is fully differentiable and optimizable with disentangled shape and pose latent spaces.

3D Reconstruction motion retargeting +1

Non-local Recurrent Regularization Networks for Multi-view Stereo

no code implementations13 Oct 2021 Qingshan Xu, Martin R. Oswald, Wenbing Tao, Marc Pollefeys, Zhaopeng Cui

However, existing recurrent methods only model the local dependencies in the depth domain, which greatly limits the capability of capturing the global scene context along the depth dimension.

Depth Estimation

Learning Object-Compositional Neural Radiance Field for Editable Scene Rendering

no code implementations ICCV 2021 Bangbang Yang, yinda zhang, Yinghao Xu, Yijin Li, Han Zhou, Hujun Bao, Guofeng Zhang, Zhaopeng Cui

In this paper, we present a novel neural scene rendering system, which learns an object-compositional neural radiance field and produces realistic rendering with editing capability for a clustered and real-world scene.

Neural Rendering Novel View Synthesis +1

DeepPanoContext: Panoramic 3D Scene Understanding with Holistic Scene Context Graph and Relation-based Optimization

1 code implementation ICCV 2021 Cheng Zhang, Zhaopeng Cui, Cai Chen, Shuaicheng Liu, Bing Zeng, Hujun Bao, yinda zhang

Panorama images have a much larger field-of-view thus naturally encode enriched scene context information compared to standard perspective images, which however is not well exploited in the previous scene understanding methods.

Graph Neural Network Object +2

Deep Hybrid Self-Prior for Full 3D Mesh Generation

no code implementations ICCV 2021 Xingkui Wei, Zhengqing Chen, Yanwei Fu, Zhaopeng Cui, yinda zhang

We present a deep learning pipeline that leverages network self-prior to recover a full 3D model consisting of both a triangular mesh and a texture map from the colored 3D point cloud.

Surface Reconstruction

Vis2Mesh: Efficient Mesh Reconstruction from Unstructured Point Clouds of Large Scenes with Learned Virtual View Visibility

1 code implementation ICCV 2021 Shuang Song, Zhaopeng Cui, Rongjun Qin

Then the visibility information of multiple views is aggregated to generate a 3D mesh model by solving an optimization problem considering visibility in which a novel adaptive visibility weighting in surface determination is also introduced to suppress line of sight with a large incident angle.

Binary Classification Depth Completion +1

End-to-End Rotation Averaging With Multi-Source Propagation

1 code implementation CVPR 2021 Luwei Yang, Heng Li, Jamal Ahmed Rahim, Zhaopeng Cui, Ping Tan

These methods can suffer from bad initializations due to the noisy spanning tree or outliers in input relative rotations.

Graph Neural Network

Towards Efficient Graph Convolutional Networks for Point Cloud Handling

no code implementations ICCV 2021 Yawei Li, He Chen, Zhaopeng Cui, Radu Timofte, Marc Pollefeys, Gregory Chirikjian, Luc van Gool

In this paper, we aim at improving the computational efficiency of graph convolutional networks (GCNs) for learning on point clouds.

Computational Efficiency

Riggable 3D Face Reconstruction via In-Network Optimization

1 code implementation CVPR 2021 Ziqian Bai, Zhaopeng Cui, Xiaoming Liu, Ping Tan

This paper presents a method for riggable 3D face reconstruction from monocular images, which jointly estimates a personalized face rig and per-image parameters including expressions, poses, and illuminations.

3D Face Reconstruction Decoder

Holistic 3D Scene Understanding from a Single Image with Implicit Representation

1 code implementation CVPR 2021 Cheng Zhang, Zhaopeng Cui, yinda zhang, Bing Zeng, Marc Pollefeys, Shuaicheng Liu

We not only propose an image-based local structured implicit network to improve the object shape estimation, but also refine the 3D object pose and scene layout via a novel implicit scene graph neural network that exploits the implicit local object features.

 Ranked #1 on Monocular 3D Object Detection on SUN RGB-D (using extra training data)

3D Shape Reconstruction Graph Neural Network +5

The Card Shuffling Hypotheses: Building a Time and Memory Efficient Graph Convolutional Network

no code implementations1 Jan 2021 Yawei Li, He Chen, Zhaopeng Cui, Radu Timofte, Marc Pollefeys, Gregory Chirikjian, Luc van Gool

State-of-the-art GCNs adopt $K$-nearest neighbor (KNN) searches for local feature aggregation and feature extraction operations from layer to layer.

3D Classification Point Cloud Classification +2

Sat2Vid: Street-view Panoramic Video Synthesis from a Single Satellite Image

no code implementations ICCV 2021 Zuoyue Li, Zhenqiang Li, Zhaopeng Cui, Rongjun Qin, Marc Pollefeys, Martin R. Oswald

For geometrical and temporal consistency, our approach explicitly creates a 3D point cloud representation of the scene and maintains dense 3D-2D correspondences across frames that reflect the geometric scene configuration inferred from the satellite view.

Image Generation

4D Human Body Capture from Egocentric Video via 3D Scene Grounding

no code implementations26 Nov 2020 Miao Liu, Dexin Yang, Yan Zhang, Zhaopeng Cui, James M. Rehg, Siyu Tang

We introduce a novel task of reconstructing a time series of second-person 3D human body meshes from monocular egocentric videos.

Time Series Time Series Analysis

OmniSLAM: Omnidirectional Localization and Dense Mapping for Wide-baseline Multi-camera Systems

no code implementations18 Mar 2020 Changhee Won, Hochang Seok, Zhaopeng Cui, Marc Pollefeys, Jongwoo Lim

In this paper, we present an omnidirectional localization and dense mapping system for a wide-baseline multiview stereo setup with ultra-wide field-of-view (FOV) fisheye cameras, which has a 360 degrees coverage of stereo observations of the environment.

Depth Estimation Visual Odometry

Reflection Separation using a Pair of Unpolarized and Polarized Images

1 code implementation NeurIPS 2019 Youwei Lyu, Zhaopeng Cui, Si Li, Marc Pollefeys, Boxin Shi

When we take photos through glass windows or doors, the transmitted background scene is often blended with undesirable reflection.

DIST: Rendering Deep Implicit Signed Distance Function with Differentiable Sphere Tracing

1 code implementation CVPR 2020 Shaohui Liu, yinda zhang, Songyou Peng, Boxin Shi, Marc Pollefeys, Zhaopeng Cui

We propose a differentiable sphere tracing algorithm to bridge the gap between inverse graphics methods and the recently proposed deep learning based implicit signed distance function.

Polarimetric Relative Pose Estimation

no code implementations ICCV 2019 Zhaopeng Cui, Viktor Larsson, Marc Pollefeys

In this paper we consider the problem of relative pose estimation from two images with per-pixel polarimetric information.

Pose Estimation

Efficient 2D-3D Matching for Multi-Camera Visual Localization

no code implementations17 Sep 2018 Marcel Geppert, Peidong Liu, Zhaopeng Cui, Marc Pollefeys, Torsten Sattler

This results in a system that provides reliable and drift-less pose estimations for high speed autonomous driving.

Robotics

Polarimetric Dense Monocular SLAM

no code implementations CVPR 2018 Luwei Yang, Feitong Tan, Ao Li, Zhaopeng Cui, Yasutaka Furukawa, Ping Tan

This paper presents a novel polarimetric dense monocular SLAM (PDMS) algorithm based on a polarization camera.

Polarimetric Multi-View Stereo

no code implementations CVPR 2017 Zhaopeng Cui, Jinwei Gu, Boxin Shi, Ping Tan, Jan Kautz

Multi-view stereo relies on feature correspondences for 3D reconstruction, and thus is fundamentally flawed in dealing with featureless scenes.

3D Reconstruction

Global Structure-From-Motion by Similarity Averaging

no code implementations ICCV 2015 Zhaopeng Cui, Ping Tan

Depth images help to upgrade an essential matrix to a similarity transformation, which can determine the scale of relative translation.

Translation

Linear Global Translation Estimation with Feature Tracks

no code implementations6 Mar 2015 Zhaopeng Cui, Nianjuan Jiang, Chengzhou Tang, Ping Tan

This paper derives a novel linear position constraint for cameras seeing a common scene point, which leads to a direct linear method for global camera translation estimation.

Position Translation

Cannot find the paper you are looking for? You can Submit a new open access paper.