Search Results for author: Hujun Bao

Found 63 papers, 34 papers with code

I$^2$-SDF: Intrinsic Indoor Scene Reconstruction and Editing via Raytracing in Neural SDFs

no code implementations14 Mar 2023 Jingsen Zhu, Yuchi Huo, Qi Ye, Fujun Luan, Jifan Li, Dianbing Xi, Lisha Wang, Rui Tang, Wei Hua, Hujun Bao, Rui Wang

In this work, we present I$^2$-SDF, a new method for intrinsic indoor scene reconstruction and editing using differentiable Monte Carlo raytracing on neural signed distance fields (SDFs).

Indoor Scene Reconstruction Novel View Synthesis

PATS: Patch Area Transportation with Subdivision for Local Feature Matching

no code implementations14 Mar 2023 Junjie Ni, Yijin Li, Zhaoyang Huang, Hongsheng Li, Hujun Bao, Zhaopeng Cui, Guofeng Zhang

However, estimating scale differences between these patches is non-trivial since the scale differences are determined by both relative camera poses and scene structures, and thus spatially varying over image pairs.

Graph Matching Optical Flow Estimation +2

Learning Neural Volumetric Representations of Dynamic Humans in Minutes

no code implementations23 Feb 2023 Chen Geng, Sida Peng, Zhen Xu, Hujun Bao, Xiaowei Zhou

In this paper, we propose a novel method for learning neural volumetric videos of dynamic humans from sparse view videos in minutes with competitive visual quality.

EC-SfM: Efficient Covisibility-based Structure-from-Motion for Both Sequential and Unordered Images

1 code implementation21 Feb 2023 Zhichao Ye, Chong Bao, Xin Zhou, Haomin Liu, Hujun Bao, Guofeng Zhang

Based on this general image connection, we propose a unified framework to efficiently reconstruct sequential images, unordered images, and the mixture of these two.

Improving Feature-based Visual Localization by Geometry-Aided Matching

1 code implementation16 Nov 2022 Hailin Yu, Youji Feng, Weicai Ye, Mingxuan Jiang, Hujun Bao, Guofeng Zhang

We apply GAM to a new hierarchical visual localization pipeline and show that GAM can effectively improve the robustness and accuracy of localization.

3D Feature Matching Pose Estimation +1

Learning-based Inverse Rendering of Complex Indoor Scenes with Differentiable Monte Carlo Raytracing

no code implementations6 Nov 2022 Jingsen Zhu, Fujun Luan, Yuchi Huo, Zihao Lin, Zhihua Zhong, Dianbing Xi, Jiaxiang Zheng, Rui Tang, Hujun Bao, Rui Wang

Indoor scenes typically exhibit complex, spatially-varying appearance from global illumination, making inverse rendering a challenging ill-posed problem.

IntrinsicNeRF: Learning Intrinsic Neural Radiance Fields for Editable Novel View Synthesis

1 code implementation2 Oct 2022 Weicai Ye, Shuo Chen, Chong Bao, Hujun Bao, Marc Pollefeys, Zhaopeng Cui, Guofeng Zhang

Existing inverse rendering combined with neural rendering methods~\cite{zhang2021physg, zhang2022modeling} can only perform editable novel view synthesis on object-specific scenes, while we present intrinsic neural radiance fields, dubbed IntrinsicNeRF, which introduce intrinsic decomposition into the NeRF-based~\cite{mildenhall2020nerf} neural rendering method and can extend its application to room-scale scenes.

Neural Rendering Novel View Synthesis

DELTAR: Depth Estimation from a Light-weight ToF Sensor and RGB Image

no code implementations27 Sep 2022 Yijin Li, Xinyang Liu, Wenqi Dong, Han Zhou, Hujun Bao, Guofeng Zhang, yinda zhang, Zhaopeng Cui

Light-weight time-of-flight (ToF) depth sensors are small, cheap, low-energy and have been massively deployed on mobile devices for the purposes like autofocus, obstacle detection, etc.

3D Reconstruction Depth Completion +2

Vox-Surf: Voxel-based Implicit Surface Representation

no code implementations21 Aug 2022 Hai Li, Xingrui Yang, Hongjia Zhai, Yuqian Liu, Hujun Bao, Guofeng Zhang

Virtual content creation and interaction play an important role in modern 3D applications such as AR and VR.

NeuMesh: Learning Disentangled Neural Mesh-based Implicit Field for Geometry and Texture Editing

no code implementations25 Jul 2022 Bangbang Yang, Chong Bao, Junyi Zeng, Hujun Bao, yinda zhang, Zhaopeng Cui, Guofeng Zhang

Very recently neural implicit rendering techniques have been rapidly evolved and shown great advantages in novel view synthesis and 3D scene reconstruction.

3D Scene Reconstruction Neural Rendering +1

DeFlowSLAM: Self-Supervised Scene Motion Decomposition for Dynamic Dense SLAM

1 code implementation18 Jul 2022 Weicai Ye, Xingyuan Yu, Xinyue Lan, Yuhang Ming, Jinyu Li, Hujun Bao, Zhaopeng Cui, Guofeng Zhang

We present a novel dual-flow representation of scene motion that decomposes the optical flow into a static flow field caused by the camera motion and another dynamic flow field caused by the objects' movements in the scene.

Pose Estimation Simultaneous Localization and Mapping

Factorized and Controllable Neural Re-Rendering of Outdoor Scene for Photo Extrapolation

no code implementations14 Jul 2022 Boming Zhao, Bangbang Yang, Zhenyang Li, Zuoyue Li, Guofeng Zhang, Jiashu Zhao, Dawei Yin, Zhaopeng Cui, Hujun Bao

Expanding an existing tourist photo from a partially captured scene to a full scene is one of the desired experiences for photography applications.

PVO: Panoptic Visual Odometry

1 code implementation4 Jul 2022 Weicai Ye, Xinyue Lan, Shuo Chen, Yuhang Ming, Xingyuan Yu, Hujun Bao, Zhaopeng Cui, Guofeng Zhang

PVO models visual odometry (VO) and video panoptic segmentation (VPS) in a unified view, enabling the two tasks to facilitate each other.

Optical Flow Estimation Panoptic Segmentation +3

VIP-SLAM: An Efficient Tightly-Coupled RGB-D Visual Inertial Planar SLAM

no code implementations4 Jul 2022 Danpeng Chen, Shuai Wang, Weijian Xie, Shangjin Zhai, Nan Wang, Hujun Bao, Guofeng Zhang

Even if the plane parameters are involved in the optimization, we effectively simplify the back-end map by using planar structures.


Neural 3D Scene Reconstruction with the Manhattan-world Assumption

1 code implementation CVPR 2022 Haoyu Guo, Sida Peng, Haotong Lin, Qianqian Wang, Guofeng Zhang, Hujun Bao, Xiaowei Zhou

Based on the Manhattan-world assumption, planar constraints are employed to regularize the geometry in floor and wall regions predicted by a 2D semantic segmentation network.

2D Semantic Segmentation 3D Reconstruction +2

Neural Rendering in a Room: Amodal 3D Understanding and Free-Viewpoint Rendering for the Closed Scene Composed of Pre-Captured Objects

no code implementations5 May 2022 Bangbang Yang, yinda zhang, Yijin Li, Zhaopeng Cui, Sean Fanello, Hujun Bao, Guofeng Zhang

We, as human beings, can understand and picture a familiar scene from arbitrary viewpoints given a single image, whereas this is still a grand challenge for computers.

Data Augmentation Neural Rendering +1

A Hybrid Mesh-neural Representation for 3D Transparent Object Reconstruction

no code implementations23 Mar 2022 Jiamin Xu, Zihan Zhu, Hujun Bao, Weiwei Xu

We propose a novel method to reconstruct the 3D shapes of transparent objects using hand-held captured images under natural light conditions.

Image Matting Object Reconstruction +1

Animatable Implicit Neural Representations for Creating Realistic Avatars from Videos

1 code implementation15 Mar 2022 Sida Peng, Zhen Xu, Junting Dong, Qianqian Wang, Shangzhan Zhang, Qing Shuai, Hujun Bao, Xiaowei Zhou

Some recent works have proposed to decompose a non-rigidly deforming scene into a canonical neural radiance field and a set of deformation fields that map observation-space points to the canonical space, thereby enabling them to learn the dynamic scene from images.

Normal and Visibility Estimation of Human Face from a Single Image

no code implementations9 Mar 2022 Fuzhi Zhong, Rui Wang, Yuchi Huo, Hujun Bao

Recent work on the intrinsic image of humans starts to consider the visibility of incident illumination and encodes the light transfer function by spherical harmonics.

Hybrid Tracker with Pixel and Instance for Video Panoptic Segmentation

no code implementations2 Mar 2022 Weicai Ye, Xinyue Lan, Ge Su, Hujun Bao, Zhaopeng Cui, Guofeng Zhang

Existing methods are mainly based on the trained instance embedding to maintain consistent panoptic segmentation.

Association Optical Flow Estimation +2

SelfRecon: Self Reconstruction Your Digital Avatar from Monocular Video

1 code implementation CVPR 2022 Boyi Jiang, Yang Hong, Hujun Bao, Juyong Zhang

Meanwhile, the explicit mesh is updated periodically to adjust its topology changes, and a consistency loss is designed to match both representations.

Neural Rendering

NICE-SLAM: Neural Implicit Scalable Encoding for SLAM

1 code implementation CVPR 2022 Zihan Zhu, Songyou Peng, Viktor Larsson, Weiwei Xu, Hujun Bao, Zhaopeng Cui, Martin R. Oswald, Marc Pollefeys

Neural implicit representations have recently shown encouraging results in various domains, including promising progress in simultaneous localization and mapping (SLAM).

Simultaneous Localization and Mapping

Geometry-aware Two-scale PIFu Representation for Human Reconstruction

no code implementations3 Dec 2021 Zheng Dong, Ke Xu, Ziheng Duan, Hujun Bao, Weiwei Xu, Rynson W. H. Lau

Our key idea is to exploit the complementary properties of depth denoising and 3D reconstruction, for learning a two-scale PIFu representation to reconstruct high-frequency facial details and consistent bodies separately.

3D Human Reconstruction 3D Reconstruction +2

Efficient Neural Radiance Fields for Interactive Free-viewpoint Video

no code implementations2 Dec 2021 Haotong Lin, Sida Peng, Zhen Xu, Yunzhi Yan, Qing Shuai, Hujun Bao, Xiaowei Zhou

We propose a novel scene representation, called ENeRF, for the fast creation of interactive free-viewpoint videos.

Depth Estimation Depth Prediction +1

LatentHuman: Shape-and-Pose Disentangled Latent Representation for Human Bodies

no code implementations30 Nov 2021 Sandro Lombardi, Bangbang Yang, Tianxing Fan, Hujun Bao, Guofeng Zhang, Marc Pollefeys, Zhaopeng Cui

In this work, we propose a novel neural implicit representation for the human body, which is fully differentiable and optimizable with disentangled shape and pose latent spaces.

3D Reconstruction motion retargeting +1

CPT: A Pre-Trained Unbalanced Transformer for Both Chinese Language Understanding and Generation

1 code implementation13 Sep 2021 Yunfan Shao, Zhichao Geng, Yitao Liu, Junqi Dai, Hang Yan, Fei Yang, Li Zhe, Hujun Bao, Xipeng Qiu

In this paper, we take the advantage of previous pre-trained models (PTMs) and propose a novel Chinese Pre-trained Unbalanced Transformer (CPT).

Denoising Language Modelling +3

Learning Object-Compositional Neural Radiance Field for Editable Scene Rendering

no code implementations ICCV 2021 Bangbang Yang, yinda zhang, Yinghao Xu, Yijin Li, Han Zhou, Hujun Bao, Guofeng Zhang, Zhaopeng Cui

In this paper, we present a novel neural scene rendering system, which learns an object-compositional neural radiance field and produces realistic rendering with editing capability for a clustered and real-world scene.

Neural Rendering Novel View Synthesis

DeepPanoContext: Panoramic 3D Scene Understanding with Holistic Scene Context Graph and Relation-based Optimization

1 code implementation ICCV 2021 Cheng Zhang, Zhaopeng Cui, Cai Chen, Shuaicheng Liu, Bing Zeng, Hujun Bao, yinda zhang

Panorama images have a much larger field-of-view thus naturally encode enriched scene context information compared to standard perspective images, which however is not well exploited in the previous scene understanding methods.

Scene Understanding

MINERVAS: Massive INterior EnviRonments VirtuAl Synthesis

no code implementations13 Jul 2021 Haocheng Ren, Hao Zhang, Jia Zheng, Jiaxiang Zheng, Rui Tang, Yuchi Huo, Hujun Bao, Rui Wang

With the rapid development of data-driven techniques, data has played an essential role in various computer vision tasks.

2D Semantic Segmentation Depth Estimation +1

Attention-guided Temporally Coherent Video Object Matting

1 code implementation24 May 2021 Yunke Zhang, Chi Wang, Miaomiao Cui, Peiran Ren, Xuansong Xie, Xian-Sheng Hua, Hujun Bao, QiXing Huang, Weiwei Xu

Experimental results show that our method can generate high-quality alpha mattes for various videos featuring appearance change, occlusion, and fast motion.

Image Matting Semantic Segmentation +3

VS-Net: Voting with Segmentation for Visual Localization

1 code implementation CVPR 2021 Zhaoyang Huang, Han Zhou, Yijin Li, Bangbang Yang, Yan Xu, Xiaowei Zhou, Hujun Bao, Guofeng Zhang, Hongsheng Li

To address this problem, we propose a novel visual localization framework that establishes 2D-to-3D correspondences between the query image and the 3D map with a series of learnable scene-specific landmarks.

Semantic Segmentation Visual Localization

Animatable Neural Radiance Fields for Modeling Dynamic Human Bodies

1 code implementation ICCV 2021 Sida Peng, Junting Dong, Qianqian Wang, Shangzhan Zhang, Qing Shuai, Xiaowei Zhou, Hujun Bao

Moreover, the learned blend weight fields can be combined with input skeletal motions to generate new deformation fields to animate the human model.

StereoPIFu: Depth Aware Clothed Human Digitization via Stereo Vision

1 code implementation CVPR 2021 Yang Hong, Juyong Zhang, Boyi Jiang, Yudong Guo, Ligang Liu, Hujun Bao

In this paper, we propose StereoPIFu, which integrates the geometric constraints of stereo vision with implicit function representation of PIFu, to recover the 3D shape of the clothed human from a pair of low-cost rectified images.

Reconstructing 3D Human Pose by Watching Humans in the Mirror

1 code implementation CVPR 2021 Qi Fang, Qing Shuai, Junting Dong, Hujun Bao, Xiaowei Zhou

In this paper, we introduce the new task of reconstructing 3D human pose from a single image in which we can see the person and the person's image through a mirror.

3D Pose Estimation

AD-NeRF: Audio Driven Neural Radiance Fields for Talking Head Synthesis

1 code implementation ICCV 2021 Yudong Guo, Keyu Chen, Sen Liang, Yong-Jin Liu, Hujun Bao, Juyong Zhang

Generating high-fidelity talking head video by fitting with the input audio sequence is a challenging problem that receives considerable attentions recently.

Talking Face Generation

You Don't Only Look Once: Constructing Spatial-Temporal Memory for Integrated 3D Object Detection and Tracking

no code implementations ICCV 2021 Jiaming Sun, Yiming Xie, Siyu Zhang, Linghao Chen, Guofeng Zhang, Hujun Bao, Xiaowei Zhou

In this work, we propose a novel system for integrated 3D object detection and tracking, which uses a dynamic object occupancy map and previous object states as spatial-temporal memory to assist object detection in future frames.

3D Object Detection object-detection +1

Graph-Based Asynchronous Event Processing for Rapid Object Recognition

no code implementations ICCV 2021 Yijin Li, Han Zhou, Bangbang Yang, Ye Zhang, Zhaopeng Cui, Hujun Bao, Guofeng Zhang

Different from traditional video cameras, event cameras capture asynchronous events stream in which each event encodes pixel location, trigger time, and the polarity of the brightness changes.

graph construction Object Recognition

Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humans

3 code implementations CVPR 2021 Sida Peng, Yuanqing Zhang, Yinghao Xu, Qianqian Wang, Qing Shuai, Hujun Bao, Xiaowei Zhou

To this end, we propose Neural Body, a new human body representation which assumes that the learned neural representations at different frames share the same set of latent codes anchored to a deformable mesh, so that the observations across frames can be naturally integrated.

Novel View Synthesis Representation Learning

Location-aware Single Image Reflection Removal

1 code implementation ICCV 2021 Zheng Dong, Ke Xu, Yin Yang, Hujun Bao, Weiwei Xu, Rynson W. H. Lau

It is beneficial to strong reflection detection and substantially improves the quality of reflection removal results.

Reflection Removal

Recurrent Multi-view Alignment Network for Unsupervised Surface Registration

1 code implementation CVPR 2021 Wanquan Feng, Juyong Zhang, Hongrui Cai, Haofei Xu, Junhui Hou, Hujun Bao

Learning non-rigid registration in an end-to-end manner is challenging due to the inherent high degrees of freedom and the lack of labeled training data.

Deformable Object Manipulation Neural Rendering +1

SelfVoxeLO: Self-supervised LiDAR Odometry with Voxel-based Deep Neural Networks

no code implementations19 Oct 2020 Yan Xu, Zhaoyang Huang, Kwan-Yee Lin, Xinge Zhu, Jianping Shi, Hujun Bao, Guofeng Zhang, Hongsheng Li

To suit our network to self-supervised learning, we design several novel loss functions that utilize the inherent properties of LiDAR point clouds.

Self-Supervised Learning

Motion Capture from Internet Videos

2 code implementations ECCV 2020 Junting Dong, Qing Shuai, Yuanqing Zhang, Xian Liu, Xiaowei Zhou, Hujun Bao

Therefore, we propose to capture human motion by jointly analyzing these Internet videos instead of using single videos separately.

Pose Estimation

Mobile3DRecon: Real-time Monocular 3D Reconstruction on a Mobile Phone

no code implementations ISMAR 2020 Xingbin Yang, Liyang Zhou, Hanqing Jiang, Zhongliang Tang, Yuanbo Wang, Hujun Bao, Guofeng Zhang

The proposed mesh generation module incrementally fuses each estimated keyframe depth map to an online dense surface mesh, which is useful for achieving realistic AR effects such as occlusions and collisions.

3D Reconstruction Monocular Depth Estimation +1

BCNet: Learning Body and Cloth Shape from A Single Image

1 code implementation ECCV 2020 Boyi Jiang, Juyong Zhang, Yang Hong, Jinhao Luo, Ligang Liu, Hujun Bao

In this paper, we consider the problem to automatically reconstruct garment and body shapes from a single near-front view RGB image.

Audio-driven Talking Face Video Generation with Learning-based Personalized Head Pose

1 code implementation24 Feb 2020 Ran Yi, Zipeng Ye, Juyong Zhang, Hujun Bao, Yong-Jin Liu

In this paper, we address this problem by proposing a deep neural network model that takes an audio signal A of a source person and a very short video V of a target person as input, and outputs a synthesized high-quality talking face video with personalized head pose (making use of the visual information in V), expression and lip synchronization (by considering both A and V).

3D Face Animation Video Generation

Deep Snake for Real-Time Instance Segmentation

1 code implementation CVPR 2020 Sida Peng, Wen Jiang, Huaijin Pi, Xiuli Li, Hujun Bao, Xiaowei Zhou

Based on deep snake, we develop a two-stage pipeline for instance segmentation: initial contour proposal and contour deformation, which can handle errors in object localization.

Object Localization Real-time Instance Segmentation +2

GIFT: Learning Transformation-Invariant Dense Visual Descriptors via Group CNNs

1 code implementation NeurIPS 2019 Yuan Liu, Zehong Shen, Zhixuan Lin, Sida Peng, Hujun Bao, Xiaowei Zhou

Instead of feature pooling, we use group convolutions to exploit underlying structures of the extracted features on the group, resulting in descriptors that are both discriminative and provably invariant to the group of transformations.

Pose Estimation

Depth Completion from Sparse LiDAR Data with Depth-Normal Constraints

no code implementations ICCV 2019 Yan Xu, Xinge Zhu, Jianping Shi, Guofeng Zhang, Hujun Bao, Hongsheng Li

Most of existing methods directly train a network to learn a mapping from sparse depth inputs to dense depth maps, which has difficulties in utilizing the 3D geometric constraints and handling the practical sensor noises.

Autonomous Driving Depth Completion

PVNet: Pixel-wise Voting Network for 6DoF Pose Estimation

2 code implementations CVPR 2019 Sida Peng, Yu-An Liu, Qi-Xing Huang, Hujun Bao, Xiaowei Zhou

We further create a Truncation LINEMOD dataset to validate the robustness of our approach against truncation.

Ranked #2 on 6D Pose Estimation using RGB on YCB-Video (Mean AUC metric)

6D Pose Estimation using RGB

ICE-BA: Incremental, Consistent and Efficient Bundle Adjustment for Visual-Inertial SLAM

1 code implementation CVPR 2018 Hao-Min Liu, Mingyu Chen, Guofeng Zhang, Hujun Bao, Yingze Bao

However, jointly using visual and inertial measurements to optimize SLAM objective functions is a problem of high computational complexity.

Pose Estimation

Robust Keyframe-based Dense SLAM with an RGB-D Camera

8 code implementations14 Nov 2017 Hao-Min Liu, Chen Li, Guojun Chen, Guofeng Zhang, Michael Kaess, Hujun Bao

In this paper, we present RKD-SLAM, a robust keyframe-based dense SLAM approach for an RGB-D camera that can robustly handle fast motion and dense loop closure, and run without time limitation in a moderate size scene.

ENFT: Efficient Non-Consecutive Feature Tracking for Robust Structure-from-Motion

3 code implementations27 Oct 2015 Guofeng Zhang, Hao-Min Liu, Zilong Dong, Jiaya Jia, Tien-Tsin Wong, Hujun Bao

Our framework consists of steps of solving the feature `dropout' problem when indistinctive structures, noise or large image distortion exists, and of rapidly recognizing and joining common features located in different subsequences.

Cannot find the paper you are looking for? You can Submit a new open access paper.