Search Results for author:
Found 63 papers, 34 papers with code
BlinkFlow: A Dataset to Push the Limits of Event-based Optical Flow Estimation
BlinkSim consists of a configurable rendering engine and a flexible engine for event data simulation.
I$^2$-SDF: Intrinsic Indoor Scene Reconstruction and Editing via Raytracing in Neural SDFs
In this work, we present I$^2$-SDF, a new method for intrinsic indoor scene reconstruction and editing using differentiable Monte Carlo raytracing on neural signed distance fields (SDFs).
PATS: Patch Area Transportation with Subdivision for Local Feature Matching
However, estimating scale differences between these patches is non-trivial since the scale differences are determined by both relative camera poses and scene structures, and thus spatially varying over image pairs.
Perceiving Unseen 3D Objects by Poking the Objects
We present a novel approach to interactive 3D object perception for robots.
Learning Neural Volumetric Representations of Dynamic Humans in Minutes
In this paper, we propose a novel method for learning neural volumetric videos of dynamic humans from sparse view videos in minutes with competitive visual quality.
EC-SfM: Efficient Covisibility-based Structure-from-Motion for Both Sequential and Unordered Images
Based on this general image connection, we propose a unified framework to efficiently reconstruct sequential images, unordered images, and the mixture of these two.
OnePose++: Keypoint-Free One-Shot Object Pose Estimation without CAD Models
We propose a new method for object pose estimation without CAD models.
Improving Feature-based Visual Localization by Geometry-Aided Matching
We apply GAM to a new hierarchical visual localization pipeline and show that GAM can effectively improve the robustness and accuracy of localization.
Learning-based Inverse Rendering of Complex Indoor Scenes with Differentiable Monte Carlo Raytracing
Indoor scenes typically exhibit complex, spatially-varying appearance from global illumination, making inverse rendering a challenging ill-posed problem.
IntrinsicNeRF: Learning Intrinsic Neural Radiance Fields for Editable Novel View Synthesis
Existing inverse rendering combined with neural rendering methods~\cite{zhang2021physg, zhang2022modeling} can only perform editable novel view synthesis on object-specific scenes, while we present intrinsic neural radiance fields, dubbed IntrinsicNeRF, which introduce intrinsic decomposition into the NeRF-based~\cite{mildenhall2020nerf} neural rendering method and can extend its application to room-scale scenes.
DELTAR: Depth Estimation from a Light-weight ToF Sensor and RGB Image
Light-weight time-of-flight (ToF) depth sensors are small, cheap, low-energy and have been massively deployed on mobile devices for the purposes like autofocus, obstacle detection, etc.
Vox-Surf: Voxel-based Implicit Surface Representation
Virtual content creation and interaction play an important role in modern 3D applications such as AR and VR.
NeuMesh: Learning Disentangled Neural Mesh-based Implicit Field for Geometry and Texture Editing
Very recently neural implicit rendering techniques have been rapidly evolved and shown great advantages in novel view synthesis and 3D scene reconstruction.
QuickPose: Real-time Multi-view Multi-person Pose Estimation in Crowded Scenes
The key challenge of this problem is to efficiently match 2D observations across multiple views.
Ranked #2 on
3D Multi-Person Pose Estimation
on Shelf
3D Multi-Person Pose Estimation
Multi-Person Pose Estimation
DeFlowSLAM: Self-Supervised Scene Motion Decomposition for Dynamic Dense SLAM
We present a novel dual-flow representation of scene motion that decomposes the optical flow into a static flow field caused by the camera motion and another dynamic flow field caused by the objects' movements in the scene.
Factorized and Controllable Neural Re-Rendering of Outdoor Scene for Photo Extrapolation
Expanding an existing tourist photo from a partially captured scene to a full scene is one of the desired experiences for photography applications.
PVO: Panoptic Visual Odometry
PVO models visual odometry (VO) and video panoptic segmentation (VPS) in a unified view, enabling the two tasks to facilitate each other.
VIP-SLAM: An Efficient Tightly-Coupled RGB-D Visual Inertial Planar SLAM
Even if the plane parameters are involved in the optimization, we effectively simplify the back-end map by using planar structures.
Neural 3D Scene Reconstruction with the Manhattan-world Assumption
Based on the Manhattan-world assumption, planar constraints are employed to regularize the geometry in floor and wall regions predicted by a 2D semantic segmentation network.
Neural Rendering in a Room: Amodal 3D Understanding and Free-Viewpoint Rendering for the Closed Scene Composed of Pre-Captured Objects
We, as human beings, can understand and picture a familiar scene from arbitrary viewpoints given a single image, whereas this is still a grand challenge for computers.
CNN LEGO: Disassembling and Assembling Convolutional Neural Network
the feature maps are adopted to locate the critical features in each layer.
A Hybrid Mesh-neural Representation for 3D Transparent Object Reconstruction
We propose a novel method to reconstruct the 3D shapes of transparent objects using hand-held captured images under natural light conditions.
Animatable Implicit Neural Representations for Creating Realistic Avatars from Videos
Some recent works have proposed to decompose a non-rigidly deforming scene into a canonical neural radiance field and a set of deformation fields that map observation-space points to the canonical space, thereby enabling them to learn the dynamic scene from images.
Normal and Visibility Estimation of Human Face from a Single Image
Recent work on the intrinsic image of humans starts to consider the visibility of incident illumination and encodes the light transfer function by spherical harmonics.
Hybrid Tracker with Pixel and Instance for Video Panoptic Segmentation
Existing methods are mainly based on the trained instance embedding to maintain consistent panoptic segmentation.
SelfRecon: Self Reconstruction Your Digital Avatar from Monocular Video
Meanwhile, the explicit mesh is updated periodically to adjust its topology changes, and a consistency loss is designed to match both representations.
NICE-SLAM: Neural Implicit Scalable Encoding for SLAM
Neural implicit representations have recently shown encouraging results in various domains, including promising progress in simultaneous localization and mapping (SLAM).
Geometry-aware Two-scale PIFu Representation for Human Reconstruction
Our key idea is to exploit the complementary properties of depth denoising and 3D reconstruction, for learning a two-scale PIFu representation to reconstruct high-frequency facial details and consistent bodies separately.
Efficient Neural Radiance Fields for Interactive Free-viewpoint Video
We propose a novel scene representation, called ENeRF, for the fast creation of interactive free-viewpoint videos.
LatentHuman: Shape-and-Pose Disentangled Latent Representation for Human Bodies
In this work, we propose a novel neural implicit representation for the human body, which is fully differentiable and optimizable with disentangled shape and pose latent spaces.
CPT: A Pre-Trained Unbalanced Transformer for Both Chinese Language Understanding and Generation
In this paper, we take the advantage of previous pre-trained models (PTMs) and propose a novel Chinese Pre-trained Unbalanced Transformer (CPT).
Learning Object-Compositional Neural Radiance Field for Editable Scene Rendering
In this paper, we present a novel neural scene rendering system, which learns an object-compositional neural radiance field and produces realistic rendering with editing capability for a clustered and real-world scene.
DeepPanoContext: Panoramic 3D Scene Understanding with Holistic Scene Context Graph and Relation-based Optimization
Panorama images have a much larger field-of-view thus naturally encode enriched scene context information compared to standard perspective images, which however is not well exploited in the previous scene understanding methods.
MINERVAS: Massive INterior EnviRonments VirtuAl Synthesis
With the rapid development of data-driven techniques, data has played an essential role in various computer vision tasks.
Attention-guided Temporally Coherent Video Object Matting
Experimental results show that our method can generate high-quality alpha mattes for various videos featuring appearance change, occlusion, and fast motion.
VS-Net: Voting with Segmentation for Visual Localization
To address this problem, we propose a novel visual localization framework that establishes 2D-to-3D correspondences between the query image and the 3D map with a series of learnable scene-specific landmarks.
Animatable Neural Radiance Fields for Modeling Dynamic Human Bodies
Moreover, the learned blend weight fields can be combined with input skeletal motions to generate new deformation fields to animate the human model.
StereoPIFu: Depth Aware Clothed Human Digitization via Stereo Vision
In this paper, we propose StereoPIFu, which integrates the geometric constraints of stereo vision with implicit function representation of PIFu, to recover the 3D shape of the clothed human from a pair of low-cost rectified images.
LoFTR: Detector-Free Local Feature Matching with Transformers
We present a novel method for local image feature matching.
Ranked #1 on
Image Matching
on IMC PhotoTourism
(using extra training data)
NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Video
We present a novel framework named NeuralRecon for real-time 3D scene reconstruction from a monocular video.
Reconstructing 3D Human Pose by Watching Humans in the Mirror
In this paper, we introduce the new task of reconstructing 3D human pose from a single image in which we can see the person and the person's image through a mirror.
AD-NeRF: Audio Driven Neural Radiance Fields for Talking Head Synthesis
Generating high-fidelity talking head video by fitting with the input audio sequence is a challenging problem that receives considerable attentions recently.
Active Boundary Loss for Semantic Segmentation
This paper proposes a novel active boundary loss for semantic segmentation.
You Don't Only Look Once: Constructing Spatial-Temporal Memory for Integrated 3D Object Detection and Tracking
In this work, we propose a novel system for integrated 3D object detection and tracking, which uses a dynamic object occupancy map and previous object states as spatial-temporal memory to assist object detection in future frames.
Graph-Based Asynchronous Event Processing for Rapid Object Recognition
Different from traditional video cameras, event cameras capture asynchronous events stream in which each event encodes pixel location, trigger time, and the polarity of the brightness changes.
Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humans
To this end, we propose Neural Body, a new human body representation which assumes that the learned neural representations at different frames share the same set of latent codes anchored to a deformable mesh, so that the observations across frames can be naturally integrated.
Location-aware Single Image Reflection Removal
It is beneficial to strong reflection detection and substantially improves the quality of reflection removal results.
Recurrent Multi-view Alignment Network for Unsupervised Surface Registration
Learning non-rigid registration in an end-to-end manner is challenging due to the inherent high degrees of freedom and the lack of labeled training data.
SelfVoxeLO: Self-supervised LiDAR Odometry with Voxel-based Deep Neural Networks
To suit our network to self-supervised learning, we design several novel loss functions that utilize the inherent properties of LiDAR point clouds.
SMAP: Single-Shot Multi-Person Absolute 3D Pose Estimation
Recovering multi-person 3D poses with absolute scales from a single RGB image is a challenging problem due to the inherent depth and scale ambiguity from a single view.
3D Depth Estimation
3D Multi-Person Pose Estimation (absolute)
+3
Motion Capture from Internet Videos
Therefore, we propose to capture human motion by jointly analyzing these Internet videos instead of using single videos separately.
Mobile3DRecon: Real-time Monocular 3D Reconstruction on a Mobile Phone
The proposed mesh generation module incrementally fuses each estimated keyframe depth map to an online dense surface mesh, which is useful for achieving realistic AR effects such as occlusions and collisions.
Disp R-CNN: Stereo 3D Object Detection via Shape Prior Guided Instance Disparity Estimation
In this paper, we propose a novel system named Disp R-CNN for 3D object detection from stereo images.
3D Object Detection From Stereo Images
Disparity Estimation
+2
BCNet: Learning Body and Cloth Shape from A Single Image
In this paper, we consider the problem to automatically reconstruct garment and body shapes from a single near-front view RGB image.
Audio-driven Talking Face Video Generation with Learning-based Personalized Head Pose
In this paper, we address this problem by proposing a deep neural network model that takes an audio signal A of a source person and a very short video V of a target person as input, and outputs a synthesized high-quality talking face video with personalized head pose (making use of the visual information in V), expression and lip synchronization (by considering both A and V).
Deep Snake for Real-Time Instance Segmentation
Based on deep snake, we develop a two-stage pipeline for instance segmentation: initial contour proposal and contour deformation, which can handle errors in object localization.
Ranked #2 on
Semantic Contour Prediction
on Sbd val
GIFT: Learning Transformation-Invariant Dense Visual Descriptors via Group CNNs
Instead of feature pooling, we use group convolutions to exploit underlying structures of the extracted features on the group, resulting in descriptors that are both discriminative and provably invariant to the group of transformations.
Depth Completion from Sparse LiDAR Data with Depth-Normal Constraints
Most of existing methods directly train a network to learn a mapping from sparse depth inputs to dense depth maps, which has difficulties in utilizing the 3D geometric constraints and handling the practical sensor noises.
Fast and Robust Multi-Person 3D Pose Estimation from Multiple Views
This paper addresses the problem of 3D pose estimation for multiple people in a few calibrated camera views.
Ranked #11 on
3D Multi-Person Pose Estimation
on Campus
PVNet: Pixel-wise Voting Network for 6DoF Pose Estimation
We further create a Truncation LINEMOD dataset to validate the robustness of our approach against truncation.
Ranked #2 on
6D Pose Estimation using RGB
on YCB-Video
(Mean AUC metric)
ICE-BA: Incremental, Consistent and Efficient Bundle Adjustment for Visual-Inertial SLAM
However, jointly using visual and inertial measurements to optimize SLAM objective functions is a problem of high computational complexity.
Robust Keyframe-based Dense SLAM with an RGB-D Camera
In this paper, we present RKD-SLAM, a robust keyframe-based dense SLAM approach for an RGB-D camera that can robustly handle fast motion and dense loop closure, and run without time limitation in a moderate size scene.
ENFT: Efficient Non-Consecutive Feature Tracking for Robust Structure-from-Motion
Our framework consists of steps of solving the feature `dropout' problem when indistinctive structures, noise or large image distortion exists, and of rapidly recognizing and joining common features located in different subsequences.
