Search Results for author:
Found 45 papers, 16 papers with code
SINE: Semantic-driven Image-based NeRF Editing with Prior-guided Editing Field
Despite the great success in 2D editing using user-friendly tools, such as Photoshop, semantic strokes, or even text prompts, similar capabilities in 3D areas are still limited, either relying on 3D modeling skills or allowing editing within only a few categories.
BlinkFlow: A Dataset to Push the Limits of Event-based Optical Flow Estimation
BlinkSim consists of a configurable rendering engine and a flexible engine for event data simulation.
PATS: Patch Area Transportation with Subdivision for Local Feature Matching
However, estimating scale differences between these patches is non-trivial since the scale differences are determined by both relative camera poses and scene structures, and thus spatially varying over image pairs.
NICER-SLAM: Neural Implicit Scene Encoding for RGB SLAM
Neural implicit representations have recently become popular in simultaneous localization and mapping (SLAM), especially in dense visual SLAM.
RAGO: Recurrent Graph Optimizer For Multiple Rotation Averaging
Our graph optimizer iteratively refines the global camera rotations by minimizing each node's single rotation objective function.
Generative Category-Level Shape and Pose Estimation with Semantic Primitives
Then, by using a SIM(3)-invariant shape descriptor, we gracefully decouple the shape and pose of an object, thus supporting latent shape optimization of target objects in arbitrary poses.
Ranked #1 on
6D Pose Estimation using RGBD
on REAL275
IntrinsicNeRF: Learning Intrinsic Neural Radiance Fields for Editable Novel View Synthesis
Existing inverse rendering combined with neural rendering methods~\cite{zhang2021physg, zhang2022modeling} can only perform editable novel view synthesis on object-specific scenes, while we present intrinsic neural radiance fields, dubbed IntrinsicNeRF, which introduce intrinsic decomposition into the NeRF-based~\cite{mildenhall2020nerf} neural rendering method and can extend its application to room-scale scenes.
DELTAR: Depth Estimation from a Light-weight ToF Sensor and RGB Image
Light-weight time-of-flight (ToF) depth sensors are small, cheap, low-energy and have been massively deployed on mobile devices for the purposes like autofocus, obstacle detection, etc.
NeuMesh: Learning Disentangled Neural Mesh-based Implicit Field for Geometry and Texture Editing
Very recently neural implicit rendering techniques have been rapidly evolved and shown great advantages in novel view synthesis and 3D scene reconstruction.
CompNVS: Novel View Synthesis with Scene Completion
We introduce a scalable framework for novel view synthesis from RGB-D images with largely incomplete scene coverage.
DeFlowSLAM: Self-Supervised Scene Motion Decomposition for Dynamic Dense SLAM
We present a novel dual-flow representation of scene motion that decomposes the optical flow into a static flow field caused by the camera motion and another dynamic flow field caused by the objects' movements in the scene.
Factorized and Controllable Neural Re-Rendering of Outdoor Scene for Photo Extrapolation
Expanding an existing tourist photo from a partially captured scene to a full scene is one of the desired experiences for photography applications.
PVO: Panoptic Visual Odometry
We present PVO, a novel panoptic visual odometry framework to achieve more comprehensive modeling of the scene motion, geometry, and panoptic segmentation information.
TC-SfM: Robust Track-Community-Based Structure-from-Motion
Specifically, a novel structure is proposed, namely, {\textit{track-community}}, in which each community consists of a group of tracks and represents a local segment in the scene.
Neural Rendering in a Room: Amodal 3D Understanding and Free-Viewpoint Rendering for the Closed Scene Composed of Pre-Captured Objects
We, as human beings, can understand and picture a familiar scene from arbitrary viewpoints given a single image, whereas this is still a grand challenge for computers.
FD-SLAM: 3-D Reconstruction Using Features and Dense Matching
It is well known that visual SLAM systems based on dense matching are locally accurate but are also susceptible to long-term drift and map corruption.
Hybrid Tracker with Pixel and Instance for Video Panoptic Segmentation
Existing methods are mainly based on the trained instance embedding to maintain consistent panoptic segmentation.
SceneSqueezer: Learning To Compress Scene for Camera Relocalization
Standard visual localization methods build a priori 3D model of a scene which is used to establish correspondences against the 2D keypoints in a query image.
NICE-SLAM: Neural Implicit Scalable Encoding for SLAM
Neural implicit representations have recently shown encouraging results in various domains, including promising progress in simultaneous localization and mapping (SLAM).
LatentHuman: Shape-and-Pose Disentangled Latent Representation for Human Bodies
In this work, we propose a novel neural implicit representation for the human body, which is fully differentiable and optimizable with disentangled shape and pose latent spaces.
Non-local Recurrent Regularization Networks for Multi-view Stereo
However, existing recurrent methods only model the local dependencies in the depth domain, which greatly limits the capability of capturing the global scene context along the depth dimension.
Learning Object-Compositional Neural Radiance Field for Editable Scene Rendering
In this paper, we present a novel neural scene rendering system, which learns an object-compositional neural radiance field and produces realistic rendering with editing capability for a clustered and real-world scene.
DeepPanoContext: Panoramic 3D Scene Understanding with Holistic Scene Context Graph and Relation-based Optimization
Panorama images have a much larger field-of-view thus naturally encode enriched scene context information compared to standard perspective images, which however is not well exploited in the previous scene understanding methods.
Vis2Mesh: Efficient Mesh Reconstruction from Unstructured Point Clouds of Large Scenes with Learned Virtual View Visibility
Then the visibility information of multiple views is aggregated to generate a 3D mesh model by solving an optimization problem considering visibility in which a novel adaptive visibility weighting in surface determination is also introduced to suppress line of sight with a large incident angle.
Deep Hybrid Self-Prior for Full 3D Mesh Generation
We present a deep learning pipeline that leverages network self-prior to recover a full 3D model consisting of both a triangular mesh and a texture map from the colored 3D point cloud.
End-to-End Rotation Averaging With Multi-Source Propagation
These methods can suffer from bad initializations due to the noisy spanning tree or outliers in input relative rotations.
Towards Efficient Graph Convolutional Networks for Point Cloud Handling
In this paper, we aim at improving the computational efficiency of graph convolutional networks (GCNs) for learning on point clouds.
Riggable 3D Face Reconstruction via In-Network Optimization
This paper presents a method for riggable 3D face reconstruction from monocular images, which jointly estimates a personalized face rig and per-image parameters including expressions, poses, and illuminations.
Holistic 3D Scene Understanding from a Single Image with Implicit Representation
We not only propose an image-based local structured implicit network to improve the object shape estimation, but also refine the 3D object pose and scene layout via a novel implicit scene graph neural network that exploits the implicit local object features.
Ranked #1 on
3D Shape Reconstruction
on Pix3D
P2-Net: Joint Description and Detection of Local Features for Pixel and Point Matching
Accurately describing and detecting 2D and 3D keypoints is crucial to establishing correspondences across images and point clouds.
The Card Shuffling Hypotheses: Building a Time and Memory Efficient Graph Convolutional Network
State-of-the-art GCNs adopt $K$-nearest neighbor (KNN) searches for local feature aggregation and feature extraction operations from layer to layer.
Graph-Based Asynchronous Event Processing for Rapid Object Recognition
Different from traditional video cameras, event cameras capture asynchronous events stream in which each event encodes pixel location, trigger time, and the polarity of the brightness changes.
Sat2Vid: Street-view Panoramic Video Synthesis from a Single Satellite Image
For geometrical and temporal consistency, our approach explicitly creates a 3D point cloud representation of the scene and maintains dense 3D-2D correspondences across frames that reflect the geometric scene configuration inferred from the satellite view.
4D Human Body Capture from Egocentric Video via 3D Scene Grounding
We introduce a novel task of reconstructing a time series of second-person 3D human body meshes from monocular egocentric videos.
Self-Supervised Human Depth Estimation from Monocular Videos
Previous methods on estimating detailed human depth often require supervised training with `ground truth' depth data.
OmniSLAM: Omnidirectional Localization and Dense Mapping for Wide-baseline Multi-camera Systems
In this paper, we present an omnidirectional localization and dense mapping system for a wide-baseline multiview stereo setup with ultra-wide field-of-view (FOV) fisheye cameras, which has a 360 degrees coverage of stereo observations of the environment.
Reflection Separation using a Pair of Unpolarized and Polarized Images
When we take photos through glass windows or doors, the transmitted background scene is often blended with undesirable reflection.
DIST: Rendering Deep Implicit Signed Distance Function with Differentiable Sphere Tracing
We propose a differentiable sphere tracing algorithm to bridge the gap between inverse graphics methods and the recently proposed deep learning based implicit signed distance function.
Polarimetric Relative Pose Estimation
In this paper we consider the problem of relative pose estimation from two images with per-pixel polarimetric information.
DeepLiDAR: Deep Surface Normal Guided Depth Prediction for Outdoor Scene from Sparse LiDAR Data and Single Color Image
In this paper, we propose a deep learning architecture that produces accurate dense depth for the outdoor scene from a single color image and a sparse depth.
Efficient 2D-3D Matching for Multi-Camera Visual Localization
This results in a system that provides reliable and drift-less pose estimations for high speed autonomous driving.
Robotics
Polarimetric Dense Monocular SLAM
This paper presents a novel polarimetric dense monocular SLAM (PDMS) algorithm based on a polarization camera.
Polarimetric Multi-View Stereo
Multi-view stereo relies on feature correspondences for 3D reconstruction, and thus is fundamentally flawed in dealing with featureless scenes.
Global Structure-From-Motion by Similarity Averaging
Depth images help to upgrade an essential matrix to a similarity transformation, which can determine the scale of relative translation.
Linear Global Translation Estimation with Feature Tracks
This paper derives a novel linear position constraint for cameras seeing a common scene point, which leads to a direct linear method for global camera translation estimation.
