Search Results for author:
Found 46 papers, 19 papers with code
Du²Net: Learning Depth Estimation from Dual-Cameras and Dual-Pixels
Computational stereo has reached a high level of accuracy, but degrades in the presence of occlusions, repeated textures, and correspondence errors along edges.
Learning Versatile 3D Shape Generation with Improved AR Models
Auto-Regressive (AR) models have achieved impressive results in 2D image generation by modeling joint distributions in the grid space.
SINE: Semantic-driven Image-based NeRF Editing with Prior-guided Editing Field
Despite the great success in 2D editing using user-friendly tools, such as Photoshop, semantic strokes, or even text prompts, similar capabilities in 3D areas are still limited, either relying on 3D modeling skills or allowing editing within only a few categories.
DELTAR: Depth Estimation from a Light-weight ToF Sensor and RGB Image
Light-weight time-of-flight (ToF) depth sensors are small, cheap, low-energy and have been massively deployed on mobile devices for the purposes like autofocus, obstacle detection, etc.
LoRD: Local 4D Implicit Representation for High-Fidelity Dynamic Human Modeling
Recent progress in 4D implicit representation focuses on globally controlling the shape and motion with low dimensional latent vectors, which is prone to missing surface details and accumulating tracking error.
PRIF: Primary Ray-based Implicit Function
We introduce a new implicit shape representation called Primary Ray-based Implicit Function (PRIF).
NeuMesh: Learning Disentangled Neural Mesh-based Implicit Field for Geometry and Texture Editing
Very recently neural implicit rendering techniques have been rapidly evolved and shown great advantages in novel view synthesis and 3D scene reconstruction.
Neural Rendering in a Room: Amodal 3D Understanding and Free-Viewpoint Rendering for the Closed Scene Composed of Pre-Captured Objects
We, as human beings, can understand and picture a familiar scene from arbitrary viewpoints given a single image, whereas this is still a grand challenge for computers.
Density-preserving Deep Point Cloud Compression
To address this, we propose a novel deep point cloud compression method that preserves local density information.
Pixel2Mesh++: 3D Mesh Generation and Refinement from Multi-View Images
We study the problem of shape generation in 3D mesh representation from a small number of color images with or without camera poses.
Efficient Virtual View Selection for 3D Hand Pose Estimation
3D hand pose estimation from single depth is a fundamental problem in computer vision, and has wide applications. However, the existing methods still can not achieve satisfactory hand pose estimation results due to view variation and occlusion of human hand.
Ranked #1 on
Hand Pose Estimation
on ICVL Hands
H4D: Human 4D Modeling by Learning Neural Compositional Representation
A simple yet effective linear motion model is proposed to provide a rough and regularized motion estimation, followed by per-frame compensation for pose and geometry details with the residual encoded in the auxiliary code.
OmniSyn: Synthesizing 360 Videos with Wide-baseline Panoramas
Immersive maps such as Google Street View and Bing Streetside provide true-to-life views with a massive collection of panoramas.
VoLux-GAN: A Generative Model for 3D Face Synthesis with HDRI Relighting
We propose VoLux-GAN, a generative framework to synthesize 3D-aware faces with convincing relighting.
Multiresolution Deep Implicit Functions for 3D Shape Representation
To the best of our knowledge, MDIF is the first deep implicit function model that can at the same time (1) represent different levels of detail and allow progressive decoding; (2) support both encoder-decoder inference and decoder-only latent optimization, and fulfill multiple applications; (3) perform detailed decoder-only shape completion.
Learning Object-Compositional Neural Radiance Field for Editable Scene Rendering
In this paper, we present a novel neural scene rendering system, which learns an object-compositional neural radiance field and produces realistic rendering with editing capability for a clustered and real-world scene.
DeepPanoContext: Panoramic 3D Scene Understanding with Holistic Scene Context Graph and Relation-based Optimization
Panorama images have a much larger field-of-view thus naturally encode enriched scene context information compared to standard perspective images, which however is not well exploited in the previous scene understanding methods.
Deep Hybrid Self-Prior for Full 3D Mesh Generation
We present a deep learning pipeline that leverages network self-prior to recover a full 3D model consisting of both a triangular mesh and a texture map from the colored 3D point cloud.
Spatially-Varying Outdoor Lighting Estimation from Intrinsics
We train a deep neural network to regress intrinsic cues with physically-based constraints and use them to conduct global and local lightings estimation.
HumanGPS: Geodesic PreServing Feature for Dense Human Correspondences
In this paper, we address the problem of building dense correspondences between human images under arbitrary camera viewpoints and body poses.
Learning Compositional Representation for 4D Captures with Neural ODE
To model the motion, a neural Ordinary Differential Equation (ODE) is trained to update the initial state conditioned on the learned motion code, and a decoder takes the shape code and the updated state code to reconstruct the 3D model at each time stamp.
Holistic 3D Scene Understanding from a Single Image with Implicit Representation
We not only propose an image-based local structured implicit network to improve the object shape estimation, but also refine the 3D object pose and scene layout via a novel implicit scene graph neural network that exploits the implicit local object features.
Ranked #1 on
3D Shape Reconstruction
on Pix3D
Interacting Two-Hand 3D Pose and Shape Reconstruction From Single Color Image
In this paper, we propose a novel deep learning framework to reconstruct 3D hand poses and shapes of two interacting hands from a single color image.
GeoLayout: Geometry Driven Room Layout Estimation Based on Depth Maps of Planes
The task of room layout estimation is to locate the wall-floor, wall-ceiling, and wall-wall boundaries.
HITNet: Hierarchical Iterative Tile Refinement Network for Real-time Stereo Matching
Contrary to many recent neural network approaches that operate on a full cost volume and rely on 3D convolutions, our approach does not explicitly build a volume and instead relies on a fast multi-resolution initialization step, differentiable 2D geometric propagation and warping mechanisms to infer disparity hypotheses.
Ranked #1 on
Stereo Depth Estimation
on KITTI2015
Deep Implicit Volume Compression
We describe a novel approach for compressing truncated signed distance fields (TSDF) stored in 3D voxel grids, and their corresponding textures.
Du$^2$Net: Learning Depth Estimation from Dual-Cameras and Dual-Pixels
Computational stereo has reached a high level of accuracy, but degrades in the presence of occlusions, repeated textures, and correspondence errors along edges.
Neural Pose Transfer by Spatially Adaptive Instance Normalization
Pose transfer has been studied for decades, in which the pose of a source mesh is applied to a target mesh.
DeepSFM: Structure From Motion Via Deep Bundle Adjustment
The explicit constraints on both depth (structure) and pose (motion), when combined with the learning components, bring the merit from both traditional BA and emerging deep learning technology.
Neural Point Cloud Rendering via Multi-Plane Projection
The input to the network is the raw point cloud of a scene and the output are image or image sequences from a novel view or along a novel camera trajectory.
DIST: Rendering Deep Implicit Signed Distance Function with Differentiable Sphere Tracing
We propose a differentiable sphere tracing algorithm to bridge the gap between inverse graphics methods and the recently proposed deep learning based implicit signed distance function.
Pixel2Mesh++: Multi-View 3D Mesh Generation via Deformation
We study the problem of shape generation in 3D mesh representation from a few color images with known camera poses.
DeepLiDAR: Deep Surface Normal Guided Depth Prediction for Outdoor Scene from Sparse LiDAR Data and Single Color Image
In this paper, we propose a deep learning architecture that produces accurate dense depth for the outdoor scene from a single color image and a sparse depth.
ActiveStereoNet: End-to-End Self-Supervised Learning for Active Stereo Systems
In this paper we present ActiveStereoNet, the first deep learning solution for active stereo systems.
Multi-level Semantic Feature Augmentation for One-shot Learning
In semantic space, we search for related concepts, which are then projected back into the image feature spaces by the decoder portion of the TriNet.
Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images
We propose an end-to-end deep learning architecture that produces a 3D shape in triangular mesh from a single color image.
Ranked #3 on
3D Object Reconstruction
on Data3D−R2N2
(Avg F1 metric)
Deep Depth Completion of a Single RGB-D Image
The goal of our work is to complete the depth channel of an RGB-D image.
VOCABULARY-INFORMED VISUAL FEATURE AUGMENTATION FOR ONE-SHOT LEARNING
Besides directly augmenting image features, we transform the image features to semantic space using the encoder and perform the data augmentation.
Matterport3D: Learning from RGB-D Data in Indoor Environments
Access to large, diverse RGB-D datasets is critical for training RGB-D scene understanding algorithms.
Hand3D: Hand Pose Estimation using 3D Neural Network
We propose a novel 3D neural network architecture for 3D hand pose estimation from a single depth image.
Physically-Based Rendering for Indoor Scene Understanding Using Convolutional Neural Networks
One of the bottlenecks in training for better representations is the amount of available per-pixel ground truth data that is required for core scene understanding tasks such as semantic segmentation, normal prediction, and object edge detection.
Joint Hand Detection and Rotation Estimation by Using CNN
Hand detection is essential for many hand related tasks, e. g. parsing hand pose, understanding gesture, which are extremely useful for robotics and human-computer interaction.
DeepContext: Context-Encoding Neural Pathways for 3D Holistic Scene Understanding
In particular, 3D context has been shown to be an extremely important cue for scene understanding - yet very little research has been done on integrating context information with deep models.
LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop
While there has been remarkable progress in the performance of visual recognition algorithms, the state-of-the-art models tend to be exceptionally data-hungry.
TurkerGaze: Crowdsourcing Saliency with Webcam based Eye Tracking
Traditional eye tracking requires specialized hardware, which means collecting gaze data from many observers is expensive, tedious and slow.
FrameBreak: Dramatic Image Extrapolation by Guided Shift-Maps
We analyze the self-similarity of the guide image to generate a set of allowable local transformations and apply them to the input image.
