Search Results for author:
Found 70 papers, 16 papers with code
3D Fluid Flow Reconstruction Using Compact Light Field PIV
In this paper, we present a PIV solution that uses a compact lenslet-based light field camera to track dense particles floating in the fluid and reconstruct the 3D fluid flow.
ProcK: Machine Learning for Knowledge-Intensive Processes
In this work, we introduce ProcK (Process & Knowledge), a novel pipeline to build business process prediction models that take into account both sequential data in the form of event logs and rich semantic information represented in a graph-structured knowledge base.
Underwater 3D Reconstruction Using Light Fields
We also develop a fast algorithm for locating the angular patches in presence of non-linear light paths.
iButter: Neural Interactive Bullet Time Generator for Human Free-viewpoint Rendering
To fill this gap, in this paper we propose a neural interactive bullet-time generator (iButter) for photo-realistic human free-viewpoint rendering from dense RGB streams, which enables flexible and interactive design for human bullet-time visual effects.
Neural Free-Viewpoint Performance Rendering under Complex Human-object Interactions
In this paper, we propose a neural human performance capture and rendering system to generate both high-quality geometry and photo-realistic texture of both human and objects under challenging interaction scenarios in arbitrary novel views, from only sparse RGB streams.
Relightable Neural Video Portrait
With the ability to achieve simultaneous relighting and reenactment, we are able to improve the realism in a variety of virtual production and video rewrite applications.
Few-shot Neural Human Performance Rendering from Sparse RGBD Videos
To fill this gap, in this paper we propose a few-shot neural human rendering approach (FNHR) from only sparse RGBD inputs, which exploits the temporal and spatial redundancy to generate photo-realistic free-view output of human activities.
IREM: High-Resolution Magnetic Resonance (MR) Image Reconstruction via Implicit Neural Representation
For collecting high-quality high-resolution (HR) MR image, we propose a novel image reconstruction network named IREM, which is trained on multiple low-resolution (LR) MR images and achieve an arbitrary up-sampling rate for HR image reconstruction.
PIANO: A Parametric Hand Bone Model from Magnetic Resonance Imaging
Hand modeling is critical for immersive VR/AR, action understanding, or human healthcare.
Editable Free-viewpoint Video Using a Layered Neural Representation
Such layered representation supports fully perception and realistic manipulation of the dynamic scene whilst still supporting a free viewing experience in a wide range.
SportsCap: Monocular 3D Human Motion Capture and Fine-grained Understanding in Challenging Sports Videos
In this paper, we propose SportsCap -- the first approach for simultaneously capturing 3D human motions and understanding fine-grained actions from monocular challenging sports video input.
MirrorNeRF: One-shot Neural Portrait Radiance Field from Multi-mirror Catadioptric Imaging
In this paper, we propose MirrorNeRF - a one-shot neural portrait free-viewpoint rendering approach using a catadioptric imaging system with multiple sphere mirrors and a single high-resolution digital camera, which is the first to combine neural radiance field with catadioptric imaging so as to enable one-shot photo-realistic human portrait reconstruction and rendering, in a low-cost and casual capture setting.
Convolutional Neural Opacity Radiance Fields
In this paper, we propose a novel scheme to generate opacity radiance fields with a convolutional neural renderer for fuzzy objects, which is the first to combine both explicit opacity supervision and convolutional mechanism into the neural radiance field framework so as to enable high-quality appearance and global consistent alpha mattes generation in arbitrary novel views.
Neural Video Portrait Relighting in Real-time via Consistency Modeling
In this paper, we propose a neural approach for real-time, high-quality and coherent video portrait relighting, which jointly models the semantic, temporal and lighting consistency using a new dynamic OLAT dataset.
GNeRF: GAN-based Neural Radiance Field without Posed Camera
We introduce GNeRF, a framework to marry Generative Adversarial Networks (GAN) with Neural Radiance Field (NeRF) reconstruction for the complex scenarios with unknown and even randomly initialized camera poses.
MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereo
We present MVSNeRF, a novel neural rendering approach that can efficiently reconstruct neural radiance fields for view synthesis.
Non-line-of-Sight Imaging via Neural Transient Fields
We present a neural modeling framework for Non-Line-of-Sight (NLOS) imaging.
LGNN: A Context-aware Line Segment Detector
We present a novel real-time line segment detection scheme called Line Graph Neural Network (LGNN).
SofGAN: A Portrait Image Generator with Dynamic Styling
To address this issue, we propose a SofGAN image generator to decouple the latent space of portraits into two subspaces: a geometry space and a texture space.
AutoSweep: Recovering 3D Editable Objectsfrom a Single Photograph
We base our work on the assumption that most human-made objects are constituted by parts and these parts can be well represented by generalized primitives.
OSLNet: Deep Small-Sample Classification with an Orthogonal Softmax Layer
A deep neural network of multiple nonlinear layers forms a large function space, which can easily lead to overfitting when it encounters small-sample data.
Spatial-Angular Interaction for Light Field Image Super-Resolution
Specifically, spatial and angular features are first separately extracted from input LFs, and then repetitively interacted to progressively incorporate spatial and angular information.
A Neural Rendering Framework for Free-Viewpoint Relighting
We present a novel Relightable Neural Renderer (RNR) for simultaneous view synthesis and relighting using multi-view image inputs.
Deep Coarse-to-fine Dense Light Field Reconstruction with Flexible Sampling and Geometry-aware Fusion
Specifically, the coarse sub-aperture image (SAI) synthesis module first explores the scene geometry from an unstructured sparsely-sampled LF and leverages it to independently synthesize novel SAIs, in which a confidence-based blending strategy is proposed to fuse the information from different input SAIs, giving an intermediate densely-sampled LF.
Light Field Super-resolution via Attention-Guided Fusion of Hybrid Lenses
To the best of our knowledge, this is the first end-to-end deep learning method for reconstructing a high-resolution LF image with a hybrid input.
Learning Semantics-aware Distance Map with Semantics Layering Network for Amodal Instance Segmentation
Specifically, we first introduce a new representation, namely a semantics-aware distance map (sem-dist map), to serve as our target for amodal segmentation instead of the commonly used masks and heatmaps.
PIV-Based 3D Fluid Flow Reconstruction Using Light Field Camera
Particle Imaging Velocimetry (PIV) estimates the flow of fluid by analyzing the motion of injected particles.
Non-Lambertian Surface Shape and Reflectance Reconstruction Using Concentric Multi-Spectral Light Field
In this paper, we introduce a novel concentric multi-spectral light field (CMSLF) design that is able to recover the shape and reflectance of surfaces with arbitrary material in one shot.
Generic Multiview Visual Tracking
A natural solution to these challenges is to use multiple cameras with multiview inputs, though existing systems are mostly limited to specific targets (e. g. human), static cameras, and/or camera calibration.
TightCap: 3D Human Shape Capture with Clothing Tightness Field
In this paper, we present TightCap, a data-driven scheme to capture both the human shape and dressed garments accurately with only a single 3D human scan, which enables numerous applications such as virtual try-on, biometrics and body evaluation.
3D Face Reconstruction Using Color Photometric Stereo with Uncalibrated Near Point Lights
We avoid the need for spatial constancy of albedo; instead, we use a new measure for albedo similarity that is based on the albedo norm profile.
Photo-Realistic Facial Details Synthesis from Single Image
Our technique employs expression analysis for proxy face geometry generation and combines supervised and unsupervised learning for facial detail synthesis.
Hair Segmentation on Time-of-Flight RGBD Images
Robust segmentation of hair from portrait images remains challenging: hair does not conform to a uniform shape, style or even color; dark hair in particular lacks features.
Deep Surface Light Fields
A surface light field represents the radiance of rays originating from any points on the surface in any directions.
4D Human Body Correspondences from Panoramic Depth Maps
The availability of affordable 3D full body reconstruction systems has given rise to free-viewpoint video (FVV) of human shapes.
Learning to Dodge A Bullet: Concyclic View Morphing via Deep Learning
The bullet-time effect, presented in feature film ``The Matrix", has been widely adopted in feature films and TV commercials to create an amazing stopping-time illusion.
Saliency Detection in 360° Videos
Considering that the 360° videos are usually stored with equirectangular panorama, we propose to implement the spherical convolution on panorama by stretching and rotating the kernel based on the location of patch to be convolved.
A Generic Multi-Projection-Center Model and Calibration Method for Light Field Cameras
Based on the MPC model and projective transformation, we propose a calibration algorithm to verify our light field camera model.
Automatic 3D Indoor Scene Modeling From Single Panorama
The recovered layout is then used to guide shape estimation of the remaining objects using their normal information.
Focus Manipulation Detection via Photometric Histogram Analysis
With the rise of misinformation spread via social media channels, enabled by the increasing automation and realism of image manipulation tools, image forensics is an increasingly relevant problem.
Gaze Prediction in Dynamic 360° Immersive Videos
This paper explores gaze prediction in dynamic $360^circ$ immersive videos, emph{i. e.}, based on the history scan path and VR contents, we predict where a viewer will look at an upcoming time.
Semantic See-Through Rendering on Light Fields
We present a novel semantic light field (LF) refocusing technique that can achieve unprecedented see-through quality.
Robust 3D Human Motion Reconstruction Via Dynamic Template Construction
In multi-view human body capture systems, the recovered 3D geometry or even the acquired imagery data can be heavily corrupted due to occlusions, noise, limited field of- view, etc.
Deep Eyes: Binocular Depth-from-Focus on Focal Stack Pairs
We then construct three individual networks: a Focus-Net to extract depth from a single focal stack, a EDoF-Net to obtain the extended depth of field (EDoF) image from the focal stack, and a Stereo-Net to conduct stereo matching.
Sparse Photometric 3D Face Reconstruction Guided by Morphable Models
We present a novel 3D face reconstruction technique that leverages sparse photometric stereo (PS) and latest advances on face registration/modeling from a single image.
Large-Scale 3D Shape Reconstruction and Segmentation from ShapeNet Core55
We introduce a large-scale 3D shape understanding benchmark using data and annotation from ShapeNet 3D object database.
Personalized Saliency and its Prediction
Specifically, we propose to decompose a personalized saliency map (referred to as PSM) into a universal saliency map (referred to as USM) predictable by existing saliency detection models and a new discrepancy map across users that characterizes personalized saliency.
Catadioptric HyperSpectral Light Field Imaging
In this paper, we present a single camera hyperspectral light field imaging solution that we call Snapshot Plenoptic Imager (SPI).
Ray Space Features for Plenoptic Structure-From-Motion
In this paper, we explore using light fields captured by plenoptic cameras or camera arrays as inputs.
A Learning-based Framework for Hybrid Depth-from-Defocus and Stereo Matching
Depth from defocus (DfD) and stereo matching are two most studied passive depth sensing schemes.
Image Splicing Detection via Camera Response Function Analysis
Recent advances on image manipulation techniques have made image forgery detection increasingly more challenging.
Robust Guided Image Filtering
In this paper, we propose a general framework for Robust Guided Image Filtering (RGIF), which contains a data term and a smoothness term, to solve the two issues mentioned above.
Occlusion-Model Guided Anti-Occlusion Depth Estimation in Light Field
Occlusion is one of the most challenging problems in depth estimation.
Rotational Crossed-Slit Light Field
In this paper, we present a novel LF sampling scheme by exploiting a special non-centric camera called the crossed-slit or XSlit camera.
Depth Recovery From Light Field Using Focal Stack Symmetry
The other is a data consistency measure based on analysis-by-synthesis, i. e., the difference between the synthesized focal stack given the hypothesized depth map and that from the LF.
Scene-adaptive Coded Apertures Imaging
Firstly, we capture the scene with a pinhole and analyze the scene content to determine primary edge orientations.
Robust High Quality Image Guided Depth Upsampling
A popular solution is upsampling the obtained noisy low resolution depth map with the guidance of the companion high resolution color image.
Automatic Layer Separation using Light Field Imaging
We propose a novel approach that jointly removes reflection or translucent layer from a scene and estimates scene depth.
Resolving Scale Ambiguity Via XSlit Aspect Ratio Analysis
We first conduct a comprehensive analysis to characterize DDAR, infer object depth from its AR, and model recoverable depth range, sensitivity, and error.
A Weighted Sparse Coding Framework for Saliency Detection
In this paper, we present a unified saliency detection framework for handling heterogenous types of input data.
Ambient Occlusion via Compressive Visibility Estimation
This enables a sparsity-prior based solution for iteratively recovering the surface normal, the surface albedo, and the visibility function from a small number of images.
Aliasing Detection and Reduction in Plenoptic Imaging
When using plenoptic camera for digital refocusing, angular undersampling can cause severe (angular) aliasing artifacts.
Curvilinear Structure Tracking by Low Rank Tensor Approximation with Model Propagation
Robust tracking of deformable object like catheter or vascular structures in X-ray images is an important technique used in image guided medical interventions for effective motion compensation and dynamic multi-modality image fusion.
Image Pre-compensation: Balancing Contrast and Ringing
In particular, we show that linear tone mapping eliminates ringing but incurs severe contrast loss, while non-linear tone mapping functions such as Gamma curves slightly enhances contrast but introduces ringing.
Light Field Stereo Matching Using Bilateral Statistics of Surface Cameras
Our bilateral consistency metric is used to indicate the probability of occlusions by analyzing the SCams.
Saliency Detection on Light Field
Existing saliency detection approaches use images as inputs and are sensitive to foreground/background similarities, complex background textures, and occlusions.
Fast Patch-Based Denoising Using Approximated Patch Geodesic Paths
PatchGPs treat image patches as nodes and patch differences as edge weights for computing the shortest (geodesic) paths.
Manhattan Scene Understanding via XSlit Imaging
Specifically, we prove that parallel 3D lines map to 2D curves in an XSlit image and they converge at an XSlit Vanishing Point (XVP).
Reconstructing Gas Flows Using Light-Path Approximation
By observing the LF-probe through the gas flow, we acquire a dense set of ray-ray correspondences and then reconstruct their light paths.