Search Results for author:
Found 35 papers, 12 papers with code
Real Acoustic Fields: An Audio-Visual Room Acoustics Dataset and Benchmark
The dataset includes high-quality and densely captured room impulse response data paired with multi-view images, and precise 6DoF pose tracking data for sound emitters and listeners in the rooms.
ViewDiff: 3D-Consistent Image Generation with Text-to-Image Models
In this paper, we present a method that leverages pretrained text-to-image models as a prior, and learn to generate multi-view images in a single denoising process from real-world data.
IRIS: Inverse Rendering of Indoor Scenes from Low Dynamic Range Images
While numerous 3D reconstruction and novel-view synthesis methods allow for photorealistic rendering of a scene from multi-view images easily captured with consumer cameras, they bake illumination in their representations and fall short of supporting advanced applications like material editing, relighting, and virtual object insertion.
PlatoNeRF: 3D Reconstruction in Plato's Cave via Single-View Two-Bounce Lidar
3D reconstruction from a single-view is challenging because of the ambiguity from monocular cues and lack of information about occluded regions.
SpecNeRF: Gaussian Directional Encoding for Specular Reflections
We show that our Gaussian directional encoding and geometry prior significantly improve the modeling of challenging specular reflections in neural radiance fields, which helps decompose appearance into more physically meaningful components.
HybridNeRF: Efficient Neural Rendering via Adaptive Volumetric Surfaces
Neural radiance fields provide state-of-the-art view synthesis quality but tend to be slow to render.
PyNeRF: Pyramidal Neural Radiance Fields
Compared to Mip-NeRF, we reduce error rates by 20% while training over 60x faster.
VR-NeRF: High-Fidelity Virtualized Walkable Spaces
We present an end-to-end system for the high-fidelity capture, model reconstruction, and real-time rendering of walkable spaces in virtual reality using neural radiance fields.
Learning Neural Duplex Radiance Fields for Real-Time View Synthesis
Neural radiance fields (NeRFs) enable novel view synthesis with unprecedented visual quality.
HyperReel: High-Fidelity 6-DoF Video with Ray-Conditioned Sampling
Volumetric scene representations enable photorealistic view synthesis for static scenes and form the basis of several existing 6-DoF video techniques.
Ranked #1 on
Novel View Synthesis
on DONeRF: Evaluation Dataset
Neural Fields for Structured Lighting
We present an image formation model and optimization procedure that combines the advantages of neural radiance fields and structured light imaging.
360MonoDepth: High-Resolution 360deg Monocular Depth Estimation
360deg cameras can capture complete environments in a single shot, which makes 360deg imagery alluring in many computer vision tasks.
360° Optical Flow using Tangent Images
Our method leverages gnomonic projection to locally convert ERP images to perspective images, and uniformly samples the ERP image by projection to a cubemap and regular icosahedron vertices, to incrementally refine the estimated 360{\deg} flow fields even in the presence of large rotations.
360MonoDepth: High-Resolution 360° Monocular Depth Estimation
360{\deg} cameras can capture complete environments in a single shot, which makes 360{\deg} imagery alluring in many computer vision tasks.
TöRF: Time-of-Flight Radiance Fields for Dynamic Scene View Synthesis
Neural networks can represent and accurately reconstruct radiance fields for static 3D scenes (e. g., NeRF).
MatryODShka: Real-time 6DoF Video View Synthesis using Multi-Sphere Images
Our approach is to simultaneously learn depth and disocclusions via a multi-sphere image representation, which can be rendered with correct 6DoF disparity and motion parallax in VR.
Combining Task Predictors via Enhancing Joint Predictability
Predictor combination aims to improve a (target) predictor of a learning task based on the (reference) predictors of potentially relevant tasks, without having access to the internals of individual predictors.
BlockGAN: Learning 3D Object-aware Scene Representations from Unlabelled Images
Our experiments show that using explicit 3D features to represent objects allows BlockGAN to learn disentangled representations both in terms of objects (foreground and background) and their properties (pose and identity).
Neural Style-Preserving Visual Dubbing
We present a style-preserving visual dubbing approach from single video inputs, which maintains the signature style of target actors when modifying facial expressions, including mouth motions, to match foreign languages.
Real-Time Global Illumination Decomposition of Videos
We propose the first approach for the decomposition of a monocular color video into direct and indirect illumination components in real time.
HoloGAN: Unsupervised learning of 3D representations from natural images
This shows that HoloGAN is the first generative model that learns 3D representations from natural images in an entirely unsupervised manner.
Unsupervised Attention-guided Image-to-Image Translation
Current unsupervised image-to-image translation techniques struggle to focus their attention on individual objects without altering the background or the way multiple objects interact within a scene.
Unsupervised Attention-guided Image to Image Translation
Current unsupervised image-to-image translation techniques struggle to focus their attention on individual objects without altering the background or the way multiple objects interact within a scene.
Deep Video Portraits
In order to enable source-to-target video re-animation, we render a synthetic target video with the reconstructed head animation parameters from a source video, and feed it into the trained network -- thus taking full control of the target.
LIME: Live Intrinsic Material Estimation
We present the first end to end approach for real time material estimation for general object shapes with uniform material that only requires a single color image as input.
Predictor Combination at Test Time
We present an algorithm for test-time combination of a set of reference predictors with unknown parametric forms.
InverseFaceNet: Deep Monocular Inverse Face Rendering
In contrast, we propose to recover high-quality facial pose, shape, expression, reflectance and illumination using a deep neural network that is trained using a large, synthetically created training corpus.
EgoCap: Egocentric Marker-less Motion Capture with Two Fisheye Cameras (Extended Abstract)
Marker-based and marker-less optical skeletal motion-capture methods use an outside-in arrangement of cameras placed around a scene, with viewpoints converging on the center.
Real-time Halfway Domain Reconstruction of Motion and Geometry
Our extensive experiments and evaluations show that our approach produces high-quality dense reconstructions of 3D geometry and scene flow at real-time frame rates, and compares favorably to the state of the art.
Video Depth-From-Defocus
Many compelling video post-processing effects, in particular aesthetic focus editing and refocusing effects, are feasible if per-frame depth information is available.
EgoCap: Egocentric Marker-less Motion Capture with Two Fisheye Cameras
We therefore propose a new method for real-time, marker-less and egocentric motion capture which estimates the full-body skeleton pose from a lightweight stereo pair of fisheye cameras that are attached to a helmet or virtual reality headset.
Dense Wide-Baseline Scene Flow From Two Handheld Video Cameras
We finally refine the computed correspondence fields in a variational scene flow formulation.
General Automatic Human Shape and Motion Capture Using Volumetric Contour Cues
Our method uses a new image formation model with analytic visibility and analytically differentiable alignment energy.
A Versatile Scene Model with Differentiable Visibility Applied to Generative Pose Estimation
Generative reconstruction methods compute the 3D configuration (such as pose and/or geometry) of a shape by optimizing the overlap of the projected 3D shape model with images.
Megastereo: Constructing High-Resolution Stereo Panoramas
As our first contribution, we describe the necessary correction steps and a compact representation for the input images in order to achieve a highly accurate approximation to the required ray space.
