Search Results for author: Jonathan T. Barron

Found 82 papers, 48 papers with code

Continuously Differentiable Exponential Linear Units

1 code implementation24 Apr 2017 Jonathan T. Barron

Exponential Linear Units (ELUs) are a useful rectifier for constructing deep learning architectures, as they may speed up and otherwise improve learning by virtue of not have vanishing gradients and by having mean activations near zero.

Pushing the Boundaries of View Extrapolation with Multiplane Images

1 code implementation CVPR 2019 Pratul P. Srinivasan, Richard Tucker, Jonathan T. Barron, Ravi Ramamoorthi, Ren Ng, Noah Snavely

We present a theoretical analysis showing how the range of views that can be rendered from an MPI increases linearly with the MPI disparity sampling frequency, as well as a novel MPI prediction procedure that theoretically enables view extrapolations of up to $4\times$ the lateral viewpoint movement allowed by prior work.

What Matters in Unsupervised Optical Flow

5 code implementations ECCV 2020 Rico Jonschkowski, Austin Stone, Jonathan T. Barron, Ariel Gordon, Kurt Konolige, Anelia Angelova

We systematically compare and analyze a set of key components in unsupervised optical flow to identify which photometric loss, occlusion handling, and smoothness regularization is most effective.

Occlusion Handling Optical Flow Estimation

A General and Adaptive Robust Loss Function

3 code implementations CVPR 2019 Jonathan T. Barron

We present a generalization of the Cauchy/Lorentzian, Geman-McClure, Welsch/Leclerc, generalized Charbonnier, Charbonnier/pseudo-Huber/L1-L2, and L2 loss functions.

Clustering Image Generation +1

Learning Single Camera Depth Estimation using Dual-Pixels

1 code implementation ICCV 2019 Rahul Garg, Neal Wadhwa, Sameer Ansari, Jonathan T. Barron

Using our approach, existing monocular depth estimation techniques can be effectively applied to dual-pixel data, and much smaller models can be constructed that still infer high quality depth.

Monocular Depth Estimation

How to Train Neural Networks for Flare Removal

1 code implementation ICCV 2021 Yicheng Wu, Qiurui He, Tianfan Xue, Rahul Garg, Jiawen Chen, Ashok Veeraraghavan, Jonathan T. Barron

When a camera is pointed at a strong light source, the resulting photograph may contain lens flare artifacts.

Flare Removal

Baking Neural Radiance Fields for Real-Time View Synthesis

1 code implementation ICCV 2021 Peter Hedman, Pratul P. Srinivasan, Ben Mildenhall, Jonathan T. Barron, Paul Debevec

Neural volumetric representations such as Neural Radiance Fields (NeRF) have emerged as a compelling technique for learning to represent 3D scenes from images with the goal of rendering photorealistic images of the scene from unobserved viewpoints.

Zero-Shot Text-Guided Object Generation with Dream Fields

4 code implementations CVPR 2022 Ajay Jain, Ben Mildenhall, Jonathan T. Barron, Pieter Abbeel, Ben Poole

Our method, Dream Fields, can generate the geometry and color of a wide range of objects without 3D supervision.

Neural Rendering Object

NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis

36 code implementations ECCV 2020 Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, Ren Ng

Our algorithm represents a scene using a fully-connected (non-convolutional) deep network, whose input is a single continuous 5D coordinate (spatial location $(x, y, z)$ and viewing direction $(\theta, \phi)$) and whose output is the volume density and view-dependent emitted radiance at that spatial location.

Generalizable Novel View Synthesis Low-Dose X-Ray Ct Reconstruction +2

DreamFusion: Text-to-3D using 2D Diffusion

4 code implementations29 Sep 2022 Ben Poole, Ajay Jain, Jonathan T. Barron, Ben Mildenhall

Using this loss in a DeepDream-like procedure, we optimize a randomly-initialized 3D model (a Neural Radiance Field, or NeRF) via gradient descent such that its 2D renderings from random angles achieve a low loss.

Denoising Image Generation +1

NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections

1 code implementation CVPR 2021 Ricardo Martin-Brualla, Noha Radwan, Mehdi S. M. Sajjadi, Jonathan T. Barron, Alexey Dosovitskiy, Daniel Duckworth

We present a learning-based method for synthesizing novel views of complex scenes using only unstructured collections of in-the-wild photographs.

Nerfies: Deformable Neural Radiance Fields

2 code implementations ICCV 2021 Keunhong Park, Utkarsh Sinha, Jonathan T. Barron, Sofien Bouaziz, Dan B Goldman, Steven M. Seitz, Ricardo Martin-Brualla

We present the first method capable of photorealistically reconstructing deformable scenes using photos/videos captured casually from mobile phones.

3D Human Reconstruction

Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields

4 code implementations ICCV 2021 Jonathan T. Barron, Ben Mildenhall, Matthew Tancik, Peter Hedman, Ricardo Martin-Brualla, Pratul P. Srinivasan

Mip-NeRF is also able to match the accuracy of a brute-force supersampled NeRF on our multiscale dataset while being 22x faster.

Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields

1 code implementation CVPR 2022 Jonathan T. Barron, Ben Mildenhall, Dor Verbin, Pratul P. Srinivasan, Peter Hedman

Though neural radiance fields (NeRF) have demonstrated impressive view synthesis results on objects and small bounded regions of space, they struggle on "unbounded" scenes, where the camera may point in any direction and content may exist at any distance.

Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fields

1 code implementation CVPR 2022 Dor Verbin, Peter Hedman, Ben Mildenhall, Todd Zickler, Jonathan T. Barron, Pratul P. Srinivasan

Neural Radiance Fields (NeRF) is a popular view synthesis technique that represents a scene as a continuous volumetric function, parameterized by multilayer perceptrons that provide the volume density and view-dependent emitted radiance at each location.

Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains

13 code implementations NeurIPS 2020 Matthew Tancik, Pratul P. Srinivasan, Ben Mildenhall, Sara Fridovich-Keil, Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan T. Barron, Ren Ng

We show that passing input points through a simple Fourier feature mapping enables a multilayer perceptron (MLP) to learn high-frequency functions in low-dimensional problem domains.

HyperNeRF: A Higher-Dimensional Representation for Topologically Varying Neural Radiance Fields

2 code implementations24 Jun 2021 Keunhong Park, Utkarsh Sinha, Peter Hedman, Jonathan T. Barron, Sofien Bouaziz, Dan B Goldman, Ricardo Martin-Brualla, Steven M. Seitz

A common approach to reconstruct such non-rigid scenes is through the use of a learned deformation field mapping from coordinates in each input image into a canonical template coordinate space.

Novel View Synthesis

Deep Bilateral Learning for Real-Time Image Enhancement

2 code implementations10 Jul 2017 Michaël Gharbi, Jiawen Chen, Jonathan T. Barron, Samuel W. Hasinoff, Frédo Durand

For this, we introduce a new neural network architecture inspired by bilateral grid processing and local affine color transforms.

Image Enhancement Image Retouching

HumanNeRF: Free-viewpoint Rendering of Moving People from Monocular Video

1 code implementation CVPR 2022 Chung-Yi Weng, Brian Curless, Pratul P. Srinivasan, Jonathan T. Barron, Ira Kemelmacher-Shlizerman

Our method optimizes for a volumetric representation of the person in a canonical T-pose, in concert with a motion field that maps the estimated canonical representation to every frame of the video via backward warps.

Zip-NeRF: Anti-Aliased Grid-Based Neural Radiance Fields

1 code implementation ICCV 2023 Jonathan T. Barron, Ben Mildenhall, Dor Verbin, Pratul P. Srinivasan, Peter Hedman

Neural Radiance Field training can be accelerated through the use of grid-based representations in NeRF's learned mapping from spatial coordinates to colors and volumetric density.

Novel View Synthesis

Learned Initializations for Optimizing Coordinate-Based Neural Representations

3 code implementations CVPR 2021 Matthew Tancik, Ben Mildenhall, Terrance Wang, Divi Schmidt, Pratul P. Srinivasan, Jonathan T. Barron, Ren Ng

Coordinate-based neural representations have shown significant promise as an alternative to discrete, array-based representations for complex low dimensional signals.

Meta-Learning

IBRNet: Learning Multi-View Image-Based Rendering

1 code implementation CVPR 2021 Qianqian Wang, Zhicheng Wang, Kyle Genova, Pratul Srinivasan, Howard Zhou, Jonathan T. Barron, Ricardo Martin-Brualla, Noah Snavely, Thomas Funkhouser

Unlike neural scene representation work that optimizes per-scene functions for rendering, we learn a generic view interpolation function that generalizes to novel scenes.

Neural Rendering Novel View Synthesis

Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation

1 code implementation3 Mar 2015 Jordi Pont-Tuset, Pablo Arbelaez, Jonathan T. Barron, Ferran Marques, Jitendra Malik

We propose a unified approach for bottom-up hierarchical image segmentation and object proposal generation for recognition, called Multiscale Combinatorial Grouping (MCG).

Image Segmentation Object +2

Portrait Shadow Manipulation

1 code implementation18 May 2020 Xuaner Cecilia Zhang, Jonathan T. Barron, Yun-Ta Tsai, Rohit Pandey, Xiuming Zhang, Ren Ng, David E. Jacobs

We propose a way to explicitly encode facial symmetry and show that our dataset and training procedure enable the model to generalize to images taken in the wild.

Neural Light Transport for Relighting and View Synthesis

1 code implementation9 Aug 2020 Xiuming Zhang, Sean Fanello, Yun-Ta Tsai, Tiancheng Sun, Tianfan Xue, Rohit Pandey, Sergio Orts-Escolano, Philip Davidson, Christoph Rhemann, Paul Debevec, Jonathan T. Barron, Ravi Ramamoorthi, William T. Freeman

In particular, we show how to fuse previously seen observations of illuminants and views to synthesize a new image of the same scene under a desired lighting condition from a chosen viewpoint.

NeRD: Neural Reflectance Decomposition from Image Collections

1 code implementation ICCV 2021 Mark Boss, Raphael Braun, Varun Jampani, Jonathan T. Barron, Ce Liu, Hendrik P. A. Lensch

This problem is inherently more challenging when the illumination is not a single light source under laboratory conditions but is instead an unconstrained environmental illumination.

Depth Prediction Image Relighting +3

The Fast Bilateral Solver

2 code implementations10 Nov 2015 Jonathan T. Barron, Ben Poole

We present the bilateral solver, a novel algorithm for edge-aware smoothing that combines the flexibility and speed of simple filtering approaches with the accuracy of domain-specific optimization algorithms.

Colorization Semantic Segmentation

Fast Fourier Color Constancy

2 code implementations CVPR 2017 Jonathan T. Barron, Yun-Ta Tsai

We present Fast Fourier Color Constancy (FFCC), a color constancy algorithm which solves illuminant estimation by reducing it to a spatial localization task on a torus.

Color Constancy

INeRF: Inverting Neural Radiance Fields for Pose Estimation

1 code implementation10 Dec 2020 Lin Yen-Chen, Pete Florence, Jonathan T. Barron, Alberto Rodriguez, Phillip Isola, Tsung-Yi Lin

We then show that for complex real-world scenes from the LLFF dataset, iNeRF can improve NeRF by estimating the camera poses of novel images and using these images as additional training data for NeRF.

Object Pose Estimation

A Generalization of Otsu's Method and Minimum Error Thresholding

1 code implementation14 Jul 2020 Jonathan T. Barron

We present Generalized Histogram Thresholding (GHT), a simple, fast, and effective technique for histogram-based image thresholding.

Binarization

A Generalization of Otsu’s Method and Minimum Error Thresholding

1 code implementation ECCV 2020 Jonathan T. Barron

We present Generalized Histogram Thresholding (GHT), a simple, fast, and effective technique for histogram-based image thresholding.

Binarization

Sky Optimization: Semantically aware image processing of skies in low-light photography

1 code implementation15 Jun 2020 Orly Liba, Longqi Cai, Yun-Ta Tsai, Elad Eban, Yair Movshovitz-Attias, Yael Pritch, Huizhong Chen, Jonathan T. Barron

The sky is a major component of the appearance of a photograph, and its color and tone can strongly influence the mood of a picture.

Cross-Camera Convolutional Color Constancy

1 code implementation ICCV 2021 Mahmoud Afifi, Jonathan T. Barron, Chloe LeGendre, Yun-Ta Tsai, Francois Bleibel

We present "Cross-Camera Convolutional Color Constancy" (C5), a learning-based method, trained on images from multiple cameras, that accurately estimates a scene's illuminant color from raw images captured by a new camera previously unseen during training.

Color Constancy

Lighthouse: Predicting Lighting Volumes for Spatially-Coherent Illumination

1 code implementation CVPR 2020 Pratul P. Srinivasan, Ben Mildenhall, Matthew Tancik, Jonathan T. Barron, Richard Tucker, Noah Snavely

We present a deep learning solution for estimating the incident illumination at any 3D location within a scene from an input narrow-baseline stereo image pair.

Lighting Estimation

Polynomial Neural Fields for Subband Decomposition and Manipulation

1 code implementation9 Feb 2023 Guandao Yang, Sagie Benaim, Varun Jampani, Kyle Genova, Jonathan T. Barron, Thomas Funkhouser, Bharath Hariharan, Serge Belongie

We use this framework to design Fourier PNFs, which match state-of-the-art performance in signal representation tasks that use neural fields.

Convolutional Color Constancy

1 code implementation ICCV 2015 Jonathan T. Barron

Color constancy is the problem of inferring the color of the light that illuminated a scene, usually so that the illumination color can be removed.

Color Constancy object-detection +2

Squareplus: A Softplus-Like Algebraic Rectifier

1 code implementation22 Dec 2021 Jonathan T. Barron

We present squareplus, an activation function that resembles softplus, but which can be computed using only algebraic operations: addition, multiplication, and square-root.

Aperture Supervision for Monocular Depth Estimation

no code implementations CVPR 2018 Pratul P. Srinivasan, Rahul Garg, Neal Wadhwa, Ren Ng, Jonathan T. Barron

We present a novel method to train machine learning algorithms to estimate scene depths from a single image, by using the information provided by a camera's aperture as supervision.

Monocular Depth Estimation

Learning to Synthesize Motion Blur

no code implementations CVPR 2019 Tim Brooks, Jonathan T. Barron

We present a technique for synthesizing a motion blurred image from a pair of unblurred images captured in succession.

Stereoscopic Dark Flash for Low-light Photography

no code implementations5 Jan 2019 Jian Wang, Tianfan Xue, Jonathan T. Barron, Jiawen Chen

In this work, we present a camera configuration for acquiring "stereoscopic dark flash" images: a simultaneous stereo pair in which one camera is a conventional RGB sensor, but the other camera is sensitive to near-infrared and near-ultraviolet instead of R and B.

Intrinsic Scene Properties from a Single RGB-D Image

no code implementations CVPR 2013 Jonathan T. Barron, Jitendra Malik

Our model takes as input a single RGB-D image and produces as output an improved depth map, a set of surface normals, a reflectance image, a shading image, and a spatially varying model of illumination.

Multiscale Combinatorial Grouping

no code implementations CVPR 2014 Pablo Arbelaez, Jordi Pont-Tuset, Jonathan T. Barron, Ferran Marques, Jitendra Malik

We propose a unified approach for bottom-up hierarchical image segmentation and object candidate generation for recognition, called Multiscale Combinatorial Grouping (MCG).

Image Segmentation Object +2

Fast Bilateral-Space Stereo for Synthetic Defocus

no code implementations CVPR 2015 Jonathan T. Barron, Andrew Adams, YiChang Shih, Carlos Hernandez

Given a stereo pair it is possible to recover a depth map and use that depth to render a synthetically defocused image.

Handheld Mobile Photography in Very Low Light

no code implementations24 Oct 2019 Orly Liba, Kiran Murthy, Yun-Ta Tsai, Tim Brooks, Tianfan Xue, Nikhil Karnad, Qiurui He, Jonathan T. Barron, Dillon Sharlet, Ryan Geiss, Samuel W. Hasinoff, Yael Pritch, Marc Levoy

Aside from the physical limits imposed by read noise and photon shot noise, these cameras are typically handheld, have small apertures and sensors, use mass-produced analog electronics that cannot easily be cooled, and are commonly used to photograph subjects that move, like children and pets.

Tone Mapping

Boundary Cues for 3D Object Shape Recovery

no code implementations CVPR 2013 Kevin Karsch, Zicheng Liao, Jason Rock, Jonathan T. Barron, Derek Hoiem

Early work in computer vision considered a host of geometric cues for both shape reconstruction and recognition.

Object

Learning to Autofocus

no code implementations CVPR 2020 Charles Herrmann, Richard Strong Bowen, Neal Wadhwa, Rahul Garg, Qiurui He, Jonathan T. Barron, Ramin Zabih

Autofocus is an important task for digital cameras, yet current approaches often exhibit poor performance.

Depth Estimation

Learned Dual-View Reflection Removal

no code implementations1 Oct 2020 Simon Niklaus, Xuaner Cecilia Zhang, Jonathan T. Barron, Neal Wadhwa, Rahul Garg, Feng Liu, Tianfan Xue

Traditional reflection removal algorithms either use a single image as input, which suffers from intrinsic ambiguities, or use multiple images from a moving camera, which is inconvenient for users.

Reflection Removal

Shape, Illumination, and Reflectance from Shading

no code implementations7 Oct 2020 Jonathan T. Barron, Jitendra Malik

A fundamental problem in computer vision is that of inferring the intrinsic, 3D structure of the world from flat, 2D images of that world.

Color Constancy

A Convenient Generalization of Schlick's Bias and Gain Functions

no code implementations17 Oct 2020 Jonathan T. Barron

We present a generalization of Schlick's bias and gain functions -- simple parametric curve-shaped functions for inputs in [0, 1].

NeRV: Neural Reflectance and Visibility Fields for Relighting and View Synthesis

no code implementations CVPR 2021 Pratul P. Srinivasan, Boyang Deng, Xiuming Zhang, Matthew Tancik, Ben Mildenhall, Jonathan T. Barron

We present a method that takes as input a set of images of a scene illuminated by unconstrained known lighting, and produces as output a 3D representation that can be rendered from novel viewpoints under arbitrary lighting conditions.

Scalable Font Reconstruction with Dual Latent Manifolds

no code implementations EMNLP 2021 Nikita Srivatsan, Si Wu, Jonathan T. Barron, Taylor Berg-Kirkpatrick

We propose a deep generative model that performs typography analysis and font reconstruction by learning disentangled manifolds of both font style and character shape.

Style Transfer

Defocus Map Estimation and Deblurring from a Single Dual-Pixel Image

no code implementations ICCV 2021 Shumian Xin, Neal Wadhwa, Tianfan Xue, Jonathan T. Barron, Pratul P. Srinivasan, Jiawen Chen, Ioannis Gkioulekas, Rahul Garg

We use data captured with a consumer smartphone camera to demonstrate that, after a one-time calibration step, our approach improves upon prior works for both defocus map estimation and blur removal, despite being entirely unsupervised.

Deblurring

Urban Radiance Fields

no code implementations CVPR 2022 Konstantinos Rematas, Andrew Liu, Pratul P. Srinivasan, Jonathan T. Barron, Andrea Tagliasacchi, Thomas Funkhouser, Vittorio Ferrari

The goal of this work is to perform 3D reconstruction and novel view synthesis from data captured by scanning platforms commonly deployed for world mapping in urban outdoor environments (e. g., Street View).

3D Reconstruction Novel View Synthesis

RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputs

no code implementations CVPR 2022 Michael Niemeyer, Jonathan T. Barron, Ben Mildenhall, Mehdi S. M. Sajjadi, Andreas Geiger, Noha Radwan

We observe that the majority of artifacts in sparse input scenarios are caused by errors in the estimated scene geometry, and by divergent behavior at the start of training.

Novel View Synthesis

Fast and High-Quality Image Denoising via Malleable Convolutions

no code implementations2 Jan 2022 Yifan Jiang, Bartlomiej Wronski, Ben Mildenhall, Jonathan T. Barron, Zhangyang Wang, Tianfan Xue

These spatially-varying kernels are produced by an efficient predictor network running on a downsampled input, making them much more efficient to compute than per-pixel kernels produced by a full-resolution image, and also enlarging the network's receptive field compared with static kernels.

Image Denoising Image Restoration +1

NeRF-Supervision: Learning Dense Object Descriptors from Neural Radiance Fields

no code implementations3 Mar 2022 Lin Yen-Chen, Pete Florence, Jonathan T. Barron, Tsung-Yi Lin, Alberto Rodriguez, Phillip Isola

In particular, we demonstrate that a NeRF representation of a scene can be used to train dense object descriptors.

MERF: Memory-Efficient Radiance Fields for Real-time View Synthesis in Unbounded Scenes

no code implementations23 Feb 2023 Christian Reiser, Richard Szeliski, Dor Verbin, Pratul P. Srinivasan, Ben Mildenhall, Andreas Geiger, Jonathan T. Barron, Peter Hedman

We design a lossless procedure for baking the parameterization used during training into a model that achieves real-time rendering while still preserving the photorealistic view synthesis quality of a volumetric radiance field.

BakedSDF: Meshing Neural SDFs for Real-Time View Synthesis

no code implementations28 Feb 2023 Lior Yariv, Peter Hedman, Christian Reiser, Dor Verbin, Pratul P. Srinivasan, Richard Szeliski, Jonathan T. Barron, Ben Mildenhall

We present a method for reconstructing high-quality meshes of large unbounded real-world scenes suitable for photorealistic novel view synthesis.

Novel View Synthesis

Learning a Diffusion Prior for NeRFs

no code implementations27 Apr 2023 Guandao Yang, Abhijit Kundu, Leonidas J. Guibas, Jonathan T. Barron, Ben Poole

Neural Radiance Fields (NeRFs) have emerged as a powerful neural 3D representation for objects and scenes derived from 2D data.

Eclipse: Disambiguating Illumination and Materials using Unintended Shadows

no code implementations25 May 2023 Dor Verbin, Ben Mildenhall, Peter Hedman, Jonathan T. Barron, Todd Zickler, Pratul P. Srinivasan

We present a method based on differentiable Monte Carlo ray tracing that uses images of an object to jointly recover its spatially-varying materials, the surrounding illumination environment, and the shapes of the unseen light occluders who inadvertently cast shadows upon it.

Inverse Rendering

CamP: Camera Preconditioning for Neural Radiance Fields

no code implementations21 Aug 2023 Keunhong Park, Philipp Henzler, Ben Mildenhall, Jonathan T. Barron, Ricardo Martin-Brualla

We propose using a proxy problem to compute a whitening transform that eliminates the correlation between camera parameters and normalizes their effects, and we propose to use this transform as a preconditioner for the camera parameters during joint optimization.

State of the Art on Diffusion Models for Visual Computing

no code implementations11 Oct 2023 Ryan Po, Wang Yifan, Vladislav Golyanik, Kfir Aberman, Jonathan T. Barron, Amit H. Bermano, Eric Ryan Chan, Tali Dekel, Aleksander Holynski, Angjoo Kanazawa, C. Karen Liu, Lingjie Liu, Ben Mildenhall, Matthias Nießner, Björn Ommer, Christian Theobalt, Peter Wonka, Gordon Wetzstein

The field of visual computing is rapidly advancing due to the emergence of generative artificial intelligence (AI), which unlocks unprecedented capabilities for the generation, editing, and reconstruction of images, videos, and 3D scenes.

Nuvo: Neural UV Mapping for Unruly 3D Representations

no code implementations11 Dec 2023 Pratul P. Srinivasan, Stephan J. Garbin, Dor Verbin, Jonathan T. Barron, Ben Mildenhall

We present a UV mapping method designed to operate on geometry produced by 3D reconstruction and generation techniques.

3D Reconstruction valid

SMERF: Streamable Memory Efficient Radiance Fields for Real-Time Large-Scene Exploration

no code implementations12 Dec 2023 Daniel Duckworth, Peter Hedman, Christian Reiser, Peter Zhizhin, Jean-François Thibert, Mario Lučić, Richard Szeliski, Jonathan T. Barron

Recent techniques for real-time view synthesis have rapidly advanced in fidelity and speed, and modern methods are capable of rendering near-photorealistic scenes at interactive frame rates.

Novel View Synthesis

SHINOBI: Shape and Illumination using Neural Object Decomposition via BRDF Optimization In-the-wild

no code implementations18 Jan 2024 Andreas Engelhardt, Amit Raj, Mark Boss, Yunzhi Zhang, Abhishek Kar, Yuanzhen Li, Deqing Sun, Ricardo Martin Brualla, Jonathan T. Barron, Hendrik P. A. Lensch, Varun Jampani

We present SHINOBI, an end-to-end framework for the reconstruction of shape, material, and illumination from object images captured with varying lighting, pose, and background.

Inverse Rendering Object

Binary Opacity Grids: Capturing Fine Geometric Detail for Mesh-Based View Synthesis

no code implementations19 Feb 2024 Christian Reiser, Stephan Garbin, Pratul P. Srinivasan, Dor Verbin, Richard Szeliski, Ben Mildenhall, Jonathan T. Barron, Peter Hedman, Andreas Geiger

Third, we minimize the binary entropy of the opacity values, which facilitates the extraction of surface geometry by encouraging opacity values to binarize towards the end of training.

Cannot find the paper you are looking for? You can Submit a new open access paper.