Search Results for author: Gordon Wetzstein

Found 47 papers, 19 papers with code

Generative Neural Articulated Radiance Fields

no code implementations28 Jun 2022 Alexander W. Bergman, Petr Kellnhofer, Yifan Wang, Eric R. Chan, David B. Lindell, Gordon Wetzstein

Unsupervised learning of 3D-aware generative adversarial networks (GANs) using only collections of single-view 2D photographs has very recently made much progress.

Learning to Solve PDE-constrained Inverse Problems with Graph Networks

no code implementations1 Jun 2022 Qingqing Zhao, David B. Lindell, Gordon Wetzstein

Given a sparse set of measurements, we are interested in recovering the initial condition or parameters of the PDE.

Time-multiplexed Neural Holography: A flexible framework for holographic near-eye displays with fast heavily-quantized spatial light modulators

no code implementations5 May 2022 Suyeon Choi, Manu Gopakumar, YiFan, Peng, Jonghyun Kim, Matthew O'Toole, Gordon Wetzstein

Holographic near-eye displays offer unprecedented capabilities for virtual and augmented reality systems, including perceptually important focus cues.

Learning Spatially Varying Pixel Exposures for Motion Deblurring

no code implementations14 Apr 2022 Cindy M. Nguyen, Julien N. P. Martel, Gordon Wetzstein

Computationally removing the motion blur introduced by camera shake or object motion in a captured image remains a challenging task in computational photography.


3D GAN Inversion for Controllable Portrait Image Animation

no code implementations25 Mar 2022 Connor Z. Lin, David B. Lindell, Eric R. Chan, Gordon Wetzstein

Portrait image animation enables the post-capture adjustment of these attributes from a single image while maintaining a photorealistic reconstruction of the subject's likeness or identity.

Image Animation Pose Transfer

CryoAI: Amortized Inference of Poses for Ab Initio Reconstruction of 3D Molecular Volumes from Real Cryo-EM Images

no code implementations15 Mar 2022 Axel Levy, Frédéric Poitevin, Julien Martel, Youssef Nashed, Ariana Peck, Nina Miolane, Daniel Ratner, Mike Dunne, Gordon Wetzstein

We introduce cryoAI, an ab initio reconstruction algorithm for homogeneous conformations that uses direct gradient-based optimization of particle poses and the electron scattering potential from single-particle cryo-EM data.

Efficient Geometry-aware 3D Generative Adversarial Networks

1 code implementation CVPR 2022 Eric R. Chan, Connor Z. Lin, Matthew A. Chan, Koki Nagano, Boxiao Pan, Shalini De Mello, Orazio Gallo, Leonidas Guibas, Jonathan Tremblay, Sameh Khamis, Tero Karras, Gordon Wetzstein

Unsupervised generation of high-quality multi-view-consistent images and 3D shapes using only collections of single-view 2D photographs has been a long-standing challenge.

Neural Rendering

MantissaCam: Learning Snapshot High-dynamic-range Imaging with Perceptually-based In-pixel Irradiance Encoding

no code implementations9 Dec 2021 Haley M. So, Julien N. P. Martel, Piotr Dudek, Gordon Wetzstein

We demonstrate the efficacy of our method in simulation and show benefits of our algorithm on modulo images captured with a prototype implemented with a programmable sensor.

Computer Vision

BACON: Band-limited Coordinate Networks for Multiscale Scene Representation

1 code implementation CVPR 2022 David B. Lindell, Dave Van Veen, Jeong Joon Park, Gordon Wetzstein

These networks are trained to map continuous input coordinates to the value of a signal at each point.

Fast Training of Neural Lumigraph Representations using Meta Learning

no code implementations NeurIPS 2021 Alexander W. Bergman, Petr Kellnhofer, Gordon Wetzstein

Inspired by neural variants of image-based rendering, we develop a new neural rendering approach with the goal of quickly learning a high-quality representation which can also be rendered in real-time.

Computer Vision Meta-Learning +2

ACORN: Adaptive Coordinate Networks for Neural Scene Representation

1 code implementation6 May 2021 Julien N. P. Martel, David B. Lindell, Connor Z. Lin, Eric R. Chan, Marco Monteiro, Gordon Wetzstein

Here, we introduce a new hybrid implicit-explicit network architecture and training strategy that adaptively allocates resources during training and inference based on the local complexity of a signal of interest.

3D Shape Representation Representation Learning

Time-Multiplexed Coded Aperture Imaging: Learned Coded Aperture and Pixel Exposures for Compressive Imaging Systems

no code implementations ICCV 2021 Edwin Vargas, Julien N. P. Martel, Gordon Wetzstein, Henry Arguello

Compressive imaging using coded apertures (CA) is a powerful technique that can be used to recover depth, light fields, hyperspectral images and other quantities from a single snapshot.

ScanGAN360: A Generative Model of Realistic Scanpaths for 360$^{\circ}$ Images

no code implementations25 Mar 2021 Daniel Martin, Ana Serrano, Alexander W. Bergman, Gordon Wetzstein, Belen Masia

Generative adversarial approaches could alleviate this challenge by generating a large number of possible scanpaths for unseen images.

Computer Vision Dynamic Time Warping

D-VDAMP: Denoising-based Approximate Message Passing for Compressive MRI

1 code implementation25 Oct 2020 Christopher A. Metzler, Gordon Wetzstein

Plug and play (P&P) algorithms iteratively apply highly optimized image denoisers to impose priors and solve computational image reconstruction problems, to great effect.

Denoising Image Reconstruction

MetaSDF: Meta-learning Signed Distance Functions

2 code implementations NeurIPS 2020 Vincent Sitzmann, Eric R. Chan, Richard Tucker, Noah Snavely, Gordon Wetzstein

Neural implicit shape representations are an emerging paradigm that offers many potential benefits over conventional discrete representations, including memory efficiency at a high spatial resolution.


Implicit Neural Representations with Periodic Activation Functions

17 code implementations NeurIPS 2020 Vincent Sitzmann, Julien N. P. Martel, Alexander W. Bergman, David B. Lindell, Gordon Wetzstein

However, current network architectures for such implicit neural representations are incapable of modeling signals with fine detail, and fail to represent a signal's spatial and temporal derivatives, despite the fact that these are essential to many physical signals defined implicitly as the solution to partial differential equations.

Image Inpainting

State of the Art on Neural Rendering

no code implementations8 Apr 2020 Ayush Tewari, Ohad Fried, Justus Thies, Vincent Sitzmann, Stephen Lombardi, Kalyan Sunkavalli, Ricardo Martin-Brualla, Tomas Simon, Jason Saragih, Matthias Nießner, Rohit Pandey, Sean Fanello, Gordon Wetzstein, Jun-Yan Zhu, Christian Theobalt, Maneesh Agrawala, Eli Shechtman, Dan B. Goldman, Michael Zollhöfer

Neural rendering is a new and rapidly emerging field that combines generative machine learning techniques with physical knowledge from computer graphics, e. g., by the integration of differentiable rendering into network training.

Computer Vision Image Generation +2

Event Based, Near Eye Gaze Tracking Beyond 10,000Hz

1 code implementation7 Apr 2020 Anastasios N. Angelopoulos, Julien N. P. Martel, Amit P. S. Kohli, Jorg Conradt, Gordon Wetzstein

The cameras in modern gaze-tracking systems suffer from fundamental bandwidth and power limitations, constraining data acquisition speed to 300 Hz realistically.

Semantic Implicit Neural Scene Representations With Semi-Supervised Training

no code implementations28 Mar 2020 Amit Kohli, Vincent Sitzmann, Gordon Wetzstein

The recent success of implicit neural scene representations has presented a viable new method for how we capture and store 3D scenes.

3D Semantic Segmentation Representation Learning

Deep S$^3$PR: Simultaneous Source Separation and Phase Retrieval Using Deep Generative Models

1 code implementation14 Feb 2020 Christopher A. Metzler, Gordon Wetzstein

This paper introduces and solves the simultaneous source separation and phase retrieval (S$^3$PR) problem.

Keyhole Imaging: Non-Line-of-Sight Imaging and Tracking of Moving Objects Along a Single Optical Path

no code implementations13 Dec 2019 Christopher A. Metzler, David B. Lindell, Gordon Wetzstein

Non-line-of-sight (NLOS) imaging and tracking is an emerging technology that allows the shape or position of objects around corners or behind diffusers to be recovered from transient, time-of-flight measurements.

Autonomous Driving

Deep Optics for Single-shot High-dynamic-range Imaging

no code implementations CVPR 2020 Christopher A. Metzler, Hayato Ikoma, Yifan Peng, Gordon Wetzstein

High-dynamic-range (HDR) imaging is crucial for many computer graphics and vision applications.

Deep Optics for Monocular Depth Estimation and 3D Object Detection

no code implementations ICCV 2019 Julie Chang, Gordon Wetzstein

In addition, we train object detection networks on the KITTI dataset and show that the lens optimized for depth estimation also results in improved 3D object detection performance.

3D Object Detection Monocular Depth Estimation +2

LiFF: Light Field Features in Scale and Depth

1 code implementation CVPR 2019 Donald G. Dansereau, Bernd Girod, Gordon Wetzstein

Feature detectors and descriptors are key low-level vision tools that many higher-level tasks build on.

DeepVoxels: Learning Persistent 3D Feature Embeddings

1 code implementation CVPR 2019 Vincent Sitzmann, Justus Thies, Felix Heide, Matthias Nießner, Gordon Wetzstein, Michael Zollhöfer

In this work, we address the lack of 3D understanding of generative neural networks by introducing a persistent 3D feature embedding for view synthesis.

3D Reconstruction Computer Vision +1

Convolutional Sparse Coding for High Dynamic Range Imaging

no code implementations13 Jun 2018 Ana Serrano, Felix Heide, Diego Gutierrez, Gordon Wetzstein, Belen Masia

Current HDR acquisition techniques are based on either (i) fusing multibracketed, low dynamic range (LDR) images, (ii) modifying existing hardware and capturing different exposures simultaneously with multiple sensors, or (iii) reconstructing a single image with spatially-varying pixel exposures.

Reconstructing Transient Images From Single-Photon Sensors

no code implementations CVPR 2017 Matthew O'Toole, Felix Heide, David B. Lindell, Kai Zang, Steven Diamond, Gordon Wetzstein

Computer vision algorithms build on 2D images or 3D videos that capture dynamic events at the millisecond time scale.

Computer Vision

A Wide-Field-Of-View Monocentric Light Field Camera

no code implementations CVPR 2017 Donald G. Dansereau, Glenn Schuster, Joseph Ford, Gordon Wetzstein

Finally, we describe a processing toolchain, including a convenient spherical LF parameterization, and demonstrate depth estimation and post-capture refocus for indoor and outdoor panoramas with 15 x 15 x 1600 x 200 pixels (72 MPix) and a 138-degree FOV.

Computer Vision Depth Estimation

Unrolled Optimization with Deep Priors

2 code implementations22 May 2017 Steven Diamond, Vincent Sitzmann, Felix Heide, Gordon Wetzstein

A broad class of problems at the core of computational imaging, sensing, and low-level computer vision reduces to the inverse problem of extracting latent images that follow a prior distribution, from measurements taken under a known physical image formation model.

Computer Vision Deblurring +1

Snapshot Difference Imaging using Time-of-Flight Sensors

no code implementations19 May 2017 Clara Callenberg, Felix Heide, Gordon Wetzstein, Matthias Hullin

Computational photography encompasses a diversity of imaging techniques, but one of the core operations performed by many of them is to compute image differences.

Dirty Pixels: Towards End-to-End Image Processing and Perception

1 code implementation23 Jan 2017 Steven Diamond, Vincent Sitzmann, Frank Julca-Aguilar, Stephen Boyd, Gordon Wetzstein, Felix Heide

As such, conventional imaging involves processing the RAW sensor measurements in a sequential pipeline of steps, such as demosaicking, denoising, deblurring, tone-mapping and compression.

Autonomous Driving Computer Vision +11

How do people explore virtual environments?

no code implementations13 Dec 2016 Vincent Sitzmann, Ana Serrano, Amy Pavel, Maneesh Agrawala, Diego Gutierrez, Belen Masia, Gordon Wetzstein

Understanding how people explore immersive virtual environments is crucial for many applications, such as designing virtual reality (VR) content, developing new compression algorithms, or learning computational models of saliency or visual attention.

Fast and Flexible Convolutional Sparse Coding

no code implementations CVPR 2015 Felix Heide, Wolfgang Heidrich, Gordon Wetzstein

Convolutional sparse coding (CSC) has become an increasingly important tool in machine learning and computer vision.

Computer Vision General Classification

Cannot find the paper you are looking for? You can Submit a new open access paper.