Search Results for author:
Found 71 papers, 25 papers with code
Disambiguating Monocular Depth Estimation with a Single Transient
Monocular depth estimation algorithms successfully predict the relative depth order of objects in a scene.
Instant Continual Learning of Neural Radiance Fields
Neural radiance fields (NeRFs) have emerged as an effective method for novel-view synthesis and 3D scene reconstruction.
PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Tracking
Our goal is to advance the state-of-the-art by placing emphasis on long videos with naturalistic motion.
Efficient 3D Articulated Human Generation with Layered Surface Volumes
These layers are rendered using alpha compositing with fast differentiable rasterization, and they can be interpreted as a volumetric representation that allocates its capacity to a manifold of finite thickness around the template.
Articulated 3D Head Avatar Generation using Text-to-Image Diffusion Models
Recent work on text-guided 3D object generation has shown great promise in addressing these needs.
Single-Shot Implicit Morphable Faces with Consistent Texture Parameterization
To tackle this problem, we propose a novel method for constructing implicit 3D morphable face models that are both generalizable and intuitive for editing.
Learning Controllable Adaptive Simulation for Multi-resolution Physics
In this work, we introduce Learning controllable Adaptive simulation for Multi-resolution Physics (LAMP) as the first full deep learning-based surrogate model that jointly learns the evolution model and optimizes appropriate spatial resolutions that devote more compute to the highly dynamic regions.
LumiGAN: Unconditional Generation of Relightable 3D Human Faces
Unsupervised learning of 3D human faces from unstructured 2D image data is an active research area.
PixelRNN: In-pixel Recurrent Neural Networks for End-to-end-optimized Perception with Neural Sensors
Conventional image sensors digitize high-resolution images at fast frame rates, producing a large amount of data that needs to be transmitted off the sensor for further processing.
Generative Novel View Synthesis with 3D-Aware Diffusion Models
We present a diffusion-based model for 3D-aware generative novel view synthesis from as few as a single input image.
CC3D: Layout-Conditioned Generation of Compositional 3D Scenes
In this work, we introduce CC3D, a conditional generative model that synthesizes complex 3D scenes conditioned on 2D semantic scene layouts, trained using single-view images.
Compositional 3D Scene Generation using Locally Conditioned Diffusion
Designing complex 3D scenes has been a tedious, manual process requiring domain expertise.
DehazeNeRF: Multiple Image Haze Removal and 3D Shape Reconstruction using Neural Radiance Fields
Neural radiance fields (NeRFs) have demonstrated state-of-the-art performance for 3D computer vision tasks, including novel view synthesis and 3D shape reconstruction.
MELON: NeRF with Unposed Images in SO(3)
Neural radiance fields enable novel-view synthesis and scene reconstruction with photorealistic quality from a few images, but require known and accurate camera poses.
Diffusion in the Dark: A Diffusion Model for Low-Light Text Recognition
Images are indispensable for the automation of high-level tasks, such as text recognition.
PaletteNeRF: Palette-based Appearance Editing of Neural Radiance Fields
Recent advances in neural radiance fields have enabled the high-fidelity 3D reconstruction of complex scenes for novel view synthesis.
PointAvatar: Deformable Point-based Head Avatars from Videos
The ability to create realistic, animatable and relightable head avatars from casual video sequences would open up wide ranging applications in communication and entertainment.
ALTO: Alternating Latent Topologies for Implicit 3D Reconstruction
One school of thought is to encode a latent vector for each point (point latents).
SinGRAF: Learning a 3D Generative Radiance Field for a Single Scene
Generative models have shown great promise in synthesizing photorealistic 3D objects, but they require large amounts of training data.
3D Neural Field Generation using Triplane Diffusion
Diffusion models have emerged as the state-of-the-art for image generation, among other tasks.
DiffDreamer: Towards Consistent Unsupervised Single-view Scene Extrapolation with Conditional Diffusion Models
Scene extrapolation -- the idea of generating novel views by flying into a given image -- is a promising, yet challenging task.
Ranked #1 on
Perpetual View Generation
on LHQ
Scale-Agnostic Super-Resolution in MRI using Feature-Based Coordinate Networks
We propose using a coordinate network decoder for the task of super-resolution in MRI.
Amortized Inference for Heterogeneous Reconstruction in Cryo-EM
Cryo-electron microscopy (cryo-EM) is an imaging modality that provides unique insights into the dynamics of proteins and other building blocks of life.
Heterogeneous reconstruction of deformable atomic models in Cryo-EM
Cryogenic electron microscopy (cryo-EM) provides a unique opportunity to study the structural heterogeneity of biomolecules.
NeuForm: Adaptive Overfitting for Neural Shape Editing
Neural representations are popular for representing shapes, as they can be learned form sensor data and used for data cleanup, model completion, shape editing, and shape synthesis.
3D-Aware Video Generation
Generative models have emerged as an essential building block for many image synthesis and editing tasks.
Generative Neural Articulated Radiance Fields
Unsupervised learning of 3D-aware generative adversarial networks (GANs) using only collections of single-view 2D photographs has very recently made much progress.
Learning to Solve PDE-constrained Inverse Problems with Graph Networks
Given a sparse set of measurements, we are interested in recovering the initial condition or parameters of the PDE.
Time-multiplexed Neural Holography: A flexible framework for holographic near-eye displays with fast heavily-quantized spatial light modulators
Holographic near-eye displays offer unprecedented capabilities for virtual and augmented reality systems, including perceptually important focus cues.
Learning Spatially Varying Pixel Exposures for Motion Deblurring
Computationally removing the motion blur introduced by camera shake or object motion in a captured image remains a challenging task in computational photography.
3D GAN Inversion for Controllable Portrait Image Animation
Portrait image animation enables the post-capture adjustment of these attributes from a single image while maintaining a photorealistic reconstruction of the subject's likeness or identity.
CryoAI: Amortized Inference of Poses for Ab Initio Reconstruction of 3D Molecular Volumes from Real Cryo-EM Images
We introduce cryoAI, an ab initio reconstruction algorithm for homogeneous conformations that uses direct gradient-based optimization of particle poses and the electron scattering potential from single-particle cryo-EM data.
Efficient Geometry-aware 3D Generative Adversarial Networks
Unsupervised generation of high-quality multi-view-consistent images and 3D shapes using only collections of single-view 2D photographs has been a long-standing challenge.
BACON: Band-limited Coordinate Networks for Multiscale Scene Representation
These networks are trained to map continuous input coordinates to the value of a signal at each point.
MantissaCam: Learning Snapshot High-dynamic-range Imaging with Perceptually-based In-pixel Irradiance Encoding
We demonstrate the efficacy of our method in simulation and show benefits of our algorithm on modulo images captured with a prototype implemented with a programmable sensor.
Advances in Neural Rendering
The reconstruction of such a scene representation from observations using differentiable rendering losses is known as inverse graphics or inverse rendering.
Fast Training of Neural Lumigraph Representations using Meta Learning
Inspired by neural variants of image-based rendering, we develop a new neural rendering approach with the goal of quickly learning a high-quality representation which can also be rendered in real-time.
ACORN: Adaptive Coordinate Networks for Neural Scene Representation
Here, we introduce a new hybrid implicit-explicit network architecture and training strategy that adaptively allocates resources during training and inference based on the local complexity of a signal of interest.
Time-Multiplexed Coded Aperture Imaging: Learned Coded Aperture and Pixel Exposures for Compressive Imaging Systems
Compressive imaging using coded apertures (CA) is a powerful technique that can be used to recover depth, light fields, hyperspectral images and other quantities from a single snapshot.
ScanGAN360: A Generative Model of Realistic Scanpaths for 360$^{\circ}$ Images
Generative adversarial approaches could alleviate this challenge by generating a large number of possible scanpaths for unseen images.
Neural Lumigraph Rendering
Novel view synthesis is a challenging and ill-posed inverse rendering problem.
AutoInt: Automatic Integration for Fast Neural Volume Rendering
For training, we instantiate the computational graph corresponding to the derivative of the network.
pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis
We have witnessed rapid progress on 3D-aware image synthesis, leveraging recent advances in generative visual models and neural rendering.
Ranked #3 on
Scene Generation
on VizDoom
D-VDAMP: Denoising-based Approximate Message Passing for Compressive MRI
Plug and play (P&P) algorithms iteratively apply highly optimized image denoisers to impose priors and solve computational image reconstruction problems, to great effect.
SUREMap: Predicting Uncertainty in CNN-based Image Reconstruction Using Stein's Unbiased Risk Estimate
Convolutional neural networks (CNN) have emerged as a powerful tool for solving computational imaging reconstruction problems.
Single-shot Hyperspectral-Depth Imaging with Learned Diffractive Optics
Imaging depth and spectrum have been extensively studied in isolation from each other for decades.
MetaSDF: Meta-learning Signed Distance Functions
Neural implicit shape representations are an emerging paradigm that offers many potential benefits over conventional discrete representations, including memory efficiency at a high spatial resolution.
Implicit Neural Representations with Periodic Activation Functions
However, current network architectures for such implicit neural representations are incapable of modeling signals with fine detail, and fail to represent a signal's spatial and temporal derivatives, despite the fact that these are essential to many physical signals defined implicitly as the solution to partial differential equations.
State of the Art on Neural Rendering
Neural rendering is a new and rapidly emerging field that combines generative machine learning techniques with physical knowledge from computer graphics, e. g., by the integration of differentiable rendering into network training.
Event Based, Near Eye Gaze Tracking Beyond 10,000Hz
The cameras in modern gaze-tracking systems suffer from fundamental bandwidth and power limitations, constraining data acquisition speed to 300 Hz realistically.
Semantic Implicit Neural Scene Representations With Semi-Supervised Training
The recent success of implicit neural scene representations has presented a viable new method for how we capture and store 3D scenes.
Deep S$^3$PR: Simultaneous Source Separation and Phase Retrieval Using Deep Generative Models
This paper introduces and solves the simultaneous source separation and phase retrieval (S$^3$PR) problem.
Keyhole Imaging: Non-Line-of-Sight Imaging and Tracking of Moving Objects Along a Single Optical Path
Non-line-of-sight (NLOS) imaging and tracking is an emerging technology that allows the shape or position of objects around corners or behind diffusers to be recovered from transient, time-of-flight measurements.
Deep Optics for Single-shot High-dynamic-range Imaging
High-dynamic-range (HDR) imaging is crucial for many computer graphics and vision applications.
Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations
Unsupervised learning with generative models has the potential of discovering rich representations of 3D scenes.
Deep Optics for Monocular Depth Estimation and 3D Object Detection
In addition, we train object detection networks on the KITTI dataset and show that the lens optimized for depth estimation also results in improved 3D object detection performance.
Ranked #10 on
Depth Estimation
on NYU-Depth V2
LiFF: Light Field Features in Scale and Depth
Feature detectors and descriptors are key low-level vision tools that many higher-level tasks build on.
DeepVoxels: Learning Persistent 3D Feature Embeddings
In this work, we address the lack of 3D understanding of generative neural networks by introducing a persistent 3D feature embedding for view synthesis.
Convolutional Sparse Coding for High Dynamic Range Imaging
Current HDR acquisition techniques are based on either (i) fusing multibracketed, low dynamic range (LDR) images, (ii) modifying existing hardware and capturing different exposures simultaneously with multiple sensors, or (iii) reconstructing a single image with spatially-varying pixel exposures.
Deep End-to-End Time-of-Flight Imaging
We present an end-to-end image processing framework for time-of-flight (ToF) cameras.
Non-line-of-sight Imaging with Partial Occluders and Surface Normals
Imaging objects obscured by occluders is a significant challenge for many applications.
Consensus Convolutional Sparse Coding
Convolutional sparse coding (CSC) is a promising direction for unsupervised learning in computer vision.
A Wide-Field-Of-View Monocentric Light Field Camera
Finally, we describe a processing toolchain, including a convenient spherical LF parameterization, and demonstrate depth estimation and post-capture refocus for indoor and outdoor panoramas with 15 x 15 x 1600 x 200 pixels (72 MPix) and a 138-degree FOV.
Reconstructing Transient Images From Single-Photon Sensors
Computer vision algorithms build on 2D images or 3D videos that capture dynamic events at the millisecond time scale.
Unrolled Optimization with Deep Priors
A broad class of problems at the core of computational imaging, sensing, and low-level computer vision reduces to the inverse problem of extracting latent images that follow a prior distribution, from measurements taken under a known physical image formation model.
Snapshot Difference Imaging using Time-of-Flight Sensors
Computational photography encompasses a diversity of imaging techniques, but one of the core operations performed by many of them is to compute image differences.
Dirty Pixels: Towards End-to-End Image Processing and Perception
As such, conventional imaging involves processing the RAW sensor measurements in a sequential pipeline of steps, such as demosaicking, denoising, deblurring, tone-mapping and compression.
How do people explore virtual environments?
Understanding how people explore immersive virtual environments is crucial for many applications, such as designing virtual reality (VR) content, developing new compression algorithms, or learning computational models of saliency or visual attention.
Variable Aperture Light Field Photography: Overcoming the Diffraction-Limited Spatio-Angular Resolution Tradeoff
Light fields have many applications in machine vision, consumer photography, robotics, and microscopy.
Fast and Flexible Convolutional Sparse Coding
Convolutional sparse coding (CSC) has become an increasingly important tool in machine learning and computer vision.
Transparent Object Reconstruction via Coded Transport of Intensity
Capturing and understanding visual signals is one of the core interests of computer vision.
