Search Results for author:
Found 90 papers, 30 papers with code
Transparent Object Reconstruction via Coded Transport of Intensity
Capturing and understanding visual signals is one of the core interests of computer vision.
Fast and Flexible Convolutional Sparse Coding
Convolutional sparse coding (CSC) has become an increasingly important tool in machine learning and computer vision.
Variable Aperture Light Field Photography: Overcoming the Diffraction-Limited Spatio-Angular Resolution Tradeoff
Light fields have many applications in machine vision, consumer photography, robotics, and microscopy.
How do people explore virtual environments?
Understanding how people explore immersive virtual environments is crucial for many applications, such as designing virtual reality (VR) content, developing new compression algorithms, or learning computational models of saliency or visual attention.
Dirty Pixels: Towards End-to-End Image Processing and Perception
As such, conventional imaging involves processing the RAW sensor measurements in a sequential pipeline of steps, such as demosaicking, denoising, deblurring, tone-mapping and compression.
Snapshot Difference Imaging using Time-of-Flight Sensors
Computational photography encompasses a diversity of imaging techniques, but one of the core operations performed by many of them is to compute image differences.
Unrolled Optimization with Deep Priors
A broad class of problems at the core of computational imaging, sensing, and low-level computer vision reduces to the inverse problem of extracting latent images that follow a prior distribution, from measurements taken under a known physical image formation model.
Reconstructing Transient Images From Single-Photon Sensors
Computer vision algorithms build on 2D images or 3D videos that capture dynamic events at the millisecond time scale.
A Wide-Field-Of-View Monocentric Light Field Camera
Finally, we describe a processing toolchain, including a convenient spherical LF parameterization, and demonstrate depth estimation and post-capture refocus for indoor and outdoor panoramas with 15 x 15 x 1600 x 200 pixels (72 MPix) and a 138-degree FOV.
Consensus Convolutional Sparse Coding
Convolutional sparse coding (CSC) is a promising direction for unsupervised learning in computer vision.
Non-line-of-sight Imaging with Partial Occluders and Surface Normals
Imaging objects obscured by occluders is a significant challenge for many applications.
Deep End-to-End Time-of-Flight Imaging
We present an end-to-end image processing framework for time-of-flight (ToF) cameras.
Convolutional Sparse Coding for High Dynamic Range Imaging
Current HDR acquisition techniques are based on either (i) fusing multibracketed, low dynamic range (LDR) images, (ii) modifying existing hardware and capturing different exposures simultaneously with multiple sensors, or (iii) reconstructing a single image with spatially-varying pixel exposures.
DeepVoxels: Learning Persistent 3D Feature Embeddings
In this work, we address the lack of 3D understanding of generative neural networks by introducing a persistent 3D feature embedding for view synthesis.
LiFF: Light Field Features in Scale and Depth
Feature detectors and descriptors are key low-level vision tools that many higher-level tasks build on.
Deep Optics for Monocular Depth Estimation and 3D Object Detection
In addition, we train object detection networks on the KITTI dataset and show that the lens optimized for depth estimation also results in improved 3D object detection performance.
Ranked #11 on
Depth Estimation
on NYU-Depth V2
Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations
Unsupervised learning with generative models has the potential of discovering rich representations of 3D scenes.
Deep Optics for Single-shot High-dynamic-range Imaging
High-dynamic-range (HDR) imaging is crucial for many computer graphics and vision applications.
Keyhole Imaging: Non-Line-of-Sight Imaging and Tracking of Moving Objects Along a Single Optical Path
Non-line-of-sight (NLOS) imaging and tracking is an emerging technology that allows the shape or position of objects around corners or behind diffusers to be recovered from transient, time-of-flight measurements.
Deep S$^3$PR: Simultaneous Source Separation and Phase Retrieval Using Deep Generative Models
This paper introduces and solves the simultaneous source separation and phase retrieval (S$^3$PR) problem.
Semantic Implicit Neural Scene Representations With Semi-Supervised Training
The recent success of implicit neural scene representations has presented a viable new method for how we capture and store 3D scenes.
Event Based, Near Eye Gaze Tracking Beyond 10,000Hz
The cameras in modern gaze-tracking systems suffer from fundamental bandwidth and power limitations, constraining data acquisition speed to 300 Hz realistically.
State of the Art on Neural Rendering
Neural rendering is a new and rapidly emerging field that combines generative machine learning techniques with physical knowledge from computer graphics, e. g., by the integration of differentiable rendering into network training.
Implicit Neural Representations with Periodic Activation Functions
However, current network architectures for such implicit neural representations are incapable of modeling signals with fine detail, and fail to represent a signal's spatial and temporal derivatives, despite the fact that these are essential to many physical signals defined implicitly as the solution to partial differential equations.
MetaSDF: Meta-learning Signed Distance Functions
Neural implicit shape representations are an emerging paradigm that offers many potential benefits over conventional discrete representations, including memory efficiency at a high spatial resolution.
Single-shot Hyperspectral-Depth Imaging with Learned Diffractive Optics
Imaging depth and spectrum have been extensively studied in isolation from each other for decades.
D-VDAMP: Denoising-based Approximate Message Passing for Compressive MRI
Plug and play (P&P) algorithms iteratively apply highly optimized image denoisers to impose priors and solve computational image reconstruction problems, to great effect.
SUREMap: Predicting Uncertainty in CNN-based Image Reconstruction Using Stein's Unbiased Risk Estimate
Convolutional neural networks (CNN) have emerged as a powerful tool for solving computational imaging reconstruction problems.
pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis
We have witnessed rapid progress on 3D-aware image synthesis, leveraging recent advances in generative visual models and neural rendering.
Ranked #3 on
Scene Generation
on VizDoom
AutoInt: Automatic Integration for Fast Neural Volume Rendering
For training, we instantiate the computational graph corresponding to the derivative of the network.
Neural Lumigraph Rendering
Novel view synthesis is a challenging and ill-posed inverse rendering problem.
ScanGAN360: A Generative Model of Realistic Scanpaths for 360$^{\circ}$ Images
Generative adversarial approaches could alleviate this challenge by generating a large number of possible scanpaths for unseen images.
Time-Multiplexed Coded Aperture Imaging: Learned Coded Aperture and Pixel Exposures for Compressive Imaging Systems
Compressive imaging using coded apertures (CA) is a powerful technique that can be used to recover depth, light fields, hyperspectral images and other quantities from a single snapshot.
ACORN: Adaptive Coordinate Networks for Neural Scene Representation
Here, we introduce a new hybrid implicit-explicit network architecture and training strategy that adaptively allocates resources during training and inference based on the local complexity of a signal of interest.
Fast Training of Neural Lumigraph Representations using Meta Learning
Inspired by neural variants of image-based rendering, we develop a new neural rendering approach with the goal of quickly learning a high-quality representation which can also be rendered in real-time.
Advances in Neural Rendering
The reconstruction of such a scene representation from observations using differentiable rendering losses is known as inverse graphics or inverse rendering.
MantissaCam: Learning Snapshot High-dynamic-range Imaging with Perceptually-based In-pixel Irradiance Encoding
We demonstrate the efficacy of our method in simulation and show benefits of our algorithm on modulo images captured with a prototype implemented with a programmable sensor.
BACON: Band-limited Coordinate Networks for Multiscale Scene Representation
These networks are trained to map continuous input coordinates to the value of a signal at each point.
Efficient Geometry-aware 3D Generative Adversarial Networks
Unsupervised generation of high-quality multi-view-consistent images and 3D shapes using only collections of single-view 2D photographs has been a long-standing challenge.
CryoAI: Amortized Inference of Poses for Ab Initio Reconstruction of 3D Molecular Volumes from Real Cryo-EM Images
We introduce cryoAI, an ab initio reconstruction algorithm for homogeneous conformations that uses direct gradient-based optimization of particle poses and the electron scattering potential from single-particle cryo-EM data.
3D GAN Inversion for Controllable Portrait Image Animation
Portrait image animation enables the post-capture adjustment of these attributes from a single image while maintaining a photorealistic reconstruction of the subject's likeness or identity.
Learning Spatially Varying Pixel Exposures for Motion Deblurring
Computationally removing the motion blur introduced by camera shake or object motion in a captured image remains a challenging task in computational photography.
Time-multiplexed Neural Holography: A flexible framework for holographic near-eye displays with fast heavily-quantized spatial light modulators
Holographic near-eye displays offer unprecedented capabilities for virtual and augmented reality systems, including perceptually important focus cues.
Learning to Solve PDE-constrained Inverse Problems with Graph Networks
Given a sparse set of measurements, we are interested in recovering the initial condition or parameters of the PDE.
Generative Neural Articulated Radiance Fields
Unsupervised learning of 3D-aware generative adversarial networks (GANs) using only collections of single-view 2D photographs has very recently made much progress.
3D-Aware Video Generation
Generative models have emerged as an essential building block for many image synthesis and editing tasks.
NeuForm: Adaptive Overfitting for Neural Shape Editing
Neural representations are popular for representing shapes, as they can be learned form sensor data and used for data cleanup, model completion, shape editing, and shape synthesis.
Heterogeneous reconstruction of deformable atomic models in Cryo-EM
Cryogenic electron microscopy (cryo-EM) provides a unique opportunity to study the structural heterogeneity of biomolecules.
Amortized Inference for Heterogeneous Reconstruction in Cryo-EM
Cryo-electron microscopy (cryo-EM) is an imaging modality that provides unique insights into the dynamics of proteins and other building blocks of life.
Scale-Agnostic Super-Resolution in MRI using Feature-Based Coordinate Networks
We propose using a coordinate network decoder for the task of super-resolution in MRI.
DiffDreamer: Towards Consistent Unsupervised Single-view Scene Extrapolation with Conditional Diffusion Models
Scene extrapolation -- the idea of generating novel views by flying into a given image -- is a promising, yet challenging task.
Ranked #1 on
Perpetual View Generation
on LHQ
SinGRAF: Learning a 3D Generative Radiance Field for a Single Scene
Generative models have shown great promise in synthesizing photorealistic 3D objects, but they require large amounts of training data.
3D Neural Field Generation using Triplane Diffusion
Diffusion models have emerged as the state-of-the-art for image generation, among other tasks.
ALTO: Alternating Latent Topologies for Implicit 3D Reconstruction
One school of thought is to encode a latent vector for each point (point latents).
PointAvatar: Deformable Point-based Head Avatars from Videos
The ability to create realistic, animatable and relightable head avatars from casual video sequences would open up wide ranging applications in communication and entertainment.
PaletteNeRF: Palette-based Appearance Editing of Neural Radiance Fields
Recent advances in neural radiance fields have enabled the high-fidelity 3D reconstruction of complex scenes for novel view synthesis.
Diffusion in the Dark: A Diffusion Model for Low-Light Text Recognition
Capturing images is a key part of automation for high-level tasks such as scene text recognition.
MELON: NeRF with Unposed Images in SO(3)
Neural radiance fields enable novel-view synthesis and scene reconstruction with photorealistic quality from a few images, but require known and accurate camera poses.
DehazeNeRF: Multiple Image Haze Removal and 3D Shape Reconstruction using Neural Radiance Fields
Neural radiance fields (NeRFs) have demonstrated state-of-the-art performance for 3D computer vision tasks, including novel view synthesis and 3D shape reconstruction.
CC3D: Layout-Conditioned Generation of Compositional 3D Scenes
In this work, we introduce CC3D, a conditional generative model that synthesizes complex 3D scenes conditioned on 2D semantic scene layouts, trained using single-view images.
Compositional 3D Scene Generation using Locally Conditioned Diffusion
Designing complex 3D scenes has been a tedious, manual process requiring domain expertise.
Generative Novel View Synthesis with 3D-Aware Diffusion Models
We present a diffusion-based model for 3D-aware generative novel view synthesis from as few as a single input image.
PixelRNN: In-pixel Recurrent Neural Networks for End-to-end-optimized Perception with Neural Sensors
Conventional image sensors digitize high-resolution images at fast frame rates, producing a large amount of data that needs to be transmitted off the sensor for further processing.
LumiGAN: Unconditional Generation of Relightable 3D Human Faces
Unsupervised learning of 3D human faces from unstructured 2D image data is an active research area.
Learning Controllable Adaptive Simulation for Multi-resolution Physics
In this work, we introduce Learning controllable Adaptive simulation for Multi-resolution Physics (LAMP) as the first full deep learning-based surrogate model that jointly learns the evolution model and optimizes appropriate spatial resolutions that devote more compute to the highly dynamic regions.
Single-Shot Implicit Morphable Faces with Consistent Texture Parameterization
To tackle this problem, we propose a novel method for constructing implicit 3D morphable face models that are both generalizable and intuitive for editing.
Articulated 3D Head Avatar Generation using Text-to-Image Diffusion Models
Recent work on text-guided 3D object generation has shown great promise in addressing these needs.
Efficient 3D Articulated Human Generation with Layered Surface Volumes
These layers are rendered using alpha compositing with fast differentiable rasterization, and they can be interpreted as a volumetric representation that allocates its capacity to a manifold of finite thickness around the template.
PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Tracking
Our goal is to advance the state-of-the-art by placing emphasis on long videos with naturalistic motion.
Ranked #1 on
Point Tracking
on TAP-Vid
Instant Continual Learning of Neural Radiance Fields
Neural radiance fields (NeRFs) have emerged as an effective method for novel-view synthesis and 3D scene reconstruction.
Gradient Descent Provably Solves Nonlinear Tomographic Reconstruction
We show that this approach reduces metal artifacts compared to a commercial reconstruction of a human skull with metal dental crowns.
State of the Art on Diffusion Models for Visual Computing
The field of visual computing is rapidly advancing due to the emergence of generative artificial intelligence (AI), which unlocks unprecedented capabilities for the generation, editing, and reconstruction of images, videos, and 3D scenes.
Pose-to-Motion: Cross-Domain Motion Retargeting with Pose Prior
Our experiments show that our method effectively combines the motion features of the source character with the pose features of the target character, and performs robustly with small or noisy pose data sets, ranging from a few artist-created poses to noisy poses estimated directly from images.
DMV3D: Denoising Multi-View Diffusion using 3D Large Reconstruction Model
We propose \textbf{DMV3D}, a novel 3D generation approach that uses a transformer-based 3D large reconstruction model to denoise multi-view diffusion.
Volumetric Reconstruction Resolves Off-Resonance Artifacts in Static and Dynamic PROPELLER MRI
Off-resonance artifacts in magnetic resonance imaging (MRI) are visual distortions that occur when the actual resonant frequencies of spins within the imaging volume differ from the expected frequencies used to encode spatial information.
4D-fy: Text-to-4D Generation Using Hybrid Score Distillation Sampling
Recent breakthroughs in text-to-4D generation rely on pre-trained text-to-image and text-to-video models to generate dynamic 3D scenes.
Gaussian Shell Maps for Efficient 3D Human Generation
Instead of rasterizing the shells directly, we sample 3D Gaussians on the shells whose attributes are encoded in the texture features.
Generative Rendering: Controllable 4D-Guided Video Generation with 2D Diffusion Models
Traditional 3D content creation tools empower users to bring their imagination to life by giving them direct control over a scene's geometry, appearance, motion, and camera path.
Orthogonal Adaptation for Modular Customization of Diffusion Models
In this paper, we address a new problem called Modular Customization, with the goal of efficiently merging customized models that were fine-tuned independently for individual concepts.
Scalable 3D Reconstruction From Single Particle X-Ray Diffraction Images Based on Online Machine Learning
X-ray free-electron lasers (XFELs) offer unique capabilities for measuring the structure and dynamics of biomolecules, helping us understand the basic building blocks of life.
GPT-4V(ision) is a Human-Aligned Evaluator for Text-to-3D Generation
These metrics lack the flexibility to generalize to different evaluation criteria and might not align well with human preferences.
GazeGPT: Augmenting Human Capabilities using Gaze-contingent Contextual AI for Smart Eyewear
Multimodal large language models (LMMs) excel in world knowledge and problem-solving abilities.
Real-time 3D-aware Portrait Editing from a Single Image
This work presents 3DPE, a practical method that can efficiently edit a face image following given prompts, like reference images or text descriptions, in a 3D-aware manner.
Generic 3D Diffusion Adapter Using Controlled Multi-View Editing
Open-domain 3D object synthesis has been lagging behind image synthesis due to limited data and higher computational complexity.
GRM: Large Gaussian Reconstruction Model for Efficient 3D Reconstruction and Generation
We introduce GRM, a large-scale reconstructor capable of recovering a 3D asset from sparse-view images in around 0. 1s.
TC4D: Trajectory-Conditioned Text-to-4D Generation
We learn local deformations that conform to the global trajectory using supervision from a text-to-video model.
CameraCtrl: Enabling Camera Control for Text-to-Video Generation
Controllability plays a crucial role in video generation since it allows users to create desired content.
PhysAvatar: Learning the Physics of Dressed 3D Avatars from Visual Observations
This marks a significant advancement towards modeling photorealistic digital humans using physically based inverse rendering with physics in the loop.
Flying with Photons: Rendering Novel Views of Propagating Light
Combined with this dataset, we introduce an efficient neural volume rendering framework based on the transient field.
Disambiguating Monocular Depth Estimation with a Single Transient
Monocular depth estimation algorithms successfully predict the relative depth order of objects in a scene.
