Search Results for author:
Found 51 papers, 8 papers with code
Generative Quanta Color Imaging
In this paper, we explore the possibility of generating a color image from a single binary frame of a single-photon camera.
Tutorial on Diffusion Models for Imaging and Vision
The goal of this tutorial is to discuss the essential ideas underlying the diffusion models.
Resolution Limit of Single-Photon LiDAR
This presents a fundamental trade-off between the spatial resolution of the sensor array and the SNR received at each pixel.
Spatio-Temporal Turbulence Mitigation: A Translational Perspective
Building upon the intuitions of classical TM algorithms, we present the Deep Atmospheric TUrbulence Mitigation network (DATUM).
Kernel Diffusion: An Alternate Approach to Blind Deconvolution
Blind deconvolution problems are severely ill-posed because neither the underlying signal nor the forward operator are not known exactly.
The Secrets of Non-Blind Poisson Deconvolution
Non-blind image deconvolution has been studied for several decades but most of the existing work focuses on blur instead of noise.
Computational Image Formation: Simulators in the Deep Learning Era
This article introduces the concept of computational image formation (CIF).
Physics-Driven Turbulence Image Restoration with Stochastic Refinement
Although fast and physics-grounded simulation tools have been introduced to help the deep-learning models adapt to real-world turbulence conditions recently, the training of such models only relies on the synthetic data and ground truth pairs.
Spatially Varying Exposure with 2-by-2 Multiplexing: Optimality and Universality
The advancement of new digital image sensors has enabled the design of exposure multiplexing schemes where a single image capture can have multiple exposures and conversion gains in an interlaced format, similar to that of a Bayer color filter array.
HDR Imaging with Spatially Varying Signal-to-Noise Ratios
For HDR imaging in photon-limited situations, the dynamic range can be enormous and the noise within one exposure is spatially varying.
Towards Understanding the Effect of Pretraining Label Granularity
Furthermore, we perform comprehensive experiments using the label hierarchies of iNaturalist 2021 and observe that the following conditions, in addition to proper choice of label granularity, enable the transfer to work well in practice: 1) the pretraining dataset needs to have a meaningful label hierarchy, and 2) the pretraining and target label functions need to align well.
Scattering and Gathering for Spatially Varying Blurs
A spatially varying blur kernel $h(\mathbf{x},\mathbf{u})$ is specified by an input coordinate $\mathbf{u} \in \mathbb{R}^2$ and an output coordinate $\mathbf{x} \in \mathbb{R}^2$.
Structured Kernel Estimation for Photon-Limited Deconvolution
By modeling the blur kernel using a low-dimensional representation with the key points on the motion trajectory, we significantly reduce the search space and improve the regularity of the kernel estimation problem.
Real-Time Dense Field Phase-to-Space Simulation of Imaging through Atmospheric Turbulence
We show that the cross-correlation of the Zernike modes has an insignificant contribution to the statistics of the random samples.
What Does a One-Bit Quanta Image Sensor Offer?
In particular, it is theoretically found that the sensor can offer three benefits: (1) Low-light: One-bit QIS performs better at low-light because it has a low read noise, and its one-bit quantization can produce an error-free measurement.
Photon-Limited Blind Deconvolution using Unsupervised Iterative Kernel Estimation
When the noise is strong, these networks fail to simultaneously deblur and denoise; (3) While iterative schemes are known to be robust in the classical frameworks, they are seldom considered in deep neural networks because it requires a differentiable non-blind solver.
Single Frame Atmospheric Turbulence Mitigation: A Benchmark Study and A New Physics-Inspired Transformer Model
Image restoration algorithms for atmospheric turbulence are known to be much more challenging to design than traditional ones such as blur or noise because the distortion caused by the turbulence is an entanglement of spatially varying blur, geometric distortion, and sensor noise.
Imaging through the Atmosphere using Turbulence Mitigation Transformer
While existing deep-learning-based methods have demonstrated promising results in specific testing conditions, they suffer from three limitations: (1) lack of generalization capability from synthetic training data to real turbulence data; (2) failure to scale, hence causing memory and speed challenges when extending the idea to a large number of frames; (3) lack of a fast and accurate simulator to generate data for training neural networks.
Tilt-then-Blur or Blur-then-Tilt? Clarifying the Atmospheric Turbulence Model
Imaging at a long distance often requires advanced image restoration algorithms to compensate for the distortions caused by atmospheric turbulence.
On the Insensitivity of Bit Density to Read Noise in One-bit Quanta Image Sensors
An intriguing phenomenon is observed when the quanta exposure is at the unity and the threshold is $q = 0. 5$.
Exposure-Referred Signal-to-Noise Ratio for Digital Image Sensors
New theoretical results are derived for image sensors of any bit-depth and full-well capacity.
Graph-Based Depth Denoising & Dequantization for Point Cloud Enhancement
Specifically, we model depth formation as a combined process of signal-dependent noise addition and non-uniform log-based quantization.
Photon Limited Non-Blind Deblurring Using Algorithm Unrolling
Image deblurring in photon-limited conditions is ubiquitous in a variety of low-light applications such as photography, microscopy, and astronomy.
Detecting and Segmenting Adversarial Graphics Patterns from Images
Adversarial attacks pose a substantial threat to computer vision system security, but the social media industry constantly faces another form of "adversarial attack" in which the hackers attempt to upload inappropriate images and fool the automated screening systems by adding artificial graphics patterns.
Accelerating Atmospheric Turbulence Simulation via Learned Phase-to-Space Transform
Fast and accurate simulation of imaging through atmospheric turbulence is essential for developing turbulence mitigation algorithms.
Graph Signal Restoration Using Nested Deep Algorithm Unrolling
We overcome two main challenges in existing graph signal restoration methods: 1) limited performance of convex optimization algorithms due to fixed parameters which are often determined manually.
DROID: Driver-centric Risk Object Identification
Identification of high-risk driving situations is generally approached through collision risk estimation or accident pattern recognition.
Student-Teacher Learning from Clean Inputs to Noisy Inputs
Feature-based student-teacher learning, a training method that encourages the student's hidden features to mimic those of the teacher network, is empirically successful in transferring the knowledge from a pre-trained teacher network to the student network.
HDR Imaging with Quanta Image Sensors: Theoretical Limits and Optimal Reconstruction
We provide a complete theoretical characterization of the sensor in the context of HDR imaging, by proving the fundamental limits in the dynamic range that QIS can offer and the trade-offs with noise and speed.
Dynamic Low-light Imaging with Quanta Image Sensors
QIS are single-photon image sensors with photon counting capabilities.
Image Classification in the Dark using Quanta Image Sensors
In this paper, we present a new low-light image classification solution using Quanta Image Sensors (QIS).
One Size Fits All: Can We Train One Denoiser for All Noise Levels?
The de facto training protocol to achieve this goal is to train the estimator with noisy samples whose noise levels are uniformly distributed across the range of interest.
Simulating Anisoplanatic Turbulence by Sampling Inter-modal and Spatially Correlated Zernike Coefficients
Simulating atmospheric turbulence is an essential task for evaluating turbulence mitigation algorithms and training learning-based methods.
Who Make Drivers Stop? Towards Driver-centric Risk Assessment: Risk Object Identification via Causal Inference
We formulate the task as the cause-effect problem and present a novel two-stage risk object identification framework based on causal inference with the proposed object-level manipulable driving model.
Learning 3D-aware Egocentric Spatial-Temporal Interaction via Graph Convolutional Networks
First, we decompose egocentric interactions into ego-thing and ego-stuff interaction, modeled by two GCNs.
Rethinking Atmospheric Turbulence Mitigation
We demonstrate how a simple prior can outperform state-of-the-art blind deconvolution methods.
Color Filter Arrays for Quanta Image Sensors
In this paper, we discuss how to design color filter arrays for QIS and other small pixels.
Performance Analysis of Plug-and-Play ADMM: A Graph Signal Processing Perspective
The Plug-and-Play (PnP) ADMM algorithm is a powerful image restoration framework that allows advanced image denoising priors to be integrated into physical forward models to generate high quality image restoration results.
Automatic Foreground Extraction from Imperfect Backgrounds using Multi-Agent Consensus Equilibrium
The fusion framework allows us to integrate the individual strengths of alpha matting, background subtraction and image denoising to produce an overall better estimate.
Optimal Combination of Image Denoisers
Given a set of image denoisers, each having a different denoising capability, is there a provably optimal way of combining these denoisers to produce an overall better result?
Plug-and-Play Unplugged: Optimization Free Reconstruction using Consensus Equilibrium
We give examples to illustrate consensus equilibrium and the convergence properties of these algorithms and demonstrate this method on some toy problems and on a denoising example in which we use an array of convolutional neural network denoisers, none of which is tuned to match the noise level in a noisy image but which in consensus can achieve a better result than any of them individually.
Optimal Threshold Design for Quanta Image Sensor
Second, we show that around the oracle threshold there exists a set of thresholds that give asymptotically unbiased reconstructions.
Plug-and-Play ADMM for Image Restoration: Fixed Point Convergence and Applications
We compare Plug-and-Play ADMM with state-of-the-art algorithms in each problem type, and demonstrate promising experimental results of the algorithm.
Algorithm-Induced Prior for Image Restoration
Different from classical image priors which are defined before running the reconstruction algorithm, algorithm-induced priors are defined by the denoising procedure used to replace one of the two modules in the ADMM algorithm.
Adaptive Image Denoising by Mixture Adaptation
We propose an adaptive learning procedure to learn patch-based image priors for image denoising.
Understanding Symmetric Smoothing Filters: A Gaussian Mixture Model Perspective
We show that Sinkhorn-Knopp is equivalent to an Expectation-Maximization (EM) algorithm of learning a Gaussian mixture model of the image patches.
Depth Reconstruction from Sparse Samples: Representation, Algorithm, and Sampling
Experimental results show that the proposed method produces high quality dense depth estimates, and is robust to noisy measurements.
Adaptive Image Denoising by Targeted Databases
First, we determine the basis function of the denoising filter by solving a group sparsity minimization problem.
Monte Carlo non local means: Random sampling for large-scale image filtering
In particular, our error probability bounds show that, at any given sampling ratio, the probability for MCNLM to have a large deviation from the original NLM solution decays exponentially as the size of the image or database grows.
Stochastic blockmodel approximation of a graphon: Theory and consistent estimation
Non-parametric approaches for analyzing network data based on exchangeable graph models (ExGM) have recently gained interest.
