Search Results for author:
Found 79 papers, 28 papers with code
Dual Exposure Stereo for Extended Dynamic Range 3D Imaging
Achieving robust stereo 3D imaging under diverse illumination conditions is an important however challenging task, due to the limited dynamic ranges (DRs) of cameras, which are significantly smaller than real world DR. As a result, the accuracy of existing stereo depth estimation methods is often compromised by under- or over-exposed images.
SimCMF: A Simple Cross-modal Fine-tuning Strategy from Vision Foundation Models to Any Imaging Modality
To this end, this work presents a simple and effective framework, SimCMF, to study an important problem: cross-modal fine-tuning from vision foundation models trained on natural RGB images to other imaging modalities of different physical properties (e. g., polarization).
Neural Light Spheres for Implicit Image Stitching and View Synthesis
Challenging to capture, and challenging to display on a cellphone screen, the panorama paradoxically remains both a staple and underused feature of modern mobile camera applications.
SimMAT: Exploring Transferability from Vision Foundation Models to Any Image Modality
To this end, this work presents a simple and effective framework SimMAT to study an open problem: the transferability from vision foundation models trained on natural RGB images to other image modalities of different physical properties (e. g., polarization).
Polarization Wavefront Lidar: Learning Large Scene Reconstruction from Polarized Wavefronts
We leverage polarimetric wavefronts to estimate normals, distance, and material properties in outdoor scenarios with a novel learned reconstruction method.
HINT: Learning Complete Human Neural Representations from Limited Viewpoints
To address this issue, we propose HINT, a NeRF-based algorithm able to learn a detailed and complete human model from limited viewing angles.
Gated Fields: Learning Scene Reconstruction from Gated Videos
We validate the method across day and night scenarios and find that Gated Fields compares favorably to RGB and LiDAR reconstruction methods.
Cross-spectral Gated-RGB Stereo Depth Estimation
By employing nanosecond-scale gates, existing sensors are capable of capturing mega-pixel gated images, delivering dense depth improving on today's LiDAR sensors in spatial resolution and depth precision.
Radar Fields: Frequency-Space Neural Scene Representations for FMCW Radar
Neural fields have been broadly investigated as scene representations for the reproduction and novel generation of diverse outdoor scenes, including those autonomous vehicles and robots must handle.
Inverse Neural Rendering for Explainable Multi-Object Tracking
We propose to recast 3D multi-object tracking from RGB cameras as an \emph{Inverse Rendering (IR)} problem, by optimizing via a differentiable rendering pipeline over the latent space of pre-trained 3D object representations and retrieve the latents that best represent object instances in a given input image.
Robust Depth Enhancement via Polarization Prompt Fusion Tuning
Existing depth sensors are imperfect and may provide inaccurate depth values in challenging scenarios, such as in the presence of transparent or reflective objects.
CtRL-Sim: Reactive and Controllable Driving Agents with Offline Reinforcement Learning
In this work, we take an alternative approach and propose CtRL-Sim, a method that leverages return-conditioned offline reinforcement learning (RL) to efficiently generate reactive and controllable traffic agents.
Neural Exposure Fusion for High-Dynamic Range Object Detection
Computer vision in unconstrained outdoor scenarios must tackle challenging high dynamic range (HDR) scenes and rapidly changing illumination conditions.
Flow-Guided Online Stereo Rectification for Wide Baseline Stereo
Stereo rectification is widely considered "solved" due to the abundance of traditional approaches to perform rectification.
Neural Spline Fields for Burst Image Fusion and Layer Separation
Each photo in an image burst can be considered a sample of a complex 3D scene: the product of parallax, diffuse and specular materials, scene motion, and illuminant variation.
Dispersed Structured Light for Hyperspectral 3D Imaging
In this work, we present Dispersed Structured Light (DSL), a cost-effective and compact method for accurate hyperspectral 3D imaging.
Spectral and Polarization Vision: Spectro-polarimetric Real-world Dataset
Image datasets are essential not only in validating existing methods in computer vision but also in developing new methods.
In the Blink of an Eye: Event-based Emotion Recognition
To the best of our knowledge, our method is the first eye-based emotion recognition method that leverages event-based cameras and spiking neural network.
Spatially Varying Nanophotonic Neural Networks
The explosive growth of computation and energy cost of artificial intelligence has spurred strong interests in new computing modalities as potential alternatives to conventional electronic processors.
Thin On-Sensor Nanophotonic Array Cameras
The optical array is embedded on a metasurface that, at 700~nm height, is flat and sits on the sensor cover glass at 2. 5~mm focal distance from the sensor.
Stochastic Light Field Holography
The Visual Turing Test is the ultimate goal to evaluate the realism of holographic displays.
Gated Stereo: Joint Depth Estimation from Gated and Wide-Baseline Active Stereo Cues
We propose Gated Stereo, a high-resolution and long-range depth estimation technique that operates on active gated stereo images.
S$^3$Track: Self-supervised Tracking with Soft Assignment Flow
With this training approach in hand, we develop an appearance-based model for learning instance-aware object features used to construct a cost matrix based on the pairwise distances between the object features.
ScatterNeRF: Seeing Through Fog with Physically-Based Inverse Neural Rendering
With data as bottleneck and most of today's training data relying on good weather conditions with inclement weather as outlier, we rely on an inverse rendering approach to reconstruct the scene content.
Instance Segmentation in the Dark
Existing instance segmentation techniques are primarily tailored for high-visibility inputs, but their performance significantly deteriorates in extremely low-light environments.
Multi-view Spectral Polarization Propagation for Video Glass Segmentation
In this paper, we present the first polarization-guided video glass segmentation propagation solution (PGVS-Net) that can robustly and coherently propagate glass segmentation in RGB-P video sequences.
LiDAR-in-the-Loop Hyperparameter Optimization
The resulting point clouds output by these DSPs are input to downstream 3D vision models -- both, in the form of training datasets or as input at inference time.
Seeing With Sound: Long-range Acoustic Beamforming for Multimodal Scene Understanding
We validate the benefit of adding sound detections to existing RGB cameras in challenging automotive scenarios, where camera-only approaches fail or do not deliver the ultra-fast rates of pressure sensors.
Single Depth-image 3D Reflection Symmetry and Shape Prediction
In this paper, we present Iterative Symmetry Completion Network (ISCNet), a single depth-image shape completion method that exploits reflective symmetry cues to obtain more detailed shapes.
Shakes on a Plane: Unsupervised Depth Estimation from Unstabilized Photography
Modern mobile burst photography pipelines capture and merge a short sequence of frames to recover an enhanced image, but often disregard the 3D nature of the scene they capture, treating pixel motion between images as a 2D aggregation problem.
Neural Volume Super-Resolution
This allows us to super-resolve the 3D scene representation by applying 2D convolutional networks on the 2D feature planes.
The Differentiable Lens: Compound Lens Search over Glass Surfaces and Materials for Object Detection
Recently, joint optimization approaches that design lenses alongside other components of the image acquisition and processing pipeline -- notably, downstream neural networks -- have achieved improved imaging quality or better performance on vision tasks.
Diffusion-SDF: Conditional Generative Modeling of Signed Distance Functions
Probabilistic diffusion models have achieved state-of-the-art results for image synthesis, inpainting, and text-to-image tasks.
GenSDF: Two-Stage Learning of Generalizable Signed Distance Functions
The first stage uses an episodic training scheme to simulate training on unlabeled data and meta-learns initial shape priors.
Differentiable Point-Based Radiance Fields for Efficient View Synthesis
We propose a differentiable rendering algorithm for efficient novel view synthesis.
LiDAR Snowfall Simulation for Robust 3D Object Detection
Due to the difficulty of collecting and annotating training data in this setting, we propose a physically based method to simulate the effect of snowfall on real clear-weather LiDAR point clouds.
Ranked #1 on
3D Object Detection
on Heavy Snowfall
Glass Segmentation Using Intensity and Spectral Polarization Cues
Transparent and semi-transparent materials pose significant challenges for existing scene understanding and segmentation algorithms due to their lack of RGB texture which impedes the extraction of meaningful features.
Ranked #2 on
Camouflaged Object Segmentation
on PCOD_1200
Spiking Transformers for Event-Based Single Object Tracking
In particular, the proposed architecture features a transformer module to provide global spatial information and a spiking neural network (SNN) module for extracting temporal cues.
All-photon Polarimetric Time-of-Flight Imaging
Conventional ToF imaging methods estimate the depth by sending pulses of light into a scene and measuring the ToF of the first-arriving photons directly reflected from a scene surface without any temporal delay.
All You Need is RAW: Defending Against Adversarial Attacks with Camera Image Pipelines
Various defense methods have proposed image-to-image mapping methods, either including these perturbations in the training process or removing them in a preprocessing denoising step.
Gated2Gated: Self-Supervised Depth Estimation from Gated Images
Gated cameras hold promise as an alternative to scanning LiDAR sensors with high-resolution 3D depth that is robust to back-scatter in fog, snow, and rain.
Neural Point Light Fields
We introduce Neural Point Light Fields that represent scenes implicitly with a light field living on a sparse point cloud.
The Implicit Values of A Good Hand Shake: Handheld Multi-Frame Neural Depth Refinement
Modern smartphones can continuously stream multi-megapixel RGB images at 60Hz, synchronized with high-quality 3D pose information and low-resolution LiDAR-driven depth estimates.
Neural Étendue Expander for Ultra-Wide-Angle High-Fidelity Holographic Display
As a result, modern holographic displays possess low \'{e}tendue, which is the product of the display area and the maximum solid angle of diffracted light.
A survey on computational spectral reconstruction methods from RGB to hyperspectral imaging
We present a thorough investigation of these state-of-the-art spectral reconstruction methods from the widespread RGB images.
End-to-End High Dynamic Range Camera Pipeline Optimization
Traditionally, hardware ISP settings used by downstream vision modules have been chosen by domain experts.
Neural Auto-Exposure for High-Dynamic Range Object Detection
In this work, we revisit auto-exposure control as an alternative to HDR sensors.
Polarimetric Spatio-Temporal Light Transport Probing
This allows us, for the first time, to decompose scene light transport into temporal, spatial, and complete polarimetric dimensions that unveil scene properties hidden to conventional methods.
Centimeter-Wave Free-Space Time-of-Flight Imaging
In this work, we close this gap and propose a computational imaging method for all-optical free-space correlation before photo-conversion that achieves micron-scale depth resolution with robustness to surface reflectance and ambient light with conventional silicon intensity sensors.
Neural Ray-Tracing: Learning Surfaces and Reflectance for Relighting and View Synthesis
Recent neural rendering methods have demonstrated accurate view interpolation by predicting volumetric density and color with a neural network.
Mask-ToF: Learning Microlens Masks for Flying Pixel Correction in Time-of-Flight Imaging
We introduce Mask-ToF, a method to reduce flying pixels (FP) in time-of-flight (ToF) depth captures.
Latent Variable Sequential Set Transformers For Joint Multi-Agent Motion Prediction
AutoBots can produce either the trajectory of one ego-agent or a distribution over the future trajectories for all agents in the scene.
ZeroScatter: Domain Transfer for Long Distance Imaging and Vision through Scattering Media
Most of today's supervised imaging and vision approaches, however, rely on training data collected in the real world that is biased towards good weather conditions, with dense fog, snow, and heavy rain as outliers in these datasets.
Adversarial Imaging Pipelines
As a result, optimized patterns can become adversarial for the classifier after being transformed by a certain camera ISP and optic but not for others.
Gated3D: Monocular 3D Object Detection From Temporal Illumination Cues
Today's state-of-the-art methods for 3D object detection are based on lidar, stereo, or monocular cameras.
Polka Lines: Learning Structured Illumination and Reconstruction for Active Stereo
Active stereo cameras that recover depth from structured light captures have become a cornerstone sensor modality for 3D scene reconstruction and understanding tasks across application domains.
Neural Scene Graphs for Dynamic Scenes
Recent implicit neural rendering methods have demonstrated that it is possible to learn accurate view synthesis for complex scenes by predicting their volumetric density and color supervised solely by a set of RGB images.
Seeing Around Street Corners: Non-Line-of-Sight Detection and Tracking In-the-Wild Using Doppler Radar
In this work, we depart from visible-wavelength approaches and demonstrate detection, classification, and tracking of hidden objects in large-scale dynamic environments using Doppler radars that can be manufactured at low-cost in series production.
Pixel-Accurate Depth Evaluation in Realistic Driving Scenarios
This work introduces an evaluation benchmark for depth estimation and completion using high-resolution depth measurements with angular resolution of up to 25" (arcsecond), akin to a 50 megapixel camera with per-pixel depth available.
Defending Against Universal Attacks Through Selective Feature Regeneration
Deep neural network (DNN) predictions have been shown to be vulnerable to carefully crafted adversarial perturbations.
Hyperparameter Optimization in Black-box Image Processing using Differentiable Proxies
We present a fully automatic system to optimize the parameters of black-box hardware and software image processing pipelines according to any arbitrary (i. e., application-specific) metric.
Seeing Through Fog Without Seeing Fog: Deep Multimodal Sensor Fusion in Unseen Adverse Weather
The fusion of multimodal sensor streams, such as camera, lidar, and radar measurements, plays a critical role in object detection for autonomous vehicles, which base their decision making on these inputs.
Ranked #2 on
2D Object Detection
on Clear Weather
Gated2Depth: Real-time Dense Lidar from Gated Images
The proposed replacement for scanning lidar systems is real-time, handles back-scatter and provides dense depth at long ranges.
DeepVoxels: Learning Persistent 3D Feature Embeddings
In this work, we address the lack of 3D understanding of generative neural networks by introducing a persistent 3D feature embedding for view synthesis.
Steady-state Non-Line-of-Sight Imaging
Relying on consumer color image sensors, with high fill factor, high quantum efficiency and low read-out noise, we demonstrate high-fidelity color NLOS imaging for scene configurations tackled before with picosecond time resolution.
Convolutional Sparse Coding for High Dynamic Range Imaging
Current HDR acquisition techniques are based on either (i) fusing multibracketed, low dynamic range (LDR) images, (ii) modifying existing hardware and capturing different exposures simultaneously with multiple sensors, or (iii) reconstructing a single image with spatially-varying pixel exposures.
Deep End-to-End Time-of-Flight Imaging
We present an end-to-end image processing framework for time-of-flight (ToF) cameras.
Non-line-of-sight Imaging with Partial Occluders and Surface Normals
Imaging objects obscured by occluders is a significant challenge for many applications.
Consensus Convolutional Sparse Coding
Convolutional sparse coding (CSC) is a promising direction for unsupervised learning in computer vision.
Reconstructing Transient Images From Single-Photon Sensors
Computer vision algorithms build on 2D images or 3D videos that capture dynamic events at the millisecond time scale.
Unrolled Optimization with Deep Priors
A broad class of problems at the core of computational imaging, sensing, and low-level computer vision reduces to the inverse problem of extracting latent images that follow a prior distribution, from measurements taken under a known physical image formation model.
Snapshot Difference Imaging using Time-of-Flight Sensors
Computational photography encompasses a diversity of imaging techniques, but one of the core operations performed by many of them is to compute image differences.
Discriminative Transfer Learning for General Image Restoration
Recently, several discriminative learning approaches have been proposed for effective image restoration, achieving convincing trade-off between image quality and computational efficiency.
Dirty Pixels: Towards End-to-End Image Processing and Perception
As such, conventional imaging involves processing the RAW sensor measurements in a sequential pipeline of steps, such as demosaicking, denoising, deblurring, tone-mapping and compression.
Material Classification Using Raw Time-Of-Flight Measurements
We propose a material classification method using raw time-of-flight (ToF) measurements.
Fast and Flexible Convolutional Sparse Coding
Convolutional sparse coding (CSC) has become an increasingly important tool in machine learning and computer vision.
Defocus Deblurring and Superresolution for Time-of-Flight Depth Cameras
Continuous-wave time-of-flight (ToF) cameras show great promise as low-cost depth image sensors in mobile applications.
Diffuse Mirrors: 3D Reconstruction from Diffuse Indirect Illumination Using Inexpensive Time-of-Flight Sensors
The functional difference between a diffuse wall and a mirror is well understood: one scatters back into all directions, and the other one preserves the directionality of reflected light.
Stochastic Deconvolution
We present a novel stochastic framework for non-blind deconvolution based on point samples obtained from random walks.
