Search Results for author:
Found 101 papers, 29 papers with code
Conditional Image Repainting via Semantic Bridge and Piecewise Value Function
We study conditional image repainting where a model is trained to generate visual content conditioned on user inputs, and composite the generated content seamlessly onto a user provided image while preserving the semantics of users' inputs.
FHDe²Net: Full High Definition Demoireing Network
Frequency aliasing in the digital capture of display screens leads to the moir´e pattern, appearing as stripe-shaped distortions in images.
What is Learned in Deep Uncalibrated Photometric Stereo?
Based on this insight, we propose a guided calibration network, named GCNet, that explicitly leverages object shape and shading information for improved lighting estimation.
Imaging Interiors: An Implicit Solution to Electromagnetic Inverse Scattering Problems
By representing the scatterer's relative permittivity as a continuous implicit representation, our method is able to address the low-resolution problems arising from discretization.
E2VIDiff: Perceptual Events-to-Video Reconstruction using Diffusion Priors
Event cameras, mimicking the human retina, capture brightness changes with unparalleled temporal resolution and dynamic range.
Technique Report of CVPR 2024 PBDL Challenges
The intersection of physics-based vision and deep learning presents an exciting frontier for advancing computer vision technologies.
NeRSP: Neural 3D Reconstruction for Reflective Objects with Sparse Polarized Images
We present NeRSP, a Neural 3D reconstruction technique for Reflective surfaces with Sparse Polarized images.
Reconstructing Satellites in 3D from Amateur Telescope Images
This paper proposes a framework for the 3D reconstruction of satellites in low-Earth orbit, utilizing videos captured by small amateur telescopes.
SpikeNVS: Enhancing Novel View Synthesis from Blurry Images via Spike Camera
Since the spike camera relies on temporal integration instead of temporal differentiation used by event cameras, our proposed TfS loss maintains manageable training costs.
Spin-UP: Spin Light for Natural Light Uncalibrated Photometric Stereo
Natural Light Uncalibrated Photometric Stereo (NaUPS) relieves the strict environment and light assumptions in classical Uncalibrated Photometric Stereo (UPS) methods.
Complementing Event Streams and RGB Frames for Hand Mesh Reconstruction
Experiments on real-world data demonstrate that EvRGBHand can effectively solve the challenging issues when using either type of camera alone via retaining the merits of both, and shows the potential of generalization to outdoor scenes and another type of event camera.
Colorizing Monochromatic Radiance Fields
Though Neural Radiance Fields (NeRF) can produce colorful 3D representations of the world by using a set of 2D images, such ability becomes non-existent when only monochromatic images are provided.
Language-guided Image Reflection Separation
This paper studies the problem of language-guided reflection separation, which aims at addressing the ill-posed reflection separation problem by introducing language descriptions to provide layer content.
Real-time 3D-aware Portrait Video Relighting
We demonstrate the effectiveness and interactivity of our method on various portrait videos with diverse lighting and viewing conditions.
EventPS: Real-Time Photometric Stereo Using an Event Camera
Photometric stereo is a well-established technique to estimate the surface normal of an object.
Towards HDR and HFR Video from Rolling-Mixed-Bit Spikings
However reconstructing HDR videos in high-speed conditions using single-bit spikings presents challenges due to the limited bit depth.
NB-GTR: Narrow-Band Guided Turbulence Removal
The removal of atmospheric turbulence is crucial for long-distance imaging.
EvDiG: Event-guided Direct and Global Components Separation
In this paper we leverage the event camera to facilitate the separation of direct and global components enabling video-rate separation of high quality.
VMINer: Versatile Multi-view Inverse Rendering with Near- and Far-field Light Sources
This paper introduces a versatile multi-view inverse rendering framework with near- and far-field light sources.
DiLiGenRT: A Photometric Stereo Dataset with Quantified Roughness and Translucency
Photometric stereo faces challenges from non-Lambertian reflectance in real-world scenarios.
Neural Underwater Scene Representation
Among the numerous efforts towards digitally recovering the physical world Neural Radiance Fields (NeRFs) have proved effective in most cases.
Latency Correction for Event-guided Deblurring and Frame Interpolation
Event cameras with their high temporal resolution dynamic range and low power consumption are particularly good at time-sensitive applications like deblurring and frame interpolation.
EvPlug: Learn a Plug-and-Play Module for Event and Image Fusion
The learned fusion module integrates event streams with image features in the form of a plug-in, endowing the RGB-based model to be robust to HDR and fast motion scenes while enabling high temporal resolution inference.
Pano-NeRF: Synthesizing High Dynamic Range Novel Views with Geometry from Sparse Low Dynamic Range Panoramic Images
However, in the case of inputting sparse Low Dynamic Range (LDR) panoramic images, NeRF often degrades with under-constrained geometry and is unable to reconstruct HDR radiance from LDR inputs.
EventAid: Benchmarking Event-aided Image/Video Enhancement Algorithms with Real-captured Hybrid Dataset
Event cameras are emerging imaging technology that offers advantages over conventional frame-based imaging sensors in dynamic range and sensing speed.
Surface Geometry Processing: An Efficient Normal-based Detail Representation
With the rapid development of high-resolution 3D vision applications, the traditional way of manipulating surface detail requires considerable memory and computing time.
Self-Supervised Scene Dynamic Recovery from Rolling Shutter Images and Events
Scene Dynamic Recovery (SDR) by inverting distorted Rolling Shutter (RS) images to an undistorted high frame-rate Global Shutter (GS) video is a severely ill-posed problem due to the missing temporal dynamic information in both RS intra-frame scanlines and inter-frame exposures, particularly when prior knowledge about camera/object motions is unavailable.
Spike Stream Denoising via Spike Camera Simulation
Based on our noise model, the first benchmark for spike stream denoising is proposed which includes clear (noisy) spike stream.
DANI-Net: Uncalibrated Photometric Stereo by Differentiable Shadow Handling, Anisotropic Reflectance Modeling, and Neural Inverse Rendering
Although the ambiguity is alleviated on non-Lambertian objects, the problem is still difficult to solve for more general objects with complex shapes introducing irregular shadows and general materials with complex reflectance like anisotropic reflectance.
EvHandPose: Event-based 3D Hand Pose Estimation with Sparse Supervision
Experiments on EvRealHands demonstrate that EvHandPose outperforms previous event-based methods under all evaluation scenes, achieves accurate and stable hand pose estimation with high temporal resolution in fast motion and strong light scenes compared with RGB-based methods, generalizes well to outdoor scenes and another type of event camera, and shows the potential for the hand gesture recognition task.
Occlusion-Free Scene Recovery via Neural Radiance Fields
Our everyday lives are filled with occlusions that we strive to see through.
1000 FPS HDR Video With a Spike-RGB Hybrid Camera
Capturing high frame rate and high dynamic range (HFR&HDR) color videos in high-speed scenes with conventional frame-based cameras is very challenging.
High-Fidelity Event-Radiance Recovery via Transient Event Frequency
High-fidelity radiance recovery plays a crucial role in scene information reconstruction and understanding.
L-CoIns: Language-Based Colorization With Instance Awareness
Language-based colorization produces plausible colors consistent with the language description provided by the user.
Learning Event Guided High Dynamic Range Video Reconstruction
Limited by the trade-off between frame rate and exposure time when capturing moving scenes with conventional cameras, frame based HDR video reconstruction suffers from scene-dependent exposure ratio balancing and ghosting artifacts.
Complementary Intrinsics From Neural Radiance Fields and CNNs for Outdoor Scene Relighting
Relighting an outdoor scene is challenging due to the diverse illuminations and salient cast shadows.
ReLeaPS : Reinforcement Learning-based Illumination Planning for Generalized Photometric Stereo
Illumination planning in photometric stereo aims to find a balance between tween surface normal estimation accuracy and image capturing efficiency by selecting optimal light configurations.
DiLiGenT-Pi: Photometric Stereo for Planar Surfaces with Rich Details - Benchmark Dataset and Beyond
Photometric stereo aims to recover detailed surface shapes from images captured under varying illuminations.
Coherent Event Guided Low-Light Video Enhancement
With frame-based cameras, capturing fast-moving scenes without suffering from blur often comes at the cost of low SNR and low contrast.
Affective Image Filter: Reflecting Emotions from Text to Images
In this work, we propose Affective Image Filter (AIF), a novel model that is able to understand the visually-abstract emotions from the text and reflect them to visually-concrete images with appropriate colors and textures.
All-in-Focus Imaging From Event Focal Stack
Given an RGB image focused at an arbitrary distance, we explore the high temporal resolution of event streams, from which we automatically select refocusing timestamps and reconstruct corresponding refocused images with events to form a focal stack.
Deep Learning Methods for Calibrated Photometric Stereo and Beyond
We summarize the performance of deep learning photometric stereo models on the most widely-used benchmark data set.
NeuralMPS: Non-Lambertian Multispectral Photometric Stereo via Spectral Reflectance Decomposition
In this paper, we propose a deep neural network named NeuralMPS to solve the MPS problem under general non-Lambertian spectral reflectances.
CT2: Colorization Transformer via Color Tokens
Automatic image colorization is an ill-posed problem with multi-modal uncertainty, and there remains two main challenges with previous methods: incorrect semantic colors and under-saturation.
AdsCVLR: Commercial Visual-Linguistic Representation Modeling in Sponsored Search
Sponsored search advertisements (ads) appear next to search results when consumers look for products and services on search engines.
Ranked #3 on
Image-text matching
on CommercialAdsDataset
NeIF: Representing General Reflectance as Neural Intrinsics Fields for Uncalibrated Photometric Stereo
Uncalibrated photometric stereo (UPS) is challenging due to the inherent ambiguity brought by unknown light.
Edge-preserving Near-light Photometric Stereo with Neural Surfaces
This paper presents a near-light photometric stereo method that faithfully preserves sharp depth edges in the 3D reconstruction.
Self-Supervised Low-Light Image Enhancement Using Discrepant Untrained Network Priors
The enhancement is done by jointly optimizing the Retinex decomposition and the illumination adjustment.
1000x Faster Camera and Machine Vision with Ordinary Devices
By treating vidar as spike trains in biological vision, we have further developed a spiking neural network-based machine vision system that combines the speed of the machine and the mechanism of biological vision, achieving high-speed object detection and tracking 1, 000x faster than human vision.
UniCoRN: A Unified Conditional Image Repainting Network
Conditional image repainting (CIR) is an advanced image editing task, which requires the model to generate visual content in user-specified regions conditioned on multiple cross-modality constraints, and composite the visual content with the provided background seamlessly.
DiLiGenT102: A Photometric Stereo Benchmark Dataset With Controlled Shape and Material Variation
Evaluating photometric stereo using real-world dataset is important yet difficult.
EvUnroll: Neuromorphic Events Based Rolling Shutter Image Correction
This paper proposes to use neuromorphic events for correcting rolling shutter (RS) images as consecutive global shutter (GS) frames.
Learning to dehaze with polarization
Haze, a common kind of bad weather caused by atmospheric scattering, decreases the visibility of scenes and degenerates the performance of computer vision algorithms.
Optical Flow Estimation for Spiking Camera
Further, for training SCFlow, we synthesize two sets of optical flow data for the spiking camera, SPIkingly Flying Things and Photo-realistic High-speed Motion, denoted as SPIFT and PHM respectively, corresponding to random high-speed and well-designed scenes.
Label Distribution Amendment with Emotional Semantic Correlations for Facial Expression Recognition
Based on this, we propose a new method that amends the label distribution of each facial image by leveraging correlations among expressions in the semantic space.
Facial Expression Recognition
Facial Expression Recognition (FER)
Panoramic Image Reflection Removal
This paper studies the problem of panoramic image reflection removal, aiming at reliving the content ambiguity between reflection and transmission scenes.
EventZoom: Learning To Denoise and Super Resolve Neuromorphic Events
EventZoom is trained in a noise-to-noise fashion where the two ends of the network are unfiltered noisy events, enforcing noise-free event restoration.
High-Speed Image Reconstruction Through Short-Term Plasticity for Spiking Cameras
Mimicking the sampling mechanism of the fovea, a retina-inspired camera, named spiking camera, is developed to record the external information with a sampling rate of 40, 000 Hz, and outputs asynchronous binary spike streams.
Normal Integration via Inverse Plane Fitting With Minimum Point-to-Plane Distance
Experimental results on analytically computed, synthetic, and real-world surfaces show that our method yields accurate and stable reconstruction for both orthographic and perspective normal maps.
Single Image Reflection Removal With Absorption Effect
In this paper, we consider the absorption effect for the problem of single image reflection removal.
Multispectral Photometric Stereo for Spatially-Varying Spectral Reflectances: A Well Posed Problem?
To make the problem well-posed, existing MPS methods rely on restrictive assumptions, such as shape prior, surfaces having a monochromatic with uniform albedo.
DeRenderNet: Intrinsic Image Decomposition of Urban Scenes with Shape-(In)dependent Shading Rendering
We propose DeRenderNet, a deep neural network to decompose the albedo and latent lighting, and render shape-(in)dependent shadings, given a single image of an outdoor urban scene, trained in a self-supervised manner.
Spatially-Varying Outdoor Lighting Estimation from Intrinsics
We train a deep neural network to regress intrinsic cues with physically-based constraints and use them to conduct global and local lightings estimation.
EvIntSR-Net: Event Guided Multiple Latent Frames Reconstruction and Super-Resolution
To reconstruct high-resolution intensity images from event data, we propose EvIntSR-Net that converts event data to multiple latent intensity frames to achieve super-resolution on intensity images in this paper.
GPS-Net: Graph-based Photometric Stereo Network
For all-pixel operation, we propose the Normal Regression Network to make efficient use of the intra-image spatial information for predicting a surface normal map with rich details.
UnModNet: Learning to Unwrap a Modulo Image for High Dynamic Range Imaging
A conventional camera often suffers from over- or under-exposure when recording a real-world scene with a very high dynamic range (HDR).
Group Contextual Encoding for 3D Point Clouds
In this work, we extended the contextual encoding layer that was originally designed for 2D tasks to 3D Point Cloud scenarios.
RIFE: Real-Time Intermediate Flow Estimation for Video Frame Interpolation
We propose RIFE, a Real-time Intermediate Flow Estimation algorithm for Video Frame Interpolation (VFI).
Deep Photometric Stereo for Non-Lambertian Surfaces
To deal with the uncalibrated scenario where light directions are unknown, we introduce a new convolutional network, named LCNet, to estimate light directions from input images.
A Semi-Supervised Assessor of Neural Architectures
A graph convolutional neural network is introduced to predict the performance of architectures based on the learned representations and their relation modeled by the graph.
Distilling portable Generative Adversarial Networks for Image Translation
To promote the capability of student generator, we include a student discriminator to measure the distances between real images, and images generated by student and teacher generators.
Beyond Dropout: Feature Map Distortion to Regularize Deep Neural Networks
On one hand, massive trainable parameters significantly enhance the performance of these deep networks.
On Positive-Unlabeled Classification in GAN
In contrast, it is more reasonable to treat the generated data as unlabeled, which could be positive or negative according to their quality.
Context-Aware Cross-Attention for Skeleton-Based Human Action Recognition
To facilitate end-to-end training, we further develop a scenario context information extraction branch to extract context information from raw RGB video directly.
Ranked #87 on
Skeleton Based Action Recognition
on NTU RGB+D
Multi-View Photometric Stereo: A Robust Solution and Benchmark Dataset for Spatially Varying Isotropic Materials
From a single viewpoint, we use a set of photometric stereo images to identify surface points with the same distance to the camera.
AdderNet: Do We Really Need Multiplications in Deep Learning?
The widely-used convolutions in deep neural networks are exactly cross-correlation to measure the similarity between input feature and convolution filters, which involves massive multiplications between float values.
Learning from Bad Data via Generation
A worst-case formulation can be developed over this distribution set, and then be interpreted as a generation task in an adversarial manner.
Reflection Separation using a Pair of Unpolarized and Polarized Images
When we take photos through glass windows or doors, the transmitted background scene is often blended with undesirable reflection.
DIST: Rendering Deep Implicit Signed Distance Function with Differentiable Sphere Tracing
We propose a differentiable sphere tracing algorithm to bridge the gap between inverse graphics methods and the recently proposed deep learning based implicit signed distance function.
CARS: Continuous Evolution for Efficient Neural Architecture Search
Architectures in the population that share parameters within one SuperNet in the latest generation will be tuned over the training dataset with a few epochs.
Hyperspectral City V1.0 Dataset and Benchmark
This document introduces the background and the usage of the Hyperspectral City Dataset and the benchmark.
Bringing Giant Neural Networks Down to Earth with Unlabeled Data
Specifically, we exploit the unlabeled data to mimic the classification characteristics of giant networks, so that the original capacity can be preserved nicely.
SPLINE-Net: Sparse Photometric Stereo through Lighting Interpolation and Normal Estimation Networks
This paper solves the Sparse Photometric stereo through Lighting Interpolation and Normal Estimation using a generative Network (SPLINE-Net).
Data-Free Learning of Student Networks
Learning portable neural networks is very essential for computer vision for the purpose that pre-trained heavy deep models can be well applied on edge devices such as mobile phones and micro sensors.
Deep Shape from Polarization
This paper makes a first attempt to bring the Shape from Polarization (SfP) problem to the realm of deep learning.
Self-calibrating Deep Photometric Stereo Networks
This paper proposes an uncalibrated photometric stereo method for non-Lambertian scenes based on deep learning.
Face Image Reflection Removal
Face images captured through the glass are usually contaminated by reflections.
Event-driven Video Frame Synthesis
Temporal Video Frame Synthesis (TVFS) aims at synthesizing novel frames at timestamps different from existing frames, which has wide applications in video codec, editing and analysis.
Robust Student Network Learning
In this way, a portable student network with significantly fewer parameters can achieve a considerable accuracy which is comparable to that of teacher network.
Depth-Aware Stereo Video Retargeting
As compared with traditional video retargeting, stereo video retargeting poses new challenges because stereo video contains the depth information of salient objects and its time dynamics.
Self-Calibrating Polarising Radiometric Calibration
We present a self-calibrating polarising radiometric calibration method.
Uncalibrated Photometric Stereo Under Natural Illumination
This paper presents a photometric stereo method that works with unknown natural illuminations without any calibration object.
CRRN: Multi-Scale Guided Concurrent Reflection Removal Network
Removing the undesired reflections from images taken through the glass is of broad application to various computer vision tasks.
Benchmarking Single-Image Reflection Removal Algorithms
Removing undesired reflections from a photo taken in front of a glass is of great importance for enhancing the efficiency of visual computing systems.
A Microfacet-Based Reflectance Model for Photometric Stereo With Highly Specular Surfaces
Recent developments in the field have enabled shape recovery techniques for surfaces of various types, but an effective solution to directly estimating the surface normal in the presence of highly specular reflectance remains elusive.
Radiometric Calibration for Internet Photo Collections
Radiometrically calibrating the images from Internet photo collections brings photometric analysis from lab data to big image data in the wild, but conventional calibration methods cannot be directly applied to such image data.
Polarimetric Multi-View Stereo
Multi-view stereo relies on feature correspondences for 3D reconstruction, and thus is fundamentally flawed in dealing with featureless scenes.
Saliency Detection by Forward and Backward Cues in Deep-CNNs
As prior knowledge of objects or object features helps us make relations for similar objects on attentional tasks, pre-trained deep convolutional neural networks (CNNs) can be used to detect salient objects on images regardless of the object class is in the network knowledge or not.
A Benchmark Dataset and Evaluation for Non-Lambertian and Uncalibrated Photometric Stereo
Recent progress on photometric stereo extends the technique to deal with general materials and unknown illumination conditions.
Photometric Stereo With Small Angular Variations
This paper studies the effect of small angular variations in illumination directions to photometric stereo.
Polarized 3D: High-Quality Depth Sensing With Polarization Cues
We propose a framework to overcome these key challenges, allowing the benefits of polarization to be used to enhance depth maps.
