Search Results for author:
Found 62 papers, 47 papers with code
Learning Factorized Weight Matrix for Joint Image Filtering
In this work, we propose to learn the weight matrix for joint image filtering.
Deep Surface Normal Estimation on the 2-Sphere with Confidence Guided Semantic Attention
We propose a deep convolutional neural network (CNN) to estimate surface normal from a single color image accompanied with a low-quality depth channel.
Q-Ground: Image Quality Grounding with Large Multi-modality Models
Recent advances of large multi-modality models (LMM) have greatly improved the ability of image quality assessment (IQA) method to evaluate and explain the quality of visual content.
Towards Open-ended Visual Quality Comparison
Comparative settings (e. g. pairwise choice, listwise ranking) have been adopted by a wide range of subjective studies for image quality assessment (IQA), as it inherently standardizes the evaluation criteria across different observers and offer more clear-cut responses.
Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levels
The explosion of visual content available online underscores the requirement for an accurate machine assessor to robustly evaluate scores across diverse types of visual contents.
Ranked #1 on
Video Quality Assessment
on LIVE-FB LSVQ
Iterative Token Evaluation and Refinement for Real-World Super-Resolution
Distortion removal involves simple HQ token prediction with LQ images, while texture generation uses a discrete diffusion model to iteratively refine the distortion removal output with a token refinement network.
Enhancing Diffusion Models with Text-Encoder Reinforcement Learning
While fine-tuning the U-Net can partially improve performance, it remains suffering from the suboptimal text encoder.
Q-Instruct: Improving Low-level Visual Abilities for Multi-modality Foundation Models
Multi-modality foundation models, as represented by GPT-4V, have brought a new paradigm for low-level visual perception and understanding tasks, that can respond to a broad range of natural human instructions in a model.
Q-Bench: A Benchmark for General-Purpose Foundation Models on Low-level Vision
To address this gap, we present Q-Bench, a holistic benchmark crafted to systematically evaluate potential abilities of MLLMs on three realms: low-level visual perception, low-level visual description, and overall visual quality assessment.
TOPIQ: A Top-down Approach from Semantics to Distortions for Image Quality Assessment
Our approach to IQA involves the design of a heuristic coarse-to-fine network (CFANet) that leverages multi-scale features and progressively propagates multi-level semantic information to low-level representations in a top-down manner.
Ranked #11 on
Video Quality Assessment
on MSU SR-QA Dataset
MIPI 2023 Challenge on Nighttime Flare Removal: Methods and Results
In this paper, we summarize and review the Nighttime Flare Removal track on MIPI 2023.
Towards Explainable In-the-Wild Video Quality Assessment: A Database and a Language-Prompted Approach
Though subjective studies have collected overall quality scores for these videos, how the abstract quality scores relate with specific factors is still obscure, hindering VQA methods from more concrete quality evaluations (e. g. sharpness of a video).
Towards Robust Text-Prompted Semantic Criterion for In-the-Wild Video Quality Assessment
The proliferation of videos collected during in-the-wild natural settings has pushed the development of effective Video Quality Assessment (VQA) methodologies.
MIPI 2023 Challenge on RGB+ToF Depth Completion: Methods and Results
To evaluate the performance of different depth completion methods, we organized an RGB+sparse ToF depth completion competition.
MIPI 2023 Challenge on RGBW Fusion: Methods and Results
Developing and integrating advanced image sensors with novel algorithms in camera systems are prevalent with the increasing demand for computational photography and imaging on mobile platforms.
MIPI 2023 Challenge on RGBW Remosaic: Methods and Results
Developing and integrating advanced image sensors with novel algorithms in camera systems are prevalent with the increasing demand for computational photography and imaging on mobile platforms.
Exploring Opinion-unaware Video Quality Assessment with Semantic Affinity Criterion
Recent learning-based video quality assessment (VQA) algorithms are expensive to implement due to the cost of data collection of human quality opinions, and are less robust across various scenarios due to the biases of these opinions.
Exploring Video Quality Assessment on User Generated Contents from Aesthetic and Technical Perspectives
In light of this, we propose the Disentangled Objective Video Quality Evaluator (DOVER) to learn the quality of UGC videos based on the two perspectives.
Ranked #1 on
Video Quality Assessment
on YouTube-UGC
Neighbourhood Representative Sampling for Efficient End-to-end Video Quality Assessment
On the other hand, existing practices, such as resizing and cropping, will change the quality of original videos due to the loss of details and contents, and are therefore harmful to quality assessment.
Ranked #2 on
Video Quality Assessment
on KoNViD-1k
(using extra training data)
MIPI 2022 Challenge on RGB+ToF Depth Completion: Dataset and Report
A detailed description of all models developed in this challenge is provided in this paper.
MIPI 2022 Challenge on Quad-Bayer Re-mosaic: Dataset and Report
A detailed description of all models developed in this challenge is provided in this paper.
MIPI 2022 Challenge on Under-Display Camera Image Restoration: Methods and Results
In this paper, we summarize and review the Under-Display Camera (UDC) Image Restoration track on MIPI 2022.
MIPI 2022 Challenge on RGBW Sensor Fusion: Dataset and Report
A detailed description of all models developed in this challenge is provided in this paper.
MIPI 2022 Challenge on RGBW Sensor Re-mosaic: Dataset and Report
A detailed description of all models developed in this challenge is provided in this paper.
Exploring the Effectiveness of Video Perceptual Representation in Blind Video Quality Assessment
Experiments show that the perceptual representation in the HVS is an effective way of predicting subjective temporal quality, and thus TPQI can, for the first time, achieve comparable performance to the spatial quality metric and be even more effective in assessing videos with large temporal variations.
FAST-VQA: Efficient End-to-end Video Quality Assessment with Fragment Sampling
Consisting of fragments and FANet, the proposed FrAgment Sample Transformer for VQA (FAST-VQA) enables efficient end-to-end deep VQA and learns effective video-quality-related representations.
Ranked #4 on
Video Quality Assessment
on LIVE-VQC
(using extra training data)
DisCoVQA: Temporal Distortion-Content Transformers for Video Quality Assessment
Based on prominent time-series modeling ability of transformers, we propose a novel and effective transformer-based VQA method to tackle these two issues.
Ranked #6 on
Video Quality Assessment
on KoNViD-1k
FuseFormer: Fusing Fine-Grained Information in Transformers for Video Inpainting
On the contrary, the soft composition operates by stitching different patches into a whole feature map where pixels in overlapping regions are summed up.
Ranked #3 on
Video Inpainting
on DAVIS
Dual-Camera Super-Resolution with Aligned Attention Modules
We present a novel approach to reference-based super-resolution (RefSR) with the focus on dual-camera super-resolution (DCSR), which utilizes reference images for high-quality and high-fidelity results.
A Categorized Reflection Removal Dataset with Diverse Real-world Scenes
Due to the lack of a large-scale reflection removal dataset with diverse real-world scenes, many existing reflection removal methods are trained on synthetic data plus a small amount of real-world data, which makes it difficult to evaluate the strengths or weaknesses of different reflection removal methods thoroughly.
Decoupled Spatial-Temporal Transformer for Video Inpainting
Seamless combination of these two novel designs forms a better spatial-temporal attention scheme and our proposed model achieves better performance than state-of-the-art video inpainting approaches with significant boosted efficiency.
Deep Animation Video Interpolation in the Wild
In the animation industry, cartoon videos are usually produced at low frame rate since hand drawing of such frames is costly and time-consuming.
Efficient Regional Memory Network for Video Object Segmentation
For the current query frame, the query regions are tracked and predicted based on the optical flow estimated from the previous frame.
Learning N:M Fine-grained Structured Sparse Neural Networks From Scratch
In this paper, we are the first to study training from scratch an N:M fine-grained structured sparse network, which can maintain the advantages of both unstructured fine-grained sparsity and structured coarse-grained sparsity simultaneously on specifically designed GPUs.
Exploiting Raw Images for Real-Scene Super-Resolution
In this paper, we study the problem of real-scene single image super-resolution to bridge the gap between synthetic data and real captured images.
Learning Spatial and Spatio-Temporal Pixel Aggregations for Image and Video Denoising
We present a spatial pixel aggregation network and learn the pixel sampling and averaging strategies for image denoising.
Semi-synthesis: A fast way to produce effective datasets for stereo matching
Though synthetic dataset is proposed to fill the gaps of large data demand, the fine-tuning on real dataset is still needed due to the domain variances between synthetic data and real data.
Enhanced Quadratic Video Interpolation
In this work, we further improve the performance of QVI from three facets and propose an enhanced quadratic video interpolation (EQVI) model.
Towards Geometry Guided Neural Relighting with Flash Photography
By incorporating the depth map, our approach is able to extrapolate realistic high-frequency effects under novel lighting via geometry guided image decomposition from the flashlight image, and predict the cast shadow map from the shadow-encoding transformed depth map.
Pix2Vox++: Multi-scale Context-aware 3D Object Reconstruction from Single and Multiple Images
A multi-scale context-aware fusion module is then introduced to adaptively select high-quality reconstructions for different parts from all coarse 3D volumes to obtain a fused 3D volume.
Ranked #3 on
3D Object Reconstruction
on Data3D−R2N2
GRNet: Gridding Residual Network for Dense Point Cloud Completion
In particular, we devise two novel differentiable layers, named Gridding and Gridding Reverse, to convert between point clouds and 3D grids without losing structural information.
Ranked #1 on
Point Cloud Completion
on Completion3D
StereoGAN: Bridging Synthetic-to-Real Domain Gap by Joint Optimization of Domain Translation and Stereo Matching
Large-scale synthetic datasets are beneficial to stereo matching but usually introduce known domain bias.
Polarized Reflection Removal with Perfect Alignment in the Wild
We present a novel formulation to removing reflection from polarized images in the wild.
Quadratic video interpolation
Video interpolation is an important problem in computer vision, which helps overcome the temporal limitation of camera sensors.
Toward 3D Object Reconstruction from Stereo Images
Inferring the 3D shape of an object from an RGB image has shown impressive results, however, existing methods rely primarily on recognizing the most similar 3D model from the training set to solve the problem.
Deep End-to-End Alignment and Refinement for Time-of-Flight RGB-D Module
Recently, it is increasingly popular to equip mobile RGB cameras with Time-of-Flight (ToF) sensors for active depth sensing.
Towards Real Scene Super-Resolution with Raw Images
Most existing super-resolution methods do not perform well in real scenarios due to lack of realistic training data and information loss of the model input.
Learning Deformable Kernels for Image and Video Denoising
Most of the classical denoising methods restore clear results by selecting and averaging pixels in the noisy input.
Deep Surface Normal Estimation with Hierarchical RGB-D Fusion
The growing availability of commodity RGB-D cameras has boosted the applications in the field of scene understanding.
DSR: Direct Self-rectification for Uncalibrated Dual-lens Cameras
Our method is evaluated on both real-istic and synthetic stereo image pairs, and produces supe-rior results compared to the calibrated rectification or otherself-rectification approaches
Confidence Inference for Focused Learning in Stereo Matching
Intuitively, the vari-ance in the Laplacian distribution is large for low confidentpixels while small for high-confidence pixels.
Monocular Depth Estimation with Affinity, Vertical Pooling, and Label Enhancement
While significant progress has been made in monocular depth estimation with Convolutional Neural Networks (CNNs) extracting absolute features, such as edges and textures, the depth constraint of neighboring pixels, namely relative features, has been mostly ignored by recent methods.
Deep Graph Laplacian Regularization for Robust Denoising of Real Images
In this work, we combine the robustness merit of model-based approaches and the learning power of data-driven approaches for real image denoising.
Zoom and Learn: Generalizing Deep Stereo Matching to Novel Domains
By feeding real stereo pairs of different domains to stereo models pre-trained with synthetic data, we see that: i) a pre-trained model does not generalize well to the new domain, producing artifacts at boundaries and ill-posed regions; however, ii) feeding an up-sampled stereo pair leads to a disparity map with extra details.
Single View Stereo Matching
The resulting model outperforms all the previous monocular depth estimation methods as well as the stereo block matching method in the challenging KITTI dataset by only using a small number of real training data.
Ranked #50 on
Monocular Depth Estimation
on KITTI Eigen split
LSTM Pose Machines
Such suboptimal results are mainly attributed to the inability of imposing sequential geometric consistency, handling severe image quality degradation (e. g. motion blur and occlusion) as well as the inability of capturing the temporal correlation among video frames.
Ranked #3 on
Pose Estimation
on J-HMDB
Image Dehazing using Bilinear Composition Loss Function
In this paper, we introduce a bilinear composition loss function to address the problem of image dehazing.
Cascade Residual Learning: A Two-stage Convolutional Neural Network for Stereo Matching
As opposed to directly learning the disparity at the second stage, we show that residual learning provides more effective refinement.
Robust Tracking Using Region Proposal Networks
Recent advances in visual tracking showed that deep Convolutional Neural Networks (CNN) trained for image classification can be strong feature extractors for discriminative trackers.
Accurate Single Stage Detector Using Recurrent Rolling Convolution
In this paper, we proposed a novel single stage end-to-end trainable object detection network to overcome this limitation.
Look, Listen and Learn - A Multimodal LSTM for Speaker Identification
This task not only requires collective perception over both visual and auditory signals, the robustness to handle severe quality degradations and unconstrained content variations are also indispensable.
Shepard Convolutional Neural Networks
In this paper, we draw on Shepard interpolation and design Shepard Convolutional Neural Networks (ShCNN) which efficiently realizes end-to-end trainable TVI operators in the network.
