Search Results for author:
Found 291 papers, 177 papers with code
NTIRE 2025 Challenge on Short-form UGC Video Quality Assessment and Enhancement: KwaiSR Dataset and Study
In this work, we build the first benchmark dataset for short-form UGC Image Super-resolution in the wild, termed KwaiSR, intending to advance the research on developing image super-resolution algorithms for short-form UGC platforms.
NTIRE 2025 Challenge on Image Super-Resolution ($\times$4): Methods and Results
This paper presents the NTIRE 2025 image super-resolution ($\times$4) challenge, one of the associated competitions of the 10th NTIRE Workshop at CVPR 2025.
NTIRE 2025 Challenge on Real-World Face Restoration: Methods and Results
This paper provides a review of the NTIRE 2025 challenge on real-world face restoration, highlighting the proposed solutions and the resulting outcomes.
Any Image Restoration via Efficient Spatial-Frequency Degradation Adaptation
Restoring any degraded image efficiently via just one model has become increasingly significant and impactful, especially with the proliferation of mobile devices.
NTIRE 2025 Challenge on Short-form UGC Video Quality Assessment and Enhancement: Methods and Results
This paper presents a review for the NTIRE 2025 Challenge on Short-form UGC Video Quality Assessment and Enhancement.
NTIRE 2025 Challenge on Day and Night Raindrop Removal for Dual-Focused Images: Methods and Results
This paper reviews the NTIRE 2025 Challenge on Day and Night Raindrop Removal for Dual-Focused Images.
The Tenth NTIRE 2025 Image Denoising Challenge Report
This paper presents an overview of the NTIRE 2025 Image Denoising Challenge ({\sigma} = 50), highlighting the proposed methodologies and corresponding results.
NTIRE 2025 Challenge on Event-Based Image Deblurring: Methods and Results
This paper presents an overview of NTIRE 2025 the First Challenge on Event-Based Image Deblurring, detailing the proposed methodologies and corresponding results.
NTIRE 2025 Challenge on Cross-Domain Few-Shot Object Detection: Methods and Results
Cross-Domain Few-Shot Object Detection (CD-FSOD) poses significant challenges to existing object detection and few-shot detection models when applied across domains.
Cross-Domain Few-Shot
Cross-Domain Few-Shot Object Detection
+3
The Tenth NTIRE 2025 Efficient Super-Resolution Challenge Report
This paper presents a comprehensive review of the NTIRE 2025 Challenge on Single-Image Efficient Super-Resolution (ESR).
LEMUR Neural Network Dataset: Towards Seamless AutoML
We introduce LEMUR, an open source dataset of neural network models with well-structured code for diverse architectures across tasks such as object detection, image classification, segmentation, and natural language processing.
Optuna vs Code Llama: Are LLMs a New Paradigm for Hyperparameter Tuning?
Optimal hyperparameter selection is critical for maximizing neural network performance, especially as models grow in complexity.
Rip Current Segmentation: A Novel Benchmark and YOLOv8 Baseline Results
The best results were achieved by the YOLOv8-nano model (runnable on a portable device), with an mAP50 of $88. 94%$ on the validation dataset and $81. 21%$ macro average on the test dataset.
RipVIS: Rip Currents Video Instance Segmentation Benchmark for Beach Monitoring and Safety
To address these issues, we present RipVIS, a large-scale video instance segmentation benchmark explicitly designed for rip current segmentation.
Bokehlicious: Photorealistic Bokeh Rendering with Controllable Apertures
Bokeh rendering methods play a key role in creating the visually appealing, softly blurred backgrounds seen in professional photography.
CTSR: Controllable Fidelity-Realness Trade-off Distillation for Real-World Image Super Resolution
Real-world image super-resolution is a critical image processing task, where two key evaluation criteria are the fidelity to the original image and the visual realness of the generated results.
Illuminating Darkness: Enhancing Real-world Low-light Scenes with Smartphone Images
To address this limitation, we propose a large-scale high-resolution (i. e., beyond 4k) pair Single-Shot Low-Light Enhancement (SLLIE) dataset.
AdaptSR: Low-Rank Adaptation for Efficient and Scalable Real-World Super-Resolution
Recovering high-frequency details and textures from low-resolution images remains a fundamental challenge in super-resolution (SR), especially when real-world degradations are complex and unknown.
ContextFormer: Redefining Efficiency in Semantic Segmentation
We propose ContextFormer, a hybrid framework leveraging the strengths of CNNs and ViTs in the bottleneck to balance efficiency, accuracy, and robustness for real-time semantic segmentation.
MatIR: A Hybrid Mamba-Transformer Image Restoration Model
In recent years, Transformers-based models have made significant progress in the field of image restoration by leveraging their inherent ability to capture complex contextual features.
Centurio: On Drivers of Multilingual Ability of Large Vision-Language Model
Most Large Vision-Language Models (LVLMs) to date are trained predominantly on English data, which makes them struggle to understand non-English input and fail to generate output in the desired target language.
ReCap: Better Gaussian Relighting with Cross-Environment Captures
Accurate 3D objects relighting in diverse unseen environments is crucial for realistic virtual object placement.
INRetouch: Context Aware Implicit Neural Representation for Photography Retouching
We develop a context-aware Implicit Neural Representation that learns to apply edits adaptively based on image content and context, and is capable of learning from a single example.
Complexity Experts are Task-Discriminative Learners for Any Image Restoration
To address this, we introduce ``complexity experts" -- flexible expert blocks with varying computational complexity and receptive fields.
Ranked #2 on
Blind All-in-One Image Restoration
on 5-Degradations
Enhanced Super-Resolution Training via Mimicked Alignment for Real-World Scenes
In this study, we propose a novel plug-and-play module designed to mitigate these misalignment issues by aligning LR inputs with HR images during training.
AIM 2024 Challenge on Video Super-Resolution Quality Assessment: Methods and Results
This paper presents the Video Super-Resolution (SR) Quality Assessment (QA) Challenge that was part of the Advances in Image Manipulation (AIM) workshop, held in conjunction with ECCV 2024.
Steering Prediction via a Multi-Sensor System for Autonomous Racing
Our goal is to fuse the 2D LiDAR data with event data in an end-to-end learning framework for steering prediction, which is crucial for autonomous racing.
Toward Efficient Deep Blind RAW Image Restoration
In this work we tackle image restoration directly in the RAW domain.
Streaming Neural Images
Implicit Neural Representations (INRs) are a novel paradigm for signal representation that have attracted considerable interest for image compression.
NTIRE 2024 Challenge on Stereo Image Super-Resolution: Methods and Results
This paper summarizes the 3rd NTIRE challenge on stereo image super-resolution (SR) with a focus on new solutions and results.
AIM 2024 Challenge on Efficient Video Super-Resolution for AV1 Compressed Content
Video super-resolution (VSR) is a critical task for enhancing low-bitrate and low-resolution videos, particularly in streaming applications.
AIM 2024 Challenge on UHD Blind Photo Quality Assessment
By pushing the boundaries of NR-IQA for high-resolution photos, the UHD-IQA Challenge aims to stimulate the development of practical models that can keep pace with the rapidly evolving landscape of digital photography.
Compressed Depth Map Super-Resolution and Restoration: AIM 2024 Challenge Results
The increasing demand for augmented reality (AR) and virtual reality (VR) applications highlights the need for efficient depth information processing.
AIM 2024 Sparse Neural Rendering Challenge: Dataset and Benchmark
A recurring problem in sparse rendering literature is the lack of an homogeneous, up-to-date, dataset and evaluation protocol.
AIM 2024 Sparse Neural Rendering Challenge: Methods and Results
In this challenge, 5 teams submitted final results to Track 1 and 4 teams submitted final results to Track 2.
AIM 2024 Challenge on Video Saliency Prediction: Methods and Results
The goal of the participants was to develop a method for predicting accurate saliency maps for the provided set of video sequences.
AIM 2024 Challenge on Compressed Video Quality Assessment: Methods and Results
The challenge aimed to evaluate the performance of VQA methods on a diverse dataset of 459 videos, encoded with 14 codecs of various compression standards (AVC/H. 264, HEVC/H. 265, AV1, and VVC/H. 266) and containing a comprehensive collection of compression artifacts.
PIV3CAMS: a multi-camera dataset for multiple computer vision problems and its application to novel view-point synthesis
In this thesis, we introduce Paired Image and Video data from three CAMeraS, namely PIV3CAMS, aimed at multiple computer vision tasks.
Restore Anything Model via Efficient Degradation Adaptation
With the proliferation of mobile devices, the need for an efficient model to restore any degraded image has become increasingly significant and impactful.
Ranked #5 on
5-Degradation Blind All-in-One Image Restoration
on 5-Degradation Blind All-in-One Image Restoration
5-Degradation Blind All-in-One Image Restoration
Benchmarking
+2
Stereo Risk: A Continuous Modeling Approach to Stereo Matching
Stereo Risk departs from the conventional discretization approach by formulating the scene disparity as an optimal solution to a continuous risk minimization problem, hence the name "stereo risk".
African or European Swallow? Benchmarking Large Vision-Language Models for Fine-Grained Object Classification
We benchmark 12 public LVLMs on \texttt{FOCI} and show that it tests for a \textit{complementary skill} to established image understanding and reasoning benchmarks.
Does Object Grounding Really Reduce Hallucination of Large Vision-Language Models?
Large vision-language models (LVLMs) have recently dramatically pushed the state of the art in image captioning and many image understanding tasks (e. g., visual question answering).
NTIRE 2024 Challenge on Night Photography Rendering
The top ranking participants' solutions effectively represent the state-of-the-art in nighttime photography rendering.
XTrack: Multimodal Training Boosts RGB-X Video Object Trackers
Technically, we achieve this by routing samples from one modality to the expert of the others, within a mixture-of-experts framework designed for multimodal video object tracking.
Dataset Growth
To tackle this challenge, we propose InfoGrowth, an efficient online algorithm for data cleaning and selection, resulting in a growing dataset that keeps up to date with awareness of cleanliness and diversity.
Efficient Degradation-aware Any Image Restoration
Reconstructing missing details from degraded low-quality inputs poses a significant challenge.
Ranked #4 on
5-Degradation Blind All-in-One Image Restoration
on 5-Degradation Blind All-in-One Image Restoration
5-Degradation Blind All-in-One Image Restoration
Blind All-in-One Image Restoration
+1
MuDreamer: Learning Predictive World Models without Reconstruction
In this paper, we present MuDreamer, a robust reinforcement learning agent that builds upon the DreamerV3 algorithm by learning a predictive world model without the need for reconstructing input signals.
NTIRE 2024 Restore Any Image Model (RAIM) in the Wild Challenge
In this paper, we review the NTIRE 2024 challenge on Restore Any Image Model (RAIM) in the Wild.
MIPI 2024 Challenge on Nighttime Flare Removal: Methods and Results
The increasing demand for computational photography and imaging on mobile platforms has led to the widespread development and integration of advanced image sensors with novel algorithms in camera systems.
Real-Time 4K Super-Resolution of Compressed AVIF Images. AIS 2024 Challenge Survey
This paper introduces a novel benchmark as part of the AIS 2024 Real-Time Image Super-Resolution (RTSR) Challenge, which aims to upscale compressed images from 540p to 4K resolution (4x factor) in real-time on commercial GPUs.
NTIRE 2024 Quality Assessment of AI-Generated Content Challenge
A total of 196 participants have registered in the video track.
Deep RAW Image Super-Resolution. A NTIRE 2024 Challenge Survey
This paper reviews the NTIRE 2024 RAW Image Super-Resolution Challenge, highlighting the proposed solutions and results.
AIS 2024 Challenge on Video Quality Assessment of User-Generated Content: Methods and Results
The performance of the top-5 submissions is reviewed and provided here as a survey of diverse deep models for efficient video quality assessment of user-generated content.
NTIRE 2024 Challenge on Low Light Image Enhancement: Methods and Results
This paper reviews the NTIRE 2024 low light image enhancement challenge, highlighting the proposed solutions and results.
Simple Image Signal Processing using Global Context Guidance
First, we propose a novel module that can be integrated into any neural ISP to capture the global context information from the full RAW images.
Deep Portrait Quality Assessment. A NTIRE 2024 Challenge Survey
This paper reviews the NTIRE 2024 Portrait Quality Assessment Challenge, highlighting the proposed solutions and results.
NTIRE 2024 Challenge on Short-form UGC Video Quality Assessment: Methods and Results
This paper reviews the NTIRE 2024 Challenge on Shortform UGC Video Quality Assessment (S-UGC VQA), where various excellent solutions are submitted and evaluated on the collected dataset KVQ from popular short-form video platform, i. e., Kuaishou/Kwai Platform.
Event-Based Eye Tracking. AIS 2024 Challenge Survey
This survey reviews the AIS 2024 Event-Based Eye Tracking (EET) Challenge.
The Ninth NTIRE 2024 Efficient Super-Resolution Challenge Report
In sub-track 1, the practical runtime performance of the submissions was evaluated, and the corresponding score was used to determine the ranking.
Rawformer: Unpaired Raw-to-Raw Translation for Learnable Camera ISPs
Neural-based end-to-end learnable ISPs offer promising advancements, potentially replacing traditional ISPs with their ability to adapt without requiring extensive tuning for each new camera model, as is often the case for nearly every module in traditional ISPs.
NTIRE 2024 Challenge on Image Super-Resolution ($\times$4): Methods and Results
This paper reviews the NTIRE 2024 challenge on image super-resolution ($\times$4), highlighting the solutions proposed and the outcomes obtained.
Virtually Enriched NYU Depth V2 Dataset for Monocular Depth Estimation: Do We Need Artificial Augmentation?
We present ANYU, a new virtually augmented version of the NYU depth v2 dataset, designed for monocular depth estimation.
Empowering Image Recovery_ A Multi-Attention Approach
Integration of attention mechanisms across feature and positional dimensions further enhances the recovery of fine details.
Towards Image Ambient Lighting Normalization
However, existing works often simplify this task within the context of shadow removal, limiting the light sources to one and oversimplifying the scene, thus excluding complex self-shadows and restricting surface classes to smooth ones.
Towards Online Real-Time Memory-based Video Inpainting Transformers
Video inpainting tasks have seen significant improvements in recent years with the rise of deep neural networks and, in particular, vision transformers.
PASTA: Towards Flexible and Efficient HDR Imaging Via Progressively Aggregated Spatio-Temporal Alignment
Leveraging Transformer attention has led to great advancements in HDR deghosting.
Multilingual Audio-Visual Speech Recognition with Hybrid CTC/RNN-T Fast Conformer
Humans are adept at leveraging visual cues from lip movements for recognizing speech in adverse listening conditions.
Audio-Visual Speech Recognition
Robust Speech Recognition
+2
See More Details: Efficient Image Super-Resolution by Experts Mining
Subsequently, the model delves into the subtleties of rank choice by leveraging a mixture of low-rank experts.
InstructIR: High-Quality Image Restoration Following Human Instructions
All-In-One image restoration models can effectively restore images from various types and levels of degradation using degradation-specific information as prompts to guide the restoration model.
Real-World Mobile Image Denoising Dataset with Efficient Baselines
Despite the latest advancements in camera hardware the mobile camera sensor area cannot be increased significantly due to physical constraints leading to a pixel size of 0. 6--2. 0 \mum which results in strong image noise even in moderate lighting conditions.
BSRAW: Improving Blind RAW Image Super-Resolution
Our BSRAW models trained with our pipeline can upscale real-scene RAW images and improve their quality.
Single-Model and Any-Modality for Video Object Tracking
In practice, most existing RGB trackers learn a single set of parameters to use them across datasets and applications.
Ranked #29 on
Rgb-T Tracking
on LasHeR
Deep Equilibrium Diffusion Restoration with Parallel Sampling
Due to the inherent property of diffusion models, most existing methods need long serial sampling chains to restore HQ images step-by-step, resulting in expensive sampling time and high computation costs.
MoVideo: Motion-Aware Video Generation with Diffusion Models
While recent years have witnessed great progress on using diffusion models for video generation, most of them are simple extensions of image generation frameworks, which fail to explicitly consider one of the key differences between videos and images, i. e., motion.
A Study of Forward-Forward Algorithm for Self-Supervised Learning
This may be caused by a combination of factors, including having a loss function for each layer and the way the supervised training is realized in the forward-forward paradigm.
Neural Gradient Regularizer
NGR is applicable to various image types and different image processing tasks, functioning in a zero-shot learning fashion, making it a versatile and plug-and-play regularizer.
When Super-Resolution Meets Camouflaged Object Detection: A Comparison Study
Super Resolution (SR) and Camouflaged Object Detection (COD) are two hot topics in computer vision with various joint applications.
NTIRE 2023 Quality Assessment of Video Enhancement Challenge
61 participating teams submitted their prediction results during the development phase, with a total of 3168 submissions.
mBLIP: Efficient Bootstrapping of Multilingual Vision-LLMs
Modular vision-language models (Vision-LLMs) align pretrained image encoders with (frozen) large language models (LLMs) and post-hoc condition LLMs to `understand' the image input.
NILUT: Conditional Neural Implicit 3D Lookup Tables for Image Enhancement
Moreover, a NILUT can be extended to incorporate multiple styles into a single network with the ability to blend styles implicitly.
Babel-ImageNet: Massively Multilingual Evaluation of Vision-and-Language Representations
Vision-and-language (VL) models with separate encoders for each modality (e. g., CLIP) have become the go-to models for zero-shot image classification and image-text retrieval.
Towards Real-Time 4K Image Super-Resolution
Over the past few years, high-definition videos and images in 720p (HD), 1080p (FHD), and 4K (UHD) resolution have become standard.
Efficient Deep Models for Real-Time 4K Image Super-Resolution. NTIRE 2023 Benchmark and Report
This paper introduces a novel benchmark for efficient upscaling as part of the NTIRE 2023 Real-Time Image Super-Resolution (RTSR) Challenge, which aimed to upscale images from 720p and 1080p resolution to native 4K (x2 and x3 factors) in real-time on commercial GPUs.
Lens-to-lens bokeh effect transformation. NTIRE 2023 challenge report
We present the new Bokeh Effect Transformation Dataset (BETD), and review the proposed solutions for this novel task at the NTIRE 2023 Bokeh Effect Transformation Challenge.
Efficient multi-lens bokeh effect rendering and transformation
Our method can render Bokeh from an all-in-focus image, or transform the Bokeh of one lens to the effect of another lens without harming the sharp foreground regions in the image.
Alignment-free HDR Deghosting with Semantics Consistent Transformer
The spatial attention aims to deal with the intra-image correlation to model the dynamic motion, while the channel attention enables the inter-image intertwining to enhance the semantic consistency across frames.
Equivariant Multi-Modality Image Fusion
These components enable the net training to follow the principles of the natural sensing-imaging process while satisfying the equivariant imaging prior.
Denoising Diffusion Models for Plug-and-Play Image Restoration
Although diffusion models have shown impressive performance for high-quality image synthesis, their potential to serve as a generative denoiser prior to the plug-and-play IR methods remains to be further explored.
StyleGenes: Discrete and Efficient Latent Distributions for GANs
Thus, by independently sampling a variant for each gene and combining them into the final latent vector, our approach can represent a vast number of unique latent samples from a compact set of learnable parameters.
Event-Free Moving Object Segmentation from Moving Ego Vehicle
Subsequently, we devise EmoFormer, a novel network able to exploit the event data.
NTIRE 2023 Challenge on Light Field Image Super-Resolution: Dataset, Methods and Results
In this report, we summarize the first NTIRE challenge on light field (LF) image super-resolution (SR), which aims at super-resolving LF images under the standard bicubic degradation with a magnification factor of 4.
Single Image Depth Prediction Made Better: A Multivariate Gaussian Take
Accordingly, we introduce an approach that performs continuous modeling of per-pixel depth, where we can predict and reason about the per-pixel depth and its distribution.
Spherical Space Feature Decomposition for Guided Depth Map Super-Resolution
Then, the extracted features are mapped to the spherical space to complete the separation of private features and the alignment of shared features.
Graph Transformer GANs for Graph-Constrained House Generation
We present a novel graph Transformer generative adversarial network (GTGAN) to learn effective graph node relations in an end-to-end fashion for the challenging graph-constrained house generation task.
DDFM: Denoising Diffusion Model for Multi-Modality Image Fusion
To leverage strong generative priors and address challenges such as unstable training and lack of interpretability for GAN-based generative methods, we propose a novel fusion algorithm based on the denoising diffusion probabilistic model (DDPM).
Retinexformer: One-stage Retinex-based Transformer for Low-light Image Enhancement
When enhancing low-light images, many deep learning algorithms are based on the Retinex theory.
Ranked #1 on
Low-Light Image Enhancement
on SMID
Low-light Image Deblurring and Enhancement
Low-Light Image Enhancement
+3
Efficient and Explicit Modelling of Image Hierarchies for Image Restoration
The aim of this paper is to propose a mechanism to efficiently and explicitly model image hierarchies in the global, regional, and local range for image restoration.
Ranked #1 on
Image Defocus Deblurring
on DPD (Dual-view)
VA-DepthNet: A Variational Approach to Single Image Depth Prediction
While state-of-the-art deep neural network methods for SIDP learn the scene depth from images in a supervised setting, they often overlook the invaluable invariances and priors in the rigid scene space, such as the regularity of the scene.
Ranked #31 on
Monocular Depth Estimation
on NYU-Depth V2
Audio-Visual Efficient Conformer for Robust Speech Recognition
We improve previous lip reading methods using an Efficient Conformer back-end on top of a ResNet-18 visual front-end and by adding intermediate CTC losses between blocks.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
SQAD: Automatic Smartphone Camera Quality Assessment and Benchmarking
Smartphone photography is becoming increasingly popular, but fitting high-performing camera systems within the given space limitations remains a challenge for manufacturers.
Source-free Depth for Object Pop-out
In this work, we adapt such depth inference models for object segmentation using the objects' "pop-out" prior in 3D.
CiaoSR: Continuous Implicit Attention-in-Attention Network for Arbitrary-Scale Image Super-Resolution
We explicitly design an implicit attention network to learn the ensemble weights for the nearby local features.
Knowledge Distillation based Degradation Estimation for Blind Super-Resolution
It consists of a knowledge distillation based implicit degradation estimator network (KD-IDE) and an efficient SR network.
CDDFuse: Correlation-Driven Dual-Branch Feature Decomposition for Multi-Modality Image Fusion
We then introduce a dual-branch Transformer-CNN feature extractor with Lite Transformer (LT) blocks leveraging long-range attention to handle low-frequency global features and Invertible Neural Networks (INN) blocks focusing on extracting high-frequency local information.
Real-Time Under-Display Cameras Image Restoration and HDR on Mobile Devices
We propose a lightweight model for blind UDC Image Restoration and HDR, and we also provide a benchmark comparing the performance and runtime of different methods on smartphones.
Advancing Learned Video Compression with In-loop Frame Prediction
In this paper, we propose an Advanced Learned Video Compression (ALVC) approach with the in-loop frame prediction module, which is able to effectively predict the target frame from the previously compressed frames, without consuming any bit-rate.
PyNet-V2 Mobile: Efficient On-Device Photo Processing With Neural Networks
The increased importance of mobile photography created a need for fast and performant RAW image processing pipelines capable of producing good visual results in spite of the mobile camera sensor limitations.
MicroISP: Processing 32MP Photos on Mobile Devices with Deep Learning
While neural networks-based photo processing solutions can provide a better image quality compared to the traditional ISP systems, their application to mobile devices is still very limited due to their very high computational complexity.
Efficient and Accurate Quantized Image Super-Resolution on Mobile NPUs, Mobile AI & AIM 2022 challenge: Report
While numerous solutions have been proposed for this problem in the past, they are usually not compatible with low-power mobile NPUs having many computational and memory constraints.
Power Efficient Video Super-Resolution on Mobile NPUs with Deep Learning, Mobile AI & AIM 2022 challenge: Report
While numerous solutions have been proposed for this problem, they are usually quite computationally demanding, demonstrating low FPS rates and power efficiency on mobile devices.
Efficient Single-Image Depth Estimation on Mobile Devices, Mobile AI & AIM 2022 Challenge: Report
Various depth estimation models are now widely used on many mobile and IoT devices for image segmentation, bokeh effect rendering, object tracking and many other mobile tasks.
Realistic Bokeh Effect Rendering on Mobile GPUs, Mobile AI & AIM 2022 challenge: Report
In this Mobile AI challenge, the target was to develop an efficient end-to-end AI-based bokeh effect rendering approach that can run on modern smartphone GPUs using TensorFlow Lite.
Learned Smartphone ISP on Mobile GPUs with Deep Learning, Mobile AI & AIM 2022 Challenge: Report
The role of mobile cameras increased dramatically over the past few years, leading to more and more research in automatic image quality enhancement and RAW photo processing.
Perceptual Image Enhancement for Smartphone Real-Time Applications
Our experiments show that, with much fewer parameters and operations, our model can deal with the mentioned artifacts and achieve competitive performance compared with state-of-the-art methods on standard benchmarks.
Reversed Image Signal Processing and RAW Reconstruction. AIM 2022 Challenge Report
Cameras capture sensor RAW images and transform them into pleasant RGB images, suitable for the human eyes, using their integrated Image Signal Processor (ISP).
AIM 2022 Challenge on Instagram Filter Removal: Methods and Results
This paper introduces the methods and the results of AIM 2022 challenge on Instagram Filter Removal.
SiNeRF: Sinusoidal Neural Radiance Fields for Joint Pose Estimation and Scene Reconstruction
NeRFmm is the Neural Radiance Fields (NeRF) that deal with Joint Optimization tasks, i. e., reconstructing real-world scenes and registering camera parameters simultaneously.
Basic Binary Convolution Unit for Binarized Image Restoration Network
In this study, we reconsider components in binary convolution, such as residual connection, BatchNorm, activation function, and structure, for IR tasks.
Swin2SR: SwinV2 Transformer for Compressed Image Super-Resolution and Restoration
Using this method we can tackle the major issues in training transformer vision models, such as training instability, resolution gaps between pre-training and fine-tuning, and hunger on data.
Learning Task-Oriented Flows to Mutually Guide Feature Alignment in Synthesized and Real Video Denoising
To this end, we propose a new multi-scale refined optical flow-guided video denoising method, which is more robust to different noise levels.
Ranked #1 on
Video Denoising
on VideoLQ
AIM 2022 Challenge on Super-Resolution of Compressed Image and Video: Dataset, Methods and Results
The homepage of this challenge is at https://github. com/RenYang-home/AIM22_CompressSR.
3D-Aware Video Generation
Generative models have emerged as an essential building block for many image synthesis and editing tasks.
NTIRE 2022 Challenge on Perceptual Image Quality Assessment
This challenge is divided into two tracks, a full-reference IQA track similar to the previous NTIRE IQA challenge and a new track that focuses on the no-reference IQA methods.
Recurrent Video Restoration Transformer with Guided Deformable Attention
Specifically, RVRT divides the video into multiple clips and uses the previously inferred clip feature to estimate the subsequent clip feature.
NTIRE 2022 Challenge on High Dynamic Range Imaging: Methods and Results
The challenge is composed of two tracks with an emphasis on fidelity and complexity constraints: In Track 1, participants are asked to optimize objective fidelity scores while imposing a low-complexity constraint (i. e. solutions can not exceed a given number of operations).
Degradation-Aware Unfolding Half-Shuffle Transformer for Spectral Compressive Imaging
In coded aperture snapshot spectral compressive imaging (CASSI) systems, hyperspectral image (HSI) reconstruction methods are employed to recover the spatial-spectral signal from a compressed measurement.
Ranked #1 on
Spectral Reconstruction
on Real HSI
Revisiting Random Channel Pruning for Neural Network Compression
The proposed approach provides a new way to compare different methods, namely how well they behave compared with random pruning.
NTIRE 2022 Challenge on Efficient Super-Resolution: Methods and Results
The aim was to design a network for single image super-resolution that achieved improvement of efficiency measured according to several metrics including runtime, parameters, FLOPs, activations, and memory consumption while at least maintaining the PSNR of 29. 00dB on DIV2K validation set.
Conformer and Blind Noisy Students for Improved Image Quality Assessment
Learning-based approaches for perceptual image quality assessment (IQA) usually require both the distorted and reference image for measuring the perceptual quality accurately.
NTIRE 2022 Challenge on Super-Resolution and Quality Enhancement of Compressed Video: Dataset, Methods and Results
This challenge includes three tracks.
NTIRE 2022 Challenge on Stereo Image Super-Resolution: Methods and Results
In this paper, we summarize the 1st NTIRE challenge on stereo image super-resolution (restoration of rich details in a pair of low-resolution stereo images) with a focus on new solutions and results.
MST++: Multi-stage Spectral-wise Transformer for Efficient Spectral Reconstruction
Existing leading methods for spectral reconstruction (SR) focus on designing deeper or wider convolutional neural networks (CNNs) to learn the end-to-end mapping from the RGB image to its hyperspectral image (HSI).
Ranked #1 on
Spectral Reconstruction
on ARAD-1K
Arbitrary-Scale Image Synthesis
Positional encodings have enabled recent works to train a single adversarial network that can generate images of different scales.
Practical Blind Image Denoising via Swin-Conv-UNet and Data Synthesis
While recent years have witnessed a dramatic upsurge of exploiting deep neural networks toward solving image denoising, existing methods mostly rely on simple noise assumptions, such as additive white Gaussian noise (AWGN), JPEG compression noise and camera sensor noise, and a general-purpose blind denoising method for real images remains unsolved.
Ranked #2 on
Image Denoising
on urban100 sigma15
Transform your Smartphone into a DSLR Camera: Learning the ISP in the Wild
We propose a trainable Image Signal Processing (ISP) framework that produces DSLR quality images given RAW images captured by a smartphone.
Coarse-to-Fine Sparse Transformer for Hyperspectral Image Reconstruction
Many algorithms have been developed to solve the inverse problem of coded aperture snapshot spectral imaging (CASSI), i. e., recovering the 3D hyperspectral images (HSIs) from a 2D compressive measurement.
Ranked #4 on
Spectral Reconstruction
on Real HSI
HDNet: High-resolution Dual-domain Learning for Spectral Compressive Imaging
On the one hand, the proposed HR spatial-spectral attention module with its efficient feature fusion provides continuous and fine pixel-level features.
Ranked #7 on
Spectral Reconstruction
on Real HSI
Fast Online Video Super-Resolution with Deformable Attention Pyramid
Our DAP aligns and integrates information from the recurrent state into the current frame prediction.
VRT: A Video Restoration Transformer
Besides, parallel warping is used to further fuse information from neighboring frames by parallel feature warping.
Ranked #1 on
Deblurring
on BASED
RePaint: Inpainting using Denoising Diffusion Probabilistic Models
In this work, we propose RePaint: A Denoising Diffusion Probabilistic Model (DDPM) based inpainting approach that is applicable to even extreme masks.
Flow-Guided Sparse Transformer for Video Deblurring
Exploiting similar and sharper scene patches in spatio-temporal neighborhoods is critical for video deblurring.
Ranked #1 on
Deblurring
on DVD
Predict, Prevent, and Evaluate: Disentangled Text-Driven Image Manipulation Empowered by Pre-Trained Vision-Language Model
We propose a novel framework, i. e., Predict, Prevent, and Evaluate (PPE), for disentangled text-driven image manipulation that requires little manual annotation while being applicable to a wide variety of manipulations.
Global and Local Alignment Networks for Unpaired Image-to-Image Translation
The global alignment network aims to transfer the input image from the source domain to the target domain.
Mask-guided Spectral-wise Transformer for Efficient Hyperspectral Image Reconstruction
The HSI representations are highly similar and correlated across the spectral dimension.
Ranked #2 on
Spectral Reconstruction
on ARAD-1K
Normalizing Flow as a Flexible Fidelity Objective for Photo-Realistic Super-resolution
Super-resolution is an ill-posed problem, where a ground-truth high-resolution image represents only one possibility in the space of plausible solutions.
Towards Flexible Blind JPEG Artifacts Removal
Training a single deep blind model to handle different quality factors for JPEG image artifacts removal has been attracting considerable attention due to its convenience for practical usage.
Best Practices in Pool-based Active Learning for Image Classification
We emphasize aspects such as the importance of using data augmentation, the need of separating the contribution of a classification network and the acquisition strategy to the overall performance, the advantages that a proper initialization of the network can bring to AL.
PDC-Net+: Enhanced Probabilistic Dense Correspondence Network
In order to apply dense methods to real-world applications, such as pose estimation, image manipulation, or 3D reconstruction, it is therefore crucial to estimate the confidence of the predicted matches.
Perceptual Learned Video Compression with Recurrent Conditional GAN
This paper proposes a Perceptual Learned Video Compression (PLVC) approach with recurrent conditional GAN.
Generalized Real-World Super-Resolution through Adversarial Robustness
Instead of learning a dataset-specific degradation, we employ adversarial attacks to create difficult examples that target the model's weaknesses.
SwinIR: Image Restoration Using Swin Transformer
In particular, the deep feature extraction module is composed of several residual Swin Transformer blocks (RSTB), each of which has several Swin Transformer layers together with a residual connection.
Ranked #2 on
Grayscale Image Denoising
on Urban100 sigma15
Deep Reparametrization of Multi-Frame Super-Resolution and Denoising
The deep reparametrization allows us to directly model the image formation process in the latent space, and to integrate learned image priors into the prediction.
Ranked #6 on
Burst Image Super-Resolution
on BurstSR
Mutual Affine Network for Spatially Variant Kernel Estimation in Blind Image Super-Resolution
Extensive experiments on synthetic and real images show that the proposed MANet not only performs favorably for both spatially variant and invariant kernel estimation, but also leads to state-of-the-art blind SR performance when combined with non-blind SR methods.
Hierarchical Conditional Flow: A Unified Framework for Image Super-Resolution and Image Rescaling
More specifically, HCFlow learns a bijective mapping between HR and LR image pairs by modelling the distribution of the LR image and the rest high-frequency component simultaneously.
Ranked #4 on
Image Rescaling
on DIV2K val-q30-4x
NTIRE 2021 Multi-modal Aerial View Object Classification Challenge
In this paper, we introduce the first Challenge on Multi-modal Aerial View Object Classification (MAVOC) in conjunction with the NTIRE 2021 workshop at CVPR.
Deep Homography for Efficient Stereo Image Compression
To fully explore the mutual information across two stereo images, we use a deep regression model to estimate the homography matrix, i. e., H matrix.
Generative Flows with Invertible Attentions
The key idea is to exploit a masked scheme of these two attentions to learn long-range data dependencies in the context of generative flows.
NTIRE 2021 Challenge on Burst Super-Resolution: Methods and Results
This paper reviews the NTIRE2021 challenge on burst super-resolution.
NTIRE 2021 Challenge on High Dynamic Range Imaging: Dataset, Methods and Results
This paper reviews the first challenge on high-dynamic range (HDR) imaging that was part of the New Trends in Image Restoration and Enhancement (NTIRE) workshop, held in conjunction with CVPR 2021.
Fourier Space Losses for Efficient Perceptual Image Super-Resolution
As large models are often not practical in real-world applications, we investigate and propose novel loss functions, to enable SR with high perceptual quality from much more efficient models.
Fast and Accurate Quantized Camera Scene Detection on Smartphones, Mobile AI 2021 Challenge: Report
To address this problem, we introduce the first Mobile AI challenge, where the target is to develop quantized deep learning-based camera scene classification solutions that can demonstrate a real-time performance on smartphones and IoT platforms.
Real-Time Video Super-Resolution on Smartphones with Deep Learning, Mobile AI 2021 Challenge: Report
Video super-resolution has recently become one of the most important mobile-related problems due to the rise of video communication and streaming services.
Real-Time Quantized Image Super-Resolution on Mobile NPUs, Mobile AI 2021 Challenge: Report
Image super-resolution is one of the most popular computer vision problems with many important applications to mobile devices.
Learned Smartphone ISP on Mobile NPUs with Deep Learning, Mobile AI 2021 Challenge: Report
As the quality of mobile cameras starts to play a crucial role in modern smartphones, more and more attention is now being paid to ISP algorithms used to improve various perceptual aspects of mobile photos.
Fast and Accurate Camera Scene Detection on Smartphones
AI-powered automatic camera scene detection mode is nowadays available in nearly any modern smartphone, though the problem of accurate scene prediction has not yet been addressed by the research community.
Fast and Accurate Single-Image Depth Estimation on Mobile Devices, Mobile AI 2021 Challenge: Report
While many solutions have been proposed for this task, they are usually very computationally expensive and thus are not applicable for on-device inference.
Fast Camera Image Denoising on Mobile GPUs with Deep Learning, Mobile AI 2021 Challenge: Report
A detailed description of all models developed in the challenge is provided in this paper.
NTIRE 2021 Challenge on Perceptual Image Quality Assessment
This paper reports on the NTIRE 2021 challenge on perceptual image quality assessment (IQA), held in conjunction with the New Trends in Image Restoration and Enhancement workshop (NTIRE) workshop at CVPR 2021.
NTIRE 2021 Challenge on Image Deblurring
In this challenge report, we describe the challenge specifics and the evaluation results from the 2 competition tracks with the proposed solutions.
NTIRE 2021 Challenge on Video Super-Resolution
Super-Resolution (SR) is a fundamental computer vision task that aims to obtain a high-resolution clean image from the given low-resolution counterpart.
NTIRE 2021 Depth Guided Image Relighting Challenge
In this paper, we review the NTIRE 2021 depth guided image relighting challenge.
NTIRE 2021 Challenge on Quality Enhancement of Compressed Video: Methods and Results
This paper reviews the first NTIRE challenge on quality enhancement of compressed video, with a focus on the proposed methods and results.
NTIRE 2021 Challenge on Quality Enhancement of Compressed Video: Dataset and Study
In our study, we analyze the proposed methods of the challenge and several methods in previous works on the proposed LDV dataset.
Towards Efficient Graph Convolutional Networks for Point Cloud Handling
In this paper, we aim at improving the computational efficiency of graph convolutional networks (GCNs) for learning on point clouds.
LocalViT: Bringing Locality to Vision Transformers
The importance of locality mechanisms is validated in two ways: 1) A wide range of design choices (activation function, layer placement, expansion ratio) are available for incorporating locality mechanisms and all proper choices can lead to a performance gain over the baseline, and 2) The same locality mechanism is successfully applied to 4 vision transformers, which shows the generalization of the locality concept.
Ranked #680 on
Image Classification
on ImageNet
Flow-based Kernel Prior with Application to Blind Super-Resolution
Kernel estimation is generally one of the key problems for blind image super-resolution (SR).
Designing a Practical Degradation Model for Deep Blind Image Super-Resolution
It is widely acknowledged that single image super-resolution (SISR) methods would not perform well if the assumed degradation model deviates from those in real images.
Editorial: Introduction to the Issue on Deep Learning for Image/Video Restoration and Compression
Recent works have shown that learned models can achieve significant performance gains, especially in terms of perceptual quality measures, over traditional methods.
Deep Burst Super-Resolution
We propose a novel architecture for the burst super-resolution task.
Ranked #8 on
Burst Image Super-Resolution
on SyntheticBurst
Burst Image Super-Resolution
Multi-Frame Super-Resolution
+1
Trilevel Neural Architecture Search for Efficient Single Image Super-Resolution
Modern solutions to the single image super-resolution (SISR) problem using deep neural networks aim not only at better performance accuracy but also at a lighter and computationally efficient model.
DeFlow: Learning Complex Image Degradations from Unpaired Data with Conditional Flows
We propose DeFlow, a method for learning stochastic image degradations from unpaired data.
Generating Masks from Boxes by Mining Spatio-Temporal Consistencies in Videos
This effectively limits the performance and generalization capabilities of existing video segmentation methods.
Learning Accurate Dense Correspondences and When to Trust Them
Establishing dense correspondences between a pair of images is an important and general problem.
Local Memory Attention for Fast Video Semantic Segmentation
Our approach aggregates a rich representation of the semantic information in past frames into a memory module.
The Card Shuffling Hypotheses: Building a Time and Memory Efficient Graph Convolutional Network
State-of-the-art GCNs adopt $K$-nearest neighbor (KNN) searches for local feature aggregation and feature extraction operations from layer to layer.
An Efficient Recurrent Adversarial Framework for Unsupervised Real-Time Video Enhancement
Video enhancement is a challenging problem, more than that of stills, mainly due to high computational cost, larger data volumes and the difficulty of achieving consistency in the spatio-temporal domain.
Zero-Pair Image to Image Translation using Domain Conditional Normalization
In this paper, we propose an approach based on domain conditional normalization (DCN) for zero-pair image-to-image translation, i. e., translating between two domains which have no paired training data available but each have paired training data with a third domain.
AIM 2020 Challenge on Learned Image Signal Processing Pipeline
This paper reviews the second AIM learned ISP challenge and provides the description of the proposed solutions and results.
AIM 2020 Challenge on Rendering Realistic Bokeh
This paper reviews the second AIM realistic bokeh effect rendering challenge and provides the description of the proposed solutions and results.
A Weakly Supervised Convolutional Network for Change Segmentation and Classification
The core part of our model, the Change Segmentation and Classification (CSC) module, learns an accurate change mask at a hidden layer by using a custom Remapping Block and then segmenting the current input image with the change mask.
Self-Supervised Shadow Removal
Shadow removal is an important computer vision task aiming at the detection and successful removal of the shadow produced by an occluded light source and a photo-realistic restoration of the image contents.
SMILE: Semantically-guided Multi-attribute Image and Layout Editing
Additionally, our method is capable of adding, removing or changing either fine-grained or coarse attributes by using an image as a reference or by exploring the style distribution space, and it can be easily extended to head-swapping and face-reenactment applications without being trained on videos.
AIM 2020 Challenge on Image Extreme Inpainting
This report focuses on proposed solutions and results for two different tracks on extreme image inpainting: classical image inpainting and semantically guided image inpainting.
Fast Few-Shot Classification by Few-Iteration Meta-Learning
By employing an efficient initialization module and a Steepest Descent based optimization algorithm, our base learner predicts a powerful classifier within only a few iterations.
MS-RANAS: Multi-Scale Resource-Aware Neural Architecture Search
Neural Architecture Search (NAS) has proved effective in offering outperforming alternatives to handcrafted neural networks.
AIM 2020 Challenge on Video Temporal Super-Resolution
Videos in the real-world contain various dynamics and motions that may look unnaturally discontinuous in time when the recordedframe rate is low.
Learning to Improve Image Compression without Changing the Standard Decoder
Therefore, we propose learning to improve the encoding performance with the standard decoder.
AIM 2020: Scene Relighting and Illumination Estimation Challenge
The first track considered one-to-one relighting; the objective was to relight an input photo of a scene with a different color temperature and illuminant orientation (i. e., light source position).
AIM 2020 Challenge on Real Image Super-Resolution: Methods and Results
This paper introduces the real image Super-Resolution (SR) challenge that was part of the Advances in Image Manipulation (AIM) workshop, held in conjunction with ECCV 2020.
