Search Results for author:
Found 240 papers, 147 papers with code
RePaint: Inpainting using Denoising Diffusion Probabilistic Models
In this work, we propose RePaint: A Denoising Diffusion Probabilistic Model (DDPM) based inpainting approach that is applicable to even extreme masks.
SwinIR: Image Restoration Using Swin Transformer
In particular, the deep feature extraction module is composed of several residual Swin Transformer blocks (RSTB), each of which has several Swin Transformer layers together with a residual connection.
Ranked #2 on
Color Image Denoising
on urban100 sigma15
Mask-guided Spectral-wise Transformer for Efficient Hyperspectral Image Reconstruction
The HSI representations are highly similar and correlated across the spectral dimension.
Ranked #2 on
Spectral Reconstruction
on ARAD-1K
MST++: Multi-stage Spectral-wise Transformer for Efficient Spectral Reconstruction
Existing leading methods for spectral reconstruction (SR) focus on designing deeper or wider convolutional neural networks (CNNs) to learn the end-to-end mapping from the RGB image to its hyperspectral image (HSI).
Ranked #1 on
Spectral Reconstruction
on ARAD-1K
Retinexformer: One-stage Retinex-based Transformer for Low-light Image Enhancement
When enhancing low-light images, many deep learning algorithms are based on the Retinex theory.
Ranked #1 on
Low-Light Image Enhancement
on SDSD-outdoor
Low-light Image Deblurring and Enhancement
Low-Light Image Enhancement
+3
Probabilistic Regression for Visual Tracking
In this work, we therefore propose a probabilistic regression formulation and apply it to tracking.
Ranked #4 on
Object Tracking
on FE108
Learning Discriminative Model Prediction for Tracking
The current strive towards end-to-end trainable computer vision systems imposes major challenges for the task of visual tracking.
Ranked #5 on
Object Tracking
on FE108
Know Your Surroundings: Exploiting Scene Information for Object Tracking
Such approaches are however prone to fail in case of e. g. fast appearance changes or presence of distractor objects, where a target appearance model alone is insufficient for robust tracking.
Ranked #8 on
Object Tracking
on FE108
Learning What to Learn for Video Object Segmentation
This allows us to achieve a rich internal representation of the target in the current frame, significantly increasing the segmentation accuracy of our approach.
Generating Masks from Boxes by Mining Spatio-Temporal Consistencies in Videos
This effectively limits the performance and generalization capabilities of existing video segmentation methods.
AIM 2020 Challenge on Efficient Super-Resolution: Methods and Results
This paper reviews the AIM 2020 challenge on efficient single image super-resolution with focus on the proposed solutions and results.
DSLR-Quality Photos on Mobile Devices with Deep Convolutional Networks
Despite a rapid rise in the quality of built-in smartphone cameras, their physical limitations - small sensor size, compact lenses and the lack of specific hardware, - impede them to achieve the quality results of DSLR cameras.
VRT: A Video Restoration Transformer
Besides, parallel warping is used to further fuse information from neighboring frames by parallel feature warping.
Ranked #1 on
Deblurring
on BASED
Designing a Practical Degradation Model for Deep Blind Image Super-Resolution
It is widely acknowledged that single image super-resolution (SISR) methods would not perform well if the assumed degradation model deviates from those in real images.
NTIRE 2020 Challenge on Real-World Image Super-Resolution: Methods and Results
This paper reviews the NTIRE 2020 challenge on real world super-resolution.
DeepSVG: A Hierarchical Generative Network for Vector Graphics Animation
Scalable Vector Graphics (SVG) are ubiquitous in modern 2D interfaces due to their ability to scale to different resolutions.
Ranked #1 on
Vector Graphics Animation
on SVG-Icons8
Deep Unfolding Network for Image Super-Resolution
As a result, the proposed network inherits the flexibility of model-based methods to super-resolve blurry, noisy images for different scale factors via a single model, while maintaining the advantages of learning-based methods.
SRFlow: Learning the Super-Resolution Space with Normalizing Flow
SRFlow therefore directly accounts for the ill-posed nature of the problem, and learns to predict diverse photo-realistic high-resolution images.
Ranked #6 on
Image Super-Resolution
on DIV2K val - 4x upscaling
(using extra training data)
Learning Accurate Dense Correspondences and When to Trust Them
Establishing dense correspondences between a pair of images is an important and general problem.
PDC-Net+: Enhanced Probabilistic Dense Correspondence Network
In order to apply dense methods to real-world applications, such as pose estimation, image manipulation, or 3D reconstruction, it is therefore crucial to estimate the confidence of the predicted matches.
Plug-and-Play Image Restoration with Deep Denoiser Prior
Recent works on plug-and-play image restoration have shown that a denoiser can implicitly serve as the image prior for model-based methods to solve many inverse problems.
Practical Blind Image Denoising via Swin-Conv-UNet and Data Synthesis
While recent years have witnessed a dramatic upsurge of exploiting deep neural networks toward solving image denoising, existing methods mostly rely on simple noise assumptions, such as additive white Gaussian noise (AWGN), JPEG compression noise and camera sensor noise, and a general-purpose blind denoising method for real images remains unsolved.
Ranked #1 on
Image Denoising
on urban100 sigma15
The 2018 PIRM Challenge on Perceptual Image Super-resolution
This paper reports on the 2018 PIRM challenge on perceptual super-resolution (SR), held in conjunction with the Perceptual Image Restoration and Manipulation (PIRM) workshop at ECCV 2018.
Ranked #33 on
Video Quality Assessment
on MSU SR-QA Dataset
AIM 2022 Challenge on Super-Resolution of Compressed Image and Video: Dataset, Methods and Results
The homepage of this challenge is at https://github. com/RenYang-home/AIM22_CompressSR.
Swin2SR: SwinV2 Transformer for Compressed Image Super-Resolution and Restoration
Using this method we can tackle the major issues in training transformer vision models, such as training instability, resolution gaps between pre-training and fine-tuning, and hunger on data.
Generative Adversarial Networks for Extreme Learned Image Compression
We present a learned image compression system based on GANs, operating at extremely low bitrates.
HDNet: High-resolution Dual-domain Learning for Spectral Compressive Imaging
On the one hand, the proposed HR spatial-spectral attention module with its efficient feature fusion provides continuous and fine pixel-level features.
Ranked #5 on
Spectral Reconstruction
on Real HSI
Coarse-to-Fine Sparse Transformer for Hyperspectral Image Reconstruction
Many algorithms have been developed to solve the inverse problem of coded aperture snapshot spectral imaging (CASSI), i. e., recovering the 3D hyperspectral images (HSIs) from a 2D compressive measurement.
Ranked #2 on
Spectral Reconstruction
on Real HSI
Degradation-Aware Unfolding Half-Shuffle Transformer for Spectral Compressive Imaging
In coded aperture snapshot spectral compressive imaging (CASSI) systems, hyperspectral image (HSI) reconstruction methods are employed to recover the spatial-spectral signal from a compressed measurement.
Ranked #1 on
Spectral Reconstruction
on Real HSI
NTIRE 2024 Challenge on Low Light Image Enhancement: Methods and Results
This paper reviews the NTIRE 2024 low light image enhancement challenge, highlighting the proposed solutions and results.
Towards Flexible Blind JPEG Artifacts Removal
Training a single deep blind model to handle different quality factors for JPEG image artifacts removal has been attracting considerable attention due to its convenience for practical usage.
Ranked #1 on
JPEG Artifact Correction
on LIVE1 (Quality 30 Color)
Practical Full Resolution Learned Lossless Image Compression
We propose the first practical learned lossless image compression system, L3C, and show that it outperforms the popular engineered codecs, PNG, WebP and JPEG 2000.
Ranked #3 on
Image Compression
on ImageNet32
NTIRE 2020 Challenge on Spectral Reconstruction from an RGB Image
This paper reviews the second challenge on spectral reconstruction from RGB images, i. e., the recovery of whole-scene hyperspectral (HS) information from a 3-channel RGB image.
InstructIR: High-Quality Image Restoration Following Human Instructions
All-In-One image restoration models can effectively restore images from various types and levels of degradation using degradation-specific information as prompts to guide the restoration model.
Replacing Mobile Camera ISP with a Single Deep Learning Model
The model is trained to convert RAW Bayer data obtained directly from mobile camera sensor into photos captured with a professional high-end DSLR camera, making the solution independent of any particular mobile ISP implementation.
Efficient and Explicit Modelling of Image Hierarchies for Image Restoration
The aim of this paper is to propose a mechanism to efficiently and explicitly model image hierarchies in the global, regional, and local range for image restoration.
Ranked #1 on
Image Defocus Deblurring
on DPD (Dual-view)
Recurrent Video Restoration Transformer with Guided Deformable Attention
Specifically, RVRT divides the video into multiple clips and uses the previously inferred clip feature to estimate the subsequent clip feature.
CDDFuse: Correlation-Driven Dual-Branch Feature Decomposition for Multi-Modality Image Fusion
We then introduce a dual-branch Transformer-CNN feature extractor with Lite Transformer (LT) blocks leveraging long-range attention to handle low-frequency global features and Invertible Neural Networks (INN) blocks focusing on extracting high-frequency local information.
DDFM: Denoising Diffusion Model for Multi-Modality Image Fusion
To leverage strong generative priors and address challenges such as unstable training and lack of interpretability for GAN-based generative methods, we propose a novel fusion algorithm based on the denoising diffusion probabilistic model (DDPM).
Equivariant Multi-Modality Image Fusion
These components enable the net training to follow the principles of the natural sensing-imaging process while satisfying the equivariant imaging prior.
Denoising Diffusion Models for Plug-and-Play Image Restoration
Although diffusion models have shown impressive performance for high-quality image synthesis, their potential to serve as a generative denoiser prior to the plug-and-play IR methods remains to be further explored.
Learned Smartphone ISP on Mobile NPUs with Deep Learning, Mobile AI 2021 Challenge: Report
As the quality of mobile cameras starts to play a crucial role in modern smartphones, more and more attention is now being paid to ISP algorithms used to improve various perceptual aspects of mobile photos.
Reversed Image Signal Processing and RAW Reconstruction. AIM 2022 Challenge Report
Cameras capture sensor RAW images and transform them into pleasant RGB images, suitable for the human eyes, using their integrated Image Signal Processor (ISP).
Perceptual Image Enhancement for Smartphone Real-Time Applications
Our experiments show that, with much fewer parameters and operations, our model can deal with the mentioned artifacts and achieve competitive performance compared with state-of-the-art methods on standard benchmarks.
Real-Time Under-Display Cameras Image Restoration and HDR on Mobile Devices
We propose a lightweight model for blind UDC Image Restoration and HDR, and we also provide a benchmark comparing the performance and runtime of different methods on smartphones.
Efficient multi-lens bokeh effect rendering and transformation
Our method can render Bokeh from an all-in-focus image, or transform the Bokeh of one lens to the effect of another lens without harming the sharp foreground regions in the image.
BSRAW: Improving Blind RAW Image Super-Resolution
Our BSRAW models trained with our pipeline can upscale real-scene RAW images and improve their quality.
Simple Image Signal Processing using Global Context Guidance
First, we propose a novel module that can be integrated into any neural ISP to capture the global context information from the full RAW images.
AIM 2020 Challenge on Image Extreme Inpainting
This report focuses on proposed solutions and results for two different tracks on extreme image inpainting: classical image inpainting and semantically guided image inpainting.
GLU-Net: Global-Local Universal Network for Dense Flow and Correspondences
Establishing dense correspondences between a pair of images is an important and general problem, covering geometric matching, optical flow and semantic correspondences.
GOCor: Bringing Globally Optimized Correspondence Volumes into Your Neural Network
We propose GOCor, a fully differentiable dense matching module, acting as a direct replacement to the feature correlation layer.
Dense Pixel Correspondence Estimation
Feature Correlation
+2
Learning for Video Compression with Hierarchical Quality and Recurrent Enhancement
In our HLVC approach, the hierarchical quality benefits the coding efficiency, since the high quality information facilitates the compression and enhancement of low quality frames at encoder and decoder sides, respectively.
Perceptual Learned Video Compression with Recurrent Conditional GAN
This paper proposes a Perceptual Learned Video Compression (PLVC) approach with recurrent conditional GAN.
Hierarchical Conditional Flow: A Unified Framework for Image Super-Resolution and Image Rescaling
More specifically, HCFlow learns a bijective mapping between HR and LR image pairs by modelling the distribution of the LR image and the rest high-frequency component simultaneously.
Conditional Probability Models for Deep Image Compression
During training, the auto-encoder makes use of the context model to estimate the entropy of its representation, and the context model is concurrently updated to learn the dependencies between the symbols in the latent representation.
Deep Burst Super-Resolution
We propose a novel architecture for the burst super-resolution task.
Ranked #6 on
Burst Image Super-Resolution
on SyntheticBurst
Burst Image Super-Resolution
Multi-Frame Super-Resolution
+1
Deep Reparametrization of Multi-Frame Super-Resolution and Denoising
The deep reparametrization allows us to directly model the image formation process in the latent space, and to integrate learned image priors into the prediction.
Ranked #4 on
Burst Image Super-Resolution
on BurstSR
Night-to-Day Image Translation for Retrieval-based Localization
We then compare the daytime and translated night images to obtain a pose estimate for the night image using the known 6-DOF position of the closest day image.
Mutual Affine Network for Spatially Variant Kernel Estimation in Blind Image Super-Resolution
Extensive experiments on synthetic and real images show that the proposed MANet not only performs favorably for both spatially variant and invariant kernel estimation, but also leads to state-of-the-art blind SR performance when combined with non-blind SR methods.
Rendering Natural Camera Bokeh Effect with Deep Learning
Bokeh is an important artistic effect used to highlight the main object of interest on the photo by blurring all out-of-focus areas.
AIM 2019 Challenge on Real-World Image Super-Resolution: Methods and Results
For training, only one set of source input images is therefore provided in the challenge.
Frequency Separation for Real-World Super-Resolution
Furthermore, we propose to separate the low and high image frequencies and treat them differently during training.
AIM 2020: Scene Relighting and Illumination Estimation Challenge
The first track considered one-to-one relighting; the objective was to relight an input photo of a scene with a different color temperature and illuminant orientation (i. e., light source position).
NTIRE 2021 Depth Guided Image Relighting Challenge
In this paper, we review the NTIRE 2021 depth guided image relighting challenge.
Flow-Guided Sparse Transformer for Video Deblurring
Exploiting similar and sharper scene patches in spatio-temporal neighborhoods is critical for video deblurring.
Ranked #1 on
Deblurring
on DVD
Flow-based Kernel Prior with Application to Blind Super-Resolution
Kernel estimation is generally one of the key problems for blind image super-resolution (SR).
NTIRE 2021 Challenge on High Dynamic Range Imaging: Dataset, Methods and Results
This paper reviews the first challenge on high-dynamic range (HDR) imaging that was part of the New Trends in Image Restoration and Enhancement (NTIRE) workshop, held in conjunction with CVPR 2021.
Real-Time Quantized Image Super-Resolution on Mobile NPUs, Mobile AI 2021 Challenge: Report
Image super-resolution is one of the most popular computer vision problems with many important applications to mobile devices.
ComboGAN: Unrestrained Scalability for Image Domain Translation
This year alone has seen unprecedented leaps in the area of learning-based image translation, namely CycleGAN, by Zhu et al.
Ranked #6 on
Facial Expression Translation
on CelebA
Knowledge Distillation based Degradation Estimation for Blind Super-Resolution
It consists of a knowledge distillation based implicit degradation estimator network (KD-IDE) and an efficient SR network.
NTIRE 2022 Challenge on Efficient Super-Resolution: Methods and Results
The aim was to design a network for single image super-resolution that achieved improvement of efficiency measured according to several metrics including runtime, parameters, FLOPs, activations, and memory consumption while at least maintaining the PSNR of 29. 00dB on DIV2K validation set.
Basic Binary Convolution Unit for Binarized Image Restoration Network
In this study, we reconsider components in binary convolution, such as residual connection, BatchNorm, activation function, and structure, for IR tasks.
VA-DepthNet: A Variational Approach to Single Image Depth Prediction
While state-of-the-art deep neural network methods for SIDP learn the scene depth from images in a supervised setting, they often overlook the invaluable invariances and priors in the rigid scene space, such as the regularity of the scene.
Ranked #18 on
Monocular Depth Estimation
on NYU-Depth V2
LocalViT: Bringing Locality to Vision Transformers
The importance of locality mechanisms is validated in two ways: 1) A wide range of design choices (activation function, layer placement, expansion ratio) are available for incorporating locality mechanisms and all proper choices can lead to a performance gain over the baseline, and 2) The same locality mechanism is successfully applied to 4 vision transformers, which shows the generalization of the locality concept.
Ranked #623 on
Image Classification
on ImageNet
DeepSEE: Deep Disentangled Semantic Explorative Extreme Super-Resolution
To the best of our knowledge, DeepSEE is the first method to leverage semantic maps for explorative super-resolution.
Efficient and Accurate Quantized Image Super-Resolution on Mobile NPUs, Mobile AI & AIM 2022 challenge: Report
While numerous solutions have been proposed for this problem in the past, they are usually not compatible with low-power mobile NPUs having many computational and memory constraints.
Efficient Deep Models for Real-Time 4K Image Super-Resolution. NTIRE 2023 Benchmark and Report
This paper introduces a novel benchmark for efficient upscaling as part of the NTIRE 2023 Real-Time Image Super-Resolution (RTSR) Challenge, which aimed to upscale images from 720p and 1080p resolution to native 4K (x2 and x3 factors) in real-time on commercial GPUs.
CiaoSR: Continuous Implicit Attention-in-Attention Network for Arbitrary-Scale Image Super-Resolution
We explicitly design an implicit attention network to learn the ensemble weights for the nearby local features.
DeFlow: Learning Complex Image Degradations from Unpaired Data with Conditional Flows
We propose DeFlow, a method for learning stochastic image degradations from unpaired data.
How to Train Your Energy-Based Model for Regression
While they are commonly employed for generative image modeling, recent work has applied EBMs also for regression tasks, achieving state-of-the-art performance on object detection and visual tracking.
Ranked #1 on
Visual Object Tracking
on OTB-100
NTIRE 2023 Challenge on Light Field Image Super-Resolution: Dataset, Methods and Results
In this report, we summarize the first NTIRE challenge on light field (LF) image super-resolution (SR), which aims at super-resolving LF images under the standard bicubic degradation with a magnification factor of 4.
3D Appearance Super-Resolution with Deep Learning
Experimental results demonstrate that our proposed networks successfully incorporate the 3D geometric information and super-resolve the texture maps.
mBLIP: Efficient Bootstrapping of Multilingual Vision-LLMs
To this end, we \textit{re-align} an image encoder previously tuned to an English LLM to a new, multilingual LLM -- for this, we leverage multilingual data from a mix of vision-and-language tasks, which we obtain by machine-translating high-quality English data to 95 languages.
Efficient Video Super-Resolution through Recurrent Latent Space Propagation
However, as the motion estimation problem is a highly challenging problem, inaccurate motion compensation may affect the performance of VSR algorithms.
Video Super-Resolution
Image and Video Processing
Edge Guided GANs with Contrastive Learning for Semantic Image Synthesis
We propose a novel ECGAN for the challenging semantic image synthesis task.
Audio-Visual Efficient Conformer for Robust Speech Recognition
We improve previous lip reading methods using an Efficient Conformer back-end on top of a ResNet-18 visual front-end and by adding intermediate CTC losses between blocks.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Towards Real-Time 4K Image Super-Resolution
Over the past few years, high-definition videos and images in 720p (HD), 1080p (FHD), and 4K (UHD) resolution have become standard.
Deep Portrait Quality Assessment. A NTIRE 2024 Challenge Survey
This paper reviews the NTIRE 2024 Portrait Quality Assessment Challenge, highlighting the proposed solutions and results.
3D-Aware Video Generation
Generative models have emerged as an essential building block for many image synthesis and editing tasks.
NTIRE 2021 Challenge on Quality Enhancement of Compressed Video: Dataset and Study
In our study, we analyze the proposed methods of the challenge and several methods in previous works on the proposed LDV dataset.
NTIRE 2021 Challenge on Quality Enhancement of Compressed Video: Methods and Results
This paper reviews the first NTIRE challenge on quality enhancement of compressed video, with a focus on the proposed methods and results.
Learning Filter Basis for Convolutional Neural Network Compression
Convolutional neural networks (CNNs) based solutions have achieved state-of-the-art performances for many computer vision tasks, including classification and super-resolution of images.
Group Sparsity: The Hinge Between Filter Pruning and Decomposition for Network Compression
In this paper, we analyze two popular network compression techniques, i. e. filter pruning and low-rank decomposition, in a unified sense.
SESAME: Semantic Editing of Scenes by Adding, Manipulating or Erasing Objects
In contrast to previous methods that employ a discriminator that trivially concatenates semantics and image as an input, the SESAME discriminator is composed of two input streams that independently process the image and its semantics, using the latter to manipulate the results of the former.
DHP: Differentiable Meta Pruning via HyperNetworks
Passing the sparsified latent vectors through the hypernetworks, the corresponding slices of the generated weight parameters can be removed, achieving the effect of network pruning.
Dense Haze: A benchmark for image dehazing with dense-haze and haze-free images
Characterized by dense and homogeneous hazy scenes, Dense-Haze contains 33 pairs of real hazy and corresponding haze-free images of various outdoor scenes.
NILUT: Conditional Neural Implicit 3D Lookup Tables for Image Enhancement
Moreover, a NILUT can be extended to incorporate multiple styles into a single network with the ability to blend styles implicitly.
Deep Homography for Efficient Stereo Image Compression
To fully explore the mutual information across two stereo images, we use a deep regression model to estimate the homography matrix, i. e., H matrix.
An efficient CNN for spectral reconstruction from RGB images
Recently, the example-based single image spectral reconstruction from RGB images task, aka, spectral super-resolution was approached by means of deep learning by Galliani et al.
Deep Equilibrium Diffusion Restoration with Parallel Sampling
Due to the inherent property of diffusion models, most existing methods need long serial sampling chains to restore HQ images step-by-step, resulting in expensive sampling time and high computation costs.
Arbitrary-Scale Image Synthesis
Positional encodings have enabled recent works to train a single adversarial network that can generate images of different scales.
Learned Smartphone ISP on Mobile GPUs with Deep Learning, Mobile AI & AIM 2022 Challenge: Report
The role of mobile cameras increased dramatically over the past few years, leading to more and more research in automatic image quality enhancement and RAW photo processing.
Event-Free Moving Object Segmentation from Moving Ego Vehicle
Subsequently, we devise EmoFormer, a novel network able to exploit the event data.
SMIT: Stochastic Multi-Label Image-to-Image Translation
This problem is highly challenging due to three main reasons: (i) unpaired datasets, (ii) multiple attributes, and (iii) the multimodality (e. g., style) associated with the translation.
Predict, Prevent, and Evaluate: Disentangled Text-Driven Image Manipulation Empowered by Pre-Trained Vision-Language Model
We propose a novel framework, i. e., Predict, Prevent, and Evaluate (PPE), for disentangled text-driven image manipulation that requires little manual annotation while being applicable to a wide variety of manipulations.
Single-Model and Any-Modality for Video Object Tracking
In practice, most existing RGB trackers learn a single set of parameters to use them across datasets and applications.
SMILE: Semantically-guided Multi-attribute Image and Layout Editing
Additionally, our method is capable of adding, removing or changing either fine-grained or coarse attributes by using an image as a reference or by exploring the style distribution space, and it can be easily extended to head-swapping and face-reenactment applications without being trained on videos.
Global and Local Alignment Networks for Unpaired Image-to-Image Translation
The global alignment network aims to transfer the input image from the source domain to the target domain.
AIM 2020 Challenge on Learned Image Signal Processing Pipeline
This paper reviews the second AIM learned ISP challenge and provides the description of the proposed solutions and results.
SiNeRF: Sinusoidal Neural Radiance Fields for Joint Pose Estimation and Scene Reconstruction
NeRFmm is the Neural Radiance Fields (NeRF) that deal with Joint Optimization tasks, i. e., reconstructing real-world scenes and registering camera parameters simultaneously.
Learning to Improve Image Compression without Changing the Standard Decoder
Therefore, we propose learning to improve the encoding performance with the standard decoder.
NTIRE 2024 Challenge on Short-form UGC Video Quality Assessment: Methods and Results
This paper reviews the NTIRE 2024 Challenge on Shortform UGC Video Quality Assessment (S-UGC VQA), where various excellent solutions are submitted and evaluated on the collected dataset KVQ from popular short-form video platform, i. e., Kuaishou/Kwai Platform.
Lens-to-lens bokeh effect transformation. NTIRE 2023 challenge report
We present the new Bokeh Effect Transformation Dataset (BETD), and review the proposed solutions for this novel task at the NTIRE 2023 Bokeh Effect Transformation Challenge.
NTIRE 2022 Challenge on Super-Resolution and Quality Enhancement of Compressed Video: Dataset, Methods and Results
This challenge includes three tracks.
O-HAZE: a dehazing benchmark with real hazy and haze-free outdoor images
Haze removal or dehazing is a challenging ill-posed problem that has drawn a significant attention in the last few years.
I-HAZE: a dehazing benchmark with real hazy and haze-free indoor images
This represents an important advantage of the I-HAZE dataset that allows us to objectively compare the existing image dehazing techniques using traditional image quality metrics such as PSNR and SSIM.
PyNet-V2 Mobile: Efficient On-Device Photo Processing With Neural Networks
The increased importance of mobile photography created a need for fast and performant RAW image processing pipelines capable of producing good visual results in spite of the mobile camera sensor limitations.
SQAD: Automatic Smartphone Camera Quality Assessment and Benchmarking
Smartphone photography is becoming increasingly popular, but fitting high-performing camera systems within the given space limitations remains a challenge for manufacturers.
Babel-ImageNet: Massively Multilingual Evaluation of Vision-and-Language Representations
We evaluate 8 different publicly available multilingual CLIP models on zero-shot image classification (ZS-IC) for each of the 92 Babel-ImageNet languages, demonstrating a significant gap between English ImageNet performance and that of high-resource languages (e. g., German or Chinese), and an even bigger gap for low-resource languages (e. g., Sinhala or Lao).
Exemplar Guided Face Image Super-Resolution without Facial Landmarks
Nowadays, due to the ubiquitous visual media there are vast amounts of already available high-resolution (HR) face images.
Ranked #1 on
Image Super-Resolution
on VggFace2 - 8x upscaling
A Weakly Supervised Convolutional Network for Change Segmentation and Classification
The core part of our model, the Change Segmentation and Classification (CSC) module, learns an accurate change mask at a hidden layer by using a custom Remapping Block and then segmenting the current input image with the change mask.
Generalized Real-World Super-Resolution through Adversarial Robustness
Instead of learning a dataset-specific degradation, we employ adversarial attacks to create difficult examples that target the model's weaknesses.
Source-free Depth for Object Pop-out
In this work, we adapt such depth inference models for object segmentation using the objects' "pop-out" prior in 3D.
Advancing Learned Video Compression with In-loop Frame Prediction
In this paper, we propose an Advanced Learned Video Compression (ALVC) approach with the in-loop frame prediction module, which is able to effectively predict the target frame from the previously compressed frames, without consuming any bit-rate.
The Ninth NTIRE 2024 Efficient Super-Resolution Challenge Report
In sub-track 1, the practical runtime performance of the submissions was evaluated, and the corresponding score was used to determine the ranking.
Fast Perceptual Image Enhancement
We extend upon the results of Ignatov et al., where they are able to translate images from compact mobile cameras into images with comparable quality to high-resolution photos taken by DSLR cameras.
Conformer and Blind Noisy Students for Improved Image Quality Assessment
Learning-based approaches for perceptual image quality assessment (IQA) usually require both the distorted and reference image for measuring the perceptual quality accurately.
The Heterogeneity Hypothesis: Finding Layer-Wise Differentiated Network Architectures
Based on that, we articulate the heterogeneity hypothesis: with the same training protocol, there exists a layer-wise differentiated network architecture (LW-DNA) that can outperform the original network with regular channel configurations but with a lower level of model complexity.
Towards Image Understanding from Deep Compression without Decoding
Motivated by recent work on deep neural network (DNN)-based image compression methods showing potential improvements in image quality, savings in storage, and bandwidth reduction, we propose to perform image understanding tasks such as classification and segmentation directly on the compressed representations produced by these compression methods.
Local Memory Attention for Fast Video Semantic Segmentation
Our approach aggregates a rich representation of the semantic information in past frames into a memory module.
NTIRE 2020 Challenge on Real Image Denoising: Dataset, Methods and Results
This challenge is based on a newly collected validation and testing image datasets, and hence, named SIDD+.
Object Segmentation by Mining Cross-Modal Semantics
In this paper, we propose a novel approach by mining the Cross-Modal Semantics to guide the fusion and decoding of multimodal features, with the aim of controlling the modal contribution based on relative entropy.
NTIRE 2024 Challenge on Image Super-Resolution ($\times$4): Methods and Results
This paper reviews the NTIRE 2024 challenge on image super-resolution ($\times$4), highlighting the solutions proposed and the outcomes obtained.
Fast Few-Shot Classification by Few-Iteration Meta-Learning
By employing an efficient initialization module and a Steepest Descent based optimization algorithm, our base learner predicts a powerful classifier within only a few iterations.
Extremely Weak Supervised Image-to-Image Translation for Semantic Segmentation
Recent advances in generative models and adversarial training have led to a flourishing image-to-image (I2I) translation literature.
AIM 2022 Challenge on Instagram Filter Removal: Methods and Results
This paper introduces the methods and the results of AIM 2022 challenge on Instagram Filter Removal.
Neural Gradient Regularizer
NGR is applicable to various image types and different image processing tasks, functioning in a zero-shot learning fashion, making it a versatile and plug-and-play regularizer.
Event-Based Eye Tracking. AIS 2024 Challenge Survey
This survey reviews the AIS 2024 Event-Based Eye Tracking (EET) Challenge.
Zero-Pair Image to Image Translation using Domain Conditional Normalization
In this paper, we propose an approach based on domain conditional normalization (DCN) for zero-pair image-to-image translation, i. e., translating between two domains which have no paired training data available but each have paired training data with a third domain.
WESPE: Weakly Supervised Photo Enhancer for Digital Cameras
Low-end and compact mobile cameras demonstrate limited photo quality mainly due to space, hardware and budget constraints.
AI Benchmark: Running Deep Neural Networks on Android Smartphones
Over the last years, the computational power of mobile devices such as smartphones and tablets has grown dramatically, reaching the level of desktop computers available not long ago.
Revisiting Random Channel Pruning for Neural Network Compression
The proposed approach provides a new way to compare different methods, namely how well they behave compared with random pruning.
Optimal transport maps for distribution preserving operations on latent spaces of Generative Models
Generative models such as Variational Auto Encoders (VAEs) and Generative Adversarial Networks (GANs) are typically trained for a fixed prior distribution in the latent space, such as uniform or Gaussian.
Logo Synthesis and Manipulation with Clustered Generative Adversarial Networks
We propose the use of synthetic labels obtained through clustering to disentangle and stabilize GAN training.
Evaluating race and sex diversity in the world's largest companies using deep neural networks
Diversity is one of the fundamental properties for the survival of species, populations, and organizations.
Soft-to-Hard Vector Quantization for End-to-End Learning Compressible Representations
We present a new approach to learn compressible representations in deep architectures with an end-to-end training strategy.
On the Relation between Color Image Denoising and Classification
Inspired by classification models, we propose a novel deep learning architecture for color (multichannel) image denoising and report on thousands of images from ImageNet dataset as well as commonly used imagery.
Single Image Super Resolution - When Model Adaptation Matters
The internal contents of the low resolution input image is neglected with deep modeling despite the earlier works showing the power of using such internal priors.
Failure Detection for Facial Landmark Detectors
Most face applications depend heavily on the accuracy of the face and facial landmarks detectors employed.
Generic 3D Convolutional Fusion for image restoration
Unlike other methods adapted to different tasks, our method uses the exact same convolutional network architecture to address both image denois- ing and single image super-resolution.
k2-means for fast and accurate large scale clustering
k^2-means builds upon the standard k-means (Lloyd's algorithm) and combines a new strategy to accelerate the convergence with a new low time complexity divisive initialization.
Fast Optical Flow using Dense Inverse Search
Most recent works in optical flow extraction focus on the accuracy and neglect the time complexity.
Some like it hot - visual guidance for preference prediction
Our computational pipeline comprises a face detector, convolutional neural networks for the extraction of deep features, standard support vector regression for gender, age and facial beauty, and - as the main novelties - visual regularized collaborative filtering to infer inter-person preferences as well as a novel regression technique for handling visual queries without rating history.
Seven ways to improve example-based single image super resolution
In this paper we present seven techniques that everybody should know to improve example-based single image super resolution (SR): 1) augmentation of data, 2) use of large dictionaries with efficient search structures, 3) cascading, 4) image self-similarities, 5) back projection refinement, 6) enhanced prediction by consistency check, and 7) context reasoning.
Ranked #45 on
Image Super-Resolution
on BSD100 - 4x upscaling
Multi-bin Trainable Linear Unit for Fast Image Restoration Networks
Tremendous advances in image restoration tasks such as denoising and super-resolution have been achieved using neural networks.
Towards Closing the Gap in Weakly Supervised Semantic Segmentation with DCNNs: Combining Local and Global Models
We reduce the gap by 64. 2% whereas the current state-of-the-art reduces it only by 57. 5%.
Weakly supervised Semantic Segmentation
Weakly-Supervised Semantic Segmentation
Adversarial Sampling for Active Learning
This paper proposes asal, a new GAN based active learning method that generates high entropy samples.
PIRM Challenge on Perceptual Image Enhancement on Smartphones: Report
This paper reviews the first challenge on efficient perceptual image enhancement with the focus on deploying deep learning models on smartphones.
Towards Spectral Estimation from a Single RGB Image in the Wild
In contrast to the current literature, we address the problem of estimating the spectrum from a single common trichromatic RGB image obtained under unconstrained settings (e. g. unknown camera parameters, unknown scene radiance, unknown scene contents).
Spectral Estimation From A Single Rgb Image
Spectral Reconstruction
Metric Imitation by Manifold Transfer for Efficient Vision Applications
In particular, MI consists of: 1) quantifying the properties of source metrics as manifold geometry, 2) transferring the manifold from source domain to target domain, and 3) learning a mapping of TFs so that the manifold is approximated as well as possible in the mapped feature domain.
Anchored Regression Networks Applied to Age Estimation and Super Resolution
We propose the Anchored Regression Network (ARN), a nonlinear regression network which can be seamlessly integrated into various networks or can be used stand-alone when the features have already been fixed.
PIRM2018 Challenge on Spectral Image Super-Resolution: Dataset and Study
This paper introduces a newly collected and novel dataset (StereoMSI) for example-based single and colour-guided spectral image super-resolution.
Unsupervised Learning for Real-World Super-Resolution
Instead of directly addressing this problem, most works employ the popular bicubic downsampling strategy to artificially generate a corresponding low resolution image.
AI Benchmark: All About Deep Learning on Smartphones in 2019
The performance of mobile AI accelerators has been evolving rapidly in the past two years, nearly doubling with each new generation of SoCs.
Divide-and-Conquer Adversarial Learning for High-Resolution Image and Video Enhancement
This paper introduces a divide-and-conquer inspired adversarial learning (DACAL) approach for photo enhancement.
AIM 2019 Challenge on Image Demoireing: Dataset and Study
In addition to describing the dataset and its creation, this paper also reviews the challenge tracks, competition, and results, the latter summarizing the current state-of-the-art on this dataset.
AIM 2019 Challenge on Image Demoireing: Methods and Results
A new dataset, called LCDMoire was created for this challenge, and consists of 10, 200 synthetically generated image pairs (moire and clean ground truth).
Adversarial Feature Distribution Alignment for Semi-Supervised Learning
We test our method on CIFAR10 and SVHN.
Efficient Video Semantic Segmentation with Labels Propagation and Refinement
(ii) On the GPU, two Convolutional Neural Networks: A main segmentation network that is used to predict dense semantic labels from scratch, and a Refiner that is designed to improve predictions from previous frames with the help of a fast Inconsistencies Attention Module (IAM).
Self-Supervised 2D Image to 3D Shape Translation with Disentangled Representations
In this paper, we propose SIST, a Self-supervised Image to Shape Translation framework that fulfills three tasks: (i) reconstructing the 3D shape from a single image; (ii) learning disentangled representations for shape, appearance and viewpoint; and (iii) generating a realistic RGB image from these independent factors.
Unsupervised Multimodal Video-to-Video Translation via Self-Supervised Learning
Existing unsupervised video-to-video translation methods fail to produce translated videos which are frame-wise realistic, semantic information preserving and video-level consistent.
NTIRE 2020 Challenge on Image and Video Deblurring
This paper reviews the NTIRE 2020 Challenge on Image and Video Deblurring.
AIM 2019 Challenge on Video Temporal Super-Resolution: Methods and Results
Videos contain various types and strengths of motions that may look unnaturally discontinuous in time when the recorded frame rate is low.
NTIRE 2020 Challenge on Perceptual Extreme Super-Resolution: Methods and Results
This paper reviews the NTIRE 2020 challenge on perceptual extreme super-resolution with focus on proposed solutions and results.
NTIRE 2020 Challenge on Video Quality Mapping: Methods and Results
For track 2, some existing methods are evaluated, showing promising solutions to the weakly-supervised video quality mapping problem.
NH-HAZE: An Image Dehazing Benchmark with Non-Homogeneous Hazy and Haze-Free Images
The non-homogeneous haze has been introduced in the scene using a professional haze generator that imitates the real conditions of hazy scenes.
NTIRE 2020 Challenge on NonHomogeneous Dehazing
We focus on the proposed solutions and their results evaluated on NH-Haze, a novel dataset consisting of 55 pairs of real haze free and nonhomogeneous hazy images recorded outdoor.
NTIRE 2020 Challenge on Image Demoireing: Methods and Results
Track 1 targeted the single image demoireing problem, which seeks to remove moire patterns from a single image.
Learning Context-Based Non-local Entropy Modeling for Image Compression
To address this issue, we propose a non-local operation for context modeling by employing the global similarity within the context.
Flexible Example-based Image Enhancement with Task Adaptive Global Feature Self-Guided Network
We propose the first practical multitask image enhancement network, that is able to learn one-to-many and many-to-one image mappings.
Video Super Resolution Based on Deep Learning: A Comprehensive Survey
To the best of our knowledge, this work is the first systematic review on VSR tasks, and it is expected to make a contribution to the development of recent studies in this area and potentially deepen our understanding to the VSR techniques based on deep learning.
AIM 2020 Challenge on Video Extreme Super-Resolution: Methods and Results
Missing information can be restored well in this region, especially in HR videos, where the high-frequency content mostly consists of texture details.
AIM 2020 Challenge on Real Image Super-Resolution: Methods and Results
This paper introduces the real image Super-Resolution (SR) challenge that was part of the Advances in Image Manipulation (AIM) workshop, held in conjunction with ECCV 2020.
AIM 2020 Challenge on Video Temporal Super-Resolution
Videos in the real-world contain various dynamics and motions that may look unnaturally discontinuous in time when the recordedframe rate is low.
MS-RANAS: Multi-Scale Resource-Aware Neural Architecture Search
Neural Architecture Search (NAS) has proved effective in offering outperforming alternatives to handcrafted neural networks.
The Card Shuffling Hypotheses: Building a Time and Memory Efficient Graph Convolutional Network
State-of-the-art GCNs adopt $K$-nearest neighbor (KNN) searches for local feature aggregation and feature extraction operations from layer to layer.
Self-Supervised Shadow Removal
Shadow removal is an important computer vision task aiming at the detection and successful removal of the shadow produced by an occluded light source and a photo-realistic restoration of the image contents.
AIM 2020 Challenge on Rendering Realistic Bokeh
This paper reviews the second AIM realistic bokeh effect rendering challenge and provides the description of the proposed solutions and results.
An Efficient Recurrent Adversarial Framework for Unsupervised Real-Time Video Enhancement
Video enhancement is a challenging problem, more than that of stills, mainly due to high computational cost, larger data volumes and the difficulty of achieving consistency in the spatio-temporal domain.
Trilevel Neural Architecture Search for Efficient Single Image Super-Resolution
Modern solutions to the single image super-resolution (SISR) problem using deep neural networks aim not only at better performance accuracy but also at a lighter and computationally efficient model.
Towards Efficient Graph Convolutional Networks for Point Cloud Handling
In this paper, we aim at improving the computational efficiency of graph convolutional networks (GCNs) for learning on point clouds.
NTIRE 2021 Challenge on Image Deblurring
In this challenge report, we describe the challenge specifics and the evaluation results from the 2 competition tracks with the proposed solutions.
NTIRE 2021 Challenge on Video Super-Resolution
Super-Resolution (SR) is a fundamental computer vision task that aims to obtain a high-resolution clean image from the given low-resolution counterpart.
Editorial: Introduction to the Issue on Deep Learning for Image/Video Restoration and Compression
Recent works have shown that learned models can achieve significant performance gains, especially in terms of perceptual quality measures, over traditional methods.
NTIRE 2021 Challenge on Perceptual Image Quality Assessment
This paper reports on the NTIRE 2021 challenge on perceptual image quality assessment (IQA), held in conjunction with the New Trends in Image Restoration and Enhancement workshop (NTIRE) workshop at CVPR 2021.
Fast and Accurate Camera Scene Detection on Smartphones
AI-powered automatic camera scene detection mode is nowadays available in nearly any modern smartphone, though the problem of accurate scene prediction has not yet been addressed by the research community.
Fast and Accurate Single-Image Depth Estimation on Mobile Devices, Mobile AI 2021 Challenge: Report
While many solutions have been proposed for this task, they are usually very computationally expensive and thus are not applicable for on-device inference.
