Search Results for author:
Found 61 papers, 24 papers with code
The Power of Context: How Multimodality Improves Image Super-Resolution
Single-image super-resolution (SISR) remains challenging due to the inherent difficulty of recovering fine-grained details and preserving perceptual quality from low-resolution inputs.
On the Relation Between Linear Diffusion and Power Iteration
These models learn the implicit distribution given by the training dataset, and sample new data by transforming random noise through the reverse process, which can be thought of as gradual denoising.
Stochastic Deep Restoration Priors for Imaging Inverse Problems
We introduce Stochastic deep Restoration Priors (ShaRP), a novel method that leverages an ensemble of such restoration models to regularize inverse problems.
A Survey on Diffusion Models for Inverse Problems
Diffusion models have become increasingly popular for generative modeling due to their ability to generate high-quality samples.
Denoising: A Powerful Building-Block for Imaging, Inverse Problems, and Machine Learning
Denoising, the process of reducing random fluctuations in a signal to emphasize essential patterns, has been a fundamental problem of interest since the dawn of modern scientific inquiry.
ArtVLM: Attribute Recognition Through Vision-Based Prefix Language Modeling
Recognizing and disentangling visual attributes from objects is a foundation to many computer vision applications.
Bigger is not Always Better: Scaling Properties of Latent Diffusion Models
We study the scaling properties of latent diffusion models (LDMs) with an emphasis on their sampling efficiency.
SPIRE: Semantic Prompt-Driven Image Restoration
In this paper, we develop SPIRE, a Semantic and restoration Prompt-driven Image Restoration framework that leverages natural language as a user-friendly interface to control the image restoration process.
A Restoration Network as an Implicit Prior
Image denoisers have been shown to be powerful priors for solving inverse problems in imaging.
CoDi: Conditional Diffusion Distillation for Higher-Fidelity and Faster Image Generation
Our conditional-task learning and distillation approach outperforms previous distillation methods, achieving a new state-of-the-art in producing high-quality images with very few steps (e. g., 1-4) across multiple tasks, including super-resolution, text-guided image editing, and depth-to-image generation.
Prompt-tuning latent diffusion models for inverse problems
We propose a new method for solving imaging inverse problems using text-to-image latent diffusion models as general priors.
MULLER: Multilayer Laplacian Resizer for Vision
Specifically, we select a state-of-the-art vision Transformer, MaxViT, as the baseline, and show that, if trained with MULLER, MaxViT gains up to 0. 6% top-1 accuracy, and meanwhile enjoys 36% inference cost saving to achieve similar top-1 accuracy on ImageNet-1k, as compared to the standard training scheme.
VILA: Learning Image Aesthetics from User Comments with Vision-Language Pretraining
Our results show that our pretrained aesthetic vision-language model outperforms prior works on image aesthetic captioning over the AVA-Captions dataset, and it has powerful zero-shot capability for aesthetic tasks such as zero-shot style classification and zero-shot IAA, surpassing many supervised baselines.
Ranked #46 on
Video Quality Assessment
on MSU SR-QA Dataset
SVDiff: Compact Parameter Space for Diffusion Fine-Tuning
Diffusion models have achieved remarkable success in text-to-image generation, enabling the creation of high-quality images from text prompts or other modalities.
Inversion by Direct Iteration: An Alternative to Denoising Diffusion for Image Restoration
In conditional denoising diffusion image restoration the denoising network generates the restored image by repeatedly denoising an initial image of pure noise, conditioned on the degraded input.
MRET: Multi-resolution Transformer for Video Quality Assessment
No-reference video quality assessment (NR-VQA) for user generated content (UGC) is crucial for understanding and improving visual experience.
Multiscale Structure Guided Diffusion for Image Deblurring
We evaluate a single-dataset trained model on diverse datasets and demonstrate more robust deblurring results with fewer artifacts on unseen data.
Soft Diffusion: Score Matching for General Corruptions
To reverse these general diffusions, we propose a new objective called Soft Score Matching that provably learns the score function for any linear corruption process and yields state of the art results for CelebA.
Ranked #8 on
Image Generation
on CelebA 64x64
MaxViT: Multi-Axis Vision Transformer
We also show that our proposed model expresses strong generative modeling capability on ImageNet, demonstrating the superior potential of MaxViT blocks as a universal vision module.
Ranked #1 on
Object Detection
on COCO 2017
MAXIM: Multi-Axis MLP for Image Processing
In this work, we present a multi-axis MLP based architecture called MAXIM, that can serve as an efficient and flexible general-purpose vision backbone for image processing tasks.
Ranked #1 on
Deblurring
on HIDE (trained on GOPRO)
Deblurring via Stochastic Refinement
Unlike existing techniques, we train a stochastic sampler that refines the output of a deterministic predictor and is capable of producing a diverse set of plausible reconstructions for a given input.
Solving Image PDEs with a Shallow Network
Partial differential equations (PDEs) are typically used as models of physical processes but are also of great interest in PDE-based image processing.
MUSIQ: Multi-scale Image Quality Transformer
To accommodate this, the input images are usually resized and cropped to a fixed shape, causing image quality degradation.
Ranked #3 on
Image Quality Assessment
on MSU NR VQA Database
Rich Features for Perceptual Quality Assessment of UGC Videos
Besides the subjective ratings and content labels of the dataset, we also propose a DNN-based framework to thoroughly analyze importance of content, technical quality, and compression level in perceptual quality.
COMISR: Compression-Informed Video Super-Resolution
Most video super-resolution methods focus on restoring high-resolution video frames from low-resolution videos without taking into account compression.
Deep 3D-to-2D Watermarking: Embedding Messages in 3D Meshes and Extracting Them from 2D Renderings
Digital watermarking is widely used for copyright protection.
Interpretable Unsupervised Diversity Denoising and Artefact Removal
This work presents an interpretable approach for unsupervised and diverse image restoration.
Patch Craft: Video Denoising by Deep Modeling and Patch Matching
Our algorithm augments video sequences with patch-craft frames and feeds them to a CNN.
Ranked #4 on
Video Denoising
on DAVIS sigma20
Learning to Resize Images for Computer Vision Tasks
Moreover, we show that the proposed resizer can also be useful for fine-tuning the classification baselines for other vision tasks.
High Perceptual Quality Image Denoising with a Posterior Sampling CGAN
We showcase our proposed method with a novel denoiser architecture that achieves the reformed denoising goal and produces vivid and diverse outcomes in immoderate noise levels.
Deep Perceptual Image Quality Assessment for Compression
In this paper we propose the largest image compression quality dataset to date with human perceptual preferences, enabling the use of deep learning, and we develop a full reference perceptual quality assessment metric for lossy image compression that outperforms the existing state-of-the-art methods.
Mobile Computational Photography: A Tour
The first mobile camera phone was sold only 20 years ago, when taking pictures with one's phone was an oddity, and sharing pictures online was unheard of.
Projected Distribution Loss for Image Enhancement
More explicitly, we show that in imaging applications such as denoising, super-resolution, demosaicing, deblurring and JPEG artifact removal, the proposed learning loss outperforms the current state-of-the-art on reference-based perceptual losses.
Polyblur: Removing mild blur by polynomial reblurring
The proposed method estimates and removes mild blur from a 12MP image on a modern mobile phone in a fraction of a second.
Learning to Reduce Defocus Blur by Realistically Modeling Dual-Pixel Data
Leveraging these realistic synthetic DP images, we introduce a recurrent convolutional network (RCN) architecture that improves deblurring results and is suitable for use with single-frame and multi-frame data (e. g., video) captured by DP sensors.
Ranked #13 on
Image Defocus Deblurring
on DPD (Dual-view)
(using extra training data)
Rank-smoothed Pairwise Learning In Perceptual Quality Assessment
Training a model on these pairwise preferences is a common deep learning approach.
Multi-path Neural Networks for On-device Multi-domain Visual Classification
This approach effectively reduces the total number of parameters and FLOPS, encouraging positive knowledge transfer while mitigating negative interference across domains.
The Rate-Distortion-Accuracy Tradeoff: JPEG Case Study
As a case study, we focus on the design of the quantization tables in the JPEG compression standard.
Regularization by Denoising via Fixed-Point Projection (RED-PRO)
Two such methods are the Plug-and-Play Prior (PnP) and Regularization by Denoising (RED).
GIFnets: Differentiable GIF Encoding Framework
DitherNet manipulates the input image to reduce color banding artifacts and provides an alternative to traditional dithering.
Creating High Resolution Images with a Latent Adversarial Generator
That is to say, instead of generating an arbitrary image as a sample from the manifold of natural images, we propose to sample images from a particular "subspace" of natural images, directed by a low-resolution image from the same subspace.
Super-Resolving Commercial Satellite Imagery Using Realistic Training Data
We propose a realistic training data generation model for commercial satellite imagery products, which includes not only the imaging process on satellites but also the post-process on the ground.
Image Stylization: From Predefined to Personalized
We present a framework for interactive design of new image stylizations using a wide range of predefined filter blocks.
Better Compression with Deep Pre-Editing
In this work we aim to break the unholy connection between bit-rate and image quality, and propose a way to circumvent compression artifacts by pre-editing the incoming image and modifying its content to fit the given bits.
Distortion Agnostic Deep Watermarking
Watermarking is the process of embedding information into an image that can survive under distortions, while requiring the encoded image to have little or no perceptual difference from the original image.
LIDIA: Lightweight Learned Image Denoising with Instance Adaptation
This work proposes a novel lightweight learnable architecture for image denoising, and presents a combination of supervised and unsupervised training of it, the first aiming for a universal denoiser and the second for adapting it to the incoming image.
Deep K-SVD Denoising
The question we address in this paper is whether K-SVD was brought to its peak in its original conception, or whether it can be made competitive again.
Handheld Multi-Frame Super-Resolution
In this paper, we supplant the use of traditional demosaicing in single-frame and burst photography pipelines with a multiframe super-resolution algorithm that creates a complete RGB image directly from a burst of CFA raw images.
DeepRED: Deep Image Prior Powered by RED
Inverse problems in imaging are extensively studied, with a variety of strategies, tools, and theory that have been accumulated over the years.
Ranked #7 on
Image Super-Resolution
on Set14 - 8x upscaling
Rendition: Reclaiming what a black box takes away
The premise of our work is deceptively familiar: A black box $f(\cdot)$ has altered an image $\mathbf{x} \rightarrow f(\mathbf{x})$.
Local Kernels that Approximate Bayesian Regularization and Proximal Operators
In this work, we broadly connect kernel-based filtering (e. g. approaches such as the bilateral filters and nonlocal means, but also many more) with general variational formulations of Bayesian regularized least squares, and the related concept of proximal operators.
Graphic Narrative with Interactive Stylization Design
In parallel to this manual design, we propose a novel procedural approach that automatically assembles sequences of filters for innovative results.
Graphics
Learned Perceptual Image Enhancement
Learning a typical image enhancement pipeline involves minimization of a loss function between enhanced and reference images.
BLADE: Filter Learning for General Purpose Computational Photography
The Rapid and Accurate Image Super Resolution (RAISR) method of Romano, Isidoro, and Milanfar is a computationally efficient image upscaling method using a trained set of filters.
NIMA: Neural Image Assessment
Automatically learned quality assessment for images has recently become a hot topic due to its usefulness in a wide variety of applications such as evaluating image capture pipelines, storage techniques and sharing media.
Ranked #4 on
Aesthetics Quality Assessment
on AVA
The Little Engine that Could: Regularization by Denoising (RED)
As opposed to the $P^3$ method, we offer Regularization by Denoising (RED): using the denoising engine in defining the regularization of the inverse problem.
Linear Support Tensor Machine: Pedestrian Detection in Thermal Infrared Images
Pedestrian detection in thermal infrared images poses unique challenges because of the low resolution and noisy nature of the image.
Style-Transfer via Texture-Synthesis
Recent work on this problem adopting Convolutional Neural-networks (CNN) ignited a renewed interest in this field, due to the very impressive results obtained.
Fast Multi-Layer Laplacian Enhancement
A novel, fast and practical way of enhancing images is introduced in this paper.
RAISR: Rapid and Accurate Image Super Resolution
Our approach additionally includes an extremely efficient way to produce an image that is significantly sharper than the input blurry one, without introducing artifacts such as halos and noise amplification.
