Search Results for author:
Found 98 papers, 51 papers with code
R-Bench: Are your Large Multimodal Model Robust to Real-world Corruptions?
However, various corruptions in the real world mean that images will not be as ideal as in simulations, presenting significant challenges for the practical application of LMMs.
AIM 2024 Challenge on Video Super-Resolution Quality Assessment: Methods and Results
This paper presents the Video Super-Resolution (SR) Quality Assessment (QA) Challenge that was part of the Advances in Image Manipulation (AIM) workshop, held in conjunction with ECCV 2024.
Q-Bench-Video: Benchmarking the Video Quality Understanding of LMMs
With the rising interest in research on Large Multi-modal Models (LMMs) for video understanding, many studies have emphasized general video comprehension capabilities, neglecting the systematic exploration into video quality understanding.
Subjective and Objective Quality-of-Experience Evaluation Study for Live Video Streaming
For the subjective QoE study, we introduce the first live video streaming QoE dataset, TaoLive QoE, which consists of $42$ source videos collected from real live broadcasts and $1, 155$ corresponding distorted ones degraded due to a variety of streaming distortions, including conventional streaming distortions such as compression, stalling, as well as live streaming-specific distortions like frame skipping, variable frame rate, etc.
AIM 2024 Challenge on Video Saliency Prediction: Methods and Results
The goal of the participants was to develop a method for predicting accurate saliency maps for the provided set of video sequences.
Towards Effective User Attribution for Latent Diffusion Models via Watermark-Informed Blending
Rapid advancements in multimodal large language models have enabled the creation of hyper-realistic images from textual descriptions.
Explore the Hallucination on Low-level Perception for MLLMs
However, these models also exhibit hallucinations, which limit their reliability as AI systems, especially in tasks involving low-level visual perception and understanding.
3DGCQA: A Quality Assessment Database for 3D AI-Generated Contents
To provide a valuable resource for future research and development in 3D content generation and quality assessment, the dataset has been open-sourced in https://github. com/zyj-2000/3DGCQA.
Exploring Rich Subjective Quality Information for Image Quality Assessment in the Wild
RichIQA is characterized by two key novel designs: (1) a three-stage image quality prediction network which exploits the powerful feature representation capability of the Convolutional vision Transformer (CvT) and mimics the short-term and long-term memory mechanisms of human brain; (2) a multi-label training strategy in which rich subjective quality information like MOS, SOS and DOS are concurrently used to train the quality prediction network.
Assessing UHD Image Quality from Aesthetics, Distortions, and Saliency
To address this problem, we design a multi-branch deep neural network (DNN) to assess the quality of UHD images from three perspectives: global aesthetic characteristics, local technical distortions, and salient content perception.
LMM-VQA: Advancing Video Quality Assessment with Large Multimodal Models
Then, we design a spatiotemporal vision encoder to extract spatial and temporal features to represent the quality characteristics of videos, which are subsequently mapped into the language space by the spatiotemporal projector for modality alignment.
AIM 2024 Challenge on Compressed Video Quality Assessment: Methods and Results
The challenge aimed to evaluate the performance of VQA methods on a diverse dataset of 459 videos, encoded with 14 codecs of various compression standards (AVC/H. 264, HEVC/H. 265, AV1, and VVC/H. 266) and containing a comprehensive collection of compression artifacts.
How Does Audio Influence Visual Attention in Omnidirectional Videos? Database and Model
To advance the research on audio-visual saliency prediction for ODVs, we further establish a new benchmark based on the AVS-ODV database by testing numerous state-of-the-art saliency models, including visual-only models and audio-visual models.
SG-JND: Semantic-Guided Just Noticeable Distortion Predictor For Image Compression
Just noticeable distortion (JND), representing the threshold of distortion in an image that is minimally perceptible to the human visual system (HVS), is crucial for image compression algorithms to achieve a trade-off between transmission bit rate and image quality.
ESIQA: Perceptual Quality Assessment of Vision-Pro-based Egocentric Spatial Images
However, the corresponding image quality assessment (IQA) research for egocentric spatial images is still lacking.
Benchmarking AIGC Video Quality Assessment: A Dataset and Unified Model
For the objective perspective, we establish a benchmark for evaluating existing quality assessment metrics on the LGVQ dataset, which reveals that current metrics perform poorly on the LGVQ dataset.
UniProcessor: A Text-induced Unified Low-level Image Processor
In this paper, we propose a text-induced unified image processor for low-level vision tasks, termed UniProcessor, which can effectively process various degradation types and levels, and support multimodal control.
UNQA: Unified No-Reference Quality Assessment for Audio, Image, Video, and Audio-Visual Content
In this paper, we propose the Unified No-reference Quality Assessment model (UNQA) for audio, image, video, and A/V content, which tries to train a single QA model across different media modalities.
HazeCLIP: Towards Language Guided Real-World Image Dehazing
Existing methods have achieved remarkable performance in single image dehazing, particularly on synthetic datasets.
GLARE: Low Light Image Enhancement via Generative Latent Feature based Codebook Retrieval
Most existing Low-light Image Enhancement (LLIE) methods either directly map Low-Light (LL) to Normal-Light (NL) images or use semantic or illumination maps as guides.
Ranked #1 on
Low-Light Image Enhancement
on LOLv2-synthetic
Quality-guided Skin Tone Enhancement for Portrait Photography
In this paper, we propose a quality-guided image enhancement paradigm that enables image enhancement models to learn the distribution of images with various quality ratings.
CMC-Bench: Towards a New Paradigm of Visual Signal Compression
With the development of Large Multimodal Models (LMMs), a Cross Modality Compression (CMC) paradigm of Image-Text-Image has emerged.
GAIA: Rethinking Action Quality Assessment for AI-Generated Videos
Assessing action quality is both imperative and challenging due to its significant impact on the quality of AI-generated videos, further complicated by the inherently ambiguous nature of actions within AI-generated video (AIGV).
A-Bench: Are LMMs Masters at Evaluating AI-generated Images?
How to accurately and efficiently assess AI-generated images (AIGIs) remains a critical challenge for generative models.
Dual-Branch Network for Portrait Image Quality Assessment
We employ the fidelity loss to train the model via a learning-to-rank manner to mitigate inconsistencies in quality scores in the portrait image quality assessment dataset PIQ.
Enhancing Blind Video Quality Assessment with Rich Quality-aware Features
Motivated by previous researches that leverage pre-trained features extracted from various computer vision models as the feature representation for BVQA, we further explore rich quality-aware features from pre-trained blind image quality assessment (BIQA) and BVQA models as auxiliary features to help the BVQA model to handle complex distortions and diverse content of social media videos.
Understanding and Evaluating Human Preferences for AI Generated Images with Instruction Tuning
Artificial Intelligence Generated Content (AIGC) has grown rapidly in recent years, among which AI-based image generation has gained widespread attention due to its efficient and imaginative image creation ability.
Compression-Realized Deep Structural Network for Video Quality Enhancement
This paper focuses on the task of quality enhancement for compressed videos.
LMM-PCQA: Assisting Point Cloud Quality Assessment with LMM
Although large multi-modality models (LMMs) have seen extensive exploration and application in various quality assessment studies, their integration into Point Cloud Quality Assessment (PCQA) remains unexplored.
Large Multi-modality Model Assisted AI-Generated Image Quality Assessment
Traditional deep neural network (DNN)-based image quality assessment (IQA) models leverage convolutional neural networks (CNN) or Transformer to learn the quality-aware feature representation, achieving commendable performance on natural scene images.
NTIRE 2024 Quality Assessment of AI-Generated Content Challenge
A total of 196 participants have registered in the video track.
AIS 2024 Challenge on Video Quality Assessment of User-Generated Content: Methods and Results
The performance of the top-5 submissions is reviewed and provided here as a survey of diverse deep models for efficient video quality assessment of user-generated content.
NTIRE 2024 Challenge on Short-form UGC Video Quality Assessment: Methods and Results
This paper reviews the NTIRE 2024 Challenge on Shortform UGC Video Quality Assessment (S-UGC VQA), where various excellent solutions are submitted and evaluated on the collected dataset KVQ from popular short-form video platform, i. e., Kuaishou/Kwai Platform.
THQA: A Perceptual Quality Assessment Database for Talking Heads
In the realm of media technology, digital humans have gained prominence due to rapid advancements in computer technology.
How is Visual Attention Influenced by Text Guidance? Database and Model
Finally, considering the effect of text descriptions on visual attention, while most existing saliency models ignore this impact, we further propose a text-guided saliency (TGSal) prediction model, which extracts and integrates both image features and text features to predict the image saliency under various text-description conditions.
AIGIQA-20K: A Large Database for AI-Generated Image Quality Assessment
With the rapid advancements in AI-Generated Content (AIGC), AI-Generated Images (AIGIs) have been widely applied in entertainment, education, and social media.
AIGCOIQA2024: Perceptual Quality Assessment of AI Generated Omnidirectional Images
Finally, we conduct a benchmark experiment to evaluate the performance of state-of-the-art IQA models on our database.
Subjective-Aligned Dataset and Metric for Text-to-Video Quality Assessment
Based on T2VQA-DB, we propose a novel transformer-based model for subjective-aligned Text-to-Video Quality Assessment (T2VQA).
Perceptual Video Quality Assessment: A Survey
Perceptual video quality assessment plays a vital role in the field of video processing due to the existence of quality degradations introduced in various stages of video signal acquisition, compression, transmission and display.
Q-Refine: A Perceptual Quality Refiner for AI-Generated Image
With the rapid evolution of the Text-to-Image (T2I) model in recent years, their unsatisfactory generation result has become a challenge.
Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levels
The explosion of visual content available online underscores the requirement for an accurate machine assessor to robustly evaluate scores across diverse types of visual contents.
Ranked #1 on
Video Quality Assessment
on LIVE-FB LSVQ
Perceptual Quality Assessment for Video Frame Interpolation
To evaluate the quality of VFI frames without reference videos, a no-reference perceptual quality assessment method is proposed in this paper.
Q-Boost: On Visual Quality Assessment Ability of Low-level Multi-Modality Foundation Models
Recent advancements in Multi-modality Large Language Models (MLLMs) have demonstrated remarkable capabilities in complex high-level vision tasks.
Exploring the Naturalness of AI-Generated Images
In this paper, we take the first step to benchmark and assess the visual naturalness of AI-generated images.
FS-BAND: A Frequency-Sensitive Banding Detector
Banding artifact, as known as staircase-like contour, is a common quality annoyance that happens in compression, transmission, etc.
BAND-2k: Banding Artifact Noticeable Database for Banding Detection and Quality Assessment
The quality score of a banding image is generated by pooling the banding detection maps masked by the spatial frequency filters.
Audio-visual Saliency for Omnidirectional Videos
Visual saliency prediction for omnidirectional videos (ODVs) has shown great significance and necessity for omnidirectional videos to help ODV coding, ODV transmission, ODV rendering, etc..
Simple Baselines for Projection-based Full-reference and No-reference Point Cloud Quality Assessment
Point clouds are widely used in 3D content representation and have various applications in multimedia.
A No-Reference Quality Assessment Method for Digital Human Head
In this paper, we develop a novel no-reference (NR) method based on Transformer to deal with DHQA in a multi-task manner.
Geometry-Aware Video Quality Assessment for Dynamic Digital Human
Usually, DDHs are displayed as 2D rendered animation videos and it is natural to adapt video quality assessment (VQA) methods to DDH quality assessment (DDH-QA) tasks.
Joint Gaze-Location and Gaze-Object Detection
In contrast, we reframe the gaze following detection task as detecting human head locations and their gaze followings simultaneously, aiming at jointly detect human gaze location and gaze object in a unified and single-stage pipeline.
StableVQA: A Deep No-Reference Quality Assessment Model for Video Stability
Indeed, most existing quality assessment models evaluate video quality as a whole without specifically taking the subjective experience of video stability into consideration.
Analysis of Video Quality Datasets via Design of Minimalistic Video Quality Models
By minimalistic, we restrict our family of BVQA models to build only upon basic blocks: a video preprocessor (for aggressive spatiotemporal downsampling), a spatial quality analyzer, an optional temporal quality analyzer, and a quality regressor, all with the simplest possible instantiations.
Perceptual Quality Assessment of Omnidirectional Audio-visual Signals
Omnidirectional videos (ODVs) play an increasingly important role in the application fields of medical, education, advertising, tourism, etc.
NTIRE 2023 Quality Assessment of Video Enhancement Challenge
61 participating teams submitted their prediction results during the development phase, with a total of 3168 submissions.
Blind Image Quality Assessment: A Fuzzy Neural Network for Opinion Score Distribution Prediction
On the other hand, we also prove the feasibility of the proposed method in predicting the MOS of image quality on several popular IQA databases, including CSIQ, TID2013, LIVE MD, and LIVE Challenge.
Subjective and Objective Audio-Visual Quality Assessment for User Generated Content
Then, to facilitate the development of AVQA fields, we construct a benchmark of AVQA models on the proposed SJTU-UAV database and other two AVQA databases, of which the benchmark models consist of AVQA models designed for synthetically distorted A/V sequences and AVQA models built through combining the popular VQA methods and audio features via support vector regressor (SVR).
Advancing Zero-Shot Digital Human Quality Assessment through Text-Prompted Evaluation
To address this gap, we propose SJTU-H3D, a subjective quality assessment database specifically designed for full-body digital humans.
AIGCIQA2023: A Large-scale Image Quality Assessment Database for AI Generated Images: from the Perspectives of Quality, Authenticity and Correspondence
In this paper, in order to get a better understanding of the human visual preferences for AIGIs, a large-scale IQA database for AIGC is established, which is named as AIGCIQA2023.
GMS-3DQA: Projection-based Grid Mini-patch Sampling for 3D Model Quality Assessment
Model-based 3DQA methods extract features directly from the 3D models, which are characterized by their high degree of complexity.
AGIQA-3K: An Open Database for AI-Generated Image Quality Assessment
With the rapid advancements of the text-to-image generative model, AI-generated images (AGIs) have been widely applied to entertainment, education, social media, etc.
Masked Autoencoders as Image Processors
Recently, masked autoencoders (MAE) for feature pre-training have further unleashed the potential of Transformers, leading to state-of-the-art performances on various high-level vision tasks.
Ranked #4 on
Image Defocus Deblurring
on DPD (Dual-view)
MD-VQA: Multi-Dimensional Quality Assessment for UGC Live Videos
User-generated content (UGC) live videos are often bothered by various distortions during capture procedures and thus exhibit diverse visual qualities.
A Perceptual Quality Assessment Exploration for AIGC Images
\underline{AI} \underline{G}enerated \underline{C}ontent (\textbf{AIGC}) has gained widespread attention with the increasing efficiency of deep learning in content creation.
VDPVE: VQA Dataset for Perceptual Video Enhancement
Few researchers have specifically proposed a video quality assessment method for video enhancement, and there is also no comprehensive video quality assessment dataset available in public.
Subjective and Objective Quality Assessment for in-the-Wild Computer Graphics Images
Computer graphics images (CGIs) are artificially generated by means of computer programs and are widely perceived under various scenarios, such as games, streaming media, etc.
Audio-Visual Quality Assessment for User Generated Content: Database and Method
Specifically, we construct the first UGC AVQA database named the SJTU-UAV database, which includes 520 in-the-wild UGC audio and video (A/V) sequences, and conduct a user study to obtain the mean opinion scores of the A/V sequences.
EEP-3DQA: Efficient and Effective Projection-based 3D Model Quality Assessment
Currently, great numbers of efforts have been put into improving the effectiveness of 3D model quality assessment (3DQA) methods.
DDH-QA: A Dynamic Digital Humans Quality Assessment Database
In recent years, large amounts of effort have been put into pushing forward the real-world application of dynamic digital human (DDH).
Perceptual Attacks of No-Reference Image Quality Models with Human-in-the-Loop
No-reference image quality assessment (NR-IQA) aims to quantify how humans perceive visual distortions of digital images without access to their undistorted references.
Perceptual Quality Assessment for Digital Human Heads
Digital humans are attracting more and more research interest during the last decade, the generation, representation, rendering, and animation of which have been put into large amounts of effort.
MM-PCQA: Multi-Modal Learning for No-reference Point Cloud Quality Assessment
In specific, we split the point clouds into sub-models to represent local geometry distortions such as point shift and down-sampling.
Ranked #1 on
Point Cloud Quality Assessment
on SJTU-PCQA
Evaluating Point Cloud from Moving Camera Videos: A No-Reference Metric
To tackle the challenge of point cloud quality assessment (PCQA), many PCQA methods have been proposed to evaluate the visual quality levels of point clouds by assessing the rendered static 2D projections.
Perceptual Quality Assessment of Omnidirectional Images
The original and distorted omnidirectional images, subjective quality ratings, and the head and eye movement data together constitute the OIQA database.
Subjective Quality Assessment for Images Generated by Computer Graphics
However, limited work has been put forward to tackle the problem of computer graphics generated images' quality assessment (CG-IQA).
A No-reference Quality Assessment Metric for Point Cloud Based on Captured Video Sequences
Point cloud is one of the most widely used digital formats of 3D models, the visual quality of which is quite sensitive to distortions such as downsampling, noise, and compression.
Blind Surveillance Image Quality Assessment via Deep Neural Network Combined with the Visual Saliency
In this paper, we first conduct an example experiment (i. e. the face detection task) to demonstrate that the quality of the SIs has a crucial impact on the performance of the IVSS, and then propose a saliency-based deep neural network for the blind quality assessment of the SIs, which helps IVSS to filter the low-quality SIs and improve the detection and recognition performance.
Deep Neural Network for Blind Visual Quality Assessment of 4K Content
In this paper, we propose a deep learning-based BIQA model for 4K content, which on one hand can recognize true and pseudo 4K content and on the other hand can evaluate their perceptual visual quality.
A No-Reference Deep Learning Quality Assessment Method for Super-resolution Images Based on Frequency Maps
Therefore, in this paper, we propose a no-reference deep-learning image quality assessment method based on frequency maps because the artifacts caused by SISR algorithms are quite sensitive to frequency information.
Perceptual Quality Assessment for Fine-Grained Compressed Images
Nowadays, the mainstream full-reference (FR) metrics are effective to predict the quality of compressed images at coarse-grained levels (the bit rates differences of compressed images are obvious), however, they may perform poorly for fine-grained compressed images whose bit rates differences are quite subtle.
Video-based Human-Object Interaction Detection from Tubelet Tokens
We present a novel vision Transformer, named TUTOR, which is able to learn tubelet tokens, served as highly-abstracted spatiotemporal representations, for video-based human-object interaction (V-HOI) detection.
Deep Decomposition and Bilinear Pooling Network for Blind Night-Time Image Quality Evaluation
The DDB-Net contains three modules, i. e., an image decomposition module, a feature encoding module, and a bilinear pooling module.
A Deep Learning based No-reference Quality Assessment Model for UGC Videos
The proposed model utilizes very sparse frames to extract spatial features and dense frames (i. e. the video chunk) with a very low spatial resolution to extract motion features, which thereby has low computational complexity.
Ranked #5 on
Video Quality Assessment
on YouTube-UGC
Saliency in Augmented Reality
Therefore, in this paper, we mainly analyze the interaction effect between background (BG) scenes and AR contents, and study the saliency prediction problem in AR.
Confusing Image Quality Assessment: Towards Better Augmented Reality Experience
An objective metric termed CFIQA is also proposed to better evaluate the confusing image quality.
End-to-End Human-Gaze-Target Detection with Transformers
In contrast, we redefine the HGT detection task as detecting human head locations and their gaze targets, simultaneously.
Iwin: Human-Object Interaction Detection via Transformer with Irregular Windows
Iwin Transformer is a hierarchical Transformer which progressively performs token representation learning and token agglomeration within irregular windows.
Parameterized Image Quality Score Distribution Prediction
Experimental results verifythe feasibility of using alpha stable model to describe the IQSD, and prove the effectiveness of objective alpha stable model basedIQSD prediction method.
A Brief Survey on Adaptive Video Streaming Quality Assessment
In this paper, we provide a brief overview of adaptive video streaming quality assessment.
Learning Invisible Markers for Hidden Codes in Offline-to-Online Photography
In addition, the automatic localization of hidden codes significantly reduces the time of manually correcting geometric distortions for photos, which is a revolutionary innovation for information hiding in mobile applications.
Self-Conditioned Probabilistic Learning of Video Rescaling
After optimization, the downscaled video by our framework preserves more meaningful information, which is beneficial for both the upscaling step and the downstream tasks, e. g., video action recognition task.
No-Reference Quality Assessment for 3D Colored Point Cloud and Mesh Models
Therefore, many related studies such as point cloud quality assessment (PCQA) and mesh quality assessment (MQA) have been carried out to measure the visual quality degradations of 3D models.
Ranked #3 on
Point Cloud Quality Assessment
on WPC
Deep Learning based Full-reference and No-reference Quality Assessment Models for Compressed UGC Videos
The proposed VQA framework consists of three modules, the feature extraction module, the quality regression module, and the quality pooling module.
Ranked #12 on
Video Quality Assessment
on MSU NR VQA Database
Blurry Video Frame Interpolation
Existing works reduce motion blur and up-convert frame rate through two separate ways, including frame deblurring and frame interpolation.
Toward Better Understanding of Saliency Prediction in Augmented 360 Degree Videos
The saliency annotations of head and eye movements for both original and augmented videos are collected and together constitute the ARVR dataset.
How is Gaze Influenced by Image Transformations? Dataset and Model
Data size is the bottleneck for developing deep saliency models, because collecting eye-movement data is very time consuming and expensive.
Invariance Analysis of Saliency Models versus Human Gaze During Scene Free Viewing
Most of current studies on human gaze and saliency modeling have used high-quality stimuli.
Terahertz Security Image Quality Assessment by No-reference Model Observers
Subsequently, the existing no-reference IQA algorithms, which were 5 opinion-aware approaches viz., NFERM, GMLF, DIIVINE, BRISQUE and BLIINDS2, and 8 opinion-unaware approaches viz., QAC, SISBLIM, NIQE, FISBLIM, CPBD, S3 and Fish_bb, were executed for the evaluation of the THz security image quality.
