Search Results for author:
Found 83 papers, 44 papers with code
LokiTalk: Learning Fine-Grained and Generalizable Correspondences to Enhance NeRF-based Talking Head Synthesis
Despite significant progress in talking head synthesis since the introduction of Neural Radiance Fields (NeRF), visual artifacts and high training costs persist as major obstacles to large-scale commercial adoption.
Human-Activity AGV Quality Assessment: A Benchmark Dataset and an Objective Evaluation Metric
To address this challenge, we conduct a pioneering study on human activity AGV quality assessment, focusing on visual quality evaluation and the identification of semantic distortions.
MEMO-Bench: A Multiple Benchmark for Text-to-Image and Multimodal Large Language Models on Human Emotion Analysis
Artificial Intelligence (AI) has demonstrated significant capabilities in various fields, and in areas such as human-computer interaction (HCI), embodied intelligence, and the design and animation of virtual digital humans, both practitioners and users are increasingly concerned with AI's ability to understand and express emotion.
DR-BFR: Degradation Representation with Diffusion Models for Blind Face Restoration
DR-BFR comprises two modules: 1) Degradation Representation Module (DRM): This module extracts degradation representation with content-irrelevant features from LQ faces and estimates a reasonable distribution in the degradation space through contrastive learning and a specially designed LQ reconstruction.
VQA$^2$: Visual Question Answering for Video Quality Assessment
To address this gap, we introduce the VQA2 Instruction Dataset - the first visual question answering instruction dataset that focuses on video quality assessment.
R-Bench: Are your Large Multimodal Model Robust to Real-world Corruptions?
However, various corruptions in the real world mean that images will not be as ideal as in simulations, presenting significant challenges for the practical application of LMMs.
Q-Bench-Video: Benchmarking the Video Quality Understanding of LMMs
With the rising interest in research on Large Multi-modal Models (LMMs) for video understanding, many studies have emphasized general video comprehension capabilities, neglecting the systematic exploration into video quality understanding.
Explore the Hallucination on Low-level Perception for MLLMs
However, these models also exhibit hallucinations, which limit their reliability as AI systems, especially in tasks involving low-level visual perception and understanding.
3DGCQA: A Quality Assessment Database for 3D AI-Generated Contents
To provide a valuable resource for future research and development in 3D content generation and quality assessment, the dataset has been open-sourced in https://github. com/zyj-2000/3DGCQA.
Assessing UHD Image Quality from Aesthetics, Distortions, and Saliency
To address this problem, we design a multi-branch deep neural network (DNN) to assess the quality of UHD images from three perspectives: global aesthetic characteristics, local technical distortions, and salient content perception.
Focus on Neighbors and Know the Whole: Towards Consistent Dense Multiview Text-to-Image Generator for 3D Creation
To further enhance cross-view consistency and alleviate content drift, CoSER rapidly scan all views in spiral bidirectional manner to aware holistic information and then scores each point based on semantic material.
SAMBO-RL: Shifts-aware Model-based Offline Reinforcement Learning
Furthermore, we introduce Shifts-aware Model-based Offline Reinforcement Learning (SAMBO-RL), a practical framework that efficiently trains classifiers to approximate the SAR for policy optimization.
AIM 2024 Challenge on Compressed Video Quality Assessment: Methods and Results
The challenge aimed to evaluate the performance of VQA methods on a diverse dataset of 459 videos, encoded with 14 codecs of various compression standards (AVC/H. 264, HEVC/H. 265, AV1, and VVC/H. 266) and containing a comprehensive collection of compression artifacts.
Quality Assessment in the Era of Large Models: A Survey
Quality assessment, which evaluates the visual quality level of multimedia experiences, has garnered significant attention from researchers and has evolved substantially through dedicated efforts.
SG-JND: Semantic-Guided Just Noticeable Distortion Predictor For Image Compression
Just noticeable distortion (JND), representing the threshold of distortion in an image that is minimally perceptible to the human visual system (HVS), is crucial for image compression algorithms to achieve a trade-off between transmission bit rate and image quality.
Benchmarking AIGC Video Quality Assessment: A Dataset and Unified Model
For the objective perspective, we establish a benchmark for evaluating existing quality assessment metrics on the LGVQ dataset, which reveals that current metrics perform poorly on the LGVQ dataset.
Q-Ground: Image Quality Grounding with Large Multi-modality Models
Recent advances of large multi-modality models (LMM) have greatly improved the ability of image quality assessment (IQA) method to evaluate and explain the quality of visual content.
HazeCLIP: Towards Language Guided Real-World Image Dehazing
Existing methods have achieved remarkable performance in single image dehazing, particularly on synthetic datasets.
CMC-Bench: Towards a New Paradigm of Visual Signal Compression
With the development of Large Multimodal Models (LMMs), a Cross Modality Compression (CMC) paradigm of Image-Text-Image has emerged.
GAIA: Rethinking Action Quality Assessment for AI-Generated Videos
Assessing action quality is both imperative and challenging due to its significant impact on the quality of AI-generated videos, further complicated by the inherently ambiguous nature of actions within AI-generated video (AIGV).
A-Bench: Are LMMs Masters at Evaluating AI-generated Images?
How to accurately and efficiently assess AI-generated images (AIGIs) remains a critical challenge for generative models.
Adaptive Image Quality Assessment via Teaching Large Multimodal Model to Compare
Extensive experiments on nine IQA datasets validate that the Compare2Score effectively bridges text-defined comparative levels during training with converted single image quality score for inference, surpassing state-of-the-art IQA models across diverse scenarios.
Dual-Branch Network for Portrait Image Quality Assessment
We employ the fidelity loss to train the model via a learning-to-rank manner to mitigate inconsistencies in quality scores in the portrait image quality assessment dataset PIQ.
Enhancing Blind Video Quality Assessment with Rich Quality-aware Features
Motivated by previous researches that leverage pre-trained features extracted from various computer vision models as the feature representation for BVQA, we further explore rich quality-aware features from pre-trained blind image quality assessment (BIQA) and BVQA models as auxiliary features to help the BVQA model to handle complex distortions and diverse content of social media videos.
Light-VQA+: A Video Quality Assessment Model for Exposure Correction with Vision-Language Guidance
Equally important to the VEC is the Video Quality Assessment (VQA).
G-Refine: A General Quality Refiner for Text-to-Image Generation
Based on the mechanisms of the Human Visual System (HVS) and syntax trees, the first two indicators can respectively identify the perception and alignment deficiencies, and the last module can apply targeted quality enhancement accordingly.
LMM-PCQA: Assisting Point Cloud Quality Assessment with LMM
Although large multi-modality models (LMMs) have seen extensive exploration and application in various quality assessment studies, their integration into Point Cloud Quality Assessment (PCQA) remains unexplored.
Large Multi-modality Model Assisted AI-Generated Image Quality Assessment
Traditional deep neural network (DNN)-based image quality assessment (IQA) models leverage convolutional neural networks (CNN) or Transformer to learn the quality-aware feature representation, achieving commendable performance on natural scene images.
NTIRE 2024 Quality Assessment of AI-Generated Content Challenge
A total of 196 participants have registered in the video track.
AIS 2024 Challenge on Video Quality Assessment of User-Generated Content: Methods and Results
The performance of the top-5 submissions is reviewed and provided here as a survey of diverse deep models for efficient video quality assessment of user-generated content.
NTIRE 2024 Challenge on Short-form UGC Video Quality Assessment: Methods and Results
This paper reviews the NTIRE 2024 Challenge on Shortform UGC Video Quality Assessment (S-UGC VQA), where various excellent solutions are submitted and evaluated on the collected dataset KVQ from popular short-form video platform, i. e., Kuaishou/Kwai Platform.
THQA: A Perceptual Quality Assessment Database for Talking Heads
In the realm of media technology, digital humans have gained prominence due to rapid advancements in computer technology.
AIGIQA-20K: A Large Database for AI-Generated Image Quality Assessment
With the rapid advancements in AI-Generated Content (AIGC), AI-Generated Images (AIGIs) have been widely applied in entertainment, education, and social media.
Graph Neural Aggregation-diffusion with Metastability
Due to the connection between graph diffusion and message passing, diffusion-based models have been widely studied.
Ranked #10 on
Node Classification
on Texas
Subjective-Aligned Dataset and Metric for Text-to-Video Quality Assessment
Based on T2VQA-DB, we propose a novel transformer-based model for subjective-aligned Text-to-Video Quality Assessment (T2VQA).
Language-Driven Visual Consensus for Zero-Shot Semantic Segmentation
To mitigate these issues, we propose a Language-Driven Visual Consensus (LDVC) approach, fostering improved alignment of semantic and visual information. Specifically, we leverage class embeddings as anchors due to their discrete and abstract nature, steering vision features toward class embeddings.
BlazeBVD: Make Scale-Time Equalization Great Again for Blind Video Deflickering
Developing blind video deflickering (BVD) algorithms to enhance video temporal consistency, is gaining importance amid the flourish of image processing and video generation.
Learning Dynamic Tetrahedra for High-Quality Talking Head Synthesis
Recent works in implicit representations, such as Neural Radiance Fields (NeRF), have advanced the generation of realistic and animatable head avatars from video sequences.
Towards Open-ended Visual Quality Comparison
Comparative settings (e. g. pairwise choice, listwise ranking) have been adopted by a wide range of subjective studies for image quality assessment (IQA), as it inherently standardizes the evaluation criteria across different observers and offer more clear-cut responses.
MISC: Ultra-low Bitrate Image Semantic Compression Driven by Large Multimodal Model
With the evolution of storage and communication protocols, ultra-low bitrate image compression has become a highly demanding topic.
Q-Bench+: A Benchmark for Multi-modal Foundation Models on Low-level Vision from Single Images to Pairs
To this end, we design benchmark settings to emulate human language responses related to low-level vision: the low-level visual perception (A1) via visual question answering related to low-level attributes (e. g. clarity, lighting); and the low-level visual description (A2), on evaluating MLLMs for low-level text descriptions.
Towards Optimal Adversarial Robust Q-learning with Bellman Infinity-error
Establishing robust policies is essential to counter attacks or disturbances affecting deep reinforcement learning (DRL) agents.
AttentionLut: Attention Fusion-based Canonical Polyadic LUT for Real-time Image Enhancement
Subsequently, the attention fusion module integrates the image feature with the priori attention feature obtained during training to generate image-adaptive canonical polyadic tensors.
Q-Refine: A Perceptual Quality Refiner for AI-Generated Image
With the rapid evolution of the Text-to-Image (T2I) model in recent years, their unsatisfactory generation result has become a challenge.
Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levels
The explosion of visual content available online underscores the requirement for an accurate machine assessor to robustly evaluate scores across diverse types of visual contents.
Ranked #1 on
Video Quality Assessment
on LIVE-FB LSVQ
Q-Boost: On Visual Quality Assessment Ability of Low-level Multi-Modality Foundation Models
Recent advancements in Multi-modality Large Language Models (MLLMs) have demonstrated remarkable capabilities in complex high-level vision tasks.
Exploring the Naturalness of AI-Generated Images
In this paper, we take the first step to benchmark and assess the visual naturalness of AI-generated images.
FS-BAND: A Frequency-Sensitive Banding Detector
Banding artifact, as known as staircase-like contour, is a common quality annoyance that happens in compression, transmission, etc.
BAND-2k: Banding Artifact Noticeable Database for Banding Detection and Quality Assessment
The quality score of a banding image is generated by pooling the banding detection maps masked by the spatial frequency filters.
Q-Instruct: Improving Low-level Visual Abilities for Multi-modality Foundation Models
Multi-modality foundation models, as represented by GPT-4V, have brought a new paradigm for low-level visual perception and understanding tasks, that can respond to a broad range of natural human instructions in a model.
Simple Baselines for Projection-based Full-reference and No-reference Point Cloud Quality Assessment
Point clouds are widely used in 3D content representation and have various applications in multimedia.
A No-Reference Quality Assessment Method for Digital Human Head
In this paper, we develop a novel no-reference (NR) method based on Transformer to deal with DHQA in a multi-task manner.
Geometry-Aware Video Quality Assessment for Dynamic Digital Human
Usually, DDHs are displayed as 2D rendered animation videos and it is natural to adapt video quality assessment (VQA) methods to DDH quality assessment (DDH-QA) tasks.
Q-Bench: A Benchmark for General-Purpose Foundation Models on Low-level Vision
To address this gap, we present Q-Bench, a holistic benchmark crafted to systematically evaluate potential abilities of MLLMs on three realms: low-level visual perception, low-level visual description, and overall visual quality assessment.
A Consumer-tier based Visual-Brain Machine Interface for Augmented Reality Glasses Interactions
Here we develop a consumer-tier Visual-Brain Machine Inteface(V-BMI) system specialized for Augmented Reality(AR) glasses interactions.
MRA-GNN: Minutiae Relation-Aware Model over Graph Neural Network for Fingerprint Embedding
However, previous works including CNN-based and Transformer-based approaches fail to exploit the nonstructural data, such as topology and correlation in fingerprints, which is essential to facilitate the identifiability and robustness of embedding.
RAWIW: RAW Image Watermarking Robust to ISP Pipeline
To achieve end-to-end training of the framework, we integrate a neural network that simulates the ISP pipeline to handle the RAW-to-RGB conversion process.
Advancing Zero-Shot Digital Human Quality Assessment through Text-Prompted Evaluation
To address this gap, we propose SJTU-H3D, a subjective quality assessment database specifically designed for full-body digital humans.
GMS-3DQA: Projection-based Grid Mini-patch Sampling for 3D Model Quality Assessment
Model-based 3DQA methods extract features directly from the 3D models, which are characterized by their high degree of complexity.
AGIQA-3K: An Open Database for AI-Generated Image Quality Assessment
With the rapid advancements of the text-to-image generative model, AI-generated images (AGIs) have been widely applied to entertainment, education, social media, etc.
DiffBFR: Bootstrapping Diffusion Model Towards Blind Face Restoration
This design is implemented with two key components: 1) Identity Restoration Module (IRM) for preserving the face details in results.
MD-VQA: Multi-Dimensional Quality Assessment for UGC Live Videos
User-generated content (UGC) live videos are often bothered by various distortions during capture procedures and thus exhibit diverse visual qualities.
Transforming Radiance Field with Lipschitz Network for Photorealistic 3D Scene Stylization
Simply coupling NeRF with photorealistic style transfer (PST) will result in cross-view inconsistency and degradation of stylized view syntheses.
A Perceptual Quality Assessment Exploration for AIGC Images
\underline{AI} \underline{G}enerated \underline{C}ontent (\textbf{AIGC}) has gained widespread attention with the increasing efficiency of deep learning in content creation.
Subjective and Objective Quality Assessment for in-the-Wild Computer Graphics Images
Computer graphics images (CGIs) are artificially generated by means of computer programs and are widely perceived under various scenarios, such as games, streaming media, etc.
StyO: Stylize Your Face in Only One-Shot
And it introduces a novel triple reconstruction loss to fine-tune the pre-trained LDM for encoding style and content into corresponding identifiers; 2) Fine-grained Content Controller (FCC) for the recombination phase.
EEP-3DQA: Efficient and Effective Projection-based 3D Model Quality Assessment
Currently, great numbers of efforts have been put into improving the effectiveness of 3D model quality assessment (3DQA) methods.
DDH-QA: A Dynamic Digital Humans Quality Assessment Database
In recent years, large amounts of effort have been put into pushing forward the real-world application of dynamic digital human (DDH).
CoupAlign: Coupling Word-Pixel with Sentence-Mask Alignments for Referring Image Segmentation
Then in the Sentence-Mask Alignment (SMA) module, the masks are weighted by the sentence embedding to localize the referred object, and finally projected back to aggregate the pixels for the target.
Perceptual Quality Assessment for Digital Human Heads
Digital humans are attracting more and more research interest during the last decade, the generation, representation, rendering, and animation of which have been put into large amounts of effort.
Generalized One-shot Domain Adaptation of Generative Adversarial Networks
While previous works mainly focus on style transfer, we propose a novel and concise framework to address the \textit{generalized one-shot adaptation} task for both style and entity transfer, in which a reference image and its binary entity mask are provided.
MM-PCQA: Multi-Modal Learning for No-reference Point Cloud Quality Assessment
In specific, we split the point clouds into sub-models to represent local geometry distortions such as point shift and down-sampling.
Ranked #1 on
Point Cloud Quality Assessment
on SJTU-PCQA
Evaluating Point Cloud from Moving Camera Videos: A No-Reference Metric
To tackle the challenge of point cloud quality assessment (PCQA), many PCQA methods have been proposed to evaluate the visual quality levels of point clouds by assessing the rendered static 2D projections.
Subjective Quality Assessment for Images Generated by Computer Graphics
However, limited work has been put forward to tackle the problem of computer graphics generated images' quality assessment (CG-IQA).
Blind Surveillance Image Quality Assessment via Deep Neural Network Combined with the Visual Saliency
In this paper, we first conduct an example experiment (i. e. the face detection task) to demonstrate that the quality of the SIs has a crucial impact on the performance of the IVSS, and then propose a saliency-based deep neural network for the blind quality assessment of the SIs, which helps IVSS to filter the low-quality SIs and improve the detection and recognition performance.
A No-reference Quality Assessment Metric for Point Cloud Based on Captured Video Sequences
Point cloud is one of the most widely used digital formats of 3D models, the visual quality of which is quite sensitive to distortions such as downsampling, noise, and compression.
A No-Reference Deep Learning Quality Assessment Method for Super-resolution Images Based on Frequency Maps
Therefore, in this paper, we propose a no-reference deep-learning image quality assessment method based on frequency maps because the artifacts caused by SISR algorithms are quite sensitive to frequency information.
Deep Neural Network for Blind Visual Quality Assessment of 4K Content
In this paper, we propose a deep learning-based BIQA model for 4K content, which on one hand can recognize true and pseudo 4K content and on the other hand can evaluate their perceptual visual quality.
Perceptual Quality Assessment for Fine-Grained Compressed Images
Nowadays, the mainstream full-reference (FR) metrics are effective to predict the quality of compressed images at coarse-grained levels (the bit rates differences of compressed images are obvious), however, they may perform poorly for fine-grained compressed images whose bit rates differences are quite subtle.
PetsGAN: Rethinking Priors for Single Image Generation
Moreover, we apply our method to other image manipulation tasks (e. g., style transfer, harmonization), and the results further prove the effectiveness and efficiency of our method.
No-Reference Quality Assessment for 3D Colored Point Cloud and Mesh Models
Therefore, many related studies such as point cloud quality assessment (PCQA) and mesh quality assessment (MQA) have been carried out to measure the visual quality degradations of 3D models.
Ranked #3 on
Point Cloud Quality Assessment
on WPC
ExSinGAN: Learning an Explainable Generative Model from a Single Image
Generating images from a single sample, as a newly developing branch of image synthesis, has attracted extensive attention.
Multi-Agent Semi-Siamese Training for Long-tail and Shallow Face Learning
However, unlike the existing public face datasets, in many real-world scenarios of face recognition, the depth of training dataset is shallow, which means only two face images are available for each ID.
