Search Results for author:
Found 147 papers, 45 papers with code
Hybrid Agents for Image Restoration
In this work, we present HybridAgent, intending to incorporate multiple restoration modes into a unified image restoration model and achieve intelligent and efficient user interaction through our proposed hybrid agents.
StreamMind: Unlocking Full Frame Rate Streaming Video Dialogue through Event-Gated Cognition
With the rise of real-world human-AI interaction applications, such as AI assistants, the need for Streaming Video Dialogue is critical.
Q&C: When Quantization Meets Cache in Efficient Image Generation
In this work, we take advantage of these two acceleration mechanisms and propose a hybrid acceleration method by tackling the above challenges, aiming to further improve the efficiency of DiTs while maintaining excellent generation capability.
InternVQA: Advancing Compressed Video Quality Assessment with Distilling Large Foundation Model
Video quality assessment tasks rely heavily on the rich features required for video understanding, such as semantic information, texture, and temporal motion.
AR4D: Autoregressive 4D Generation from Monocular Videos
To avoid appearance drift introduced by autoregressive generation, we further incorporate a refinement stage based on a global deformation field and the geometry of each frame's 3D representation.
Semantics Disentanglement and Composition for Versatile Codec toward both Human-eye Perception and Machine Vision Task
While learned image compression methods have achieved impressive results in either human visual perception or machine vision tasks, they are often specialized only for one domain.
GSemSplat: Generalizable Semantic 3D Gaussian Splatting from Uncalibrated Image Pairs
To ensure effective and reliable learning of semantic features in 3D space, we employ a dual-feature approach that leverages both region-specific and context-aware semantic features as supervision in the 2D space.
TIV-Diffusion: Towards Object-Centric Movement for Text-driven Image to Video Generation
Text-driven Image to Video Generation (TI2V) aims to generate controllable video given the first frame and corresponding textual description.
Light Field Image Quality Assessment With Auxiliary Learning Based on Depthwise and Anglewise Separable Convolutions
Second, we further theoretically extend the LF-DSC to the angular space of LFI and introduce the novel concept of "light field anglewise separable convolution (LF-ASC)", which is capable of extracting both the spatial and angular features for comprehensive quality assessment with low complexity.
UniMIC: Towards Universal Multi-modality Perceptual Image Compression
We present UniMIC, a universal multi-modality image compression framework, intending to unify the rate-distortion-perception (RDP) optimization for multiple image codecs simultaneously through excavating cross-modality generative priors.
LossAgent: Towards Any Optimization Objectives for Image Processing with LLM Agents
We present the first loss agent, dubbed LossAgent, for low-level image processing tasks, e. g., image super-resolution and restoration, intending to achieve any customized optimization objectives of low-level image processing in different practical applications.
Towards Defining an Efficient and Expandable File Format for AI-Generated Contents
Recently, AI-generated content (AIGC) has gained significant traction due to its powerful creation capability.
Compositional 3D-aware Video Generation with LLM Director
Specifically, given an input textual prompt, our scheme consists of three stages: 1) We leverage LLM as the director to first decompose the complex query into several sub-prompts that indicate individual concepts within the video~(\textit{e. g.}, scene, objects, motions), then we let LLM to invoke pre-trained expert models to obtain corresponding 3D representations of concepts.
MambaCSR: Dual-Interleaved Scanning for Compressed Image Super-Resolution With SSMs
We present MambaCSR, a simple but effective framework based on Mamba for the challenging compressed image super-resolution (CSR) task.
Tell Codec What Worth Compressing: Semantically Disentangled Image Coding for Machine with LMMs
We present a new image compression paradigm to achieve ``intelligently coding for machine'' by cleverly leveraging the common sense of Large Multimodal Models (LMMs).
Rethinking Domain Adaptation and Generalization in the Era of CLIP
In recent studies on domain adaptation, significant emphasis has been placed on the advancement of learning shared knowledge from a source domain to a target domain.
UCIP: A Universal Framework for Compressed Image Super-Resolution using Dynamic Prompt
Compressed Image Super-resolution (CSR) aims to simultaneously super-resolve the compressed images and tackle the challenging hybrid distortions caused by compression.
Rate-Distortion-Cognition Controllable Versatile Neural Image Compression
Recently, the field of Image Coding for Machines (ICM) has garnered heightened interest and significant advances thanks to the rapid progress of learning-based techniques for image compression and analysis.
MoE-DiffIR: Task-customized Diffusion Priors for Universal Compressed Image Restoration
We present MoE-DiffIR, an innovative universal compressed image restoration (CIR) method with task-customized diffusion priors.
Priorformer: A UGC-VQA Method with content and distortion priors
User Generated Content (UGC) videos are susceptible to complicated and variant degradations and contents, which prevents the existing blind video quality assessment (BVQA) models from good performance since the lack of the adapability of distortions and contents.
Training-free Camera Control for Video Generation
Instead, it can be plugged and played with most pretrained video diffusion models and generate camera controllable videos with a single image or text prompt as input.
Q-Mamba: On First Exploration of Vision Mamba for Image Quality Assessment
Consequently, we propose Q-Mamba by revisiting and adapting the Mamba model for three crucial IQA tasks, i. e., task-specific, universal, and transferable IQA, which reveals that the Mamba model has obvious advantages compared with existing foundational models, e. g., Swin Transformer, ViT, and CNNs, in terms of perception and computational cost for IQA.
CoNo: Consistency Noise Injection for Tuning-free Long Video Diffusion
Tuning-free long video diffusion has been proposed to generate extended-duration videos with enriched content by reusing the knowledge from pre-trained short video diffusion model without retraining.
Survey on Visual Signal Coding and Processing with Generative Models: Technologies, Standards and Optimization
This paper provides a survey of the latest developments in visual signal coding and processing with generative models.
Light Field Compression Based on Implicit Neural Representation
We store the information of a light field image implicitly in an neural network and adopt model compression methods to further compress the implicit representation.
PromptCIR: Blind Compressed Image Restoration with Prompt Learning
Existing works on blind CIR often seek assistance from a quality factor prediction network to facilitate their network to restore compressed images.
NTIRE 2024 Quality Assessment of AI-Generated Content Challenge
A total of 196 participants have registered in the video track.
NTIRE 2024 Challenge on Short-form UGC Video Quality Assessment: Methods and Results
This paper reviews the NTIRE 2024 Challenge on Shortform UGC Video Quality Assessment (S-UGC VQA), where various excellent solutions are submitted and evaluated on the collected dataset KVQ from popular short-form video platform, i. e., Kuaishou/Kwai Platform.
Learn from Failure: Fine-Tuning LLMs with Trial-and-Error Data for Intuitionistic Propositional Logic Proving
Recent advances in Automated Theorem Proving have shown the effectiveness of leveraging a (large) language model that generates tactics (i. e. proof steps) to search through proof states.
End-to-End Rate-Distortion Optimized 3D Gaussian Representation
3D Gaussian Splatting (3DGS) has become an emerging technique with remarkable potential in 3D representation and image rendering.
VisualCritic: Making LMMs Perceive Visual Quality Like Humans
In this paper, we explore this question and provide the answer "Yes!".
Is Vanilla MLP in Neural Radiance Field Enough for Few-shot View Synthesis?
Neural Radiance Field (NeRF) has achieved superior performance for novel view synthesis by modeling the scene with a Multi-Layer Perception (MLP) and a volume rendering procedure, however, when fewer known views are given (i. e., few-shot view synthesis), the model is prone to overfit the given views.
SeD: Semantic-Aware Discriminator for Image Super-Resolution
In particular, one discriminator is utilized to enable the SR network to learn the distribution of real-world high-quality images in an adversarial training manner.
CMC: Few-shot Novel View Synthesis via Cross-view Multiplane Consistency
Furthermore, to regularize the unseen target views, we constrain the rendered colors and depths from different input views to be the same.
KVQ: Kwai Video Quality Assessment for Short-form Videos
Short-form UGC video platforms, like Kwai and TikTok, have been an emerging and irreplaceable mainstream media form, thriving on user-friendly engagement, and kaleidoscope creation, etc.
Conditional Neural Video Coding with Spatial-Temporal Super-Resolution
This document is an expanded version of a one-page abstract originally presented at the 2024 Data Compression Conference.
Video Quality Assessment Based on Swin TransformerV2 and Coarse to Fine Strategy
Furthermore, a temporal transformer is utilized for spatiotemporal feature fusion across the video.
High-Fidelity Diffusion-based Image Editing
Diffusion models have attained remarkable success in the domains of image generation and editing.
Diagnosis-oriented Medical Image Compression with Efficient Transfer Learning
Remote medical diagnosis has emerged as a critical and indispensable technique in practical medical systems, where medical data are required to be efficiently compressed and transmitted for diagnosis by either professional doctors or intelligent diagnosis devices.
FreqAlign: Excavating Perception-oriented Transferability for Blind Image Quality Assessment from A Frequency Perspective
Based on this, we propose to improve the perception-oriented transferability of BIQA by performing feature frequency decomposition and selecting the frequency components that contained the most transferable perception knowledge for alignment.
Blind Image Quality Assessment
Unsupervised Domain Adaptation
CCEdit: Creative and Controllable Video Editing via Diffusion Models
The versatility of our framework is demonstrated through a diverse range of choices in both structure representations and personalized T2I models, as well as the option to provide the edited key frame.
GraphAdapter: Tuning Vision-Language Models With Dual Knowledge Graph
To mitigate that, we propose an effective adapter-style tuning strategy, dubbed GraphAdapter, which performs the textual adapter by explicitly modeling the dual-modality structure knowledge (i. e., the correlation of different semantics/classes in textual and visual modalities) with a dual knowledge graph.
Diffusion Models for Image Restoration and Enhancement -- A Comprehensive Survey
Image restoration (IR) has been an indispensable and challenging task in the low-level vision field, which strives to improve the subjective quality of images distorted by various forms of degradation.
Compression with Bayesian Implicit Neural Representations
Many common types of data can be represented as functions that map coordinates to signal values, such as pixel locations to RGB values in the case of an image.
NVTC: Nonlinear Vector Transform Coding
In theory, vector quantization (VQ) is always better than scalar quantization (SQ) in terms of rate-distortion (R-D) performance.
Exploring the Rate-Distortion-Complexity Optimization in Neural Image Compression
By quantifying the decoding complexity as a factor in the optimization goal, we are now able to precisely control the RDC trade-off and then demonstrate how the rate-distortion performance of neural image codecs could adapt to various complexity demands.
Semantically Structured Image Compression via Irregular Group-Based Decoupling
Nevertheless, they divide the input image into multiple rectangular regions according to semantics and ignore avoiding information interaction among them, causing waste of bitrate and distorted reconstruction of region boundaries.
Prompt-ICM: A Unified Framework towards Image Coding for Machines with Task-driven Prompts
For ICM, developing a unified codec to reduce information redundancy while empowering the compressed features to support various vision tasks is very important, which inevitably faces two core challenges: 1) How should the compression strategy be adjusted based on the downstream tasks?
Inpaint Anything: Segment Anything Meets Image Inpainting
We are also very willing to help everyone share and promote new projects based on our Inpaint Anything (IA).
Learning Distortion Invariant Representation for Image Restoration from A Causality Perspective
In this paper, we are the first to propose a novel training strategy for image restoration from the causality perspective, to improve the generalization ability of DNNs for unknown degradations.
Versatile Neural Processes for Learning Implicit Neural Representations
In this paper, we propose an efficient NP framework dubbed Versatile Neural Processes (VNP), which largely increases the capability of approximating functions.
Semantic-aware Message Broadcasting for Efficient Unsupervised Domain Adaptation
In this way, our message broadcasting encourages the group tokens to learn more informative and diverse information for effective domain alignment.
Ranked #1 on
Unsupervised Domain Adaptation
on DomainNet
Task Residual for Tuning Vision-Language Models
Large-scale vision-language models (VLMs) pre-trained on billion-level data have learned general visual representations and broad visual concepts.
MiNL: Micro-images based Neural Representation for Light Fields
Unlike explicit representation that represents light fields as Sub-Aperture Images (SAIs) based arrays or Micro-Images (MIs) based lenslet images, implicit representation treats light fields as neural networks, which is inherently a continuous representation in contrast to discrete explicit representation.
Hierarchical Reinforcement Learning Based Video Semantic Coding for Segmentation
In this paper, we take a step forward to video semantic compression and propose the Hierarchical Reinforcement Learning based task-driven Video Semantic Coding, named as HRLVSC.
Hierarchical Reinforcement Learning
reinforcement-learning
+5
Learned Lossless JPEG Transcoding via Joint Lossy and Residual Compression
Our proposed transcoding architecture shows significant superiority in the compression of JPEG images thanks to the collaboration of learned lossy transform coding and residual entropy coding.
HST: Hierarchical Swin Transformer for Compressed Image Super-resolution
Compressed Image Super-resolution has achieved great attention in recent years, where images are degraded with compression artifacts and low-resolution artifacts.
Ranked #1 on
Compressed Image Super-resolution
on DIV2K-q40-x4
StyleAM: Perception-Oriented Unsupervised Domain Adaption for Non-reference Image Quality Assessment
Specifically, we find a more compact and reliable space i. e., feature style space for perception-oriented UDA based on an interesting/amazing observation, that the feature style (i. e., the mean and variance) of the deep layer in DNNs is exactly associated with the quality score in NR-IQA.
Source-free Unsupervised Domain Adaptation for Blind Image Quality Assessment
Thanks to the development of unsupervised domain adaptation (UDA), some works attempt to transfer the knowledge from a label-sufficient source domain to a label-free target domain under domain shift with UDA.
Blind Image Quality Assessment
Unsupervised Domain Adaptation
RTN: Reinforced Transformer Network for Coronary CT Angiography Vessel-level Image Quality Assessment
Therefore, we propose a Progressive Reinforcement learning based Instance Discarding module (termed as PRID) to progressively remove quality-irrelevant/negative instances for CCTA VIQA.
Image Coding for Machines with Omnipotent Feature Learning
Learning a kind of feature that is both general (for AI tasks) and compact (for compression) is pivotal for its success.
SwinIQA: Learned Swin Distance for Compressed Image Quality Assessment
In this paper, we design a full-reference image quality assessment metric SwinIQA to measure the perceptual quality of compressed images in a learned Swin distance space.
Compressed Image Quality Assessment
Full-Reference Image Quality Assessment
+1
Deep Frequency Filtering for Domain Generalization
Improving the generalization ability of Deep Neural Networks (DNNs) is critical for their practical uses, which has been a longstanding challenge.
Active Token Mixer
In this work, we propose an innovative token-mixer, dubbed Active Token Mixer (ATM), to actively incorporate flexible contextual information distributed across different channels from other tokens into the given query token.
Ranked #65 on
Object Detection
on COCO minival
Mask-based Latent Reconstruction for Reinforcement Learning
For deep reinforcement learning (RL) from pixels, learning effective state representations is crucial for achieving high performance.
Semantically Video Coding: Instill Static-Dynamic Clues into Structured Bitstream for AI Tasks
Traditional media coding schemes typically encode image/video into a semantic-unknown binary stream, which fails to directly support downstream intelligent tasks at the bitstream level.
Learning Cross-Scale Weighted Prediction for Efficient Neural Video Compression
Neural video codecs have demonstrated great potential in video transmission and storage applications.
SelectAugment: Hierarchical Deterministic Sample Selection for Data Augmentation
Data augmentation (DA) has been widely investigated to facilitate model optimization in many tasks.
Confounder Identification-free Causal Visual Feature Learning
In this paper, we propose a novel Confounder Identification-free Causal Visual Feature Learning (CICF) method, which obviates the need for identifying confounders.
A Close Look at Few-shot Real Image Super-resolution from the Distortion Relation Perspective
Under this brand-new scenario, we propose Distortion Relation guided Transfer Learning (DRTL) for the few-shot RealSR by transferring the rich restoration knowledge from auxiliary distortions (i. e., synthetic distortions) to the target RealSR under the guidance of distortion relation.
Meta Clustering Learning for Large-scale Unsupervised Person Re-identification
Unsupervised Person Re-identification (U-ReID) with pseudo labeling recently reaches a competitive performance compared to fully-supervised ReID methods based on modern clustering algorithms.
Texture-enhanced Light Field Super-resolution with Spatio-Angular Decomposition Kernels
Despite the recent progress in light field super-resolution (LFSR) achieved by convolutional neural networks, the correlation information of light field (LF) images has not been sufficiently studied and exploited due to the complexity of 4D LF data.
Versatile Learned Video Compression
We show that the flow prediction module can largely reduce the transmission cost of voxel flows.
Unleash the Potential of Adaptation Models via Dynamic Domain Labels
In this paper, we propose an embarrassing simple yet highly effective adversarial domain adaptation (ADA) method for effectively training models for alignment.
ToAlign: Task-oriented Alignment for Unsupervised Domain Adaptation
Unsupervised domain adaptive classifcation intends to improve the classifcation performance on unlabeled target domain.
Partial Person Re-Identification With Part-Part Correspondence Learning
Driven by the success of deep learning, the last decade has seen rapid advances in person re-identification (re-ID).
PlayVirtual: Augmenting Cycle-Consistent Virtual Trajectories for Reinforcement Learning
In this work, we propose a novel method, dubbed PlayVirtual, which augments cycle-consistent virtual trajectories to enhance the data efficiency for RL feature representation learning.
Continuous Control (100k environment steps)
Continuous Control (500k environment steps)
+4
Task-driven Semantic Coding via Reinforcement Learning
However, the traditional hybrid coding framework cannot be optimized in an end-to-end manner, which makes task-driven semantic fidelity metric unable to be automatically integrated into the rate-distortion optimization process.
Adaptive Hypergraph Convolutional Network for No-Reference 360-degree Image Quality Assessment
In the hypergraph construction, we build a location-based hyperedge and a content-based hyperedge for each viewport.
Image Super-Resolution Quality Assessment: Structural Fidelity Versus Statistical Naturalness
In this paper, we assess the quality of SISR generated images in a two-dimensional (2D) space of structural fidelity versus statistical naturalness.
Soft then Hard: Rethinking the Quantization in Neural Image Compression
Quantization is one of the core components in lossy image compression.
Bayesian Graph Convolutional Network for Traffic Prediction
Under this framework, the graph structure is viewed as a random realization from a parametric generative model, and its posterior is inferred using the observed topology of the road network and traffic data.
Cloth-Changing Person Re-identification from A Single Image with Gait Prediction and Regularization
Specifically, we introduce Gait recognition as an auxiliary task to drive the Image ReID model to learn cloth-agnostic representations by leveraging personal unique and cloth-independent gait information, we name this framework as GI-ReID.
Ranked #5 on
Person Re-Identification
on VC-Clothes
Cloth-Changing Person Re-Identification
Computational Efficiency
+1
MetaAlign: Coordinating Domain Alignment and Classification for Unsupervised Domain Adaptation
For unsupervised domain adaptation (UDA), to alleviate the effect of domain shift, many approaches align the source and target domains in the feature space by adversarial learning or by explicitly aligning their statistics.
Disentanglement-based Cross-Domain Feature Augmentation for Effective Unsupervised Domain Adaptive Person Re-identification
Each recomposed feature, obtained based on the domain-invariant feature (which enables a reliable inheritance of identity) and an enhancement from a domain specific feature (which enables the approximation of real distributions), is thus an "ideal" augmentation.
Re-energizing Domain Discriminator with Sample Relabeling for Adversarial Domain Adaptation
Many unsupervised domain adaptation (UDA) methods exploit domain adversarial training to align the features to reduce domain gap, where a feature extractor is trained to fool a domain discriminator in order to have aligned feature distributions.
Local Patch AutoAugment with Multi-Agent Collaboration
We formulate it as a multi-agent reinforcement learning (MARL) problem, where each agent learns an augmentation policy for each patch based on its content together with the semantics of the whole image.
Dense Interaction Learning for Video-based Person Re-identification
The CNN encoder is responsible for efficiently extracting discriminative spatial features while the DI decoder is designed to densely model spatial-temporal inherent interaction across frames.
Ranked #1 on
Person Re-Identification
on DukeMTMC-reID
No-Reference Quality Assessment for 360-degree Images by Analysis of Multi-frequency Information and Local-global Naturalness
To our knowledge, the proposed model is the first no-reference quality assessment method for 360-degreee images that combines multi-frequency information and image naturalness.
Image-to-Image Translation: Methods and Applications
Image-to-image translation (I2I) aims to transfer images from a source domain to a target domain while preserving the content representations.
Style Normalization and Restitution for Domain Generalization and Adaptation
In this paper, we design a novel Style Normalization and Restitution module (SNR) to simultaneously ensure both high generalization and discrimination capability of the networks.
Learned Block-based Hybrid Image Compression
Recent works on learned image compression perform encoding and decoding processes in a full-resolution manner, resulting in two problems when deployed for practical applications.
Learning Omni-frequency Region-adaptive Representations for Real Image Super-Resolution
Traditional single image super-resolution (SISR) methods that focus on solving single and uniform degradation (i. e., bicubic down-sampling), typically suffer from poor performance when applied into real-world low-resolution (LR) images due to the complicated realistic degradations.
Deep Multi-Scale Features Learning for Distorted Image Quality Assessment
In this paper, motivated by the human visual system (HVS) combining multi-scale features for perception, we propose to use pyramid features learning to build a DNN with hierarchical multi-scale features for distorted image quality prediction.
Causal Contextual Prediction for Learned Image Compression
In this paper, we propose the concept of separate entropy coding to leverage a serial decoding process for causal contextual entropy prediction in the latent space.
Improving Machine Reading Comprehension with Single-choice Decision and Transfer Learning
In addition, by transferring knowledge from other kinds of MRC tasks, our model achieves a new state-of-the-art results in both single and ensemble settings.
Ranked #1 on
Reading Comprehension
on RACE
Towards Scalable Distributed Training of Deep Learning on Public Cloud Clusters
Distributed training techniques have been widely deployed in large-scale deep neural networks (DNNs) training on dense-GPU clusters.
Bayesian Spatio-Temporal Graph Convolutional Network for Traffic Forecasting
The graph structure in our network is learned from the physical topology of the road network and traffic data in an end-to-end manner, which discovers a more accurate description of the relationship among traffic flows.
Ranked #2 on
Traffic Prediction
on SZ-Taxi
Uncertainty-Aware Few-Shot Image Classification
In this work, we propose Uncertainty-Aware Few-Shot framework for image classification by modeling uncertainty of the similarities of query-support pairs and performing uncertainty-aware optimization.
FAN: Frequency Aggregation Network for Real Image Super-resolution
Specifically, we extract different frequencies of the LR image and pass them to a channel attention-grouped residual dense network (CA-GRDB) individually to output corresponding feature maps.
LIRA: Lifelong Image Restoration from Unknown Blended Distortions
Most existing image restoration networks are designed in a disposable way and catastrophically forget previously learned distortions when trained on a new distortion removal task.
Learning Disentangled Feature Representation for Hybrid-distorted Image Restoration
Hybrid-distorted image restoration (HD-IR) is dedicated to restore real distorted image that is degraded by multiple distortions.
Feature Alignment and Restoration for Domain Generalization and Adaptation
To ensure high discrimination, we propose a Feature Restoration (FR) operation to distill task-relevant features from the residual information and use them to compensate for the aligned features.
Ranked #82 on
Domain Generalization
on PACS
Beyond Triplet Loss: Meta Prototypical N-tuple Loss for Person Re-identification
There is a lack of loss design which enables the joint optimization of multiple instances (of multiple classes) within per-query optimization for person ReID.
Residual Squeeze-and-Excitation Network for Fast Image Deraining
Image deraining is an important image processing task as rain streaks not only severely degrade the visual quality of images but also significantly affect the performance of high-level vision tasks.
Global Distance-distributions Separation for Unsupervised Person Re-identification
To address this problem, we introduce a global distance-distributions separation (GDS) constraint over the two distributions to encourage the clear separation of positive and negative samples from a global view.
Style Normalization and Restitution for Generalizable Person Re-identification
Existing fully-supervised person re-identification (ReID) methods usually suffer from poor generalization capability caused by domain gaps.
Ranked #9 on
Unsupervised Domain Adaptation
on Market to Duke
Multi-scale Grouped Dense Network for VVC Intra Coding
Versatile Video Coding (H. 266/VVC) standard achieves better image quality when keeping the same bits than any other conventional image codec, such as BPG, JPEG, and etc.
Blind Quality Assessment for Image Superresolution Using Deep Two-Stream Convolutional Networks
Numerous image superresolution (SR) algorithms have been proposed for reconstructing high-resolution (HR) images from input images with lower spatial resolutions.
TuiGAN: Learning Versatile Image-to-Image Translation with Two Unpaired Images
With TuiGAN, an image is translated in a coarse-to-fine manner where the generated image is gradually refined from global structures to local details.
Multi-Granularity Reference-Aided Attentive Feature Aggregation for Video-based Person Re-identification
In this paper, we propose an attentive feature aggregation module, namely Multi-Granularity Reference-aided Attentive Feature Aggregation (MG-RAFA), to delicately aggregate spatio-temporal features into a discriminative video-level feature representation.
Prior-enlightened and Motion-robust Video Deblurring
Various blur distortions in video will cause negative impact on both human viewing and video-based applications, which makes motion-robust deblurring methods urgently needed.
SASL: Saliency-Adaptive Sparsity Learning for Neural Network Acceleration
Accelerating the inference speed of CNNs is critical to their deployment in real-world applications.
VESR-Net: The Winning Solution to Youku Video Enhancement and Super-Resolution Challenge
This paper introduces VESR-Net, a method for video enhancement and super-resolution (VESR).
Uncertainty-Aware Multi-Shot Knowledge Distillation for Image-Based Object Re-Identification
To the best of our knowledge, we are the first to make use of multi-shots of an object in a teacher-student learning manner for effectively boosting the single image based re-id.
Generative Memorize-Then-Recall framework for low bit-rate Surveillance Video Compression
In this paper, we figure out this issue by disentangling surveillance video into the structure of a global spatio-temporal feature (memory) for Group of Picture (GoP) and skeleton for each frame (clue).
Region Normalization for Image Inpainting
In this work, we show that the mean and variance shifts caused by full-spatial FN limit the image inpainting network training and we propose a spatial region-wise normalization named Region Normalization (RN) to overcome the limitation.
Reinforced Bit Allocation under Task-Driven Semantic Distortion Metrics
Rapid growing intelligent applications require optimized bit allocation in image/video coding to support specific task-driven scenarios such as detection, classification, segmentation, etc.
Tensor Oriented No-Reference Light Field Image Quality Assessment
Light field image (LFI) quality assessment is becoming more and more important, which helps to better guide the acquisition, processing and application of immersive media.
Binocular Rivalry Oriented Predictive Auto-Encoding Network for Blind Stereoscopic Image Quality Measurement
Stereoscopic image quality measurement (SIQM) has become increasingly important for guiding stereo image processing and commutation systems due to the widespread usage of 3D contents.
Multimedia Image and Video Processing
No-Reference Light Field Image Quality Assessment Based on Spatial-Angular Measurement
In this paper, we propose a No-Reference Light Field image Quality Assessment (NR-LFQA) scheme, where the main idea is to quantify the LFI quality degradation through evaluating the spatial quality and angular consistency.
Progressive Image Inpainting with Full-Resolution Residual Network
Recently, learning-based algorithms for image inpainting achieve remarkable progress dealing with squared or irregular holes.
Stereoscopic Omnidirectional Image Quality Assessment Based on Predictive Coding Theory
In this paper, based on the predictive coding theory of the human vision system (HVS), we propose a stereoscopic omnidirectional image quality evaluator (SOIQE) to cope with the characteristics of 3D 360-degree images.
A Coarse-to-Fine Framework for Learned Color Enhancement with Non-Local Attention
For current learned methods in this field, global harmonious perception and local details are hard to be well-considered in a single model simultaneously.
ZstGAN: An Adversarial Approach for Unsupervised Zero-Shot Image-to-Image Translation
Image-to-image translation models have shown remarkable ability on transferring images among different domains.
Learning to Transfer: Unsupervised Meta Domain Translation
Unsupervised domain translation has recently achieved impressive performance with Generative Adversarial Network (GAN) and sufficient (unpaired) training data.
Semantics-Aligned Representation Learning for Person Re-identification
Specifically, we build a Semantics Aligning Network (SAN) which consists of a base network as encoder (SA-Enc) for re-ID, and a decoder (SA-Dec) for reconstructing/regressing the densely semantics aligned full texture image.
Image-to-Image Translation with Multi-Path Consistency Regularization
In this work, we introduce a new kind of loss, multi-path consistency loss, which evaluates the differences between direct translation $\mathcal{D}_s\to\mathcal{D}_t$ and indirect translation $\mathcal{D}_s\to\mathcal{D}_a\to\mathcal{D}_t$ with $\mathcal{D}_a$ as an auxiliary domain, to regularize training.
CaseNet: Content-Adaptive Scale Interaction Networks for Scene Parsing
We achieve this by the context interaction among the features of different scales.
Towards Accurate One-Stage Object Detection with AP-Loss
For this purpose, we develop a novel optimization algorithm, which seamlessly combines the error-driven update scheme in perceptron learning and backpropagation algorithm in deep networks.
Relation-Aware Global Attention for Person Re-identification
For person re-identification (re-id), attention mechanisms have become attractive as they aim at strengthening discriminative features and suppressing irrelevant ones, which matches well the key of re-id, i. e., discriminative feature learning.
Asynchronous Episodic Deep Deterministic Policy Gradient: Towards Continuous Control in Computationally Complex Environments
Not limited to the control tasks in computationally complex environments, AE-DDPG also achieves higher rewards and 2- to 4-fold improvement in sample efficiency on average compared to other variants of DDPG in MuJoCo environments.
Exploring Explicit Domain Supervision for Latent Space Disentanglement in Unpaired Image-to-Image Translation
After pre-training, this network is used to extract the domain-specific features of each image.
View Invariant 3D Human Pose Estimation
The diversity of capturing viewpoints and the flexibility of the human poses, however, remain some significant challenges.
Learning based Facial Image Compression with Semantic Fidelity Metric
The experimental results verify the framework's efficiency by demonstrating performance improvement of 71. 41%, 48. 28% and 52. 67% bitrate saving separately over JPEG2000, WebP and neural network-based codecs under the same face verification accuracy distortion metric.
Densely Semantically Aligned Person Re-Identification
We propose a densely semantically aligned person re-identification framework.
Sequential Gating Ensemble Network for Noise Robust Multi-Scale Face Restoration
The SGU sequentially takes information from two different levels as inputs and decides the output based on one active input.
Layer-Wise Coordination between Encoder and Decoder for Neural Machine Translation
Neural Machine Translation (NMT) has achieved remarkable progress with the quick evolvement of model structures.
Unsupervised Single Image Deraining with Self-supervised Constraints
Most existing single image deraining methods require learning supervised models from a large set of paired synthetic training data, which limits their generality, scalability and practicality in real-world multimedia applications.
DeepIR: A Deep Semantics Driven Framework for Image Retargeting
We present \emph{Deep Image Retargeting} (\emph{DeepIR}), a coarse-to-fine framework for content-aware image retargeting.
Distribution Discrepancy Maximization for Image Privacy Preserving
Accordingly, we introduce a collaborative training scheme: a discriminator $D$ is trained to discriminate the reconstructed image from the encrypted image, and an encryption model $G_e$ is required to generate these two kinds of images to maximize the recognition rate of $D$, leading to the same training objective for both $D$ and $G_e$.
Multi-Scale Face Restoration with Sequential Gating Ensemble Network
Experiment results demonstrate that our SGEN is more effective at multi-scale human face restoration with more image details and less noise than state-of-the-art image restoration models.
Conditional Image-to-Image Translation
In this paper, we study a new problem, conditional image-to-image translation, which is to translate an image from the source domain to the target domain conditioned on a given image in the target domain.
Learning for Video Compression
One key challenge to learning-based video compression is that motion predictive coding, a very effective tool for video compression, can hardly be trained into a neural network.
Multimedia Image and Video Processing
Learning to Run challenge solutions: Adapting reinforcement learning methods for neuromusculoskeletal environments
In the NIPS 2017 Learning to Run challenge, participants were tasked with building a controller for a musculoskeletal model to make it run as fast as possible through an obstacle course.
