Search Results for author:
Found 269 papers, 179 papers with code
Cumulative Attribute Space for Age and Crowd Density Estimation
A number of computer vision problems such as human age estimation, crowd density estimation and body/face pose (view angle) estimation can be formulated as a regression problem by learning a mapping function between a high dimensional vector-formed feature input and a scalarvalued output.
Constructing Robust Affinity Graphs for Spectral Clustering
Spectral clustering requires robust and meaningful affinity graphs as input in order to form clusters with desired structures that can well support human intuition.
Learning Deep Representation for Face Alignment with Auxiliary Attributes
In this study, we show that landmark detection or face alignment task is not a single and independent problem.
Ranked #13 on
Unsupervised Facial Landmark Detection
on MAFL
Transferring Landmark Annotations for Cross-Dataset Face Alignment
We show extensive results on combining various popular databases (LFW, AFLW, LFPW, HELEN) for improved cross-dataset and unseen data alignment.
Crowd Saliency Detection via Global Similarity Structure
This paper proposes a novel framework to identify and localize salient regions in a crowd scene, by transforming low-level features extracted from crowd motion field into a global similarity structure.
Image Super-Resolution Using Deep Convolutional Networks
We further show that traditional sparse-coding-based SR methods can also be viewed as a deep convolutional network.
Ranked #2 on
Video Super-Resolution
on Xiph HD - 4x upscaling
Learning to Recognize Pedestrian Attribute
Learning to recognize pedestrian attributes at far distance is a challenging problem in visual surveillance since face and body close-shots are hardly available; instead, only far-view image frames of pedestrian are given.
Learning from Multiple Sources for Video Summarisation
Many visual surveillance tasks, e. g. video summarisation, is conventionally accomplished through analysing imagerybased features.
Compression Artifacts Reduction by a Deep Convolutional Network
Lossy compression introduces complex compression artifacts, particularly the blocking artifacts, ringing effects and blurring.
Deeply Learned Attributes for Crowded Scene Understanding
We further measure user study performance on WWW and compare this with the proposed deep models.
Boosting Optical Character Recognition: A Super-Resolution Approach
Text image super-resolution is a challenging yet open research problem in the computer vision community.
Face alignment by coarse-to-fine shape searching
We present a novel face alignment framework based on coarse-to-fine shape searching.
Ranked #20 on
Face Alignment
on AFLW-19
Reading Scene Text in Deep Convolutional Sequences
We develop a Deep-Text Recurrent Network (DTRN) that regards scene text reading as a sequence labelling problem.
A Large-Scale Car Dataset for Fine-Grained Categorization and Verification
Updated on 24/09/2015: This update provides preliminary experiment results for fine-grained classification on the surveillance data of CompCars.
Ranked #5 on
Fine-Grained Image Classification
on CompCars
Semantic Image Segmentation via Deep Parsing Network
This paper addresses semantic image segmentation by incorporating rich information into Markov Random Field (MRF), including high-order relations and mixture of label contexts.
Ranked #89 on
Semantic Segmentation
on Cityscapes test
Learning Social Relation Traits from Face Images
Social relation defines the association, e. g, warm, friendliness, and dominance, between two or more people.
From Facial Parts Responses to Face Detection: A Deep Learning Approach
In this paper, we propose a novel deep convolutional network (DCN) that achieves outstanding performance on FDDB, PASCAL Face, and AFW.
An Empirical Study of Recent Face Alignment Methods
In this paper, we carry out a rigorous evaluation of these methods by making the following contributions: 1) we proposes a new evaluation metric for face alignment on a set of images, i. e., area under error distribution curve within a threshold, AUC$_\alpha$, given the fact that the traditional evaluation measure (mean error) is very sensitive to big alignment error.
Towards Arbitrary-View Face Alignment by Recommendation Trees
The unified framework seamlessly handles different viewpoints and landmark protocols, and it is trained by optimising directly on landmark locations, thus yielding superior results on arbitrary-view face alignment.
WIDER FACE: A Face Detection Benchmark
Face detection is one of the most studied topics in the computer vision community.
Ranked #34 on
Face Detection
on WIDER Face (Medium)
Discriminative Sparse Neighbor Approximation for Imbalanced Learning
These methods further deteriorate on small, imbalanced data that has a large degree of class overlap.
Unsupervised Learning of Discriminative Attributes and Visual Representations
Attributes offer useful mid-level features to interpret visual data.
Learning Deep Representation for Imbalanced Classification
We further demonstrate that more discriminative deep representation can be learned by enforcing a deep network to maintain both inter-cluster and inter-class margins.
Deep Learning Markov Random Field for Semantic Segmentation
Semantic segmentation tasks can be well modeled by Markov Random Field (MRF).
Deep Cascaded Bi-Network for Face Hallucination
We present a novel framework for hallucinating faces of unconstrained poses and with very low resolution (face size as small as 5pxIOD).
Ranked #5 on
Image Super-Resolution
on VggFace2 - 8x upscaling
Accelerating the Super-Resolution Convolutional Neural Network
As a successful deep model applied in image super-resolution (SR), the Super-Resolution Convolutional Neural Network (SRCNN) has demonstrated superior performance to the previous hand-crafted models either in speed and restoration quality.
Deep Convolution Networks for Compression Artifacts Reduction
Lossy compression introduces complex compression artifacts, particularly blocking artifacts, ringing effects and blurring.
From Facial Expression Recognition to Interpersonal Relation Prediction
Unlike existing models that typically learn from facial expression labels alone, we devise an effective multitask network that is capable of learning from rich auxiliary attributes such as gender, age, and head pose, beyond just facial expression data.
Image Aesthetic Assessment: An Experimental Survey
This survey aims at reviewing recent computer vision techniques used in the assessment of image aesthetic quality.
Local Similarity-Aware Deep Feature Embedding
Existing deep embedding methods in vision tasks are capable of learning a compact Euclidean space from images, where Euclidean distances correspond to a similarity metric.
Ranked #27 on
Metric Learning
on CUB-200-2011
PolyNet: A Pursuit of Structural Diversity in Very Deep Networks
A number of studies have shown that increasing the depth or width of convolutional networks is a rewarding approach to improve the performance of image recognition.
Faceness-Net: Face Detection through Deep Facial Part Responses
We propose a deep convolutional neural network (CNN) for face detection leveraging on facial attributes based supervision.
Not All Pixels Are Equal: Difficulty-aware Semantic Segmentation via Deep Layer Cascade
Third, in comparison to MC, LC is an end-to-end trainable framework, allowing joint learning of all sub-models.
Ranked #22 on
Semantic Segmentation
on PASCAL VOC 2012 test
Robust and Fast Decoding of High-Capacity Color QR Codes for Mobile Applications
Particularly, we further discover a new type of chromatic distortion in high-density color QR codes, cross-module color interference, caused by the high density which also makes the geometric distortion correction more challenging.
Face Detection through Scale-Friendly Deep Convolutional Networks
Specifically, our method achieves 76. 4 average precision on the challenging WIDER FACE dataset and 96% recall rate on the FDDB dataset with 7 frames per second (fps) for 900 * 1300 input image.
Merge or Not? Learning to Group Faces via Imitation Learning
Given a large number of unlabeled face images, face grouping aims at clustering the images into individual identities present in the data.
Aesthetic-Driven Image Enhancement by Adversarial Learning
We introduce EnhanceGAN, an adversarial learning based model that performs automatic image enhancement.
Discover and Learn New Objects from Documentaries
Despite the remarkable progress in recent years, detecting objects in a new context remains a challenging task.
Video Object Segmentation with Re-identification
Specifically, our Video Object Segmentation with Re-identification (VS-ReID) model includes a mask propagation module and a ReID module.
Quantifying Facial Age by Posterior of Age Comparisons
We introduce a novel approach for annotating large quantity of in-the-wild facial images with high-quality posterior age distribution as labels.
Ranked #7 on
Age Estimation
on MORPH Album2
(using extra training data)
Be Your Own Prada: Fashion Synthesis with Structural Coherence
In the second stage, a generative model with a newly proposed compositional mapping layer is used to render the final image with precise regions and textures conditioned on this map.
Mix-and-Match Tuning for Self-Supervised Semantic Segmentation
The key of this new form of learning is to design a proxy task (e. g. image colorization), from which a discriminative loss can be formulated on unlabeled data.
Pose-Robust Face Recognition via Deep Residual Equivariant Mapping
However, many contemporary face recognition models still perform relatively poor in processing profile faces compared to frontal faces.
Ranked #1 on
Face Identification
on IJB-A
Video Object Segmentation with Joint Re-identification and Attention-Aware Mask Propagation
The problem of video object segmentation can become extremely challenging when multiple instances co-exist.
Recovering Realistic Texture in Image Super-resolution by Deep Spatial Feature Transform
In this paper, we show that it is possible to recover textures faithful to semantic classes.
Ranked #55 on
Image Super-Resolution
on BSD100 - 4x upscaling
Crafting a Toolchain for Image Restoration by Deep Reinforcement Learning
We investigate a novel approach for image restoration by reinforcement learning.
Optimizing Video Object Detection via a Scale-Time Lattice
High-performance object detection relies on expensive convolutional networks to compute features, often leading to significant challenges in applications, e. g. those that require detecting objects from video streams in real time.
LiteFlowNet: A Lightweight Convolutional Neural Network for Optical Flow Estimation
FlowNet2, the state-of-the-art convolutional neural network (CNN) for optical flow estimation, requires over 160M parameters to achieve accurate flow estimation.
Ranked #10 on
Optical Flow Estimation
on KITTI 2012
Deep Imbalanced Learning for Face Recognition and Attribute Prediction
Data for face analysis often exhibit highly-skewed class distribution, i. e., most data belong to a few majority classes, while the minority classes only contain a scarce amount of instances.
Non-Local Recurrent Network for Image Restoration
The main contributions of this work are: (1) Unlike existing methods that measure self-similarity in an isolated manner, the proposed non-local module can be flexibly integrated into existing deep networks for end-to-end training to capture deep feature correlation between each location and its neighborhood.
Ranked #1 on
Grayscale Image Denoising
on Set12 sigma30
Zoom-Net: Mining Deep Feature Interactions for Visual Relationship Recognition
We show that by encouraging deep message propagation and interactions between local object features and global predicate features, one can achieve compelling performance in recognizing complex relationships without using any linguistic priors.
ReenactGAN: Learning to Reenact Faces via Boundary Transfer
A transformer is subsequently used to adapt the boundary of source face to the boundary of target face.
The Devil of Face Recognition is in the Noise
2) With the original datasets and cleaned subsets, we profile and analyze label noise properties of MegaFace and MS-Celeb-1M.
ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks
To further enhance the visual quality, we thoroughly study three key components of SRGAN - network architecture, adversarial loss and perceptual loss, and improve each of them to derive an Enhanced SRGAN (ESRGAN).
Ranked #2 on
Face Hallucination
on FFHQ 512 x 512 - 16x upscaling
Lifelong Learning via Progressive Distillation and Retrospection
Lifelong learning aims at adapting a learned model to new tasks while retaining the knowledge gained earlier.
PSANet: Point-wise Spatial Attention Network for Scene Parsing
We notice information flow in convolutional neural networks is restricted inside local neighborhood regions due to the physical design of convolutional filters, which limits the overall understanding of complex scenes.
Ranked #51 on
Semantic Segmentation
on Cityscapes test
Consensus-Driven Propagation in Massive Unlabeled Data for Face Recognition
Face recognition has witnessed great progress in recent years, mainly attributed to the high-capacity model designed and the abundant labeled data collected.
Improving On-policy Learning with Statistical Reward Accumulation
Most methods in deep-RL achieve good results via the maximization of the reward signal provided by the environment, typically in the form of discounted cumulative returns.
Unsupervised Disentangling Structure and Appearance
It is challenging to disentangle an object into two orthogonal spaces of structure and appearance since each can influence the visual observation in a different and unpredictable way.
Learning to Steer by Mimicking Features from Heterogeneous Auxiliary Networks
In this paper, we considerably improve the accuracy and robustness of predictions through heterogeneous auxiliary networks feature mimicking, a new and effective training method that provides us with much richer contextual signals apart from steering direction.
Ranked #1 on
Steering Control
on BDD100K val
Deep Network Interpolation for Continuous Imagery Effect Transition
Deep convolutional neural network has demonstrated its capability of learning a deterministic mapping for the desired imagery effect.
Instance-level Facial Attributes Transfer with Geometry-Aware Flow
Geometry-aware flow is able to warp the source face attribute into the target face context and generate a warp-and-blend result.
An Embarrassingly Simple Approach for Knowledge Distillation
Knowledge Distillation (KD) aims at improving the performance of a low-capacity student model by inheriting knowledge from a high-capacity teacher model.
Region Proposal by Guided Anchoring
State-of-the-art detectors mostly rely on a dense anchoring scheme, where anchors are sampled uniformly over the spatial domain with a predefined set of scales and aspect ratios.
Ranked #1 on
Region Proposal
on COCO test-dev
Hybrid Task Cascade for Instance Segmentation
In exploring a more effective approach, we find that the key to a successful instance segmentation cascade is to fully leverage the reciprocal relationship between detection and segmentation.
Ranked #32 on
Object Detection
on COCO-O
WIDER Face and Pedestrian Challenge 2018: Methods and Results
This paper presents a review of the 2018 WIDER Challenge on Face and Pedestrian.
Unsupervised Bi-directional Flow-based Video Generation from one Snapshot
Imagining multiple consecutive frames given one single snapshot is challenging, since it is difficult to simultaneously predict diverse motions from a single image and faithfully generate novel frames without visual distortions.
A Lightweight Optical Flow CNN -- Revisiting Data Fidelity and Regularization
Over four decades, the majority addresses the problem of optical flow estimation using variational methods.
Ranked #6 on
Optical Flow Estimation
on KITTI 2012
Self-Supervised Learning via Conditional Motion Propagation
Instead of explicitly modeling the motion probabilities, we design the pretext task as a conditional motion propagation problem.
Dense Intrinsic Appearance Flow for Human Pose Transfer
Unlike existing methods, we propose to estimate dense and intrinsic 3D appearance flow to better guide the transfer of pixels between poses.
Learning to Cluster Faces on an Affinity Graph
Face recognition sees remarkable progress in recent years, and its performance has reached a very high level.
Prime Sample Attention in Object Detection
Our experiments demonstrate that it is often more effective to focus on prime samples than hard samples when training a detector.
TransGaGa: Geometry-Aware Unsupervised Image-to-Image Translation
Extensive experiments demonstrate the superior performance of our method to other state-of-the-art approaches, especially in the challenging near-rigid and non-rigid objects translation tasks.
Path-Restore: Learning Network Path Selection for Image Restoration
To leverage this, we propose Path-Restore, a multi-path CNN with a pathfinder that can dynamically select an appropriate route for each image region.
CARAFE: Content-Aware ReAssembly of FEatures
CARAFE introduces little computational overhead and can be readily integrated into modern network architectures.
Ranked #3 on
Feature Upsampling
on ImageNet
EDVR: Video Restoration with Enhanced Deformable Convolutional Networks
In this work, we propose a novel Video Restoration framework with Enhanced Deformable networks, termed EDVR, to address these challenges.
Ranked #2 on
Deblurring
on REDS
Deep Flow-Guided Video Inpainting
Then the synthesized flow field is used to guide the propagation of pixels to fill up the missing regions in the video.
Ranked #8 on
Video Inpainting
on DAVIS
One-shot visual object segmentation
Optical Flow Estimation
+2
Disentangling Content and Style via Unsupervised Geometry Distillation
It is challenging to disentangle an object into two orthogonal spaces of content and style since each can influence the visual observation differently and unpredictably.
MMDetection: Open MMLab Detection Toolbox and Benchmark
In this paper, we introduce the various features of this toolbox.
Learning Lightweight Lane Detection CNNs by Self Attention Distillation
Training deep models for lane detection is challenging due to the very subtle and sparse supervisory signals inherent in lane annotations.
Ranked #5 on
Lane Detection
on BDD100K val
One-shot Face Reenactment
However, in real-world scenario end-users often only have one target face at hand, rendering existing methods inapplicable.
Delving Deep Into Hybrid Annotations for 3D Human Recovery in the Wild
Specifically, we focus on the challenging task of in-the-wild 3D human recovery from single images when paired 3D annotations are not fully available.
Robust Multi-Modality Multi-Object Tracking
Multi-sensor perception is crucial to ensure the reliability and accuracy in autonomous driving system, while multi-object tracking (MOT) improves that by tracing sequential movement of dynamic objects.
Ranked #10 on
Multiple Object Tracking
on KITTI Tracking test
Learning to Synthesize Fashion Textures
Another important observation is that fashion textures are multi-modal.
Side-Aware Boundary Localization for More Precise Object Detection
To tackle the difficulty of precise localization in the presence of displacements with large variance, we further propose a two-step localization scheme, which first predicts a range of movement through bucket prediction and then pinpoints the precise position within the predicted bucket.
EcoNAS: Finding Proxies for Economical Neural Architecture Search
While many methods have been proposed to improve the efficiency of NAS, the search progress is still laborious because training and evaluating plausible architectures over large search space is time-consuming.
DeeperForensics-1.0: A Large-Scale Dataset for Real-World Face Forgery Detection
The quality of generated videos outperforms those in existing datasets, validated by user studies.
Everybody's Talkin': Let Me Talk as You Want
The audio-translated expression parameters are then used to synthesize a photo-realistic human subject in each video frame, with the movement of the mouth regions precisely mapped to the source audio.
Zero-Reference Deep Curve Estimation for Low-Light Image Enhancement
The paper presents a novel method, Zero-Reference Deep Curve Estimation (Zero-DCE), which formulates light enhancement as a task of image-specific curve estimation with a deep network.
Ranked #1 on
Color Constancy
on INTEL-TUT2
Real or Not Real, that is the Question
While generative adversarial networks (GAN) have been widely adopted in various topics, in this paper we generalize the standard GAN to a new perspective by treating realness as a random variable that can be estimated from multiple angles.
Residual Knowledge Distillation
Knowledge distillation (KD) is one of the most potent ways for model compression.
1st Place Solutions for OpenImage2019 -- Object Detection and Instance Segmentation
Given such good instance bounding box, we further design a simple instance-level semantic segmentation pipeline and achieve the 1st place on the segmentation challenge.
Exploiting Deep Generative Prior for Versatile Image Restoration and Manipulation
Learning a good image prior is a long-term goal for image restoration and manipulation.
TransMoMo: Invariance-Driven Unsupervised Video Motion Retargeting
We present a lightweight video motion retargeting approach TransMoMo that is capable of transferring motion of a person in a source video realistically to another video of a target person.
Learning to Cluster Faces via Confidence and Connectivity Estimation
With the vertex confidence and edge connectivity, we can naturally organize more relevant vertices on the affinity graph and group them into clusters.
Self-Supervised Scene De-occlusion
This is achieved via Partial Completion Network (PCNet)-mask (M) and -content (C), that learn to recover fractions of object masks and contents, respectively, in a self-supervised manner.
Feature Pyramid Grids
Feature pyramid networks have been widely adopted in the object detection literature to improve feature representations for better handling of variations in scale.
Inter-Region Affinity Distillation for Road Marking Segmentation
We study the problem of distilling knowledge from a large deep teacher network to a much smaller student network for the task of road marking segmentation.
Ranked #1 on
Semantic Segmentation
on ApolloScape
Knowledge Distillation Meets Self-Supervision
Knowledge distillation, which involves extracting the "dark knowledge" from a teacher network to guide the learning of a student network, has emerged as an important technique for model compression and transfer learning.
Ranked #32 on
Knowledge Distillation
on ImageNet
Online Deep Clustering for Unsupervised Representation Learning
In this way, labels and the network evolve shoulder-to-shoulder rather than alternatingly.
Cross-Scale Internal Graph Neural Network for Image Super-Resolution
Specifically, we dynamically construct a cross-scale graph by searching k-nearest neighboring patches in the downsampled LR image for each query patch in the LR image.
RGB-D Salient Object Detection with Cross-Modality Modulation and Selection
Second, we propose an adaptive feature selection (AFS) module to select saliency-related features and suppress the inferior ones.
Ranked #8 on
RGB-D Salient Object Detection
on NJU2K
LiteFlowNet3: Resolving Correspondence Ambiguity for More Accurate Optical Flow Estimation
The keys to success lie in the use of cost volume and coarse-to-fine flow inference.
Ranked #4 on
Optical Flow Estimation
on KITTI 2012
TSIT: A Simple and Versatile Framework for Image-to-Image Translation
We introduce a simple and versatile framework for image-to-image translation.
MessyTable: Instance Association in Multiple Camera Views
We present an interesting and challenging dataset that features a large number of scenes with messy tables captured from multiple camera views.
Seesaw Loss for Long-Tailed Instance Segmentation
Instances of head classes dominate a long-tailed dataset and they serve as negative samples of tail categories.
Delving into Inter-Image Invariance for Unsupervised Visual Representations
In this work, we present a comprehensive empirical study to better understand the role of inter-image invariance learning from three main constituting components: pseudo-label maintenance, sampling strategy, and decision boundary design.
Understanding Deformable Alignment in Video Super-Resolution
Aside from the contributions to deformable alignment, our formulation inspires a more flexible approach to introduce offset diversity to flow-based alignment, improving its performance.
Texture Memory-Augmented Deep Patch-Based Image Inpainting
By bringing together the best of both paradigms, we propose a new deep inpainting framework where texture generation is guided by a texture memory of patch samples extracted from unmasked regions.
Flexible Piecewise Curves Estimation for Photo Enhancement
This paper presents a new method, called FlexiCurve, for photo enhancement.
Do 2D GANs Know 3D Shape? Unsupervised 3D shape reconstruction from 2D Image GANs
Through our investigation, we found that such a pre-trained GAN indeed contains rich 3D knowledge and thus can be used to recover 3D shape from a single 2D image in an unsupervised manner.
GLEAN: Generative Latent Bank for Large-Factor Image Super-Resolution
We show that pre-trained Generative Adversarial Networks (GANs), e. g., StyleGAN, can be used as a latent bank to improve the restoration quality of large-factor image super-resolution (SR).
BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond
Video super-resolution (VSR) approaches tend to have more components than the image counterparts as they need to exploit the additional temporal dimension.
CARAFE++: Unified Content-Aware ReAssembly of FEatures
Feature reassembly, i. e. feature downsampling and upsampling, is a key operation in a number of modern convolutional network architectures, e. g., residual networks and feature pyramids.
Positional Encoding as Spatial Inductive Bias in GANs
In this work, taking SinGAN and StyleGAN2 as examples, we show that such capability, to a large extent, is brought by the implicit positional encoding when using zero padding in the generators.
Computation-Efficient Knowledge Distillation via Uncertainty-Aware Mixup
Our goal is to achieve a performance comparable to conventional knowledge distillation with a lower computation cost during training.
Focal Frequency Loss for Image Reconstruction and Synthesis
In this study, we show that narrowing gaps in the frequency domain can ameliorate image reconstruction and synthesis quality further.
Exploring Data Augmentation for Multi-Modality 3D Object Detection
Due to the fact that multi-modality data augmentation must maintain consistency between point cloud and images, recent methods in this field typically use relatively insufficient data augmentation.
Chasing the Tail in Monocular 3D Human Reconstruction with Prototype Memory
The reason is that most of the current models perform regression based on a single human prototype, which is similar to common poses while far from the rare poses.
DeeperForensics Challenge 2020 on Real-World Face Forgery Detection: Methods and Results
This paper reports methods and results in the DeeperForensics Challenge 2020 on real-world face forgery detection.
FASA: Feature Augmentation and Sampling Adaptation for Long-Tailed Instance Segmentation
We propose a simple yet effective method, Feature Augmentation and Sampling Adaptation (FASA), that addresses the data scarcity issue by augmenting the feature space especially for rare classes.
Learning to Enhance Low-Light Image via Zero-Reference Deep Curve Estimation
This paper presents a novel method, Zero-Reference Deep Curve Estimation (Zero-DCE), which formulates light enhancement as a task of image-specific curve estimation with a deep network.
Network Pruning via Resource Reallocation
Channel pruning is broadly recognized as an effective approach to obtain a small compact model through eliminating unimportant channels from a large cumbersome network.
Domain Generalization: A Survey
Generalization to out-of-distribution (OOD) data is a capability natural to humans yet challenging for machines to reproduce.
Deep Animation Video Interpolation in the Wild
In the animation industry, cartoon videos are usually produced at low frame rate since hand drawing of such frames is costly and time-consuming.
Everything's Talkin': Pareidolia Face Reenactment
We present a new application direction named Pareidolia Face Reenactment, which is defined as animating a static illusory face to move in tandem with a human face in the video.
Audio-Driven Emotional Video Portraits
In this work, we present Emotional Video Portraits (EVP), a system for synthesizing high-quality video portraits with vivid emotional dynamics driven by audios.
Removing Diffraction Image Artifacts in Under-Display Camera via Dynamic Skip Connection Network
Recent development of Under-Display Camera (UDC) systems provides a true bezel-less and notch-free viewing experience on smartphones (and TV, laptops, tablets), while allowing images to be captured from the selfie camera embedded underneath.
Low-Light Image and Video Enhancement Using Deep Learning: A Survey
Low-light image enhancement (LLIE) aims at improving the perception or interpretability of an image captured in an environment with poor illumination.
NTIRE 2021 Challenge on Quality Enhancement of Compressed Video: Methods and Results
This paper reviews the first NTIRE challenge on quality enhancement of compressed video, with a focus on the proposed methods and results.
Pose-Controllable Talking Face Generation by Implicitly Modularized Audio-Visual Representation
While speech content information can be defined by learning the intrinsic synchronization between audio-visual modalities, we identify that a pose code will be complementarily learned in a modulated convolution-based reconstruction framework.
Unsupervised 3D Shape Completion through GAN Inversion
In contrast to previous fully supervised approaches, in this paper we present ShapeInversion, which introduces Generative Adversarial Network (GAN) inversion to shape completion for the first time.
BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment
We show that by empowering the recurrent framework with the enhanced propagation and alignment, one can exploit spatiotemporal information across misaligned video frames more effectively.
Ranked #1 on
Video Enhancement
on MFQE v2
Semi-Supervised Domain Generalization with Stochastic StyleMatch
We find that the DG methods, which by design are unable to handle unlabeled data, perform poorly with limited labels in SSDG; the SSL methods, especially FixMatch, obtain much better results but are still far away from the basic vanilla model trained using full labels.
Robust Reference-based Super-Resolution via C2-Matching
However, performing local transfer is difficult because of two gaps between input and reference images: the transformation gap (e. g. scale and rotation) and the resolution gap (e. g. HR and LR).
Pareidolia Face Reenactment
We present a new application direction named Pareidolia Face Reenactment, which is defined as animating a static illusory face to move in tandem with a human face in the video.
Unsupervised Object-Level Representation Learning from Scene Images
Extensive experiments on COCO show that ORL significantly improves the performance of self-supervised learning on scene images, even surpassing supervised ImageNet pre-training on several downstream tasks.
K-Net: Towards Unified Image Segmentation
The framework, named K-Net, segments both instances and semantic categories consistently by a group of learnable kernels, where each kernel is responsible for generating a mask for either a potential instance or a stuff class.
Ranked #7 on
Panoptic Segmentation
on COCO test-dev
Learning to Prompt for Vision-Language Models
Large pre-trained vision-language models like CLIP have shown great potential in learning representations that are transferable across a wide range of downstream tasks.
Ranked #2 on
Few-shot Age Estimation
on MORPH Album2
3D Human Texture Estimation from a Single Image with Transformers
We propose a Transformer-based framework for 3D human texture estimation from a single image.
Talk-to-Edit: Fine-Grained Facial Editing via Dialog
In this work, we propose Talk-to-Edit, an interactive facial editing framework that performs fine-grained attribute manipulation through dialog between the user and the system.
Ranked #1 on
Fine-Grained Facial Editing
on CelebA-Dialog
ReconfigISP: Reconfigurable Camera Image Processing Pipeline
Image Signal Processor (ISP) is a crucial component in digital cameras that transforms sensor signals into images for us to perceive and understand.
A Comprehensive Overhaul of Distilling Unconditional GANs
To further enhance the semantic consistency between the teacher and student model, we present another latent-direction-based distillation loss that preserves the semantic relations in latent space.
MeshInversion: 3D textured mesh reconstruction with generative prior
Reconstruction is achieved by searching for a latent space in the 3D GAN that best resembles the target mesh in accordance with the single view observation.
SiT: Simulation Transformer for Particle-based Physics Simulation
However, they force particles to interact with all neighbors without selection, and they fall short in capturing material semantics for different particles, leading to unsatisfactory performance, especially in generalization.
Temporally Consistent Video Colorization with Deep Feature Propagation and Self-regularization Learning
We address this problem from a new perspective, by jointly considering colorization and temporal consistency in a unified framework.
Playing for 3D Human Recovery
Specifically, we contribute GTA-Human, a large-scale 3D human dataset generated with the GTA-V game engine, featuring a highly diverse set of subjects, actions, and scenarios.
Self-Supervised Representation Learning: Introduction, Advances and Challenges
Self-supervised representation learning methods aim to provide powerful deep feature learning without the requirement of large annotated datasets, thus alleviating the annotation bottleneck that is one of the main barriers to practical deployment of deep learning today.
The Nuts and Bolts of Adopting Transformer in GANs
Transformer becomes prevalent in computer vision, especially for high-level vision tasks.
A Shading-Guided Generative Implicit Model for Shape-Accurate 3D-Aware Image Synthesis
Motivated by the observation that a 3D object should look realistic from multiple viewpoints, these methods introduce a multi-view constraint as regularization to learn valid 3D radiance fields from 2D images.
Monocular 3D Reconstruction of Interacting Hands via Collision-Aware Factorized Refinements
In this paper, we make the first attempt to reconstruct 3D interacting hands from monocular single RGB images.
Deceive D: Adaptive Pseudo Augmentation for GAN Training with Limited Data
Generative adversarial networks (GANs) typically require ample data for training in order to synthesize high-fidelity images.
Investigating Tradeoffs in Real-World Video Super-Resolution
The diversity and complexity of degradations in real-world video super-resolution (VSR) pose non-trivial challenges in inference and training.
Extract Free Dense Labels from CLIP
Contrastive Language-Image Pre-training (CLIP) has made a remarkable breakthrough in open-vocabulary zero-shot image recognition.
MoCaNet: Motion Retargeting in-the-wild via Canonicalization Networks
Trained with the canonicalization operations and the derived regularizations, our method learns to factorize a skeleton sequence into three independent semantic subspaces, i. e., motion, structure, and view angle.
LEDNet: Joint Low-light Enhancement and Deblurring in the Dark
With the pipeline, we present the first large-scale dataset for joint low-light enhancement and deblurring.
Ranked #2 on
Low-Light Image Enhancement
on Sony-Total-Dark
Conditional Prompt Learning for Vision-Language Models
With the rise of powerful pre-trained vision-language models like CLIP, it becomes essential to investigate ways to adapt these models to downstream datasets.
Ranked #3 on
Prompt Engineering
on ImageNet V2
Dense Siamese Network for Dense Unsupervised Learning
It also extracts a batch of region embeddings that correspond to some sub-regions in the overlapped area to be contrasted for region consistency.
Ranked #2 on
Unsupervised Semantic Segmentation
on COCO-All
(mIoU metric)
Open-Vocabulary DETR with Conditional Matching
To this end, we propose a novel open-vocabulary detector based on DETR -- hence the name OV-DETR -- which, once trained, can detect any object given its class name or an exemplar image.
Ranked #21 on
Open Vocabulary Object Detection
on MSCOCO
Pastiche Master: Exemplar-Based High-Resolution Portrait Style Transfer
Recent studies on StyleGAN show high performance on artistic portrait generation by transfer learning with limited data.
Bailando: 3D Dance Generation by Actor-Critic GPT with Choreographic Memory
With the learned choreographic memory, dance generation is realized on the quantized units that meet high choreography standards, such that the generated dancing sequences are confined within the spatial constraints.
Ranked #1 on
Motion Synthesis
on AIST++
TransEditor: Transformer-Based Dual-Space GAN for Highly Controllable Facial Editing
In this study, we highlight the importance of interaction in a dual-space GAN for more controllable editing.
Unsupervised Image-to-Image Translation with Generative Prior
In this work, we present a novel framework, Generative Prior-guided UNsupervised Image-to-image Translation (GP-UNIT), to improve the overall quality and applicability of the translation algorithm.
Video K-Net: A Simple, Strong, and Unified Baseline for Video Segmentation
We hope this simple, yet effective method can serve as a new, flexible baseline in unified video segmentation design.
Ranked #1 on
Video Panoptic Segmentation
on KITTI-STEP
(using extra training data)
On the Generalization of BasicVSR++ to Video Deblurring and Denoising
The exploitation of long-term information has been a long-standing problem in video restoration.
StyleGAN-Human: A Data-Centric Odyssey of Human Generation
In addition, a model zoo and human editing applications are demonstrated to facilitate future research in the community.
HuMMan: Multi-Modal 4D Human Dataset for Versatile Sensing and Modeling
4D human sensing and modeling are fundamental tasks in vision and graphics with numerous applications.
Delving into High-Quality Synthetic Face Occlusion Segmentation Datasets
This paper performs comprehensive analysis on datasets for occlusion-aware face segmentation, a task that is crucial for many downstream applications.
Text2Human: Text-Driven Controllable Human Image Generation
In this work, we present a text-driven controllable framework, Text2Human, for a high-quality and diverse human generation.
Point-to-Voxel Knowledge Distillation for LiDAR Semantic Segmentation
This article addresses the problem of distilling knowledge from a large teacher model to a slim student network for LiDAR semantic segmentation.
Ranked #8 on
LIDAR Semantic Segmentation
on nuScenes
(val mIoU metric)
Masked Frequency Modeling for Self-Supervised Visual Pre-Training
We present Masked Frequency Modeling (MFM), a unified frequency-domain-based approach for self-supervised pre-training of visual models.
Towards Robust Blind Face Restoration with Codebook Lookup Transformer
In this paper, we demonstrate that a learned discrete codebook prior in a small proxy space largely reduces the uncertainty and ambiguity of restoration mapping by casting blind face restoration as a code prediction task, while providing rich visual atoms for generating high-quality faces.
Ranked #1 on
Blind Face Restoration
on CelebA-Test
BRACE: The Breakdancing Competition Dataset for Dance Motion Synthesis
Estimating human keypoints from these videos is difficult due to the complexity of the dance, as well as the multiple moving cameras recording setup.
Monocular 3D Object Reconstruction with GAN Inversion
Reconstruction is achieved by searching for a latent space in the 3D GAN that best resembles the target mesh in accordance with the single view observation.
Transformer with Implicit Edges for Particle-based Physics Simulation
Consequently, in this paper we propose a novel Transformer-based method, dubbed as Transformer with Implicit Edges (TIE), to capture the rich semantics of particle interactions in an edge-free manner.
Exploring CLIP for Assessing the Look and Feel of Images
Measuring the perception of visual content is a long-standing problem in computer vision.
Ranked #9 on
Video Quality Assessment
on MSU SR-QA Dataset
CelebV-HQ: A Large-Scale Video Facial Attributes Dataset
Large-scale datasets have played indispensable roles in the recent success of face generation/editing and significantly facilitated the advances of emerging research fields.
Ranked #1 on
Unconditional Video Generation
on CelebV-HQ
CuDi: Curve Distillation for Efficient and Controllable Exposure Adjustment
Our method inherits the zero-reference learning and curve-based framework from an effective low-light image enhancement method, Zero-DCE, with further speed up in its inference speed, reduction in its model size, and extension to controllable exposure adjustment.
GLEAN: Generative Latent Bank for Image Super-Resolution and Beyond
While most existing perceptual-oriented approaches attempt to generate realistic outputs through learning with adversarial loss, our method, Generative LatEnt bANk (GLEAN), goes beyond existing practices by directly leveraging rich and diverse priors encapsulated in a pre-trained GAN.
StyleLight: HDR Panorama Generation for Lighting Estimation and Editing
To tackle this problem, we propose a coupled dual-StyleGAN panorama synthesis network (StyleLight) that integrates LDR and HDR panorama synthesis into a unified framework.
Mind the Gap in Distilling StyleGANs
To further enhance the semantic consistency between the teacher and student model, we present a latent-direction-based distillation loss that preserves the semantic relations in latent space.
On-Device Domain Generalization
Another interesting observation is that the teacher-student gap on out-of-distribution data is bigger than that on in-distribution data, which highlights the capacity mismatch issue as well as the shortcoming of KD.
MIPI 2022 Challenge on Quad-Bayer Re-mosaic: Dataset and Report
A detailed description of all models developed in this challenge is provided in this paper.
MIPI 2022 Challenge on RGB+ToF Depth Completion: Dataset and Report
A detailed description of all models developed in this challenge is provided in this paper.
MIPI 2022 Challenge on Under-Display Camera Image Restoration: Methods and Results
In this paper, we summarize and review the Under-Display Camera (UDC) Image Restoration track on MIPI 2022.
MIPI 2022 Challenge on RGBW Sensor Re-mosaic: Dataset and Report
A detailed description of all models developed in this challenge is provided in this paper.
MIPI 2022 Challenge on RGBW Sensor Fusion: Dataset and Report
A detailed description of all models developed in this challenge is provided in this paper.
VToonify: Controllable High-Resolution Portrait Video Style Transfer
Although a series of successful portrait image toonification models built upon the powerful StyleGAN have been proposed, these image-oriented methods have obvious limitations when applied to videos, such as the fixed frame size, the requirement of face alignment, missing non-facial details and temporal inconsistency.
Deep Fourier Up-Sampling
Existing convolutional neural networks widely adopt spatial down-/up-sampling for multi-scale modeling.
Flare7K: A Phenomenological Nighttime Flare Removal Dataset
In this paper, we introduce, Flare7K, the first nighttime flare removal dataset, which is generated based on the observation and statistics of real-world nighttime lens flares.
Ranked #2 on
Flare Removal
on Flare7K
Unified Vision and Language Prompt Learning
Prompt tuning, a parameter- and data-efficient transfer learning paradigm that tunes only a small number of parameters in a model's input space, has become a trend in the vision community since the emergence of large vision-language models like CLIP.
AnimeRun: 2D Animation Visual Correspondence from Open Source 3D Movies
Existing correspondence datasets for two-dimensional (2D) cartoon suffer from simple frame composition and monotonic movements, making them insufficient to simulate real animations.
BeautyREC: Robust, Efficient, and Content-preserving Makeup Transfer
The key insights of this study are modeling component-specific correspondence for local makeup transfer, capturing long-range dependencies for global makeup transfer, and enabling efficient makeup transfer via a single-path structure.
DifFace: Blind Face Restoration with Diffused Error Contraction
Moreover, the transition distribution can contract the error of the restoration backbone and thus makes our method more robust to unknown degradations.
Ranked #5 on
Blind Face Restoration
on CelebA-Test
Self-Supervised Geometry-Aware Encoder for Style-Based 3D GAN Inversion
In this paper, we study the challenging problem of 3D GAN inversion where a latent code is predicted given a single face image to faithfully recover its 3D shapes and detailed textures.
Reference-based Image and Video Super-Resolution via C2-Matching
To tackle these challenges, we propose C2-Matching in this work, which performs explicit robust matching crossing transformation and resolution.
Correspondence Distillation from NeRF-based GAN
The neural radiance field (NeRF) has shown promising results in preserving the fine details of objects and scenes.
DeformToon3D: Deformable Neural Radiance Fields for 3D Toonification
In this paper, we address the challenging problem of 3D toonification, which involves transferring the style of an artistic domain onto a target 3D face with stylized geometry and texture.
Betrayed by Captions: Joint Caption Grounding and Generation for Open Vocabulary Instance Segmentation
Experiments on the COCO dataset with two settings: Open Vocabulary Instance Segmentation (OVIS) and Open Set Panoptic Segmentation (OSPS) demonstrate the superiority of the CGG.
