Search Results for author:
Found 271 papers, 180 papers with code
Video Object Segmentation with Joint Re-identification and Attention-Aware Mask Propagation
The problem of video object segmentation can become extremely challenging when multiple instances co-exist.
Mix-and-Match Tuning for Self-Supervised Semantic Segmentation
The key of this new form of learning is to design a proxy task (e. g. image colorization), from which a discriminative loss can be formulated on unlabeled data.
From Facial Expression Recognition to Interpersonal Relation Prediction
Unlike existing models that typically learn from facial expression labels alone, we devise an effective multitask network that is capable of learning from rich auxiliary attributes such as gender, age, and head pose, beyond just facial expression data.
Be Your Own Prada: Fashion Synthesis with Structural Coherence
In the second stage, a generative model with a newly proposed compositional mapping layer is used to render the final image with precise regions and textures conditioned on this map.
Faceness-Net: Face Detection through Deep Facial Part Responses
We propose a deep convolutional neural network (CNN) for face detection leveraging on facial attributes based supervision.
Deep Learning Markov Random Field for Semantic Segmentation
Semantic segmentation tasks can be well modeled by Markov Random Field (MRF).
Face Detection through Scale-Friendly Deep Convolutional Networks
Specifically, our method achieves 76. 4 average precision on the challenging WIDER FACE dataset and 96% recall rate on the FDDB dataset with 7 frames per second (fps) for 900 * 1300 input image.
Local Similarity-Aware Deep Feature Embedding
Existing deep embedding methods in vision tasks are capable of learning a compact Euclidean space from images, where Euclidean distances correspond to a similarity metric.
Ranked #27 on
Metric Learning
on CUB-200-2011
Deep Cascaded Bi-Network for Face Hallucination
We present a novel framework for hallucinating faces of unconstrained poses and with very low resolution (face size as small as 5pxIOD).
Ranked #5 on
Image Super-Resolution
on VggFace2 - 8x upscaling
Discriminative Sparse Neighbor Approximation for Imbalanced Learning
These methods further deteriorate on small, imbalanced data that has a large degree of class overlap.
Reading Scene Text in Deep Convolutional Sequences
We develop a Deep-Text Recurrent Network (DTRN) that regards scene text reading as a sequence labelling problem.
Towards Arbitrary-View Face Alignment by Recommendation Trees
The unified framework seamlessly handles different viewpoints and landmark protocols, and it is trained by optimising directly on landmark locations, thus yielding superior results on arbitrary-view face alignment.
An Empirical Study of Recent Face Alignment Methods
In this paper, we carry out a rigorous evaluation of these methods by making the following contributions: 1) we proposes a new evaluation metric for face alignment on a set of images, i. e., area under error distribution curve within a threshold, AUC$_\alpha$, given the fact that the traditional evaluation measure (mean error) is very sensitive to big alignment error.
Semantic Image Segmentation via Deep Parsing Network
This paper addresses semantic image segmentation by incorporating rich information into Markov Random Field (MRF), including high-order relations and mixture of label contexts.
Ranked #89 on
Semantic Segmentation
on Cityscapes test
From Facial Parts Responses to Face Detection: A Deep Learning Approach
In this paper, we propose a novel deep convolutional network (DCN) that achieves outstanding performance on FDDB, PASCAL Face, and AFW.
Learning Social Relation Traits from Face Images
Social relation defines the association, e. g, warm, friendliness, and dominance, between two or more people.
Learning Deep Representation for Face Alignment with Auxiliary Attributes
In this study, we show that landmark detection or face alignment task is not a single and independent problem.
Ranked #13 on
Unsupervised Facial Landmark Detection
on MAFL
Boosting Optical Character Recognition: A Super-Resolution Approach
Text image super-resolution is a challenging yet open research problem in the computer vision community.
Learning to Recognize Pedestrian Attribute
Learning to recognize pedestrian attributes at far distance is a challenging problem in visual surveillance since face and body close-shots are hardly available; instead, only far-view image frames of pedestrian are given.
Learning from Multiple Sources for Video Summarisation
Many visual surveillance tasks, e. g. video summarisation, is conventionally accomplished through analysing imagerybased features.
Crowd Saliency Detection via Global Similarity Structure
This paper proposes a novel framework to identify and localize salient regions in a crowd scene, by transforming low-level features extracted from crowd motion field into a global similarity structure.
Transferring Landmark Annotations for Cross-Dataset Face Alignment
We show extensive results on combining various popular databases (LFW, AFLW, LFPW, HELEN) for improved cross-dataset and unseen data alignment.
Zoom-Net: Mining Deep Feature Interactions for Visual Relationship Recognition
We show that by encouraging deep message propagation and interactions between local object features and global predicate features, one can achieve compelling performance in recognizing complex relationships without using any linguistic priors.
Improving On-policy Learning with Statistical Reward Accumulation
Most methods in deep-RL achieve good results via the maximization of the reward signal provided by the environment, typically in the form of discounted cumulative returns.
Instance-level Facial Attributes Transfer with Geometry-Aware Flow
Geometry-aware flow is able to warp the source face attribute into the target face context and generate a warp-and-blend result.
Lifelong Learning via Progressive Distillation and Retrospection
Lifelong learning aims at adapting a learned model to new tasks while retaining the knowledge gained earlier.
Cumulative Attribute Space for Age and Crowd Density Estimation
A number of computer vision problems such as human age estimation, crowd density estimation and body/face pose (view angle) estimation can be formulated as a regression problem by learning a mapping function between a high dimensional vector-formed feature input and a scalarvalued output.
Constructing Robust Affinity Graphs for Spectral Clustering
Spectral clustering requires robust and meaningful affinity graphs as input in order to form clusters with desired structures that can well support human intuition.
Deeply Learned Attributes for Crowded Scene Understanding
We further measure user study performance on WWW and compare this with the proposed deep models.
Unsupervised Learning of Discriminative Attributes and Visual Representations
Attributes offer useful mid-level features to interpret visual data.
Learning Deep Representation for Imbalanced Classification
We further demonstrate that more discriminative deep representation can be learned by enforcing a deep network to maintain both inter-cluster and inter-class margins.
WIDER Face and Pedestrian Challenge 2018: Methods and Results
This paper presents a review of the 2018 WIDER Challenge on Face and Pedestrian.
Unsupervised Bi-directional Flow-based Video Generation from one Snapshot
Imagining multiple consecutive frames given one single snapshot is challenging, since it is difficult to simultaneously predict diverse motions from a single image and faithfully generate novel frames without visual distortions.
TransGaGa: Geometry-Aware Unsupervised Image-to-Image Translation
Extensive experiments demonstrate the superior performance of our method to other state-of-the-art approaches, especially in the challenging near-rigid and non-rigid objects translation tasks.
Learning to Synthesize Fashion Textures
Another important observation is that fashion textures are multi-modal.
EcoNAS: Finding Proxies for Economical Neural Architecture Search
While many methods have been proposed to improve the efficiency of NAS, the search progress is still laborious because training and evaluating plausible architectures over large search space is time-consuming.
Everybody's Talkin': Let Me Talk as You Want
The audio-translated expression parameters are then used to synthesize a photo-realistic human subject in each video frame, with the movement of the mouth regions precisely mapped to the source audio.
Residual Knowledge Distillation
Knowledge distillation (KD) is one of the most potent ways for model compression.
TransMoMo: Invariance-Driven Unsupervised Video Motion Retargeting
We present a lightweight video motion retargeting approach TransMoMo that is capable of transferring motion of a person in a source video realistically to another video of a target person.
MessyTable: Instance Association in Multiple Camera Views
We present an interesting and challenging dataset that features a large number of scenes with messy tables captured from multiple camera views.
Understanding Deformable Alignment in Video Super-Resolution
Aside from the contributions to deformable alignment, our formulation inspires a more flexible approach to introduce offset diversity to flow-based alignment, improving its performance.
Flexible Piecewise Curves Estimation for Photo Enhancement
This paper presents a new method, called FlexiCurve, for photo enhancement.
GLEAN: Generative Latent Bank for Large-Factor Image Super-Resolution
We show that pre-trained Generative Adversarial Networks (GANs), e. g., StyleGAN, can be used as a latent bank to improve the restoration quality of large-factor image super-resolution (SR).
Positional Encoding as Spatial Inductive Bias in GANs
In this work, taking SinGAN and StyleGAN2 as examples, we show that such capability, to a large extent, is brought by the implicit positional encoding when using zero padding in the generators.
CARAFE++: Unified Content-Aware ReAssembly of FEatures
Feature reassembly, i. e. feature downsampling and upsampling, is a key operation in a number of modern convolutional network architectures, e. g., residual networks and feature pyramids.
Chasing the Tail in Monocular 3D Human Reconstruction with Prototype Memory
The reason is that most of the current models perform regression based on a single human prototype, which is similar to common poses while far from the rare poses.
Unsupervised 3D Shape Completion through GAN Inversion
In contrast to previous fully supervised approaches, in this paper we present ShapeInversion, which introduces Generative Adversarial Network (GAN) inversion to shape completion for the first time.
Pareidolia Face Reenactment
We present a new application direction named Pareidolia Face Reenactment, which is defined as animating a static illusory face to move in tandem with a human face in the video.
MeshInversion: 3D textured mesh reconstruction with generative prior
Reconstruction is achieved by searching for a latent space in the 3D GAN that best resembles the target mesh in accordance with the single view observation.
SiT: Simulation Transformer for Particle-based Physics Simulation
However, they force particles to interact with all neighbors without selection, and they fall short in capturing material semantics for different particles, leading to unsatisfactory performance, especially in generalization.
A Comprehensive Overhaul of Distilling Unconditional GANs
To further enhance the semantic consistency between the teacher and student model, we present another latent-direction-based distillation loss that preserves the semantic relations in latent space.
Playing for 3D Human Recovery
Specifically, we contribute GTA-Human, a large-scale 3D human dataset generated with the GTA-V game engine, featuring a highly diverse set of subjects, actions, and scenarios.
Self-Supervised Representation Learning: Introduction, Advances and Challenges
Self-supervised representation learning methods aim to provide powerful deep feature learning without the requirement of large annotated datasets, thus alleviating the annotation bottleneck that is one of the main barriers to practical deployment of deep learning today.
The Nuts and Bolts of Adopting Transformer in GANs
Transformer becomes prevalent in computer vision, especially for high-level vision tasks.
Monocular 3D Reconstruction of Interacting Hands via Collision-Aware Factorized Refinements
In this paper, we make the first attempt to reconstruct 3D interacting hands from monocular single RGB images.
Unsupervised Disentangling Structure and Appearance
It is challenging to disentangle an object into two orthogonal spaces of structure and appearance since each can influence the visual observation in a different and unpredictable way.
MoCaNet: Motion Retargeting in-the-wild via Canonicalization Networks
Trained with the canonicalization operations and the derived regularizations, our method learns to factorize a skeleton sequence into three independent semantic subspaces, i. e., motion, structure, and view angle.
HuMMan: Multi-Modal 4D Human Dataset for Versatile Sensing and Modeling
4D human sensing and modeling are fundamental tasks in vision and graphics with numerous applications.
Point-to-Voxel Knowledge Distillation for LiDAR Semantic Segmentation
This article addresses the problem of distilling knowledge from a large teacher model to a slim student network for LiDAR semantic segmentation.
Ranked #8 on
LIDAR Semantic Segmentation
on nuScenes
(val mIoU metric)
CuDi: Curve Distillation for Efficient and Controllable Exposure Adjustment
Our method inherits the zero-reference learning and curve-based framework from an effective low-light image enhancement method, Zero-DCE, with further speed up in its inference speed, reduction in its model size, and extension to controllable exposure adjustment.
BeautyREC: Robust, Efficient, and Content-preserving Makeup Transfer
The key insights of this study are modeling component-specific correspondence for local makeup transfer, capturing long-range dependencies for global makeup transfer, and enabling efficient makeup transfer via a single-path structure.
Self-Supervised Geometry-Aware Encoder for Style-Based 3D GAN Inversion
In this paper, we study the challenging problem of 3D GAN inversion where a latent code is predicted given a single face image to faithfully recover its 3D shapes and detailed textures.
Correspondence Distillation from NeRF-based GAN
The neural radiance field (NeRF) has shown promising results in preserving the fine details of objects and scenes.
Embedding Fourier for Ultra-High-Definition Low-Light Image Enhancement
Our approach is motivated by a few unique characteristics in the Fourier domain: 1) most luminance information concentrates on amplitudes while noise is closely related to phases, and 2) a high-resolution image and its low-resolution version share similar amplitude patterns. Through embedding Fourier into our network, the amplitude and phase of a low-light image are separately processed to avoid amplifying noise when enhancing luminance.
Task-Oriented Human-Object Interactions Generation with Implicit Neural Representations
Generating task-oriented human-object interaction motions in simulation is challenging.
SparseNeRF: Distilling Depth Ranking for Few-shot Novel View Synthesis
Since coarse depth maps are not strictly scaled to the ground-truth depth maps, we propose a simple yet effective constraint, a local depth ranking method, on NeRFs such that the expected depth ranking of the NeRF is consistent with that of the coarse depth maps in local patches.
Iterative Prompt Learning for Unsupervised Backlit Image Enhancement
To solve this issue, we devise a prompt learning framework that first learns an initial prompt pair by constraining the text-image similarity between the prompt (negative/positive sample) and the corresponding image (backlit image/well-lit image) in the CLIP latent space.
MIPI 2023 Challenge on RGBW Fusion: Methods and Results
Developing and integrating advanced image sensors with novel algorithms in camera systems are prevalent with the increasing demand for computational photography and imaging on mobile platforms.
MIPI 2023 Challenge on RGBW Remosaic: Methods and Results
Developing and integrating advanced image sensors with novel algorithms in camera systems are prevalent with the increasing demand for computational photography and imaging on mobile platforms.
MIPI 2023 Challenge on RGB+ToF Depth Completion: Methods and Results
To evaluate the performance of different depth completion methods, we organized an RGB+sparse ToF depth completion competition.
Towards Multi-Layered 3D Garments Animation
In this paper, we propose a novel data-driven method, called LayersNet, to model garment-level animations as particle-wise interactions in a micro physics system.
MIPI 2023 Challenge on Nighttime Flare Removal: Methods and Results
In this paper, we summarize and review the Nighttime Flare Removal track on MIPI 2023.
Semi-Supervised and Long-Tailed Object Detection with CascadeMatch
This paper focuses on long-tailed object detection in the semi-supervised learning setting, which poses realistic challenges, but has rarely been studied in the literature.
Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translation
The framework includes two parts: key frame translation and full video translation.
Adaptive Window Pruning for Efficient Local Motion Deblurring
Existing image deblurring methods predominantly focus on global deblurring, inadvertently affecting the sharpness of backgrounds in locally blurred images and wasting unnecessary computation on sharp pixels, especially for high-resolution images.
PointHPS: Cascaded 3D Human Pose and Shape Estimation from Point Clouds
To tackle these challenges, we propose a principled framework, PointHPS, for accurate 3D HPS from point clouds captured in real-world settings, which iteratively refines point features through a cascaded architecture.
Audio-Driven Dubbing for User Generated Contents via Style-Aware Semi-Parametric Synthesis
Existing automated dubbing methods are usually designed for Professionally Generated Content (PGC) production, which requires massive training data and training time to learn a person-specific audio-video mapping.
Interpret Vision Transformers as ConvNets with Dynamic Convolutions
There has been a debate about the superiority between vision Transformers and ConvNets, serving as the backbone of computer vision models.
DeformToon3D: Deformable Neural Radiance Fields for 3D Toonification
In this paper, we address the challenging problem of 3D toonification, which involves transferring the style of an artistic domain onto a target 3D face with stylized geometry and texture.
PaintHuman: Towards High-fidelity Text-to-3D Human Texturing via Denoised Score Distillation
We first propose a novel score function, Denoised Score Distillation (DSD), which directly modifies the SDS by introducing negative gradient components to iteratively correct the gradient direction and generate high-quality textures.
VideoBooth: Diffusion-based Video Generation with Image Prompts
In this paper, we study the task of video generation with image prompts, which provide more accurate and direct content control beyond the text prompts.
Digital Life Project: Autonomous 3D Characters with Social Intelligence
In this work, we present Digital Life Project, a framework utilizing language as the universal medium to build autonomous 3D characters, who are capable of engaging in social interactions and expressing with articulated body motions, thereby simulating life in a digital environment.
Ranked #2 on
Motion Synthesis
on InterHuman
Gaussian3Diff: 3D Gaussian Diffusion for 3D Full Head Synthesis and Editing
We present a novel framework for generating photorealistic 3D human head and subsequently manipulating and reposing them with remarkable flexibility.
Upscale-A-Video: Temporal-Consistent Diffusion Model for Real-World Video Super-Resolution
Text-based diffusion models have exhibited remarkable success in generation and editing, showing great promise for enhancing visual content with their generative prior.
Towards Language-Driven Video Inpainting via Multimodal Large Language Models
We introduce a new task -- language-driven video inpainting, which uses natural language instructions to guide the inpainting process.
Control Color: Multimodal Diffusion-based Interactive Image Colorization
We also introduce a novel module based on self-attention and a content-guided deformable autoencoder to address the long-standing issues of color overflow and inaccurate coloring.
Explore In-Context Segmentation via Latent Diffusion Models
In-context segmentation has drawn more attention with the introduction of vision foundation models.
LN3Diff: Scalable Latent Neural Fields Diffusion for Speedy 3D Generation
The latent is decoded by a transformer-based decoder into a high-capacity 3D neural field.
Duolando: Follower GPT with Off-Policy Reinforcement Learning for Dance Accompaniment
We introduce a novel task within the field of 3D dance generation, termed dance accompaniment, which necessitates the generation of responsive movements from a dance partner, the "follower", synchronized with the lead dancer's movements and the underlying musical rhythm.
MOWA: Multiple-in-One Image Warping Model
To our knowledge, this is the first work that solves multiple practical warping tasks in one single model.
MIPI 2024 Challenge on Nighttime Flare Removal: Methods and Results
The increasing demand for computational photography and imaging on mobile platforms has led to the widespread development and integration of advanced image sensors with novel algorithms in camera systems.
MVIP-NeRF: Multi-view 3D Inpainting on NeRF Scenes via Diffusion Prior
Despite the emergence of successful NeRF inpainting methods built upon explicit RGB and depth 2D inpainting supervisions, these methods are inherently constrained by the capabilities of their underlying 2D inpainters.
Deep Imbalanced Learning for Face Recognition and Attribute Prediction
Data for face analysis often exhibit highly-skewed class distribution, i. e., most data belong to a few majority classes, while the minority classes only contain a scarce amount of instances.
WIDER FACE: A Face Detection Benchmark
Face detection is one of the most studied topics in the computer vision community.
Ranked #34 on
Face Detection
on WIDER Face (Medium)
Deep Network Interpolation for Continuous Imagery Effect Transition
Deep convolutional neural network has demonstrated its capability of learning a deterministic mapping for the desired imagery effect.
Deep Fourier Up-Sampling
Existing convolutional neural networks widely adopt spatial down-/up-sampling for multi-scale modeling.
An Embarrassingly Simple Approach for Knowledge Distillation
Knowledge Distillation (KD) aims at improving the performance of a low-capacity student model by inheriting knowledge from a high-capacity teacher model.
Discover and Learn New Objects from Documentaries
Despite the remarkable progress in recent years, detecting objects in a new context remains a challenging task.
Disentangling Content and Style via Unsupervised Geometry Distillation
It is challenging to disentangle an object into two orthogonal spaces of content and style since each can influence the visual observation differently and unpredictably.
Deep Convolution Networks for Compression Artifacts Reduction
Lossy compression introduces complex compression artifacts, particularly blocking artifacts, ringing effects and blurring.
A Large-Scale Car Dataset for Fine-Grained Categorization and Verification
Updated on 24/09/2015: This update provides preliminary experiment results for fine-grained classification on the surveillance data of CompCars.
Ranked #5 on
Fine-Grained Image Classification
on CompCars
Rethinking CLIP-based Video Learners in Cross-Domain Open-Vocabulary Action Recognition
To answer this, we establish a CROSS-domain Open-Vocabulary Action recognition benchmark named XOV-Action, and conduct a comprehensive evaluation of five state-of-the-art CLIP-based video learners under various types of domain gaps.
RGB-D Salient Object Detection with Cross-Modality Modulation and Selection
Second, we propose an adaptive feature selection (AFS) module to select saliency-related features and suppress the inferior ones.
Ranked #8 on
RGB-D Salient Object Detection
on NJU2K
Merge or Not? Learning to Group Faces via Imitation Learning
Given a large number of unlabeled face images, face grouping aims at clustering the images into individual identities present in the data.
DST-Det: Simple Dynamic Self-Training for Open-Vocabulary Object Detection
We refer to this approach as the self-training strategy, which enhances recall and accuracy for novel classes without requiring extra annotations, datasets, and re-training.
CLIM: Contrastive Language-Image Mosaic for Region Representation
Our experimental results demonstrate that CLIM improves different baseline open-vocabulary object detectors by a large margin on both OV-COCO and OV-LVIS benchmarks.
Ranked #6 on
Open Vocabulary Object Detection
on LVIS v1.0
ReconfigISP: Reconfigurable Camera Image Processing Pipeline
Image Signal Processor (ISP) is a crucial component in digital cameras that transforms sensor signals into images for us to perceive and understand.
StyleInV: A Temporal Style Modulated Inversion Network for Unconditional Video Generation
In this paper, we introduce a novel motion generator design that uses a learning-based inversion network for GAN.
Computation-Efficient Knowledge Distillation via Uncertainty-Aware Mixup
Our goal is to achieve a performance comparable to conventional knowledge distillation with a lower computation cost during training.
Correlational Image Modeling for Self-Supervised Visual Pre-Training
We introduce Correlational Image Modeling (CIM), a novel and surprisingly effective approach to self-supervised visual pre-training.
Mind the Gap in Distilling StyleGANs
To further enhance the semantic consistency between the teacher and student model, we present a latent-direction-based distillation loss that preserves the semantic relations in latent space.
FASA: Feature Augmentation and Sampling Adaptation for Long-Tailed Instance Segmentation
We propose a simple yet effective method, Feature Augmentation and Sampling Adaptation (FASA), that addresses the data scarcity issue by augmenting the feature space especially for rare classes.
Dense Siamese Network for Dense Unsupervised Learning
It also extracts a batch of region embeddings that correspond to some sub-regions in the overlapped area to be contrasted for region consistency.
Ranked #2 on
Unsupervised Semantic Segmentation
on COCO-All
(mIoU metric)
Transformer with Implicit Edges for Particle-based Physics Simulation
Consequently, in this paper we propose a novel Transformer-based method, dubbed as Transformer with Implicit Edges (TIE), to capture the rich semantics of particle interactions in an edge-free manner.
Quantifying Facial Age by Posterior of Age Comparisons
We introduce a novel approach for annotating large quantity of in-the-wild facial images with high-quality posterior age distribution as labels.
Ranked #7 on
Age Estimation
on MORPH Album2
(using extra training data)
Siamese DETR
In this work, we present Siamese DETR, a Siamese self-supervised pretraining approach for the Transformer architecture in DETR.
Temporally Consistent Video Colorization with Deep Feature Propagation and Self-regularization Learning
We address this problem from a new perspective, by jointly considering colorization and temporal consistency in a unified framework.
MIPI 2022 Challenge on Quad-Bayer Re-mosaic: Dataset and Report
A detailed description of all models developed in this challenge is provided in this paper.
MIPI 2022 Challenge on RGB+ToF Depth Completion: Dataset and Report
A detailed description of all models developed in this challenge is provided in this paper.
MIPI 2022 Challenge on Under-Display Camera Image Restoration: Methods and Results
In this paper, we summarize and review the Under-Display Camera (UDC) Image Restoration track on MIPI 2022.
MIPI 2022 Challenge on RGBW Sensor Fusion: Dataset and Report
A detailed description of all models developed in this challenge is provided in this paper.
MIPI 2022 Challenge on RGBW Sensor Re-mosaic: Dataset and Report
A detailed description of all models developed in this challenge is provided in this paper.
Path-Restore: Learning Network Path Selection for Image Restoration
To leverage this, we propose Path-Restore, a multi-path CNN with a pathfinder that can dynamically select an appropriate route for each image region.
Betrayed by Captions: Joint Caption Grounding and Generation for Open Vocabulary Instance Segmentation
Experiments on the COCO dataset with two settings: Open Vocabulary Instance Segmentation (OVIS) and Open Set Panoptic Segmentation (OSPS) demonstrate the superiority of the CGG.
Compression Artifacts Reduction by a Deep Convolutional Network
Lossy compression introduces complex compression artifacts, particularly the blocking artifacts, ringing effects and blurring.
Generating Aligned Pseudo-Supervision from Non-Aligned Data for Image Restoration in Under-Display Camera
Due to the difficulty in collecting large-scale and perfectly aligned paired training data for Under-Display Camera (UDC) image restoration, previous methods resort to monitor-based image systems or simulation-based methods, sacrificing the realness of the data and introducing domain gaps.
Aesthetic-Driven Image Enhancement by Adversarial Learning
We introduce EnhanceGAN, an adversarial learning based model that performs automatic image enhancement.
DeformToon3D: Deformable 3D Toonification from Neural Radiance Fields
In this paper, we address the challenging problem of 3D toonification, which involves transferring the style of an artistic domain onto a target 3D face with stylized geometry and texture.
Robust and Fast Decoding of High-Capacity Color QR Codes for Mobile Applications
Particularly, we further discover a new type of chromatic distortion in high-density color QR codes, cross-module color interference, caused by the high density which also makes the geometric distortion correction more challenging.
Learning to Steer by Mimicking Features from Heterogeneous Auxiliary Networks
In this paper, we considerably improve the accuracy and robustness of predictions through heterogeneous auxiliary networks feature mimicking, a new and effective training method that provides us with much richer contextual signals apart from steering direction.
Ranked #1 on
Steering Control
on BDD100K val
Monocular 3D Object Reconstruction with GAN Inversion
Reconstruction is achieved by searching for a latent space in the 3D GAN that best resembles the target mesh in accordance with the single view observation.
Unified Vision and Language Prompt Learning
Prompt tuning, a parameter- and data-efficient transfer learning paradigm that tunes only a small number of parameters in a model's input space, has become a trend in the vision community since the emergence of large vision-language models like CLIP.
Panoptic Video Scene Graph Generation
PVSG relates to the existing video scene graph generation (VidSGG) problem, which focuses on temporal interactions between humans and objects grounded with bounding boxes in videos.
Explore In-Context Learning for 3D Point Cloud Understanding
With the rise of large-scale models trained on broad data, in-context learning has become a new learning paradigm that has demonstrated significant potential in natural language processing and computer vision tasks.
Point-In-Context: Understanding Point Cloud via In-Context Learning
With the emergence of large-scale models trained on diverse datasets, in-context learning has emerged as a promising paradigm for multitasking, notably in natural language processing and image processing.
Unsupervised Object-Level Representation Learning from Scene Images
Extensive experiments on COCO show that ORL significantly improves the performance of self-supervised learning on scene images, even surpassing supervised ImageNet pre-training on several downstream tasks.
Masked Frequency Modeling for Self-Supervised Visual Pre-Training
We present Masked Frequency Modeling (MFM), a unified frequency-domain-based approach for self-supervised pre-training of visual models.
Nighttime Smartphone Reflective Flare Removal Using Optical Center Symmetry Prior
With the dataset, neural networks can be trained to remove the reflective flares effectively.
BRACE: The Breakdancing Competition Dataset for Dance Motion Synthesis
Estimating human keypoints from these videos is difficult due to the complexity of the dance, as well as the multiple moving cameras recording setup.
Delving Deep Into Hybrid Annotations for 3D Human Recovery in the Wild
Specifically, we focus on the challenging task of in-the-wild 3D human recovery from single images when paired 3D annotations are not fully available.
NTIRE 2021 Challenge on Quality Enhancement of Compressed Video: Methods and Results
This paper reviews the first NTIRE challenge on quality enhancement of compressed video, with a focus on the proposed methods and results.
Position-Guided Point Cloud Panoptic Segmentation Transformer
DEtection TRansformer (DETR) started a trend that uses a group of learnable queries for unified visual perception.
Ranked #1 on
Panoptic Segmentation
on SemanticKITTI
AnimeRun: 2D Animation Visual Correspondence from Open Source 3D Movies
Existing correspondence datasets for two-dimensional (2D) cartoon suffer from simple frame composition and monotonic movements, making them insufficient to simulate real animations.
When StyleGAN Meets Stable Diffusion: a $\mathscr{W}_+$ Adapter for Personalized Image Generation
Text-to-image diffusion models have remarkably excelled in producing diverse, high-quality, and photo-realistic images.
Everything's Talkin': Pareidolia Face Reenactment
We present a new application direction named Pareidolia Face Reenactment, which is defined as animating a static illusory face to move in tandem with a human face in the video.
Removing Diffraction Image Artifacts in Under-Display Camera via Dynamic Skip Connection Network
Recent development of Under-Display Camera (UDC) systems provides a true bezel-less and notch-free viewing experience on smartphones (and TV, laptops, tablets), while allowing images to be captured from the selfie camera embedded underneath.
Tube-Link: A Flexible Cross Tube Framework for Universal Video Segmentation
Our framework is a near-online approach that takes a short subclip as input and outputs the corresponding spatial-temporal tube masks.
Ranked #3 on
Video Semantic Segmentation
on VSPW
MosaicFusion: Diffusion Models as Data Augmenters for Large Vocabulary Instance Segmentation
We present MosaicFusion, a simple yet effective diffusion-based data augmentation approach for large vocabulary instance segmentation.
Not All Pixels Are Equal: Difficulty-aware Semantic Segmentation via Deep Layer Cascade
Third, in comparison to MC, LC is an end-to-end trainable framework, allowing joint learning of all sub-models.
Ranked #22 on
Semantic Segmentation
on PASCAL VOC 2012 test
Flare7K: A Phenomenological Nighttime Flare Removal Dataset
In this paper, we introduce, Flare7K, the first nighttime flare removal dataset, which is generated based on the observation and statistics of real-world nighttime lens flares.
Ranked #2 on
Flare Removal
on Flare7K
Flare7K++: Mixing Synthetic and Real Datasets for Nighttime Flare Removal and Beyond
To address this issue, we additionally provide the annotations of light sources in Flare7K++ and propose a new end-to-end pipeline to preserve the light source while removing lens flares.
Inter-Region Affinity Distillation for Road Marking Segmentation
We study the problem of distilling knowledge from a large deep teacher network to a much smaller student network for the task of road marking segmentation.
Ranked #1 on
Semantic Segmentation
on ApolloScape
StyleLight: HDR Panorama Generation for Lighting Estimation and Editing
To tackle this problem, we propose a coupled dual-StyleGAN panorama synthesis network (StyleLight) that integrates LDR and HDR panorama synthesis into a unified framework.
PGDiff: Guiding Diffusion Models for Versatile Face Restoration via Partial Guidance
When combined with a diffusion prior, this partial guidance can deliver appealing results across a range of restoration tasks.
Delving into High-Quality Synthetic Face Occlusion Segmentation Datasets
This paper performs comprehensive analysis on datasets for occlusion-aware face segmentation, a task that is crucial for many downstream applications.
CLIPSelf: Vision Transformer Distills Itself for Open-Vocabulary Dense Prediction
However, when transferring the vision-language alignment of CLIP from global image representation to local region representation for the open-vocabulary dense prediction tasks, CLIP ViTs suffer from the domain shift from full images to local image regions.
Self-Supervised Learning via Conditional Motion Propagation
Instead of explicitly modeling the motion probabilities, we design the pretext task as a conditional motion propagation problem.
A Shading-Guided Generative Implicit Model for Shape-Accurate 3D-Aware Image Synthesis
Motivated by the observation that a 3D object should look realistic from multiple viewpoints, these methods introduce a multi-view constraint as regularization to learn valid 3D radiance fields from 2D images.
Dense Intrinsic Appearance Flow for Human Pose Transfer
Unlike existing methods, we propose to estimate dense and intrinsic 3D appearance flow to better guide the transfer of pixels between poses.
Video K-Net: A Simple, Strong, and Unified Baseline for Video Segmentation
We hope this simple, yet effective method can serve as a new, flexible baseline in unified video segmentation design.
Ranked #1 on
Video Panoptic Segmentation
on KITTI-STEP
(using extra training data)
Image Aesthetic Assessment: An Experimental Survey
This survey aims at reviewing recent computer vision techniques used in the assessment of image aesthetic quality.
Face alignment by coarse-to-fine shape searching
We present a novel face alignment framework based on coarse-to-fine shape searching.
Ranked #20 on
Face Alignment
on AFLW-19
Contextual Object Detection with Multimodal Large Language Models
Moreover, we present ContextDET, a unified multimodal model that is capable of end-to-end differentiable modeling of visual-language contexts, so as to locate, identify, and associate visual objects with language inputs for human-AI interaction.
Aligning Bag of Regions for Open-Vocabulary Object Detection
The embeddings of regions in a bag are treated as embeddings of words in a sentence, and they are sent to the text encoder of a VLM to obtain the bag-of-regions embedding, which is learned to be aligned to the corresponding features extracted by a frozen VLM.
Ranked #7 on
Open Vocabulary Object Detection
on MSCOCO
(using extra training data)
PERF: Panoramic Neural Radiance Field from a Single Panorama
In this paper, we present PERF, a 360-degree novel view synthesis framework that trains a panoramic neural radiance field from a single panorama.
Non-Local Recurrent Network for Image Restoration
The main contributions of this work are: (1) Unlike existing methods that measure self-similarity in an isolated manner, the proposed non-local module can be flexibly integrated into existing deep networks for end-to-end training to capture deep feature correlation between each location and its neighborhood.
Ranked #1 on
Grayscale Image Denoising
on Set12 sigma30
Learning Generative Structure Prior for Blind Text Image Super-resolution
To restrict the generative space of StyleGAN so that it obeys the structure of characters yet remains flexible in handling different font styles, we store the discrete features for each character in a codebook.
TransEditor: Transformer-Based Dual-Space GAN for Highly Controllable Facial Editing
In this study, we highlight the importance of interaction in a dual-space GAN for more controllable editing.
Learning Inclusion Matching for Animation Paint Bucket Colorization
In this work, we introduce a new learning-based inclusion matching pipeline, which directs the network to comprehend the inclusion relationships between segments rather than relying solely on direct visual correspondences.
Unsupervised Image-to-Image Translation with Generative Prior
In this work, we present a novel framework, Generative Prior-guided UNsupervised Image-to-image Translation (GP-UNIT), to improve the overall quality and applicability of the translation algorithm.
GP-UNIT: Generative Prior for Versatile Unsupervised Image-to-Image Translation
In this paper, we introduce a novel versatile framework, Generative Prior-guided UNsupervised Image-to-image Translation (GP-UNIT), that improves the quality, applicability and controllability of the existing translation models.
One-shot Face Reenactment
However, in real-world scenario end-users often only have one target face at hand, rendering existing methods inapplicable.
LEDNet: Joint Low-light Enhancement and Deblurring in the Dark
With the pipeline, we present the first large-scale dataset for joint low-light enhancement and deblurring.
Ranked #2 on
Low-Light Image Enhancement
on Sony-Total-Dark
Robust Reference-based Super-Resolution via C2-Matching
However, performing local transfer is difficult because of two gaps between input and reference images: the transformation gap (e. g. scale and rotation) and the resolution gap (e. g. HR and LR).
Reference-based Image and Video Super-Resolution via C2-Matching
To tackle these challenges, we propose C2-Matching in this work, which performs explicit robust matching crossing transformation and resolution.
Open-Vocabulary DETR with Conditional Matching
To this end, we propose a novel open-vocabulary detector based on DETR -- hence the name OV-DETR -- which, once trained, can detect any object given its class name or an exemplar image.
Ranked #21 on
Open Vocabulary Object Detection
on MSCOCO
ReenactGAN: Learning to Reenact Faces via Boundary Transfer
A transformer is subsequently used to adapt the boundary of source face to the boundary of target face.
DNA-Rendering: A Diverse Neural Actor Repository for High-Fidelity Human-centric Rendering
Realistic human-centric rendering plays a key role in both computer vision and computer graphics.
Crafting a Toolchain for Image Restoration by Deep Reinforcement Learning
We investigate a novel approach for image restoration by reinforcement learning.
Accelerating the Super-Resolution Convolutional Neural Network
As a successful deep model applied in image super-resolution (SR), the Super-Resolution Convolutional Neural Network (SRCNN) has demonstrated superior performance to the previous hand-crafted models either in speed and restoration quality.
3D Human Texture Estimation from a Single Image with Transformers
We propose a Transformer-based framework for 3D human texture estimation from a single image.
RenderMe-360: A Large Digital Asset Library and Benchmarks Towards High-fidelity Head Avatars
It is a large-scale digital library for head avatars with three key attributes: 1) High Fidelity: all subjects are captured by 60 synchronized, high-resolution 2K cameras in 360 degrees.
LiteFlowNet3: Resolving Correspondence Ambiguity for More Accurate Optical Flow Estimation
The keys to success lie in the use of cost volume and coarse-to-fine flow inference.
Ranked #4 on
Optical Flow Estimation
on KITTI 2012
Deceive D: Adaptive Pseudo Augmentation for GAN Training with Limited Data
Generative adversarial networks (GANs) typically require ample data for training in order to synthesize high-fidelity images.
Robust Multi-Modality Multi-Object Tracking
Multi-sensor perception is crucial to ensure the reliability and accuracy in autonomous driving system, while multi-object tracking (MOT) improves that by tracing sequential movement of dynamic objects.
Ranked #10 on
Multiple Object Tracking
on KITTI Tracking test
Scenimefy: Learning to Craft Anime Scene via Semi-Supervised Image-to-Image Translation
The challenges of this task lie in the complexity of the scenes, the unique features of anime style, and the lack of high-quality datasets to bridge the domain gap.
On-Device Domain Generalization
Another interesting observation is that the teacher-student gap on out-of-distribution data is bigger than that on in-distribution data, which highlights the capacity mismatch issue as well as the shortcoming of KD.
Exploring CLIP for Assessing the Look and Feel of Images
Measuring the perception of visual content is a long-standing problem in computer vision.
Ranked #9 on
Video Quality Assessment
on MSU SR-QA Dataset
TSIT: A Simple and Versatile Framework for Image-to-Image Translation
We introduce a simple and versatile framework for image-to-image translation.
Audio-Driven Emotional Video Portraits
In this work, we present Emotional Video Portraits (EVP), a system for synthesizing high-quality video portraits with vivid emotional dynamics driven by audios.
Real or Not Real, that is the Question
While generative adversarial networks (GAN) have been widely adopted in various topics, in this paper we generalize the standard GAN to a new perspective by treating realness as a random variable that can be estimated from multiple angles.
Video Object Segmentation with Re-identification
Specifically, our Video Object Segmentation with Re-identification (VS-ReID) model includes a mask propagation module and a ReID module.
Deep Geometrized Cartoon Line Inbetweening
To preserve the precision and detail of the line drawings, we propose a new approach, AnimeInbet, which geometrizes raster line drawings into graphs of endpoints and reframes the inbetweening task as a graph fusion problem with vertex repositioning.
Cross-Scale Internal Graph Neural Network for Image Super-Resolution
Specifically, we dynamically construct a cross-scale graph by searching k-nearest neighboring patches in the downsampled LR image for each query patch in the LR image.
Talk-to-Edit: Fine-Grained Facial Editing via Dialog
In this work, we propose Talk-to-Edit, an interactive facial editing framework that performs fine-grained attribute manipulation through dialog between the user and the system.
Ranked #1 on
Fine-Grained Facial Editing
on CelebA-Dialog
Text2Performer: Text-Driven Human Video Generation
In this work, we present Text2Performer to generate vivid human videos with articulated motions from texts.
Learning to Enhance Low-Light Image via Zero-Reference Deep Curve Estimation
This paper presents a novel method, Zero-Reference Deep Curve Estimation (Zero-DCE), which formulates light enhancement as a task of image-specific curve estimation with a deep network.
CelebV-HQ: A Large-Scale Video Facial Attributes Dataset
Large-scale datasets have played indispensable roles in the recent success of face generation/editing and significantly facilitated the advances of emerging research fields.
Ranked #1 on
Unconditional Video Generation
on CelebV-HQ
Extract Free Dense Labels from CLIP
Contrastive Language-Image Pre-training (CLIP) has made a remarkable breakthrough in open-vocabulary zero-shot image recognition.
