Search Results for author:
Found 109 papers, 78 papers with code
HuMMan: Multi-Modal 4D Human Dataset for Versatile Sensing and Modeling
4D human sensing and modeling are fundamental tasks in vision and graphics with numerous applications.
Robust Face Anti-Spoofing with Dual Probabilistic Modeling
In this work, we attempt to fill this gap by automatically addressing the noise problem from both label and data perspectives in a probabilistic manner.
StyleGAN-Human: A Data-Centric Odyssey of Human Generation
In addition, a model zoo and human editing applications are demonstrated to facilitate future research in the community.
Few-shot Forgery Detection via Guided Adversarial Interpolation
Realistic visual media synthesis is becoming a critical societal issue with the surge of face manipulation models; new forgery approaches emerge at an unprecedented pace.
Full-Spectrum Out-of-Distribution Detection
In this paper, we take into account both shift types and introduce full-spectrum OOD (FS-OOD) detection, a more realistic problem setting that considers both detecting semantic shift and being tolerant to covariate shift; and designs three benchmarks.
Unsupervised Image-to-Image Translation with Generative Prior
In this work, we present a novel framework, Generative Prior-guided UNsupervised Image-to-image Translation (GP-UNIT), to improve the overall quality and applicability of the translation algorithm.
Balanced MSE for Imbalanced Visual Regression
Data imbalance exists ubiquitously in real-world visual regressions, e. g., age estimation and pose estimation, hurting the model's generalizability and fairness.
SeCo: Separating Unknown Musical Visual Sounds with Consistency Guidance
Recent years have witnessed the success of deep learning on the visual sound separation task.
Versatile Multi-Modal Pre-Training for Human-Centric Perception
To tackle the challenges, we design the novel Dense Intra-sample Contrastive Learning and Sparse Structure-aware Contrastive Learning targets by hierarchically learning a modal-invariant latent space featured with continuous and ordinal feature distribution and structure-aware semantic consistency.
Bailando: 3D Dance Generation by Actor-Critic GPT with Choreographic Memory
With the learned choreographic memory, dance generation is realized on the quantized units that meet high choreography standards, such that the generated dancing sequences are confined within the spatial constraints.
Pastiche Master: Exemplar-Based High-Resolution Portrait Style Transfer
Recent studies on StyleGAN show high performance on artistic portrait generation by transfer learning with limited data.
X-Learner: Learning Cross Sources and Tasks for Universal Visual Representation
2) Squeeze Stage: X-Learner condenses the model to a reasonable size and learns the universal and generalizable representation for various tasks transferring.
Bamboo: Building Mega-Scale Vision Dataset Continually with Human-Machine Synergy
Specifically, we contribute Bamboo Dataset, a mega-scale and information-dense dataset for both classification and detection.
LiDAR-based 4D Panoptic Segmentation via Dynamic Shifting Network
In this work, we address the task of LiDAR-based panoptic segmentation, which aims to parse both objects and scenes in a unified manner.
BiBERT: Accurate Fully Binarized BERT
The large pre-trained BERT has achieved remarkable performance on Natural Language Processing (NLP) tasks but is also computation and memory expensive.
Conditional Prompt Learning for Vision-Language Models
With the rise of powerful pre-trained vision-language models like CLIP, it becomes essential to investigate ways to adapt these models to downstream datasets.
TCTrack: Temporal Contexts for Aerial Tracking
Temporal contexts among consecutive frames are far from being fully utilized in existing visual trackers.
Visual Sound Localization in the Wild by Cross-Modal Interference Erasing
Specifically, we observe that the previous practice of learning only a single audio representation is insufficient due to the additive nature of audio signals.
Benchmarking and Analyzing Point Cloud Classification under Corruptions
3D perception, especially point cloud classification, has achieved substantial progress.
Multi-View Partial (MVP) Point Cloud Challenge 2021 on Completion and Registration: Methods and Results
Based on the MVP dataset, this paper reports methods and results in the Multi-View Partial Point Cloud Challenge 2021 on Completion and Registration.
ForgeryNet -- Face Forgery Analysis Challenge 2021: Methods and Results
The rapid progress of photorealistic synthesis techniques has reached a critical point where the boundary between real and manipulated images starts to blur.
Garment4D: Garment Reconstruction from Point Cloud Sequences
The main challenges are two-fold: 1) effective 3D feature learning for fine details, and 2) capture of garment dynamics caused by the interaction between garments and the human body, especially for loose garments like skirts.
Balanced Chamfer Distance as a Comprehensive Metric for Point Cloud Completion
We adopt DCD to evaluate the point cloud completion task, where experimental results show that DCD pays attention to both the overall structure and local geometric details and provides a more reliable evaluation even when CD and EMD contradict each other.
Robust Partial-to-Partial Point Cloud Registration in a Full Range
\textbf{3)} Based on a synergy of hierarchical graph networks and graphical modeling, we propose the {H}ierarchical {G}raphical {M}odeling (\textbf{HGM}) architecture to encode robust descriptors consisting of i) a unary term learned from {\textit{RI}} features; and ii) multiple smoothness terms encoded from neighboring point relations at different scales through our TPT modules.
Density-aware Chamfer Distance as a Comprehensive Metric for Point Cloud Completion
We adopt DCD to evaluate the point cloud completion task, where experimental results show that DCD pays attention to both the overall structure and local geometric details and provides a more reliable evaluation even when CD and EMD contradict each other.
Few-Shot Object Detection via Association and DIscrimination
1) In the association step, in contrast to implicitly leveraging multiple base classes, we construct a compact novel class feature space via explicitly imitating a specific base class feature space.
Lifting 2D Human Pose to 3D with Domain Adapted 3D Body Concept
Inspired by this, we propose a new framework that leverages the labeled 3D human poses to learn a 3D concept of the human body to reduce the ambiguity.
Monocular 3D Reconstruction of Interacting Hands via Collision-Aware Factorized Refinements
In this paper, we make the first attempt to reconstruct 3D interacting hands from monocular single RGB images.
Generalized Out-of-Distribution Detection: A Survey
In this survey, we first present a generic framework called generalized OOD detection, which encompasses the five aforementioned problems, i. e., AD, ND, OSR, OOD detection, and OD.
Playing for 3D Human Recovery
Image- and video-based 3D human recovery (i. e. pose and shape estimation) have achieved substantial progress.
TAda! Temporally-Adaptive Convolutions for Video Understanding
This work presents Temporally-Adaptive Convolutions (TAdaConv) for video understanding, which shows that adaptive weight calibration along the temporal dimension is an efficient way to facilitate modelling complex temporal dynamics in videos.
Ranked #31 on
Action Recognition
on Something-Something V2
(using extra training data)
A Comprehensive Overhaul of Distilling Unconditional GANs
To further enhance the semantic consistency between the teacher and student model, we present another latent-direction-based distillation loss that preserves the semantic relations in latent space.
Bayesian Imbalanced Regression Debiasing
Compared to imbalanced and long-tailed classification, imbalanced regression has its unique challenges as the regression label space can be continuous, boundless, and high-dimensional.
Talk-to-Edit: Fine-Grained Facial Editing via Dialog
In this work, we propose Talk-to-Edit, an interactive facial editing framework that performs fine-grained attribute manipulation through dialog between the user and the system.
Ranked #1 on
Fine-Grained Facial Editing
on CelebA-Dialog
Learning to Prompt for Vision-Language Models
Large pre-trained vision-language models like CLIP have shown great potential in learning representations that are transferable across a wide range of downstream tasks.
Semantically Coherent Out-of-Distribution Detection
The proposed UDG can not only enrich the semantic knowledge of the model by exploiting unlabeled data in an unsupervised manner, but also distinguish ID/OOD samples to enhance ID classification and OOD detection tasks simultaneously.
Energy-Based Open-World Uncertainty Modeling for Confidence Calibration
Confidence calibration is of great importance to the reliability of decisions made by machine learning systems.
Unsupervised Domain Adaptive 3D Detection with Multi-Level Consistency
In addition, existing 3D domain adaptive detection methods often assume prior access to the target domain annotations, which is rarely feasible in the real world.
Unsupervised Object-Level Representation Learning from Scene Images
Extensive experiments on COCO show that ORL significantly improves the performance of self-supervised learning on scene images, even surpassing supervised ImageNet pre-training on several downstream tasks.
Delving Deep into the Generalization of Vision Transformers under Distribution Shifts
By comprehensively investigating these GE-ViTs and comparing with their corresponding CNN models, we observe: 1) For the enhanced model, larger ViTs still benefit more for the OOD generalization.
Robust Reference-based Super-Resolution via C2-Matching
However, performing local transfer is difficult because of two gaps between input and reference images: the transformation gap (e. g. scale and rotation) and the resolution gap (e. g. HR and LR).
Semi-Supervised Domain Generalization with Stochastic StyleMatch
We find that the DG methods, which by design are unable to handle unlabeled data, perform poorly with limited labels in SSDG; the SSL methods, especially FixMatch, obtain much better results but are still far away from the basic vanilla model trained using full labels.
Iterative Human and Automated Identification of Wildlife Images
Camera trapping is increasingly used to monitor wildlife, but this technology typically requires extensive data annotation.
Pose-Controllable Talking Face Generation by Implicitly Modularized Audio-Visual Representation
While speech content information can be defined by learning the intrinsic synchronization between audio-visual modalities, we identify that a pose code will be complementarily learned in a modulated convolution-based reconstruction framework.
Variational Relational Point Completion Network
In particular, we propose a dual-path architecture to enable principled probabilistic modeling across partial and complete clouds.
Visually Informed Binaural Audio Generation without Binaural Audios
Moreover, combined with binaural recordings, our method is able to further boost the performance of binaural audio generation under supervised settings.
Deep Animation Video Interpolation in the Wild
In the animation industry, cartoon videos are usually produced at low frame rate since hand drawing of such frames is costly and time-consuming.
Adversarial Robustness under Long-Tailed Distribution
We then perform a systematic study on existing long-tailed recognition methods in conjunction with the adversarial training framework.
Incorporating Convolution Designs into Visual Transformers
Motivated by the success of Transformers in natural language processing (NLP) tasks, there emerge some attempts (e. g., ViT and DeiT) to apply Transformers to the vision domain.
Ranked #1 on
Image Classification
on Oxford-IIIT Pets
(using extra training data)
ForgeryNet: A Versatile Benchmark for Comprehensive Forgery Analysis
To counter this emerging threat, we construct the ForgeryNet dataset, an extremely large face forgery dataset with unified annotations in image- and video-level data across four tasks: 1) Image Forgery Classification, including two-way (real / fake), three-way (real / fake with identity-replaced forgery approaches / fake with identity-remained forgery approaches), and n-way (real and 15 respective forgery approaches) classification.
Domain Generalization: A Survey
Generalization to out-of-distribution (OOD) data is a capability natural to humans yet challenging for machines to reproduce.
CelebA-Spoof Challenge 2020 on Face Anti-Spoofing: Methods and Results
It is the largest face anti-spoofing dataset in terms of the numbers of the data and the subjects.
DeeperForensics Challenge 2020 on Real-World Face Forgery Detection: Methods and Results
This paper reports methods and results in the DeeperForensics Challenge 2020 on real-world face forgery detection.
BlockPlanner: City Block Generation With Vectorized Graph Representation
City modeling is the foundation for computational urban planning, navigation, and entertainment.
Differentiable Dynamic Wirings for Neural Networks
A standard practice of deploying deep neural networks is to apply the same architecture to all the input instances.
Chasing the Tail in Monocular 3D Human Reconstruction with Prototype Memory
The reason is that most of the current models perform regression based on a single human prototype, which is similar to common poses while far from the rare poses.
ShineOn: Illuminating Design Choices for Practical Video-based Virtual Clothing Try-on
We build a series of scientific experiments to isolate effective design choices in video synthesis for virtual clothing try-on.
Computation-Efficient Knowledge Distillation via Uncertainty-Aware Mixup
Our goal is to achieve a performance comparable to conventional knowledge distillation with a lower computation cost during training.
CARAFE++: Unified Content-Aware ReAssembly of FEatures
Feature reassembly, i. e. feature downsampling and upsampling, is a key operation in a number of modern convolutional network architectures, e. g., residual networks and feature pyramids.
LiDAR-based Panoptic Segmentation via Dynamic Shifting Network
2) Dynamic Shifting for complex point distributions.
Ranked #1 on
Panoptic Segmentation
on SemanticKITTI
Do 2D GANs Know 3D Shape? Unsupervised 3D shape reconstruction from 2D Image GANs
Through our investigation, we found that such a pre-trained GAN indeed contains rich 3D knowledge and thus can be used to recover 3D shape from a single 2D image in an unsupervised manner.
Long-tailed Recognition by Routing Diverse Distribution-Aware Experts
We take a dynamic view of the training data and provide a principled model bias and variance analysis as the training data fluctuates: Existing long-tail classifiers invariably increase the model variance and the head-tail model bias gap remains large, due to more and larger confusion with hard negatives for the tail.
Ranked #4 on
Long-tail Learning
on CIFAR-100-LT (ρ=100)
Person-in-Context Synthesiswith Compositional Structural Space
To handle the stark difference in input structures, we proposed two separate neural branches to attentively composite the respective (context/person) inputs into shared ``compositional structural space'', which encodes shape, location and appearance information for both context and person structures in a disentangled manner.
Delving into Inter-Image Invariance for Unsupervised Visual Representations
In this work, we present a rigorous and comprehensive study on inter-image invariance learning from three main constituting components: pseudo-label maintenance, sampling strategy, and decision boundary design.
Seesaw Loss for Long-Tailed Instance Segmentation
Instances of head classes dominate a long-tailed dataset and they serve as negative samples of tail categories.
Unsupervised Feature Learning by Cross-Level Instance-Group Discrimination
Unsupervised feature learning has made great strides with contrastive learning based on instance discrimination and invariant mapping, as benchmarked on curated class-balanced datasets.
Contrastive Learning
Semi-Supervised Image Classification
+2
CelebA-Spoof: Large-Scale Face Anti-Spoofing Dataset with Rich Annotations
The main reason is that current face anti-spoofing datasets are limited in both quantity and diversity.
Sep-Stereo: Visually Guided Stereophonic Audio Generation by Associating Source Separation
Stereophonic audio is an indispensable ingredient to enhance human auditory experience.
Distribution-Balanced Loss for Multi-Label Classification in Long-Tailed Datasets
We present a new loss function called Distribution-Balanced Loss for the multi-label recognition problems that exhibit long-tailed class distributions.
Unsupervised 3D Human Pose Representation with Viewpoint and Pose Disentanglement
Learning a good 3D human pose representation is important for human pose related tasks, e. g. human 3D pose estimation and action recognition.
Placepedia: Comprehensive Place Understanding with Multi-Faceted Annotations
Place is an important element in visual understanding.
Unsupervised Landmark Learning from Unpaired Data
Recent attempts for unsupervised landmark learning leverage synthesized image pairs that are similar in appearance but different in poses.
Online Deep Clustering for Unsupervised Representation Learning
In this way, labels and the network evolve shoulder-to-shoulder rather than alternatingly.
Knowledge Distillation Meets Self-Supervision
Knowledge distillation, which involves extracting the "dark knowledge" from a teacher network to guide the learning of a student network, has emerged as an important technique for model compression and transfer learning.
Ranked #11 on
Knowledge Distillation
on ImageNet
MMFashion: An Open-Source Toolbox for Visual Fashion Analysis
This toolbox supports a wide spectrum of fashion analysis tasks, including Fashion Attribute Prediction, Fashion Recognition and Retrieval, Fashion Landmark Detection, Fashion Parsing and Segmentation and Fashion Compatibility and Recommendation.
Self-Supervised Scene De-occlusion
This is achieved via Partial Completion Network (PCNet)-mask (M) and -content (C), that learn to recover fractions of object masks and contents, respectively, in a self-supervised manner.
Rotate-and-Render: Unsupervised Photorealistic Face Rotation from Single-View Images
Though face rotation has achieved rapid progress in recent years, the lack of high-quality paired training data remains a great hurdle for existing methods.
Learning Diverse Fashion Collocation by Neural Graph Filtering
To facilitate a comprehensive study on diverse fashion collocation, we reorganize Amazon Fashion dataset with carefully designed evaluation protocols.
When NAS Meets Robustness: In Search of Robust Architectures against Adversarial Attacks
Recent advances in adversarial attacks uncover the intrinsic vulnerability of modern deep neural networks.
Learning to Synthesize Fashion Textures
Another important observation is that fashion textures are multi-modal.
Vision-Infused Deep Audio Inpainting
Extensive experiments demonstrate that our framework is capable of inpainting realistic and varying audio segments with or without visual contexts.
Open Compound Domain Adaptation
A typical domain adaptation approach is to adapt models trained on the annotated data in a source domain (e. g., sunny weather) for achieving high performance on the test data in a target domain (e. g., rainy weather).
Delving Deep Into Hybrid Annotations for 3D Human Recovery in the Wild
Specifically, we focus on the challenging task of in-the-wild 3D human recovery from single images when paired 3D annotations are not fully available.
One-shot Face Reenactment
However, in real-world scenario end-users often only have one target face at hand, rendering existing methods inapplicable.
MaskGAN: Towards Diverse and Interactive Facial Image Manipulation
To overcome these drawbacks, we propose a novel framework termed MaskGAN, enabling diverse and interactive face manipulation.
MMDetection: Open MMLab Detection Toolbox and Benchmark
In this paper, we introduce the various features of this toolbox.
CARAFE: Content-Aware ReAssembly of FEatures
CARAFE introduces little computational overhead and can be readily integrated into modern network architectures.
Large-Scale Long-Tailed Recognition in an Open World
We define Open Long-Tailed Recognition (OLTR) as learning from such naturally distributed data and optimizing the classification accuracy over a balanced test set which include head, tail, and open classes.
Self-Supervised Learning via Conditional Motion Propagation
Instead of explicitly modeling the motion probabilities, we design the pretext task as a conditional motion propagation problem.
Hybrid Task Cascade for Instance Segmentation
In exploring a more effective approach, we find that the key to a successful instance segmentation cascade is to fully leverage the reciprocal relationship between detection and segmentation.
Ranked #27 on
Instance Segmentation
on COCO test-dev
Instance-level Facial Attributes Transfer with Geometry-Aware Flow
Geometry-aware flow is able to warp the source face attribute into the target face context and generate a warp-and-blend result.
PointGrow: Autoregressively Learned Point Cloud Generation with Self-Attention
Generating 3D point clouds is challenging yet highly desired.
Consensus-Driven Propagation in Massive Unlabeled Data for Face Recognition
Face recognition has witnessed great progress in recent years, mainly attributed to the high-capacity model designed and the abundant labeled data collected.
Talking Face Generation by Adversarially Disentangled Audio-Visual Representation
Talking face generation aims to synthesize a sequence of face images that correspond to a clip of speech.
DPatch: An Adversarial Patch Attack on Object Detectors
Successful realization of DPatch also illustrates the intrinsic vulnerability of the modern detector architectures to such patch-based adversarial attacks.
Im2Avatar: Colorful 3D Reconstruction from a Single Image
In this work, we study a new problem, that is, simultaneously recovering 3D shape and surface color from a single image, namely "colorful 3D reconstruction".
Adaptive Affinity Fields for Semantic Segmentation
Semantic segmentation has made much progress with increasingly powerful pixel-wise classifiers and incorporating structural priors via Conditional Random Fields (CRF) or Generative Adversarial Networks (GAN).
Ranked #47 on
Semantic Segmentation
on Cityscapes test
Dynamic Graph CNN for Learning on Point Clouds
Point clouds provide a flexible geometric representation suitable for countless applications in computer graphics; they also comprise the raw output of most 3D data acquisition devices.
Ranked #8 on
3D Point Cloud Classification
on ModelNet40-C
Mix-and-Match Tuning for Self-Supervised Semantic Segmentation
The key of this new form of learning is to design a proxy task (e. g. image colorization), from which a discriminative loss can be formulated on unlabeled data.
Unconstrained Fashion Landmark Detection via Hierarchical Recurrent Transformer Networks
This work addresses unconstrained fashion landmark detection, where clothing bounding boxes are not provided in both training and test.
Video Object Segmentation with Re-identification
Specifically, our Video Object Segmentation with Re-identification (VS-ReID) model includes a mask propagation module and a ReID module.
Not All Pixels Are Equal: Difficulty-aware Semantic Segmentation via Deep Layer Cascade
Third, in comparison to MC, LC is an end-to-end trainable framework, allowing joint learning of all sub-models.
Ranked #30 on
Semantic Segmentation
on PASCAL VOC 2012 test
Video Frame Synthesis using Deep Voxel Flow
We combine the advantages of these two methods by training a deep network that learns to synthesize video frames by flowing pixel values from existing ones, which we call deep voxel flow.
Semantic Facial Expression Editing using Autoencoded Flow
High-level manipulation of facial expressions in images --- such as changing a smile to a neutral expression --- is challenging because facial expression changes are highly non-linear, and vary depending on the appearance of the face.
Fashion Landmark Detection in the Wild
Fashion landmark is also compared to clothing bounding boxes and human joints in two applications, fashion attribute prediction and clothes retrieval, showing that fashion landmark is a more discriminative representation to understand fashion images.
Deep Learning Markov Random Field for Semantic Segmentation
Semantic segmentation tasks can be well modeled by Markov Random Field (MRF).
DeepFashion: Powering Robust Clothes Recognition and Retrieval With Rich Annotations
To demonstrate the advantages of DeepFashion, we propose a new deep model, namely FashionNet, which learns clothing features by jointly predicting clothing attributes and landmarks.
Semantic Image Segmentation via Deep Parsing Network
This paper addresses semantic image segmentation by incorporating rich information into Markov Random Field (MRF), including high-order relations and mixture of label contexts.
Ranked #81 on
Semantic Segmentation
on Cityscapes test
Deep Learning Face Attributes in the Wild
LNet is pre-trained by massive general object categories for face localization, while ANet is pre-trained by massive face identities for attribute prediction.
