Search Results for author:
Found 159 papers, 64 papers with code
UniPose: A Unified Multimodal Framework for Human Pose Comprehension, Generation and Editing
Human pose plays a crucial role in the digital age.
Semantic or Covariate? A Study on the Intractable Case of Out-of-Distribution Detection
The primary goal of out-of-distribution (OOD) detection tasks is to identify inputs with semantic shifts, i. e., if samples from novel classes are absent in the in-distribution (ID) dataset used for training, we should reject these OOD samples rather than misclassifying them into existing ID classes.
Out-of-Distribution Detection
Out of Distribution (OOD) Detection
UMFC: Unsupervised Multi-Domain Feature Calibration for Vision-Language Models
Under this unsupervised multi-domain setting, we have identified inherent model bias within CLIP, notably in its visual and text encoders.
Smile upon the Face but Sadness in the Eyes: Emotion Recognition based on Facial Expressions and Eye Behaviors
To understand and bridge this gap between FER and ER, we introduce eye behaviors as an important emotional cues for the creation of a new Eye-behavior-aided Multimodal Emotion Recognition (EMER) dataset.
Facial Expression Recognition
Facial Expression Recognition (FER)
+1
Confidence Aware Learning for Reliable Face Anti-spoofing
The prediction confidence for each sample is subsequently assessed using the Mahalanobis distance between the sample and the Gaussians for the "known data".
Face-MLLM: A Large Face Perception Model
To tackle this problem, we design a practical pipeline for constructing datasets, upon which we further build a novel multimodal large face perception model, namely Face-MLLM.
CtrLoRA: An Extensible and Efficient Framework for Controllable Image Generation
However, for every single condition type, ControlNet requires independent training on millions of data pairs with hundreds of GPU hours, which is quite expensive and makes it challenging for ordinary users to explore and develop new types of conditions.
HERM: Benchmarking and Enhancing Multimodal LLMs for Human-Centric Understanding
The significant advancements in visual understanding and instruction following from Multimodal Large Language Models (MLLMs) have opened up more possibilities for broader applications in diverse and universal human-centric scenarios.
Face Forgery Detection with Elaborate Backbone
Specifically, we analyze the crucial contributions of backbones with different configurations in FFD task and propose leveraging the ViT network with self-supervised learning on real-face datasets to pre-train a backbone, equipping it with superior facial representation capabilities.
UniLearn: Enhancing Dynamic Facial Expression Recognition through Unified Pre-Training and Fine-Tuning on Images and Videos
To overcome this, we introduce UniLearn, a novel unified learning paradigm that integrates static facial expression recognition (SFER) data to enhance DFER task.
Ranked #2 on
Dynamic Facial Expression Recognition
on DFEW
Dynamic Facial Expression Recognition
Facial Expression Recognition
Segment Anything for Videos: A Systematic Survey
To address this gap, this work conducts a systematic review on SAM for videos in the era of foundation models.
T2IShield: Defending Against Backdoors on Text-to-Image Diffusion Models
In this paper, for the first time, we propose a comprehensive defense method named T2IShield to detect, localize, and mitigate such attacks.
Dysca: A Dynamic and Scalable Benchmark for Evaluating Perception Ability of LVLMs
Besides, these benchmarks merely focus on evaluating LVLMs on the realistic style images and clean scenarios, leaving the multi-stylized images and noisy scenarios unexplored.
Evaluating the Quality of Hallucination Benchmarks for Large Vision-Language Models
Furthermore, based on the results of our quality measurement, we construct a High-Quality Hallucination Benchmark (HQH) for LVLMs, which demonstrates superior reliability and validity under our HQM framework.
VLBiasBench: A Comprehensive Benchmark for Evaluating Bias in Large Vision-Language Model
The emergence of Large Vision-Language Models (LVLMs) marks significant strides towards achieving general artificial intelligence.
Rethinking the Evaluation of Out-of-Distribution Detection: A Sorites Paradox
In this paper, we construct a benchmark named Incremental Shift OOD (IS-OOD) to address the issue, in which we divide the test samples into subsets with different semantic and covariate shift degrees relative to the ID dataset.
Out-of-Distribution Detection
Out of Distribution (OOD) Detection
Generalized Semi-Supervised Learning via Self-Supervised Feature Adaptation
In this paper, we propose a novel SSL setting, where unlabeled samples are drawn from a mixed distribution that deviates from the feature distribution of labeled samples.
Anonymization Prompt Learning for Facial Privacy-Preserving Text-to-Image Generation
Specifically, we train a learnable prompt prefix for text-to-image diffusion models, which forces the model to generate anonymized facial identities, even when prompted to produce images of specific individuals.
M$^3$GPT: An Advanced Multimodal, Multitask Framework for Motion Comprehension and Generation
Third, M$^3$GPT learns to model the connections and synergies among various motion-relevant tasks.
BIMM: Brain Inspired Masked Modeling for Video Representation Learning
The visual pathway of human brain includes two sub-pathways, ie, the ventral pathway and the dorsal pathway, which focus on object identification and dynamic information modeling, respectively.
Task-adaptive Q-Face
Although face analysis has achieved remarkable improvements in the past few years, designing a multi-task face analysis model is still challenging.
Image to Pseudo-Episode: Boosting Few-Shot Segmentation by Unlabeled Data
Few-shot segmentation (FSS) aims to train a model which can segment the object from novel classes with a few labeled samples.
Clothes-Changing Person Re-Identification with Feasibility-Aware Intermediary Matching
Current clothes-changing person re-identification (re-id) approaches usually perform retrieval based on clothes-irrelevant features, while neglecting the potential of clothes-relevant features.
HPNet: Dynamic Trajectory Forecasting with Historical Prediction Attention
The proposed Historical Prediction Attention together with the Agent Attention and Mode Attention is further formulated as the Triple Factorized Attention module, serving as the core design of HPNet. Experiments on the Argoverse and INTERACTION datasets show that HPNet achieves state-of-the-art performance, and generates accurate and stable future trajectories.
StylizedGS: Controllable Stylization for 3D Gaussian Splatting
As XR technology continues to advance rapidly, 3D generation and editing are increasingly crucial.
GPT as Psychologist? Preliminary Evaluations for GPT-4V on Visual Affective Computing
In conclusion, this paper provides valuable insights into the potential applications and challenges of MLLMs in human-centric computing.
Contrastive Learning of Person-independent Representations for Facial Action Unit Detection
We formulate the self-supervised AU representation learning signals in two-fold: (1) AU representation should be frame-wisely discriminative within a short video clip; (2) Facial frames sampled from different identities but show analogous facial AUs should have consistent AU representations.
Generalized Face Liveness Detection via De-spoofing Face Generator
In this paper, we conduct an Anomalous cue Guided FAS (AG-FAS) method, which leverages real faces for improving model generalization via a De-spoofing Face Generator (DFG).
Collaboratively Self-supervised Video Representation Learning for Action Recognition
Considering the close connection between action recognition and human pose estimation, we design a Collaboratively Self-supervised Video Representation (CSVR) learning framework specific to action recognition by jointly considering generative pose prediction and discriminative context matching as pretext tasks.
Pre-trained Model Guided Fine-Tuning for Zero-Shot Adversarial Robustness
Specifically, PMG-AFT minimizes the distance between the features of adversarial examples in the target model and those in the pre-trained model, aiming to preserve the generalization features already captured by the pre-trained model.
Towards Robust Semantic Segmentation against Patch-based Attack via Attention Refinement
The attention mechanism has been proven effective on various visual tasks in recent years.
FullLoRA-AT: Efficiently Boosting the Robustness of Pretrained Vision Transformers
In recent years, the Vision Transformer (ViT) model has gradually become mainstream in various computer vision tasks, and the robustness of the model has received increasing attention.
Video Harmonization with Triplet Spatio-Temporal Variation Patterns
Video harmonization is an important and challenging task that aims to obtain visually realistic composite videos by automatically adjusting the foreground's appearance to harmonize with the background.
ES3: Evolving Self-Supervised Learning of Robust Audio-Visual Speech Representations
While many recent approaches for this task primarily rely on guiding the learning process using the audio modality alone to capture information shared between audio and video we reframe the problem as the acquisition of shared unique (modality-specific) and synergistic speech information to address the inherent asymmetry between the modalities.
Ranked #1 on
Audio-Visual Speech Recognition
on CAS-VSR-S101
Tokenize Anything via Prompting
The semantic token is responsible for learning the semantic priors in a predefined concept space.
From Static to Dynamic: Adapting Landmark-Aware Image Models for Facial Expression Recognition in Videos
And the TMAs capture and model the relationships of dynamic changes in facial expressions, effectively extending the pre-trained image model for videos.
Ranked #2 on
Dynamic Facial Expression Recognition
on FERV39k
Dynamic Facial Expression Recognition
Facial Expression Recognition
+1
Cooperative Dual Attention for Audio-Visual Speech Enhancement with Facial Cues
By integrating cooperative dual attention in the visual encoder and audio-visual fusion strategy, our model effectively extracts beneficial speech information from both audio and visual cues for AVSE.
Learning Separable Hidden Unit Contributions for Speaker-Adaptive Lip-Reading
For deep layers where both the speaker's features and the speech content features are all expressed well, we introduce the speaker-adaptive features to learn for suppressing the speech content irrelevant noise for robust lip reading.
Dual Compensation Residual Networks for Class Imbalanced Learning
Learning generalizable representation and classifier for class-imbalanced data is challenging for data-driven deep models.
Patch Is Not All You Need
Vision Transformers have achieved great success in computer visions, delivering exceptional performance across various tasks.
Triplet Knowledge Distillation
Distillation then begins in an online manner, and the teacher is only allowed to express solutions within the aforementioned subspace.
Function-Consistent Feature Distillation
The core idea of FCFD is to make teacher and student features not only numerically similar, but more importantly produce similar outputs when fed to the later part of the same network.
CCLAP: Controllable Chinese Landscape Painting Generation via Latent Diffusion Model
While the style aggregator module is to generate paintings of a style corresponding to a reference image.
Real Face Foundation Representation Learning for Generalized Deepfake Detection
In this study, we propose Real Face Foundation Representation Learning (RFFR), which aims to learn a general representation from large-scale real face datasets and detect potential artifacts outside the distribution of RFFR.
Diversity-Measurable Anomaly Detection
In this paper, to better handle the tradeoff problem, we propose Diversity-Measurable Anomaly Detection (DMAD) framework to enhance reconstruction diversity while avoid the undesired generalization on anomalies.
Ranked #1 on
Anomaly Detection
on UCSD Ped2
Anomaly Detection In Surveillance Videos
Defect Detection
+2
Source-Free Adaptive Gaze Estimation by Uncertainty Reduction
In light of this, we present an unsupervised source-free domain adaptation approach for gaze estimation, which adapts a source-trained gaze estimator to unlabeled target domains without source data.
Holistic Label Correction for Noisy Multi-Label Classification
The issues are, that we do not understand why label dependence is helpful in the problem, and how to learn and utilize label dependence only using training data with noisy multiple labels.
DISC: Learning From Noisy Labels via Dynamic Instance-Specific Selection and Correction
Then we propose a dynamic threshold strategy for each instance, based on the momentum of each instance's memorization strength in previous epochs to select and correct noisy labeled data.
DandelionNet: Domain Composition with Instance Adaptive Classification for Domain Generalization
Domain generalization (DG) attempts to learn a model on source domains that can well generalize to unseen but different domains.
Hierarchical Compositional Representations for Few-shot Action Recognition
Recently action recognition has received more and more attention for its comprehensive and practical applications in intelligent surveillance and human-computer interaction.
Learning Pseudo Labels for Semi-and-Weakly Supervised Semantic Segmentation
Firstly, we introduce a class-aware cross entropy (CCE) loss for network training.
MAFW: A Large-scale, Multi-modal, Compound Affective Database for Dynamic Facial Expression Recognition in the Wild
To the best of our knowledge, MAFW is the first in-the-wild multi-modal database annotated with compound emotion annotations and emotion-related captions.
Ranked #12 on
Dynamic Facial Expression Recognition
on MAFW
Dynamic Facial Expression Recognition
Facial Expression Recognition
+1
UniCon+: ICTCAS-UCAS Submission to the AVA-ActiveSpeaker Task at ActivityNet Challenge 2022
This report presents a brief description of our winning solution to the AVA Active Speaker Detection (ASD) task at ActivityNet Challenge 2022.
Active Speaker Detection
Audio-Visual Active Speaker Detection
Clothes-Changing Person Re-identification with RGB Modality Only
In this paper, we propose a Clothes-based Adversarial Loss (CAL) to mine clothes-irrelevant features from the original RGB images by penalizing the predictive power of re-id model w. r. t.
Ranked #1 on
Multiview Gait Recognition
on CASIA-B
Clothes Changing Person Re-Identification
Multiview Gait Recognition
Joint Feature Learning and Relation Modeling for Tracking: A One-Stream Framework
The current popular two-stream, two-stage tracking framework extracts the template and the search region features separately and then performs relation modeling, thus the extracted features lack the awareness of the target and have limited target-background discriminability.
Ranked #4 on
Visual Tracking
on TNL2K
Enhancing Face Recognition With Self-Supervised 3D Reconstruction
In this paper, we propose to enhance face recognition with a bypass of self-supervised 3D reconstruction, which enforces the neural backbone to focus on the identity-related depth and albedo information while neglects the identity-irrelevant pose and illumination information.
Adaptive Perturbation for Adversarial Attack
Instead of using the sign function, we propose to directly utilize the exact gradient direction with a scaling factor for generating adversarial perturbations, which improves the attack success rates of adversarial examples even with fewer perturbations.
Adaptive Image Transformations for Transfer-based Adversarial Attack
Adversarial attacks provide a good way to study the robustness of deep learning models.
Learning Fair Face Representation With Progressive Cross Transformer
To mitigate racial bias and meantime preserve robust FR, we abstract face identity-related representation as a signal denoising problem and propose a progressive cross transformer (PCT) method for fair face recognition.
Meta Gradient Adversarial Attack
In recent years, research on adversarial attacks has become a hot spot.
UniCon: Unified Context Network for Robust Active Speaker Detection
Our solution is a novel, unified framework that focuses on jointly modeling multiple types of contextual information: spatial context to indicate the position and scale of each candidate's face, relational context to capture the visual relationships among the candidates and contrast audio-visual affinities with each other, and temporal context to aggregate long-term information and smooth out local uncertainties.
Active Speaker Detection
Audio-Visual Active Speaker Detection
Locality-aware Channel-wise Dropout for Occluded Face Recognition
Furthermore, by randomly dropping out several feature channels, our method can well simulate the occlusion of larger area.
Graph Jigsaw Learning for Cartoon Face Recognition
To mitigate this issue, we propose the GraphJigsaw that constructs jigsaw puzzles at various stages in the classification network and solves the puzzles with the graph convolutional network (GCN) in a progressive manner.
Gaze Estimation with an Ensemble of Four Architectures
This paper presents a method for gaze estimation according to face images.
MFR 2021: Masked Face Recognition Competition
These teams successfully submitted 18 valid solutions.
Feature Completion for Occluded Person Re-Identification
Our method significantly outperforms existing methods on the occlusion datasets, while remains top even superior performance on holistic datasets.
ICTCAS-UCAS-TAL Submission to the AVA-ActiveSpeaker Task at ActivityNet Challenge 2021
This report presents a brief description of our method for the AVA Active Speaker Detection (ASD) task at ActivityNet Challenge 2021.
Active Speaker Detection
Audio-Visual Active Speaker Detection
Meta Auxiliary Learning for Facial Action Unit Detection
The learned sample weights alleviate the negative transfer from two aspects: 1) balance the loss of each task automatically, and 2) suppress the weights of FE samples that have large uncertainties.
BiCnet-TKS: Learning Efficient Spatial-Temporal Representation for Video Person Re-Identification
Detail Branch processes frames at original resolution to preserve the detailed visual clues, and Context Branch with a down-sampling strategy is employed to capture long-range contexts.
EigenGAN: Layer-Wise Eigen-Learning for GANs
Via generative adversarial training to learn a target distribution, these layer-wise subspaces automatically discover a set of "eigen-dimensions" at each layer corresponding to a set of semantic attributes or interpretable variations.
Continuity-Discrimination Convolutional Neural Network for Visual Object Tracking
This paper proposes a novel model, named Continuity-Discrimination Convolutional Neural Network (CD-CNN), for visual object tracking.
Cross-Encoder for Unsupervised Gaze Representation Learning
To address the issue that the feature of gaze is always intertwined with the appearance of the eye, Cross-Encoder disentangles the features using a latent-code-swapping mechanism on eye-consistent image pairs and gaze-similar ones.
Attributes Aware Face Generation with Generative Adversarial Networks
Recent studies have shown remarkable success in face image generations.
Learn an Effective Lip Reading Model without Pains
Considering the non-negligible effects of these strategies and the existing tough status to train an effective lip reading model, we perform a comprehensive quantitative study and comparative analysis, for the first time, to show the effects of several different choices for lip reading.
Ranked #2 on
Lipreading
on CAS-VSR-W1k (LRW-1000)
IAUnet: Global Context-Aware Feature Learning for Person Re-Identification
Furthermore, a Channel IAU (CIAU) module is designed to model the semantic contextual interactions between channel features to enhance the feature representation, especially for small-scale visual cues and body parts.
Temporal Complementary Learning for Video Person Re-Identification
This paper proposes a Temporal Complementary Learning Network that extracts complementary features of consecutive video frames for video person re-identification.
Sketching Image Gist: Human-Mimetic Hierarchical Scene Graph Generation
Scene graph aims to faithfully reveal humans' perception of image content.
Video-based Remote Physiological Measurement via Cross-verified Feature Disentangling
Remote physiological measurements, e. g., remote photoplethysmography (rPPG) based heart rate (HR), heart rate variability (HRV) and respiration frequency (RF) measuring, are playing more and more important roles under the application scenarios where contact measurement is inconvenient or impossible.
PA-GAN: Progressive Attention Generative Adversarial Network for Facial Attribute Editing
Facial attribute editing aims to manipulate attributes on the human face, e. g., adding a mustache or changing the hair color.
Synchronous Bidirectional Learning for Multilingual Lip Reading
Based on this idea, we try to explore the synergized learning of multilingual lip reading in this paper, and further propose a synchronous bidirectional learning (SBL) framework for effective synergy of multilingual lip reading.
Single-Side Domain Generalization for Face Anti-Spoofing
In this work, we propose an end-to-end single-side domain generalization framework (SSDG) to improve the generalization ability of face anti-spoofing.
Self-supervised Equivariant Attention Mechanism for Weakly Supervised Semantic Segmentation
Our method is based on the observation that equivariance is an implicit constraint in fully supervised semantic segmentation, whose pixel-level labels take the same spatial transformation as the input images during data augmentation.
Data Augmentation
Weakly supervised Semantic Segmentation
+1
Cross-domain Face Presentation Attack Detection via Multi-domain Disentangled Representation Learning
In light of this, we propose an efficient disentangled representation learning for cross-domain face PAD.
Multi-Modal Graph Neural Network for Joint Reasoning on Vision and Scene Text
Then, we introduce three aggregators which guide the message passing from one graph to another to utilize the contexts in various modalities, so as to refine the features of nodes.
The 1st Challenge on Remote Physiological Signal Sensing (RePSS)
Remote measurement of physiological signals from videos is an emerging topic.
Mutual Information Maximization for Effective Lip Reading
By combining these two advantages together, the proposed method is expected to be both discriminative and robust for effective lip reading.
Ranked #8 on
Lipreading
on CAS-VSR-W1k (LRW-1000)
Deformation Flow Based Two-Stream Network for Lip Reading
Observing on the continuity in adjacent frames in the speaking process, and the consistency of the motion patterns among different speakers when they pronounce the same phoneme, we model the lip movements in the speaking process as a sequence of apparent deformations in the lip region.
Ranked #7 on
Lipreading
on CAS-VSR-W1k (LRW-1000)
Pseudo-Convolutional Policy Gradient for Sequence-to-Sequence Lip-Reading
On the one hand, we introduce the evaluation metric (refers to the character error rate in this paper) as a form of reward to optimize the model together with the original discriminative target.
Ranked #9 on
Lipreading
on CAS-VSR-W1k (LRW-1000)
Can We Read Speech Beyond the Lips? Rethinking RoI Selection for Deep Visual Speech Recognition
Recent advances in deep learning have heightened interest among researchers in the field of visual speech recognition (VSR).
Ranked #2 on
Lipreading
on GRID corpus (mixed-speech)
Emotion Recognition for In-the-wild Videos
This paper is a brief introduction to our submission to the seven basic expression classification track of Affective Behavior Analysis in-the-wild Competition held in conjunction with the IEEE International Conference on Automatic Face and Gesture Recognition (FG) 2020.
$M^3$T: Multi-Modal Continuous Valence-Arousal Estimation in the Wild
This report describes a multi-modal multi-task ($M^3$T) approach underlying our submission to the valence-arousal estimation track of the Affective Behavior Analysis in-the-wild (ABAW) Challenge, held in conjunction with the IEEE International Conference on Automatic Face and Gesture Recognition (FG) 2020.
FCSR-GAN: Joint Face Completion and Super-resolution via Multi-task Learning
Combined variations containing low-resolution and occlusion often present in face images in the wild, e. g., under the scenario of video surveillance.
Deep Heterogeneous Hashing for Face Video Retrieval
To tackle the key challenge of hashing on the manifold, a well-studied Riemannian kernel mapping is employed to project data (i. e. covariance matrices) into Euclidean space and thus enables to embed the two heterogeneous representations into a common Hamming space, where both intra-space discriminability and inter-space compatibility are considered.
RhythmNet: End-to-end Heart Rate Estimation from Face via Spatial-temporal Representation
Recently, some methods have been proposed for remote HR estimation from face videos; however, most of them focus on well-controlled scenarios, their generalization ability into less-constrained scenarios (e. g., with head movement, and bad illumination) are not known.
Multi-label Co-regularization for Semi-supervised Facial Action Unit Recognition
In this work, we propose a semi-supervised approach for AU recognition utilizing a large number of web face images without AU labels and a relatively small face dataset with AU annotations inspired by the co-training methods.
Cross Attention Network for Few-shot Classification
The unseen classes and low-data problem make few-shot classification very challenging.
Cross-modal Scene Graph Matching for Relationship-aware Image-Text Retrieval
In the light of recent success of scene graph in many CV and NLP tasks for describing complex natural scenes, we propose to represent image and text with two kinds of scene graphs: visual scene graph (VSG) and textual scene graph (TSG), each of which is exploited to jointly characterize objects and relationships in the corresponding modality.
Hierarchical Disentangle Network for Object Representation Learning
In this paper, we propose the hierarchical disentangle network (HDN) to exploit the rich hierarchical characteristics among categories to divide the disentangling process in a coarse-to-fine manner, such that each level only focuses on learning the specific representations in its granularity and finally the common and unique representations in all granularities jointly constitute the raw object.
Self-supervised Scale Equivariant Network for Weakly Supervised Semantic Segmentation
This regularized CAM can be embedded in most recent advanced weakly supervised semantic segmentation framework.
Transferable Contrastive Network for Generalized Zero-Shot Learning
Zero-shot learning (ZSL) is a challenging problem that aims to recognize the target categories without seen data, where semantic information is leveraged to transfer knowledge from some source classes.
Ranked #7 on
Zero-Shot Learning
on SUN Attribute
Temporal Knowledge Propagation for Image-to-Video Person Re-identification
With back propagation, temporal knowledge can be transferred to enhance the image features and the information asymmetry problem can be alleviated.
Ranked #9 on
Person Re-Identification
on iLIDS-VID
CRIC: A VQA Dataset for Compositional Reasoning on Vision and Commonsense
To comprehensively evaluate such abilities, we propose a VQA benchmark, CRIC, which introduces new types of questions about Compositional Reasoning on vIsion and Commonsense, and an evaluation metric integrating the correctness of answering and commonsense grounding.
VRSTC: Occlusion-Free Video Person Re-Identification
For one thing, the spatial structure of a pedestrian frame can be used to predict the occluded body parts from the unoccluded body parts of this frame.
Interaction-and-Aggregation Network for Person Re-identification
Person re-identification (reID) benefits greatly from deep convolutional neural networks (CNNs) which learn robust feature embeddings.
Cascade RetinaNet: Maintaining Consistency for Single-Stage Object Detection
Recent researches attempt to improve the detection performance by adopting the idea of cascade for single-stage detectors.
Multi-Task Learning for Audio Visual Active Speaker Detection
This report describes the approach underlying our submission to the active speaker detection task (task B-2) of ActivityNet Challenge 2019.
Ranked #18 on
Audio-Visual Active Speaker Detection
on AVA-ActiveSpeaker
(using extra training data)
Active Speaker Detection
Audio-Visual Active Speaker Detection
+3
Pose-adaptive Hierarchical Attention Network for Facial Expression Recognition
Multi-view facial expression recognition (FER) is a challenging task because the appearance of an expression varies in poses.
Facial Expression Recognition
Facial Expression Recognition (FER)
Weakly Supervised Object Detection with Segmentation Collaboration
Weakly supervised object detection aims at learning precise object detectors, given image category labels.
Fully Learnable Group Convolution for Acceleration of Deep Neural Networks
Benefitted from its great success on many tasks, deep learning is increasingly used on low-computational-cost devices, e. g. smartphone, embedded devices, etc.
WIDER Face and Pedestrian Challenge 2018: Methods and Results
This paper presents a review of the 2018 WIDER Challenge on Face and Pedestrian.
Tattoo Image Search at Scale: Joint Detection and Compact Representation Learning
To close the gap, we propose an efficient tattoo search approach that is able to learn tattoo detection and compact representation jointly in a single convolutional neural network (CNN) via multi-task learning.
LRW-1000: A Naturally-Distributed Large-Scale Benchmark for Lip Reading in the Wild
It has shown a large variation in this benchmark in several aspects, including the number of samples in each class, video resolution, lighting conditions, and speakers' attributes such as pose, age, gender, and make-up.
Ranked #2 on
Lipreading
on LRW-1000
VIPL-HR: A Multi-modal Database for Pulse Estimation from Less-constrained Face Video
We also learn a deep HR estimator (named as RhythmNet) with the proposed spatial-temporal representation, which achieves promising results on both the public-domain and our VIPL-HR HR estimation databases.
Meta-Learning with Individualized Feature Space for Few-Shot Classification
Specifically, we introduce a kernel generator as meta-learner to learn to construct feature embedding for query images.
Face Recognition with Contrastive Convolution
In current face recognition approaches with convolutional neural network (CNN), a pair of faces to compare are independently fed into the CNN for feature extraction.
Facial Expression Recognition with Inconsistently Annotated Datasets
To address the inconsistency, we propose an Inconsistent Pseudo Annotations to Latent Truth(IPA2LT) framework to train a FER model from multiple inconsistently labeled datasets and large scale unlabeled data.
Facial Expression Recognition
Facial Expression Recognition (FER)
Generative Adversarial Network with Spatial Attention for Face Attribute Editing
The generator contains an attribute manipulation network (AMN) to edit the face image, and a spatial attention network (SAN) to localize the attribute-specific region which restricts the alternation of AMN within this region.
Joint Representation and Truncated Inference Learning for Correlation Filter based Tracking
In existing off-line deep learning models for CF trackers, the model adaptation usually is either abandoned or has closed-form solution to make it feasible to learn deep representation in an end-to-end manner.
Learning Class Prototypes via Structure Alignment for Zero-Shot Recognition
Zero-shot learning (ZSL) aims to recognize objects of novel classes without any training samples of specific classes, which is achieved by exploiting the semantic information and auxiliary datasets.
Structure Inference Net: Object Detection Using Scene-Level Context and Instance-Level Relationships
Context is important for accurate visual recognition.
Duplex Generative Adversarial Network for Unsupervised Domain Adaptation
Following the similar idea of GAN, this work proposes a novel GAN architecture with duplex adversarial discriminators (referred to as DupGAN), which can achieve domain-invariant representation and domain transformation.
Mean-Variance Loss for Deep Age Estimation From a Face
Age estimation has broad application prospects of many fields, such as video surveillance, social networking, and human-computer interaction.
Ranked #3 on
Age Estimation
on ChaLearn 2016
Real-Time Rotation-Invariant Face Detection with Progressive Calibration Networks
Rotation-invariant face detection, i. e. detecting faces with arbitrary rotation-in-plane (RIP) angles, is widely required in unconstrained applications but still remains as a challenging task, due to the large variations of face appearances.
Shift-Net: Image Inpainting via Deep Feature Rearrangement
To this end, the encoder feature of the known region is shifted to serve as an estimation of the missing parts.
AttGAN: Facial Attribute Editing by Only Changing What You Want
Based on the encoder-decoder architecture, facial attribute editing is achieved by decoding the latent representation of the given face conditioned on the desired attributes.
Recursive Spatial Transformer (ReST) for Alignment-Free Face Recognition
The designed ReST has an intrinsic recursive structure and is capable of progressively aligning faces to a canonical one, even those with large variations.
Learning Discriminative Latent Attributes for Zero-Shot Classification
Zero-shot learning (ZSL) aims to transfer knowledge from observed classes to the unseen classes, based on the assumption that both the seen and unseen classes share a common semantic space, among which attributes enjoy a great popularity.
Learning Multifunctional Binary Codes for Both Category and Attribute Oriented Retrieval Tasks
In this paper we propose a unified framework to address multiple realistic image retrieval tasks concerning both category and attributes.
Discriminative Covariance Oriented Representation Learning for Face Recognition With Image Sets
For face recognition with image sets, while most existing works mainly focus on building robust set models with hand-crafted feature, it remains a research gap to learn better image representations which can closely match the subsequent image set modeling and classification.
Heterogeneous Face Attribute Estimation: A Deep Multi-Task Learning Approach
In DMTL, we tackle attribute correlation and heterogeneity with convolutional neural networks (CNNs) consisting of shared feature learning for all the attributes, and category-specific feature learning for heterogeneous attributes.
Ranked #5 on
Facial Attribute Classification
on LFWA
Funnel-Structured Cascade for Multi-View Face Detection with Alignment-Awareness
On the other hand, by using a unified MLP cascade to examine proposals of all views in a centralized style, it provides a favorable solution for multi-view face detection with high accuracy and low time-cost.
VIPLFaceNet: An Open Source Deep Face Recognition SDK
Robust face representation is imperative to highly accurate face recognition.
Geometry-aware Similarity Learning on SPD Manifolds for Visual Recognition
Specifically, by exploiting the Riemannian geometry of the manifold of fixed-rank Positive Semidefinite (PSD) matrices, we present a new solution to reduce optimizing over the space of column full-rank transformation matrices to optimizing on the PSD manifold which has a well-established Riemannian structure.
Cross Euclidean-to-Riemannian Metric Learning with Application to Face Recognition from Video
With this mapping, the problem of learning a cross-view metric between the two source heterogeneous spaces can be expressed as learning a single-view Euclidean distance metric in the target common Euclidean space.
Self-paced Learning for Weakly Supervised Evidence Discovery in Multimedia Event Search
Multimedia event detection has been receiving increasing attention in recent years.
Dual Purpose Hashing
Recent years have seen more and more demand for a unified framework to address multiple realistic image retrieval tasks concerning both category and attributes.
Occlusion-Free Face Alignment: Deep Regression Networks Coupled With De-Corrupt AutoEncoders
Face alignment or facial landmark detection plays an important role in many computer vision applications, e. g., face recognition, facial expression recognition, face animation, etc.
Multi-View Deep Network for Cross-View Classification
As a result, the representation from the topmost layers of the MvDN network is robust to view discrepancy, and also discriminative.
Deep Supervised Hashing for Fast Image Retrieval
In this paper, we present a new hashing method to learn compact binary codes for highly efficient image retrieval on large-scale datasets.
Ranked #1 on
Image Retrieval
on CIFAR-10
Leveraging Datasets With Varying Annotations for Face Alignment via Deep Regression Network
Facial landmark detection, as a vital topic in computer vision, has been studied for many decades and lots of datasets have been collected for evaluation.
Two Birds, One Stone: Jointly Learning Binary Code for Large-Scale Face Image Retrieval and Attributes Prediction
In this way, the learned binary codes can be applied to not only fine-grained face image retrieval, but also facial attributes prediction, which is the very innovation of this work, just like killing two birds with one stone.
A Unified Multiplicative Framework for Attribute Learning
Attributes are mid-level semantic properties of objects.
Bi-Shifting Auto-Encoder for Unsupervised Domain Adaptation
To alleviate the discrepancy between source and target domains, we propose a domain adaptation method, named as Bi-shifting Auto-Encoder network (BAE).
Learning Expressionlets via Universal Manifold Model for Dynamic Facial Expression Recognition
3) the local modes on each STM can be instantiated by fitting to UMM, and the corresponding expressionlet is constructed by modeling the variations in each local mode.
Dynamic Facial Expression Recognition
Facial Expression Recognition
+1
Learning Mid-level Words on Riemannian Manifold for Action Recognition
Human action recognition remains a challenging task due to the various sources of video data and large intra-class variations.
AgeNet: Deeply Learned Regressor and Classifier for Robust Apparent Age Estimation
Another key feature of the proposed AgeNet is that, to avoid the problem of over-fitting on small apparent age training set, we exploit a general-to-specific transfer learning scheme.
Ranked #4 on
Age Estimation
on ChaLearn 2015
Cross-pose Face Recognition by Canonical Correlation Analysis
We argue that one of the diffculties in this problem is the severe misalignment in face images or feature vectors with different poses.
Discriminant Analysis on Riemannian Manifold of Gaussian Distributions for Face Recognition With Image Sets
This paper presents a method named Discriminant Analysis on Riemannian manifold of Gaussian distributions (DARG) to solve the problem of face recognition with image sets.
Face Video Retrieval With Image Query via Hashing Across Euclidean Space and Riemannian Manifold
Retrieving videos of a specific person given his/her face image as query becomes more and more appealing for applications like smart movie fast-forwards and suspect searching.
Projection Metric Learning on Grassmann Manifold With Application to Video Based Face Recognition
In video based face recognition, great success has been made by representing videos as linear subspaces, which typically lie in a special type of non-Euclidean space known as Grassmann manifold.
Shape Driven Kernel Adaptation in Convolutional Neural Network for Robust Facial Traits Recognition
Comprehensive experiments on WebFace, Morph II and MultiPIE databases well validate the effectiveness of the proposed kernel adaptation method and tree-structured convolutional architecture for facial traits recognition tasks, including identity, age and gender classification.
Generalized Unsupervised Manifold Alignment
In this paper, we propose a generalized Unsupervised Manifold Alignment (GUMA) method to build the connections between different but correlated datasets without any known correspondences.
Self-Paced Learning with Diversity
Self-paced learning (SPL) is a recently proposed learning regime inspired by the learning process of humans and animals that gradually incorporates easy to more complex samples into training.