Search Results for author:
Found 60 papers, 27 papers with code
iBoot: Image-bootstrapped Self-Supervised Video Representation Learning
Video self-supervised learning (SSL) suffers from added challenges: video datasets are typically not as large as image datasets, compute is an order of magnitude larger, and the amount of spurious patterns the optimizer has to sieve through is multiplied several fold.
ContraCLIP: Interpretable GAN generation driven by pairs of contrasting sentences
This work addresses the problem of discovering non-linear interpretable paths in the latent space of pre-trained GANs in a model-agnostic manner.
From Keypoints to Object Landmarks via Self-Training Correspondence: A novel approach to Unsupervised Landmark Discovery
This paper proposes a novel paradigm for the unsupervised learning of object landmark detectors.
Knowledge Distillation Meets Open-Set Semi-Supervised Learning
The key idea is that we leverage the teacher's classifier as a semantic critic for evaluating the representations of both teacher and student and distilling the semantic knowledge with high-order structured information over all feature dimensions.
EdgeViTs: Competing Light-weight CNNs on Mobile Devices with Vision Transformers
In this work, pushing further along this under-studied direction we introduce EdgeViTs, a new family of light-weight ViTs that, for the first time, enable attention-based vision models to compete with the best light-weight CNNs in the tradeoff between accuracy and on-device efficiency.
Finding Directions in GAN's Latent Space for Neural Face Reenactment
Because GANs are characterized by weak controllability, the core of our approach is a method to discover which directions in latent GAN space are responsible for controlling facial pose and expression variations.
S3: Supervised Self-supervised Learning under Label Noise
Despite the large progress in supervised learning with Neural Networks, there are significant challenges in obtaining high-quality, large-scale and accurately labeled datasets.
Ranked #1 on
Learning with noisy labels
on ANIMAL
Subpixel Heatmap Regression for Facial Landmark Localization
Deep Learning models based on heatmap regression have revolutionized the task of facial landmark localization with existing models working robustly under large poses, non-uniform illumination and shadows, occlusions and self-occlusions, low resolution and blur.
Ranked #1 on
Face Alignment
on COFW
(using extra training data)
Defensive Tensorization
We propose defensive tensorization, an adversarial defence technique that leverages a latent high-order factorization of the network.
SAIC_Cambridge-HuPBA-FBK Submission to the EPIC-Kitchens-100 Action Recognition Challenge 2021
This report presents the technical details of our submission to the EPIC-Kitchens-100 Action Recognition Challenge 2021.
WarpedGANSpace: Finding non-linear RBF paths in GAN latent space
This work addresses the problem of discovering, in an unsupervised manner, interpretable paths in the latent space of pretrained GANs, so as to provide an intuitive and easy way of controlling the underlying generative factors.
Space-time Mixing Attention for Video Transformer
In this work, we propose a Video Transformer model the complexity of which scales linearly with the number of frames in the video sequence and hence induces no overhead compared to an image-based Transformer model.
Ranked #19 on
Action Classification
on Kinetics-600
Bit-Mixer: Mixed-precision networks with runtime bit-width selection
In this work, we propose Bit-Mixer, the very first method to train a meta-quantized network where during test time any layer can change its bid-width without affecting at all the overall network's ability for highly accurate inference.
Pre-training strategies and datasets for facial representation learning
Recent work on Deep Learning in the area of face analysis has focused on supervised learning for specific tasks of interest (e. g. face recognition, facial landmark localization etc.)
Ranked #1 on
Arousal Estimation
on AffectNet
Affective Processes: stochastic modelling of temporal context for emotion and facial expression recognition
Temporal context is key to the recognition of expressions of emotion.
Improving memory banks for unsupervised learning with large mini-batch, consistency and hard negative mining
An important component of unsupervised learning by instance-based discrimination is a memory bank for storing a feature representation for each training sample in the dataset.
Knowledge distillation via softmax regression representation learning
We advocate for a method that optimizes the output feature of the penultimate layer of the student network and hence is directly related to representation learning.
Unsupervised Learning of Object Landmarks via Self-Training Correspondence
This paper addresses the problem of unsupervised discovery of object landmarks.
Semi-supervised Facial Action Unit Intensity Estimation with Contrastive Learning
The second stage uses another dataset of randomly chosen labeled frames to train a regressor on top of our spatio-temporal model for estimating the AU intensity.
High-Capacity Expert Binary Networks
Network binarization is a promising hardware-aware direction for creating efficient deep models.
A Transfer Learning approach to Heatmap Regression for Action Unit intensity estimation
Action Units (AUs) are geometrically-based atomic facial muscle movements known to produce appearance changes at specific facial locations.
Training Binary Neural Networks with Real-to-Binary Convolutions
This paper shows how to train binary networks to within a few percent points ($\sim 3-5 \%$) of the full precision counterpart.
Knowledge distillation via adaptive instance normalization
To this end, we propose a new knowledge distillation method based on transferring feature statistics, specifically the channel-wise mean and variance, from the teacher to the student.
BATS: Binary ArchitecTure Search
We show that directly applying NAS to the binary domain provides very poor results.
Toward fast and accurate human pose estimation via soft-gated skip connections
In addition, with a reduction of 3x in model size and complexity, we show no decrease in performance when compared to the original HourGlass network.
Ranked #2 on
Pose Estimation
on MPII Human Pose
(using extra training data)
Towards Pose-invariant Lip-Reading
The proposed model significantly outperforms previous approaches on non-frontal views while retaining the superior performance on frontal and near frontal mouth views.
Object landmark discovery through unsupervised adaptation
Contrary to previous works, we do however assume that a landmark detector, which has already learned a structured representation for a given object category in a fully supervised manner, is available.
XNOR-Net++: Improved Binary Neural Networks
This paper proposes an improved training algorithm for binary neural networks in which both weights and activations are binary numbers.
Defensive Tensorization: Randomized Tensor Parametrization for Robust Neural Networks
As deep neural networks become widely adopted for solving most problems in computer vision and audio-understanding, there are rising concerns about their potential vulnerability.
AnimalWeb: A Large-Scale Hierarchical Dataset of Annotated Animal Faces
Several studies show that animal needs are often expressed through their faces.
Matrix and tensor decompositions for training binary neural networks
This paper is on improving the training of binary neural networks in which both activations and weights are binary.
Incremental multi-domain learning with network latent tensor factorization
Adapting the learned classification to new domains is a hard problem due to at least three reasons: (1) the new domains and the tasks might be drastically different; (2) there might be very limited amount of annotated data on the new domain and (3) full training of a new model for each new task is prohibitive in terms of computation and memory, due to the sheer number of parameters of deep CNNs.
Improved training of binary networks for human pose estimation and image recognition
Big neural networks trained on large datasets have advanced the state-of-the-art for a large variety of challenging problems, improving performance by a large margin.
T-Net: Parametrizing Fully Convolutional Nets with a Single High-Order Tensor
In this paper, we propose to fully parametrize Convolutional Neural Networks (CNNs) with a single high-order, low-rank tensor.
Ranked #32 on
Pose Estimation
on MPII Human Pose
Features Extraction Based on an Origami Representation of 3D Landmarks
The performance evaluation demonstrates an improvement on facial emotion classification (accuracy and F1 score) that indicates the superiority of the proposed methodology.
Learning to Infer the Depth Map of a Hand from its Color Image
We propose the first approach to the problem of inferring the depth map of a human hand based on a single RGB image.
Pushing the boundaries of audiovisual word recognition using Residual Networks and LSTMs
A further analysis on the utility of target word boundaries is provided, as well as on the capacity of the network in modeling the linguistic context of the target word.
Audio-Visual Speech Recognition With A Hybrid CTC/Attention Architecture
Therefore, we could use a CTC loss in combination with an attention-based model in order to force monotonic alignments and at the same time get rid of the conditional independence assumption.
Ranked #4 on
Audio-Visual Speech Recognition
on LRS2
Audio-Visual Speech Recognition
Automatic Speech Recognition
+3
3D Human Body Reconstruction from a Single Image via Volumetric Regression
This paper proposes the use of an end-to-end Convolutional Neural Network for direct reconstruction of the 3D geometry of humans via volumetric regression.
Hierarchical binary CNNs for landmark localization with limited resources
To this end, we make the following contributions: (a) we are the first to study the effect of neural network binarization on localization tasks, namely human pose estimation and face alignment.
Ranked #1 on
3D Face Alignment
on AFLW2000-3D
To learn image super-resolution, use a GAN to learn how to do image degradation first
This paper is on image and face super-resolution.
Zero-shot keyword spotting for visual speech recognition in-the-wild
Visual keyword spotting (KWS) is the problem of estimating whether a text query occurs in a given recording using only video information.
Joint Action Unit localisation and intensity estimation through heatmap regression
Contrary to previous works that try to learn an unsupervised representation of the Action Unit regions, we propose to directly and jointly estimate all AU intensities through heatmap regression, along with the location in the face where they cause visible changes.
End-to-end Audiovisual Speech Recognition
In presence of high levels of noise, the end-to-end audiovisual model significantly outperforms both audio-only models.
Ranked #14 on
Lipreading
on Lip Reading in the Wild
Super-FAN: Integrated facial landmark localization and super-resolution of real-world low resolution faces in arbitrary poses with GANs
This paper addresses 2 challenging tasks: improving the quality of low resolution facial images and accurately locating the facial landmarks on such poor resolution images.
Ranked #4 on
Face Hallucination
on FFHQ 512 x 512 - 16x upscaling
Deep word embeddings for visual speech recognition
In this paper we present a deep learning architecture for extracting word embeddings for visual speech recognition.
Synergy Between Face Alignment and Tracking via Discriminative Global Consensus Optimization
Tracking-by-detection is drift-free but results in low accuracy fittings.
Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric CNN Regression
Our CNN works with just a single 2D facial image, does not require accurate alignment nor establishes dense correspondence between images, works for arbitrary facial poses and expressions, and can be used to reconstruct the whole 3D facial geometry (including the non-visible parts of the face) bypassing the construction (during training) and fitting (during testing) of a 3D Morphable Model.
Ranked #2 on
3D Face Reconstruction
on Florence
How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)
To this end, we make the following 5 contributions: (a) we construct, for the first time, a very strong baseline by combining a state-of-the-art architecture for landmark localization with a state-of-the-art residual block, train it on a very large yet synthetically expanded 2D facial landmark dataset and finally evaluate it on all other 2D facial landmark datasets.
Ranked #1 on
Face Alignment
on LS3D-W Balanced
Combining Residual Networks with LSTMs for Lipreading
We propose an end-to-end deep learning architecture for word-level visual speech recognition.
Ranked #16 on
Lipreading
on Lip Reading in the Wild
Binarized Convolutional Landmark Localizers for Human Pose Estimation and Face Alignment with Limited Resources
(d) We present results for experiments on the most challenging datasets for human pose estimation and face alignment, reporting in many cases state-of-the-art performance.
Ranked #1 on
Face Alignment
on AFLW-Full
ChaLearn Looking at People and Faces of the World: Face Analysis Workshop and Challenge 2016
A custom-build application was used to collect and label data about the apparent age of people (as opposed to the real age).
Ranked #2 on
Gender Prediction
on FotW Gender
A Functional Regression approach to Facial Landmark Tracking
This paper presents a Functional Regression solution to the least squares problem, which we coin Continuous Regression, resulting in the first real-time incremental face tracker.
A CNN Cascade for Landmark Guided Semantic Part Segmentation
This paper proposes a CNN cascade for semantic part segmentation guided by pose-specific information encoded in terms of a set of landmarks (or keypoints).
Two-stage Convolutional Part Heatmap Regression for the 1st 3D Face Alignment in the Wild (3DFAW) Challenge
This paper describes our submission to the 1st 3D Face Alignment in the Wild (3DFAW) Challenge.
Ranked #1 on
Face Alignment
on 3DFAW
Convolutional aggregation of local evidence for large pose face alignment
Besides playing the role of a graphical model, CNN regression is a key feature of our system, guiding the network to rely on context for predicting the location of occluded landmarks, typically encountered in very large poses.
Ranked #1 on
Face Alignment
on AFLW-PIFA (21 points)
Human pose estimation via Convolutional Part Heatmap Regression
Our main contribution is a CNN cascaded architecture specifically designed for learning part relationships and spatial context, and robustly inferring pose even for the case of severe part occlusions.
Ranked #10 on
Pose Estimation
on Leeds Sports Poses
Cascaded Continuous Regression for Real-time Incremental Face Tracking
We then derive the incremental learning updates for CCR (iCCR) and show that it is an order of magnitude faster than standard incremental learning for cascaded regression, bringing the time required for the update from seconds down to a fraction of a second, thus enabling real-time tracking.
Project-Out Cascaded Regression With an Application to Face Alignment
Cascaded regression approaches have been recently shown to achieve state-of-the-art performance for many computer vision tasks.
Gauss-Newton Deformable Part Models for Face Alignment in-the-Wild
To address this limitation, in this paper, we propose to jointly optimize a part-based, trained in-the-wild, flexible appearance model along with a global shape model which results in a joint translational motion model for the model parts via Gauss-Newton (GN) optimization.
