Search Results for author:
Found 20 papers, 3 papers with code
Sample-Efficient Preference-based Reinforcement Learning with Dynamics Aware Rewards
Preference-based reinforcement learning (PbRL) aligns a robot behavior with human preferences via a reward function learned from binary feedback over agent behaviors.
ESPnet-SPK: full pipeline speaker embedding toolkit with reproducible recipes, self-supervised front-ends, and off-the-shelf models
First, we provide an open-source platform for researchers in the speaker recognition community to effortlessly build models.
Ranked #1 on
Speaker Verification
on VoxCeleb
(using extra training data)
REALM: Robust Entropy Adaptive Loss Minimization for Improved Single-Sample Test-Time Adaptation
In online F-TTA, a pre-trained model is adapted using a stream of test samples by minimizing a self-supervised objective, such as entropy minimization.
Spatial LibriSpeech: An Augmented Dataset for Spatial Audio Learning
We present Spatial LibriSpeech, a spatial audio dataset with over 650 hours of 19-channel audio, first-order ambisonics, and optional distractor noise.
Understanding the Robustness of Multi-Exit Models under Common Corruptions
Furthermore, the lack of calibration increases the inconsistency in the predictions of the model across exits, leading to both inefficient inference and more misclassifications compared with evaluation on in-distribution data.
Rewards Encoding Environment Dynamics Improves Preference-based Reinforcement Learning
In this work, we demonstrate that encoding environment dynamics in the reward function (REED) dramatically reduces the number of preference labels required in state-of-the-art preference-based RL frameworks.
Contrastive Self-Supervised Learning for Skeleton Representations
Human skeleton point clouds are commonly used to automatically classify and predict the behaviour of others.
Naturalistic Head Motion Generation from Speech
Synthesizing natural head motion to accompany speech for an embodied conversational agent is necessary for providing a rich interactive experience.
On the role of Lip Articulation in Visual Speech Perception
Previous research has shown that traditional metrics used to optimize and assess models for generating lip motion from speech are not a good indicator of subjective opinion of animation quality.
FedEmbed: Personalized Private Federated Learning
Federated learning enables the deployment of machine learning to problems for which centralized data collection is impractical.
Multimodal Punctuation Prediction with Contextual Dropout
Automatic speech recognition (ASR) is widely used in consumer electronics.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
MorphGAN: One-Shot Face Synthesis GAN for Detecting Recognition Bias
Images generated using MorphGAN conserve the identity of the person in the original image, and the provided control over head pose and facial expression allows test sets to be created to identify robustness issues of a facial recognition deep network with respect to pose and expression.
Modality Dropout for Improved Performance-driven Talking Faces
We use subjective testing to demonstrate: 1) the improvement of audiovisual-driven animation over the equivalent video-only approach, and 2) the improvement in the animation of speech-related facial movements after introducing modality dropout.
On the Role of Visual Cues in Audiovisual Speech Enhancement
One byproduct of this finding is that the learned visual embeddings can be used as features for other visual speech applications.
Speaker-Independent Speech-Driven Visual Speech Synthesis using Domain-Adapted Acoustic Models
We conclude that visual speech synthesis can significantly benefit from the powerful representation of speech in the ASR acoustic models.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Mirroring to Build Trust in Digital Assistants
We describe experiments towards building a conversational digital assistant that considers the preferred conversational style of the user.
Learning Sharing Behaviors with Arbitrary Numbers of Agents
We model the individual behavior for each agent in an interaction and then use a multi-agent fusion model to generate a summary over the expected actions of the group to render the model independent of the number of agents.
Some observations on computer lip-reading: moving from the dream to the reality
In the quest for greater computer lip-reading performance there are a number of tacit assumptions which are either present in the datasets (high resolution for example) or in the methods (recognition of spoken visual units called visemes for example).
Which phoneme-to-viseme maps best improve visual-only computer lip-reading?
A critical assumption of all current visual speech recognition systems is that there are visual speech units called visemes which can be mapped to units of acoustic speech, the phonemes.
Resolution limits on visual speech recognition
Visual-only speech recognition is dependent upon a number of factors that can be difficult to control, such as: lighting; identity; motion; emotion and expression.
