Search Results for author:
Found 33 papers, 16 papers with code
Opinions in Interactions : New Annotations of the SEMAINE Database
In this paper, we present the process we used in order to collect new annotations of opinions over the multimodal corpus SEMAINE composed of dyadic interactions.
TACO: Training-free Sound Prompted Segmentation via Deep Audio-visual CO-factorization
Large-scale pre-trained audio and image models demonstrate an unprecedented degree of generalization, making them suitable for a wide range of applications.
Multiple Choice Learning for Efficient Speech Separation with Many Speakers
Training speech separation models in the supervised setting raises a permutation problem: finding the best assignation between the model predictions and the ground truth separated signals.
A Contrastive Self-Supervised Learning scheme for beat tracking amenable to few-shot learning
In this paper, we propose a novel Self-Supervised-Learning scheme to train rhythm analysis systems and instantiate it for few-shot beat tracking.
An Eye for an Ear: Zero-shot Audio Description Leveraging an Image Captioner using Audiovisual Distribution Alignment
In this work, we introduce a novel methodology for bridging the audiovisual modality gap by matching the distributions of tokens produced by an audio backbone and those of an image captioner.
A sound description: Exploring prompt templates and class descriptions to enhance zero-shot audio classification
First, we find that the formatting of the prompts significantly affects performance so that simply prompting the models with properly formatted class labels performs competitively with optimized prompt templates and even prompt ensembling.
Ranked #1 on
Zero-shot Audio Classification
on ESC-50
Annealed Multiple Choice Learning: Overcoming limitations of Winner-takes-all with annealing
We introduce Annealed Multiple Choice Learning (aMCL) which combines simulated annealing with MCL.
Less Forgetting for Better Generalization: Exploring Continual-learning Fine-tuning Methods for Speech Self-supervised Representations
Despite being trained on massive and diverse datasets, speech self-supervised encoders are generally used for downstream purposes as mere frozen feature extractors or model initializers before fine-tuning.
Winner-takes-all learners are geometry-aware conditional density estimators
Winner-takes-all training is a simple learning paradigm, which handles ambiguous tasks by predicting a set of plausible hypotheses.
Online speaker diarization of meetings guided by speech separation
The results show that our system improves the state-of-the-art on the AMI headset mix, using no oracle information and under full evaluation (no collar and including overlapped speech).
On the choice of the optimal temporal support for audio classification with Pre-trained embeddings
Choosing the best one for a set of tasks is the subject of many recent publications.
Collaborating Foundation Models for Domain Generalized Semantic Segmentation
Domain Generalized Semantic Segmentation (DGSS) deals with training a model on a labeled source domain with the aim of generalizing to unseen domains during inference.
Resilient Multiple Choice Learning: A learned scoring scheme with application to audio scene analysis
Multiple Choice Learning is a simple framework to tackle multimodal density estimation, using the Winner-Takes-All (WTA) loss for a set of hypotheses.
Speech Self-Supervised Representations Benchmarking: a Case for Larger Probing Heads
Self-supervised learning (SSL) leverages large datasets of unlabeled speech to reach impressive performance with reduced amounts of annotated data.
SAMbA: Speech enhancement with Asynchronous ad-hoc Microphone Arrays
Speech enhancement in ad-hoc microphone arrays is often hindered by the asynchronization of the devices composing the microphone array.
Speech Self-Supervised Representation Benchmarking: Are We Doing it Right?
Self-supervised learning (SSL) has recently allowed leveraging large datasets of unlabeled speech signals to reach impressive performance on speech tasks using only small amounts of annotated data.
Automatic Data Augmentation for Domain Adapted Fine-Tuning of Self-Supervised Speech Representations
Self-Supervised Learning (SSL) has allowed leveraging large amounts of unlabeled speech data to improve the performance of speech recognition models even with small annotated datasets.
One-shot Unsupervised Domain Adaptation with Personalized Diffusion Models
Departing from the common notion of transferring only the target ``texture'' information, we leverage text-to-image diffusion models (e. g., Stable Diffusion) to generate a synthetic target dataset with photo-realistic images that not only faithfully depict the style of the target domain, but are also characterized by novel scenes in diverse contexts.
Data Augmentation
One-shot Unsupervised Domain Adaptation
+2
Fine-tuning Strategies for Faster Inference using Speech Self-Supervised Models: A Comparative Study
Self-supervised learning (SSL) has allowed substantial progress in Automatic Speech Recognition (ASR) performance in low-resource settings.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Automatic Data Augmentation Selection and Parametrization in Contrastive Self-Supervised Speech Representation Learning
Thus, this work introduces a conditional independance-based method which allows for automatically selecting a suitable distribution on the choice of augmentations and their parametrization from a set of predefined ones, for contrastive self-supervised pre-training.
Pretext Tasks selection for multitask self-supervised speech representation learning
Through solving pretext tasks, self-supervised learning leverages unlabeled data to extract useful latent representations replacing traditional input features in the downstream task.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+6
Attention-based distributed speech enhancement for unconstrained microphone arrays with varying number of nodes
Speech enhancement promises higher efficiency in ad-hoc microphone arrays than in constrained microphone arrays thanks to the wide spatial coverage of the devices in the acoustic scene.
Conditional independence for pretext task selection in Self-supervised speech representation learning
Through solving pretext tasks, self-supervised learning (SSL) leverages unlabeled data to extract useful latent representations replacing traditional input features in the downstream task.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+6
DNN-based mask estimation for distributed speech enhancement in spatially unconstrained microphone arrays
Deep neural network (DNN)-based speech enhancement algorithms in microphone arrays have now proven to be efficient solutions to speech understanding and speech recognition in noisy environments.
Distributed speech separation in spatially unconstrained microphone arrays
We propose a distributed algorithm that can process spatial information in a spatially unconstrained microphone array.
On-the-fly Detection of User Engagement Decrease in Spontaneous Human-Robot Interaction, International Journal of Social Robotics, 2019
In this paper, we consider the detection of a decrease of engagement by users spontaneously interacting with a socially assistive robot in a public space.
Human-Computer Interaction Robotics
DNN-Based Distributed Multichannel Mask Estimation for Speech Enhancement in Microphone Arrays
Multichannel processing is widely used for speech enhancement but several limitations appear when trying to deploy these solutions to the real-world.
From the Token to the Review: A Hierarchical Multimodal approach to Opinion Mining
The task of predicting fine grained user opinion based on spontaneous spoken language is a key problem arising in the development of Computational Agents as well as in the development of social network based opinion miners.
A multimodal movie review corpus for fine-grained opinion mining
We introduce specific categories in order to make the annotation of opinions easier for movie reviews.
Identify, locate and separate: Audio-visual object extraction in large video collections using weak supervision
We tackle the problem of audiovisual scene analysis for weakly-labeled data.
Opinion Dynamics Modeling for Movie Review Transcripts Classification with Hidden Conditional Random Fields
This model allows us to capture the dynamics of the reviewer's opinion in the transcripts of long unsegmented audio reviews that are analyzed by our system.
Weakly Supervised Representation Learning for Unsynchronized Audio-Visual Events
Audio-visual representation learning is an important task from the perspective of designing machines with the ability to understand complex events.
Structured Output Learning with Abstention: Application to Accurate Opinion Prediction
Motivated by Supervised Opinion Analysis, we propose a novel framework devoted to Structured Output Learning with Abstention (SOLA).
