Search Results for author:
Found 21 papers, 9 papers with code
Text-Speech Language Models with Improved Cross-Modal Transfer by Aligning Abstraction Levels
Text-Speech Language Models (TSLMs) -- language models trained to jointly process and generate text and speech -- aim to enable cross-modal knowledge transfer to overcome the scaling limitations of unimodal speech LMs.
TalTech-IRIT-LIS Speaker and Language Diarization Systems for DISPLACE 2024
Our team participated in the speaker diarization and language diarization tracks of the challenge.
Transfer Learning from Whisper for Microscopic Intelligibility Prediction
Macroscopic intelligibility models predict the expected human word-error-rate for a given speech-in-noise stimulus.
Scaling Properties of Speech Language Models
We establish a strong correlation between pre-training loss and downstream syntactic and semantic performance in SLMs and LLMs, which results in predictable scaling of linguistic performance.
PixIT: Joint Training of Speaker Diarization and Speech Separation from Real-world Multi-speaker Recordings
A major drawback of supervised speech separation (SSep) systems is their reliance on synthetic data, leading to poor real-world generalization.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Speech foundation models on intelligibility prediction for hearing-impaired listeners
Our method resulted in the winning submission in the CPC2, demonstrating its promise for speech perception applications.
Eiffel Tower: A Deep-Sea Underwater Dataset for Long-Term Visual Localization
This paper presents a new deep-sea dataset to benchmark underwater long-term visual localization.
SUCRe: Leveraging Scene Structure for Underwater Color Restoration
Underwater images are altered by the physical characteristics of the medium through which light rays pass before reaching the optical sensor.
Variable-rate hierarchical CPC leads to acoustic unit discovery in speech
The success of deep learning comes from its ability to capture the hierarchical structure of data by learning high-level representations defined in terms of low-level ones.
Homography-Based Loss Function for Camera Pose Regression
This paper focuses on the loss functions that embed the error between two poses to perform deep learning based camera pose regression.
Contrastive prediction strategies for unsupervised segmentation and categorization of phonemes and words
We investigate the performance on phoneme categorization and phoneme and word segmentation of several self-supervised learning (SSL) methods based on Contrastive Predictive Coding (CPC).
Information Retrieval for ZeroSpeech 2021: The Submission by University of Wroclaw
We present a number of low-resource approaches to the tasks of the Zero Resource Speech Challenge 2021.
Aligned Contrastive Predictive Coding
We investigate the possibility of forcing a self-supervised model trained using a contrastive predictive loss to extract slowly varying latent representations.
A Convolutional Deep Markov Model for Unsupervised Speech Representation Learning
Probabilistic Latent Variable Models (LVMs) provide an alternative to self-supervised learning approaches for linguistic representation learning from speech.
Robust Training of Vector Quantized Bottleneck Models
We show that the codebook learning can suffer from poor initialization and non-stationarity of clustered encoder outputs.
Deep Learning Classification With Noisy Labels
Deep Learning systems have shown tremendous accuracy in image classification, at the cost of big image datasets.
DNN driven Speaker Independent Audio-Visual Mask Estimation for Speech Separation
The process of selective attention in the brain is known to contextually exploit the available audio and visual cues to better focus on target speaker while filtering out other noises.
Unsupervised Incremental Learning and Prediction of Music Signals
A system is presented that segments, clusters and predicts musical audio in an unsupervised manner, adjusting the number of (timbre) clusters instantaneously to the audio input.
