Search Results for author:
Found 22 papers, 11 papers with code
Generative Spoken Language Modeling from Raw Audio
We introduce Generative Spoken Language Modeling, the task of learning the acoustic and linguistic characteristics of a language from raw audio (no text, no labels), and a set of metrics to automatically evaluate the learned representations at acoustic and linguistic levels for both encoding and generation.
Ranked #1 on
Resynthesis
on LibriSpeech
STOP: A dataset for Spoken Task Oriented Semantic Parsing
Furthermore, in addition to the human-recorded audio, we are releasing a TTS-generated version to benchmark the performance for low-resource domain adaptation of end-to-end SLU systems.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Text-Free Prosody-Aware Generative Spoken Language Modeling
Generative Spoken Language Modeling (GSLM) \cite{Lakhotia2021} is the only prior work addressing the generative aspects of speech pre-training, which replaces text with discovered phone-like units for language modeling and shows the ability to generate meaningful novel sentences.
AudioGen: Textually Guided Audio Generation
Finally, we explore the ability of the proposed method to generate audio continuation conditionally and unconditionally.
Ranked #12 on
Audio Generation
on AudioCaps
Simple and Controllable Music Generation
We tackle the task of conditional music generation.
Ranked #4 on
Text-to-Music Generation
on MusicCaps
Code Llama: Open Foundation Models for Code
We release Code Llama, a family of large language models for code based on Llama 2 providing state-of-the-art performance among open models, infilling capabilities, support for large input contexts, and zero-shot instruction following ability for programming tasks.
Ranked #26 on
Code Generation
on MBPP
High Fidelity Neural Audio Compression
We introduce a state-of-the-art real-time, high-fidelity, audio codec leveraging neural networks.
textless-lib: a Library for Textless Spoken Language Processing
Textless spoken language processing research aims to extend the applicability of standard NLP toolset onto spoken language and languages with few or no textual resources.
Speech Resynthesis from Discrete Disentangled Self-Supervised Representations
We propose using self-supervised discrete representations for the task of speech resynthesis.
Brouhaha: multi-task training for voice activity detection, speech-to-noise ratio, and C50 room acoustics estimation
Most automatic speech processing systems register degraded performance when applied to noisy or reverberant speech.
Textually Pretrained Speech Language Models
In this work, we propose TWIST, a method for training SpeechLMs using a warm-start from a pretrained textual language models.
Radarly : \'ecouter et analyser le web conversationnel en temps r\'eel (Real time listening and analysis of the social web using Radarly)
De par le contexte conversationnel digital, l{'}outil Radarly a {\'e}t{\'e} con{\c{c}}u pour permettre de traiter de grands volumes de donn{\'e}es h{\'e}t{\'e}rog{\`e}nes en temps r{\'e}el, de g{\'e}n{\'e}rer de nouveaux indicateurs et de les visualiser sur une interface coh{\'e}rente et confortable afin d{'}en tirer des analyses et {\'e}tudes pertinentes.
ASR4REAL: An extended benchmark for speech models
Popular ASR benchmarks such as Librispeech and Switchboard are limited in the diversity of settings and speakers they represent.
Textless Speech Emotion Conversion using Discrete and Decomposed Representations
We use a decomposition of the speech signal into discrete learned representations, consisting of phonetic-content units, prosodic features, speaker, and emotion.
Textless Speech Emotion Conversion using Decomposed & Discrete Representations
We decompose speech into discrete and disentangled learned representations, consisting of content units, F0, speaker, and emotion.
Generative Spoken Dialogue Language Modeling
We introduce dGSLM, the first "textless" model able to generate audio samples of naturalistic spoken dialogues.
Augmentation Invariant Discrete Representation for Generative Spoken Language Modeling
This work focuses on improving the robustness of discrete input representations for generative spoken language modeling.
Pushing the performances of ASR models on English and Spanish accents
Speech to text models tend to be trained and evaluated against a single target accent.
EXPRESSO: A Benchmark and Analysis of Discrete Expressive Speech Resynthesis
Recent work has shown that it is possible to resynthesize high-quality speech based, not on text, but on low bitrate discrete units that have been learned in a self-supervised fashion and can therefore capture expressive aspects of speech that are hard to transcribe (prosody, voice styles, non-verbal vocalization).
Low-Resource Self-Supervised Learning with SSL-Enhanced TTS
Self-supervised learning (SSL) techniques have achieved remarkable results in various speech processing tasks.
Generative Spoken Language Model based on continuous word-sized audio tokens
In NLP, text language models based on words or subwords are known to outperform their character-based counterparts.
Masked Audio Generation using a Single Non-Autoregressive Transformer
We introduce MAGNeT, a masked generative sequence modeling method that operates directly over several streams of audio tokens.
