Search Results for author:
Found 27 papers, 16 papers with code
An Investigation of Few-Shot Learning in Spoken Term Classification
In this paper, we investigate the feasibility of applying few-shot learning algorithms to a speech task.
MetaMix: Improved Meta-Learning with Interpolation-based Consistency Regularization
MetaMix can be integrated with any of the MAML-based algorithms and learn the decision boundaries generalizing better to new tasks.
AutoSpeech 2020: The Second Automated Machine Learning Challenge for Speech Classification
The AutoSpeech challenge calls for automated machine learning (AutoML) solutions to automate the process of applying machine learning to speech processing tasks.
Auto-KWS 2021 Challenge: Task, Datasets, and Baselines
Auto-KWS 2021 challenge calls for automated machine learning (AutoML) solutions to automate the process of applying machine learning to a customized keyword spotting task.
Exploring Machine Speech Chain for Domain Adaptation and Few-Shot Speaker Adaptation
Machine Speech Chain, which integrates both end-to-end (E2E) automatic speech recognition (ASR) and text-to-speech (TTS) into one circle for joint training, has been proven to be effective in data augmentation by leveraging large amounts of unpaired data.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Token-Level Supervised Contrastive Learning for Punctuation Restoration
Punctuation is critical in understanding natural language text.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+6
CL4AC: A Contrastive Loss for Audio Captioning
Automated Audio captioning (AAC) is a cross-modal translation task that aims to use natural language to describe the content of an audio clip.
An Encoder-Decoder Based Audio Captioning System With Transfer and Reinforcement Learning
Automated audio captioning aims to use natural language to describe the content of audio data.
Multi-View Self-Attention Based Transformer for Speaker Recognition
In this work, we propose a novel multi-view self-attention mechanism and present an empirical study of different Transformer variants with or without the proposed attention mechanism for speaker recognition.
SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processing
Motivated by the success of T5 (Text-To-Text Transfer Transformer) in pre-trained natural language processing models, we propose a unified-modal SpeechT5 framework that explores the encoder-decoder pre-training for self-supervised speech/text representation learning.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+7
LightHuBERT: Lightweight and Configurable Speech Representation Learning with Once-for-All Hidden-Unit BERT
LightHuBERT outperforms the original HuBERT on ASR and five SUPERB tasks with the HuBERT size, achieves comparable performance to the teacher model in most tasks with a reduction of 29% parameters, and obtains a $3. 5\times$ compression ratio in three SUPERB tasks, e. g., automatic speaker verification, keyword spotting, and intent classification, with a slight accuracy loss.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+6
Pre-Training Transformer Decoder for End-to-End ASR Model with Unpaired Speech Data
In this way, the decoder learns to reconstruct original speech information with codes before learning to generate correct text.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
GigaST: A 10,000-hour Pseudo Speech Translation Corpus
The training set is translated by a strong machine translation system and the test set is translated by human.
Leveraging Pseudo-labeled Data to Improve Direct Speech-to-Speech Translation
Direct Speech-to-speech translation (S2ST) has drawn more and more attention recently.
A Study of Modeling Rising Intonation in Cantonese Neural Speech Synthesis
In human speech, the attitude of a speaker cannot be fully expressed only by the textual content.
Personalized Dialogue Generation with Persona-Adaptive Attention
Persona-based dialogue systems aim to generate consistent responses based on historical context and predefined persona.
Visually-Aware Audio Captioning With Adaptive Audio-Visual Attention
Audio captioning aims to generate text descriptions of audio clips.
Leveraging per Image-Token Consistency for Vision-Language Pre-training
(2) Under-utilization of the unmasked tokens: CMLM primarily focuses on the masked token but it cannot simultaneously leverage other tokens to learn vision-language associations.
M3ST: Mix at Three Levels for Speech Translation
How to solve the data scarcity problem for end-to-end speech-to-text translation (ST)?
WavCaps: A ChatGPT-Assisted Weakly-Labelled Audio Captioning Dataset for Audio-Language Multimodal Research
To address this data scarcity issue, we introduce WavCaps, the first large-scale weakly-labelled audio captioning dataset, comprising approximately 400k audio clips with paired captions.
Ranked #1 on
Zero-Shot Environment Sound Classification
on ESC-50
(using extra training data)
DUB: Discrete Unit Back-translation for Speech Translation
The key point is to bridge the modality gap between speech and text so that useful MT techniques can be applied to ST.
CTC-based Non-autoregressive Speech Translation
Combining end-to-end speech translation (ST) and non-autoregressive (NAR) generation is promising in language and speech processing for their advantages of less error propagation and low latency.
PolyVoice: Language Models for Speech to Speech Translation
For the speech synthesis part, we adopt the existing VALL-E X approach and build a unit-based audio language model.
MOSPC: MOS Prediction Based on Pairwise Comparison
And our framework also surpasses the strong baseline in ranking accuracy on each fine-grained segment.
Recent Advances in Direct Speech-to-text Translation
Recently, speech-to-text translation has attracted more and more attention and many studies have emerged rapidly.
RepCodec: A Speech Representation Codec for Speech Tokenization
To improve the performance of these discrete speech tokens, we present RepCodec, a novel speech representation codec for semantic speech tokenization.
Speech Translation with Large Language Models: An Industrial Practice
Given the great success of large language models (LLMs) across various tasks, in this paper, we introduce LLM-ST, a novel and effective speech translation model constructed upon a pre-trained LLM.
