Search Results for author:
Found 36 papers, 12 papers with code
ESPnet How2 Speech Translation System for IWSLT 2019: Pre-training, Knowledge Distillation, and Going Deeper
In this year, we mainly build our systems based on Transformer architectures in all tasks and focus on the end-to-end speech translation (E2E-ST).
The JHU/KyotoU Speech Translation System for IWSLT 2018
This paper describes the Johns Hopkins University (JHU) and Kyoto University submissions to the Speech Translation evaluation campaign at IWSLT2018.
Seamless: Multilingual Expressive and Streaming Speech Translation
In this work, we introduce a family of models that enable end-to-end expressive and multilingual translations in a streaming fashion.
Efficient Monotonic Multihead Attention
We introduce the Efficient Monotonic Multihead Attention (EMMA), a state-of-the-art simultaneous translation model with numerically-stable and unbiased monotonic alignment estimation.
SeamlessM4T: Massively Multilingual & Multimodal Machine Translation
What does it take to create the Babel Fish, a tool that can help individuals translate speech between any two languages?
Automatic Speech Recognition
Speech-to-Speech Translation
+3
Hybrid Transducer and Attention based Encoder-Decoder Modeling for Speech-to-Text Tasks
In order to leverage strengths of both modeling methods, we propose a solution by combining Transducer and Attention based Encoder-Decoder (TAED) for speech-to-text tasks.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
ESPnet-ST-v2: Multipurpose Spoken Language Translation Toolkit
ESPnet-ST-v2 is a revamp of the open-source ESPnet-ST toolkit necessitated by the broadening interests of the spoken language translation community.
Enhancing Speech-to-Speech Translation with Multiple TTS Targets
It has been known that direct speech-to-speech translation (S2ST) models usually suffer from the data scarcity issue because of the limited existing parallel materials for both source and target speech.
UnitY: Two-pass Direct Speech-to-speech Translation with Discrete Units
We enhance the model performance by subword prediction in the first-pass decoder, advanced two-pass decoder architecture design and search strategy, and better training regularization.
Speech-to-Speech Translation For A Real-world Unwritten Language
We use English-Taiwanese Hokkien as a case study, and present an end-to-end solution from training data collection, modeling choices to benchmark dataset release.
Named Entity Detection and Injection for Direct Speech Translation
In a sentence, certain words are critical for its semantic.
Simple and Effective Unsupervised Speech Translation
The amount of labeled data to train models for speech tasks is limited for most languages, however, the data scarcity is exacerbated for speech translation which requires labeled data covering two different languages.
Non-autoregressive Error Correction for CTC-based ASR with Phone-conditioned Masked LM
Connectionist temporal classification (CTC) -based models are attractive in automatic speech recognition (ASR) because of their non-autoregressive nature.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Distilling the Knowledge of BERT for CTC-based ASR
In this study, we propose to distill the knowledge of BERT for CTC-based ASR, extending our previous study for attention-based ASR.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
A Study of Transducer based End-to-End ASR with ESPnet: Architecture, Auxiliary Loss and Decoding Strategies
In this study, we present recent developments of models trained with the RNN-T loss in ESPnet.
A Comparative Study on Non-Autoregressive Modelings for Speech-to-Text Generation
Non-autoregressive (NAR) models simultaneously generate multiple outputs in a sequence, which significantly reduces the inference speed at the cost of accuracy drop compared to autoregressive baselines.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
ASR Rescoring and Confidence Estimation with ELECTRA
We propose an ASR rescoring method for directly detecting errors with ELECTRA, which is originally a pre-training method for NLP tasks.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Fast-MD: Fast Multi-Decoder End-to-End Speech Translation with Non-Autoregressive Hidden Intermediates
We propose Fast-MD, a fast MD model that generates HI by non-autoregressive (NAR) decoding based on connectionist temporal classification (CTC) outputs followed by an ASR decoder.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Non-autoregressive End-to-end Speech Translation with Parallel Autoregressive Rescoring
We propose a unified NAR E2E-ST framework called Orthros, which has an NAR decoder and an auxiliary shallow AR decoder on top of the shared encoder.
VAD-free Streaming Hybrid CTC/Attention ASR for Unsegmented Recording
In this work, we propose novel decoding algorithms to enable streaming automatic speech recognition (ASR) on unsegmented long-form recordings without voice activity detection (VAD), based on monotonic chunkwise attention (MoChA) with an auxiliary connectionist temporal classification (CTC) objective.
StableEmit: Selection Probability Discount for Reducing Emission Latency of Streaming Monotonic Attention ASR
Previous works tackled this problem by leveraging alignment information to control the timing to emit tokens during training.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
ESPnet-ST IWSLT 2021 Offline Speech Translation System
This year we made various efforts on training data, architecture, and audio segmentation.
Source and Target Bidirectional Knowledge Distillation for End-to-end Speech Translation
To leverage the full potential of the source language information, we propose backward SeqKD, SeqKD from a target-to-source backward NMT model.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+5
Alignment Knowledge Distillation for Online Streaming Attention-based Speech Recognition
We compare CTC-ST with several methods that distill alignment knowledge from a hybrid ASR system and show that the CTC-ST can achieve a comparable tradeoff of accuracy and latency without relying on external alignment information.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Improved Mask-CTC for Non-Autoregressive End-to-End ASR
While Mask-CTC achieves remarkably fast inference speed, its recognition performance falls behind that of conventional autoregressive (AR) systems.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Orthros: Non-autoregressive End-to-end Speech Translation with Dual-decoder
Fast inference speed is an important goal towards real-world deployment of speech translation (ST) systems.
Distilling the Knowledge of BERT for Sequence-to-Sequence ASR
Experimental evaluations show that our method significantly improves the ASR performance from the seq2seq baseline on the Corpus of Spontaneous Japanese (CSJ).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Enhancing Monotonic Multihead Attention for Streaming ASR
For streaming inference, all monotonic attention (MA) heads should learn proper alignments because the next token is not generated until all heads detect the corresponding token boundaries.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
CTC-synchronous Training for Monotonic Attention Model
Monotonic chunkwise attention (MoChA) has been studied for the online streaming automatic speech recognition (ASR) based on a sequence-to-sequence framework.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
End-to-end speech-to-dialog-act recognition
In the proposed model, the dialog act recognition network is conjunct with an acoustic-to-word ASR model at its latent layer before the softmax layer, which provides a distributed representation of word-level ASR decoding information.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
ESPnet-ST: All-in-One Speech Translation Toolkit
We present ESPnet-ST, which is designed for the quick development of speech-to-speech translation systems in a single framework.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Minimum Latency Training Strategies for Streaming Sequence-to-Sequence ASR
This leads to an inevitable latency during inference.
Multilingual End-to-End Speech Translation
In this paper, we propose a simple yet effective framework for multilingual end-to-end speech translation (ST), in which speech utterances in source languages are directly translated to the desired target languages with a universal sequence-to-sequence architecture.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Improving OOV Detection and Resolution with External Language Models in Acoustic-to-Word ASR
Moreover, the A2C model can be used to recover out-of-vocabulary (OOV) words that are not covered by the A2W model, but this requires accurate detection of OOV words.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
A Comparative Study on Transformer vs RNN in Speech Applications
Sequence-to-sequence models have been widely used in end-to-end speech processing, for example, automatic speech recognition (ASR), speech translation (ST), and text-to-speech (TTS).
Ranked #12 on
Speech Recognition
on AISHELL-1
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Transfer learning of language-independent end-to-end ASR with language model fusion
This work explores better adaptation methods to low-resource languages using an external language model (LM) under the framework of transfer learning.
