Search Results for author:
Found 20 papers, 7 papers with code
Leveraging Timestamp Information for Serialized Joint Streaming Recognition and Translation
The growing need for instant spoken language transcription and translation is driven by increased global communication and cross-lingual interactions.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Improving Stability in Simultaneous Speech Translation: A Revision-Controllable Decoding Approach
Simultaneous Speech-to-Text translation serves a critical role in real-time crosslingual communication.
DiariST: Streaming Speech Translation with Speaker Diarization
End-to-end speech translation (ST) for conversation recordings involves several under-explored challenges such as speaker diarization (SD) without accurate word time stamps and handling of overlapping speech in a streaming fashion.
Token-Level Serialized Output Training for Joint Streaming ASR and ST Leveraging Textual Alignments
In real-world applications, users often require both translations and transcriptions of speech to enhance their comprehension, particularly in streaming scenarios where incremental generation is necessary.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
ERNIE-SAT: Speech and Text Joint Pretraining for Cross-Lingual Multi-Speaker Text-to-Speech
In this paper, we extend the pretraining method for cross-lingual multi-speaker speech synthesis tasks, including cross-lingual multi-speaker voice cloning and cross-lingual multi-speaker speech editing.
Is Self-Supervised Learning More Robust Than Supervised Learning?
Our insight has implications in improving the downstream robustness of supervised learning.
PaddleSpeech: An Easy-to-Use All-in-One Speech Toolkit
PaddleSpeech is an open-source all-in-one speech toolkit.
Automatic Speech Recognition (ASR)
Environmental Sound Classification
+9
Data-Driven Adaptive Simultaneous Machine Translation
In simultaneous translation (SimulMT), the most widely used strategy is the wait-k policy thanks to its simplicity and effectiveness in balancing translation quality and latency.
A$^3$T: Alignment-Aware Acoustic and Text Pretraining for Speech Synthesis and Editing
Recently, speech representation learning has improved many speech-related tasks such as speech recognition, speech classification, and speech-to-text translation.
TorchDrug: A Powerful and Flexible Machine Learning Platform for Drug Discovery
However, lacking domain knowledge (e. g., which tasks to work on), standard benchmarks and data preprocessing pipelines are the main obstacles for machine learning researchers to work in this domain.
Direct Simultaneous Speech-to-Text Translation Assisted by Synchronized Streaming ASR
Simultaneous speech-to-text translation is widely useful in many scenarios.
Fused Acoustic and Text Encoding for Multimodal Bilingual Pretraining and Speech Translation
Recently, representation learning for text and speech has successfully improved many language related tasks.
MAM: Masked Acoustic Modeling for End-to-End Speech-to-Text Translation
End-to-end Speech-to-text Translation (E2E-ST), which directly translates source language speech to target language text, is widely useful in practice, but traditional cascaded approaches (ASR+MT) often suffer from error propagation in the pipeline.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Improving Simultaneous Translation by Incorporating Pseudo-References with Fewer Reorderings
Simultaneous translation is vastly different from full-sentence translation, in the sense that it starts translation before the source sentence ends, with only a few words delay.
RNNLogic: Learning Logic Rules for Reasoning on Knowledge Graphs
Then in the E-step, we select a set of high-quality rules from all generated rules with both the rule generator and reasoning predictor via posterior inference; and in the M-step, the rule generator is updated with the rules selected in the E-step.
DropAttention: A Regularization Method for Fully-Connected Self-Attention Networks
Variants dropout methods have been designed for the fully-connected layer, convolutional layer and recurrent layer in neural networks, and shown to be effective to avoid overfitting.
VATEX: A Large-Scale, High-Quality Multilingual Dataset for Video-and-Language Research
We also introduce two tasks for video-and-language research based on VATEX: (1) Multilingual Video Captioning, aimed at describing a video in various languages with a compact unified captioning model, and (2) Video-guided Machine Translation, to translate a source language description into the target language using the video information as additional spatiotemporal context.
Exploring Shared Structures and Hierarchies for Multiple NLP Tasks
Designing shared neural architecture plays an important role in multi-task learning.
Same Representation, Different Attentions: Shareable Sentence Representation Learning from Multiple Tasks
More specifically, all tasks share the same sentence representation and each task can select the task-specific information from the shared sentence representation with attention mechanism.
Meta Multi-Task Learning for Sequence Modeling
Specifically, we use a shared meta-network to capture the meta-knowledge of semantic composition and generate the parameters of the task-specific semantic composition models.
