Search Results for author: Long Zhou

Found 26 papers, 9 papers with code

LightHuBERT: Lightweight and Configurable Speech Representation Learning with Once-for-All Hidden-Unit BERT

1 code implementation29 Mar 2022 Rui Wang, Qibing Bai, Junyi Ao, Long Zhou, Zhixiang Xiong, Zhihua Wei, Yu Zhang, Tom Ko, Haizhou Li

LightHuBERT outperforms the original HuBERT on ASR and five SUPERB tasks with the HuBERT size, achieves comparable performance to the teacher model in most tasks with a reduction of 29% parameters, and obtains a $3. 5\times$ compression ratio in three SUPERB tasks, e. g., automatic speaker verification, keyword spotting, and intent classification, with a slight accuracy loss.

Automatic Speech Recognition Intent Classification +3

SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processing

1 code implementation ACL 2022 Junyi Ao, Rui Wang, Long Zhou, Chengyi Wang, Shuo Ren, Yu Wu, Shujie Liu, Tom Ko, Qing Li, Yu Zhang, Zhihua Wei, Yao Qian, Jinyu Li, Furu Wei

Motivated by the success of T5 (Text-To-Text Transfer Transformer) in pre-trained natural language processing models, we propose a unified-modal SpeechT5 framework that explores the encoder-decoder pre-training for self-supervised speech/text representation learning.

Automatic Speech Recognition Quantization +5

Multi-View Self-Attention Based Transformer for Speaker Recognition

no code implementations11 Oct 2021 Rui Wang, Junyi Ao, Long Zhou, Shujie Liu, Zhihua Wei, Tom Ko, Qing Li, Yu Zhang

In this work, we propose a novel multi-view self-attention mechanism and present an empirical study of different Transformer variants with or without the proposed attention mechanism for speaker recognition.

Speaker Recognition

SemFace: Pre-training Encoder and Decoder with a Semantic Interface for Neural Machine Translation

no code implementations ACL 2021 Shuo Ren, Long Zhou, Shujie Liu, Furu Wei, Ming Zhou, Shuai Ma

While pre-training techniques are working very well in natural language processing, how to pre-train a decoder and effectively use it for neural machine translation (NMT) still remains a tricky issue.

Machine Translation Translation

A Configurable Multilingual Model is All You Need to Recognize All Languages

no code implementations13 Jul 2021 Long Zhou, Jinyu Li, Eric Sun, Shujie Liu

Particularly, a single CMM can be deployed to any user scenario where the users can pre-select any combination of languages.

Automatic Speech Recognition

CodeBLEU: a Method for Automatic Evaluation of Code Synthesis

no code implementations22 Sep 2020 Shuo Ren, Daya Guo, Shuai Lu, Long Zhou, Shujie Liu, Duyu Tang, Neel Sundaresan, Ming Zhou, Ambrosio Blanco, Shuai Ma

Evaluation metrics play a vital role in the growth of an area as it defines the standard of distinguishing between good and bad models.

Code Translation Translation

GraphCodeBERT: Pre-training Code Representations with Data Flow

1 code implementation ICLR 2021 Daya Guo, Shuo Ren, Shuai Lu, Zhangyin Feng, Duyu Tang, Shujie Liu, Long Zhou, Nan Duan, Alexey Svyatkovskiy, Shengyu Fu, Michele Tufano, Shao Kun Deng, Colin Clement, Dawn Drain, Neel Sundaresan, Jian Yin, Daxin Jiang, Ming Zhou

Instead of taking syntactic-level structure of code like abstract syntax tree (AST), we use data flow in the pre-training stage, which is a semantic-level structure of code that encodes the relation of "where-the-value-comes-from" between variables.

Clone Detection Code Completion +7

Improving Autoregressive NMT with Non-Autoregressive Model

no code implementations WS 2020 Long Zhou, Jiajun Zhang, Cheng-qing Zong

In this work, we propose a novel Encoder-NAD-AD framework for NMT, aiming at boosting AT with global information produced by NAT model.

Knowledge Distillation Machine Translation +1

Synchronous Speech Recognition and Speech-to-Text Translation with Interactive Decoding

no code implementations16 Dec 2019 Yuchen Liu, Jiajun Zhang, Hao Xiong, Long Zhou, Zhongjun He, Hua Wu, Haifeng Wang, Cheng-qing Zong

Speech-to-text translation (ST), which translates source language speech into target language text, has attracted intensive attention in recent years.

Automatic Speech Recognition Multi-Task Learning +2

Synchronously Generating Two Languages with Interactive Decoding

no code implementations IJCNLP 2019 Yining Wang, Jiajun Zhang, Long Zhou, Yuchen Liu, Cheng-qing Zong

In this paper, we introduce a novel interactive approach to translate a source language into two different languages simultaneously and interactively.

Machine Translation Translation

Sequence Generation: From Both Sides to the Middle

no code implementations23 Jun 2019 Long Zhou, Jiajun Zhang, Cheng-qing Zong, Heng Yu

The encoder-decoder framework has achieved promising process for many sequence generation tasks, such as neural machine translation and text summarization.

Machine Translation Text Summarization +1

Synchronous Bidirectional Neural Machine Translation

2 code implementations TACL 2019 Long Zhou, Jiajun Zhang, Cheng-qing Zong

In this paper, we introduce a synchronous bidirectional neural machine translation (SB-NMT) that predicts its outputs using left-to-right and right-to-left decoding simultaneously and interactively, in order to leverage both of the history and future information at the same time.

Machine Translation Translation

Synchronous Bidirectional Inference for Neural Sequence Generation

1 code implementation24 Feb 2019 Jiajun Zhang, Long Zhou, Yang Zhao, Cheng-qing Zong

In this work, we propose a synchronous bidirectional inference model to generate outputs using both left-to-right and right-to-left decoding simultaneously and interactively.

Abstractive Text Summarization Machine Translation +1

Language-Independent Representor for Neural Machine Translation

no code implementations1 Nov 2018 Long Zhou, Yuchen Liu, Jiajun Zhang, Cheng-qing Zong, Guoping Huang

Current Neural Machine Translation (NMT) employs a language-specific encoder to represent the source sentence and adopts a language-specific decoder to generate target translation.

Machine Translation Multi-Task Learning +1

Word, Subword or Character? An Empirical Study of Granularity in Chinese-English NMT

1 code implementation13 Nov 2017 Yining Wang, Long Zhou, Jiajun Zhang, Cheng-qing Zong

Our experiments show that subword model performs best for Chinese-to-English translation with the vocabulary which is not so big while hybrid word-character model is most suitable for English-to-Chinese translation.

Machine Translation Translation

Look-ahead Attention for Generation in Neural Machine Translation

no code implementations30 Aug 2017 Long Zhou, Jiajun Zhang, Cheng-qing Zong

The attention model has become a standard component in neural machine translation (NMT) and it guides translation process by selectively focusing on parts of the source sentence when predicting each target word.

Machine Translation Translation

Neural System Combination for Machine Translation

no code implementations ACL 2017 Long Zhou, Wenpeng Hu, Jiajun Zhang, Cheng-qing Zong

Neural machine translation (NMT) becomes a new approach to machine translation and generates much more fluent results compared to statistical machine translation (SMT).

Machine Translation Translation

Cannot find the paper you are looking for? You can Submit a new open access paper.