Search Results for author:
Found 113 papers, 48 papers with code
Non-Autoregressive Sequence Generation
Non-autoregressive sequence generation (NAR) attempts to generate the entire or partial output sequences in parallel to speed up the generation process and avoid potential issues (e. g., label bias, exposure bias) in autoregressive generation.
Machine Translation With Weakly Paired Bilingual Documents
Neural machine translation, which achieves near human-level performance in some languages, strongly relies on the availability of large amounts of parallel sentences, which hinders its applicability to low-resource language pairs.
ProphetChat: Enhancing Dialogue Generation with Simulation of Future Conversation
To enable the chatbot to foresee the dialogue future, we design a beam-search-like roll-out strategy for dialogue future simulation using a typical dialogue generation model and a dialogue selector.
TranSFormer: Slow-Fast Transformer for Machine Translation
Learning multiscale Transformer models has been evidenced as a viable approach to augmenting machine translation systems.
NAS-FM: Neural Architecture Search for Tunable and Interpretable Sound Synthesis based on Frequency Modulation
Since expert knowledge is hard to acquire, it hinders the flexibility to quickly design and tune digital synthesizers for diverse sounds.
DiffusionNER: Boundary Diffusion for Named Entity Recognition
In this paper, we propose DiffusionNER, which formulates the named entity recognition task as a boundary-denoising diffusion process and thus generates named entities from noisy spans.
GETMusic: Generating Any Music Tracks with a Unified Representation and Diffusion Framework
With separate tracks in GETScore and the non-autoregressive behavior of the model, GETMusic can explicitly control the generation of any target tracks from scratch or conditioning on source tracks.
CoMoSpeech: One-Step Speech and Singing Voice Synthesis via Consistency Model
In this paper, we propose a "Co"nsistency "Mo"del-based "Speech" synthesis method, CoMoSpeech, which achieve speech synthesis through a single diffusion sampling step while achieving high audio quality.
ResiDual: Transformer with Dual Residual Connections
In this paper, we propose ResiDual, a novel Transformer architecture with Pre-Post-LN (PPLN), which fuses the connections in Post-LN and Pre-LN together and inherits their advantages while avoids their limitations.
CLaMP: Contrastive Language-Music Pre-training for Cross-Modal Symbolic Music Information Retrieval
We introduce CLaMP: Contrastive Language-Music Pre-training, which learns cross-modal representations between natural language and symbolic music using a music encoder and a text encoder trained jointly with a contrastive loss.
NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizers
To enhance the zero-shot capability that is important to achieve diverse speech synthesis, we design a speech prompting mechanism to facilitate in-context learning in the diffusion model and the duration/pitch predictor.
AUDIT: Audio Editing by Following Instructions with Latent Diffusion Models
Recently, some diffusion-based methods achieved zero-shot audio editing by using a diffusion and denoising process conditioned on the text description of the output audio.
Ranked #1 on
Audio Generation
on AudioCaps
(FD metric, using extra
training data)
Improving Autoencoder-based Outlier Detection with Adjustable Probabilistic Reconstruction Error and Mean-shift Outlier Scoring
The MSS improved the performance of multiple autoencoder-based outlier detectors by an average of 20%.
HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face
Solving complicated AI tasks with different domains and modalities is a key step toward artificial general intelligence.
DAE-Talker: High Fidelity Speech-Driven Talking Face Generation with Diffusion Autoencoder
Additionally, we propose a novel method for generating continuous video frames with the DDIM image decoder trained on individual frames, eliminating the need for modelling the joint distribution of consecutive frames directly.
HiFace: High-Fidelity 3D Face Reconstruction by Learning Static and Dynamic Details
3D Morphable Models (3DMMs) demonstrate great potential for reconstructing faithful and animatable 3D facial surfaces from a single image.
FoundationTTS: Text-to-Speech for ASR Customization with Generative Language Model
Neural text-to-speech (TTS) generally consists of cascaded architecture with separately optimized acoustic model and vocoder, or end-to-end architecture with continuous mel-spectrograms or self-extracted speech frames as the intermediate representations to bridge acoustic model and vocoder, which suffers from two limitations: 1) the continuous acoustic frames are hard to predict with phoneme only, and acoustic information like duration or pitch is also needed to solve the one-to-many problem, which is not easy to scale on large scale and noise datasets; 2) to achieve diverse speech output based on continuous speech features, complex VAE or flow-based models are usually required.
A Study on ReLU and Softmax in Transformer
This paper sheds light on the following points: 1) Softmax and ReLU use different normalization methods over elements which lead to different variances of results, and ReLU is good at dealing with a large number of key-value slots; 2) FFN and key-value memory are equivalent, and thus the Transformer can be viewed as a memory network where FFNs and self-attention networks are both key-value memories.
N-Gram Nearest Neighbor Machine Translation
Nearest neighbor machine translation augments the Autoregressive Translation~(AT) with $k$-nearest-neighbor retrieval, by comparing the similarity between the token-level context representations of the target tokens in the query and the datastore.
ERA-Solver: Error-Robust Adams Solver for Fast Sampling of Diffusion Probabilistic Models
Different from the traditional predictor based on explicit Adams methods, we leverage a Lagrange interpolation function as the predictor, which is further enhanced with an error-robust strategy to adaptively select the Lagrange bases with lower error in the estimated noise.
Regeneration Learning: A Learning Paradigm for Data Generation
Regeneration learning extends the concept of representation learning to data generation tasks, and can be regarded as a counterpart of traditional representation learning, since 1) regeneration learning handles the abstraction (Y') of the target data Y for data generation while traditional representation learning handles the abstraction (X') of source data X for data understanding; 2) both the processes of Y'-->Y in regeneration learning and X-->X' in representation learning can be learned in a self-supervised way (e. g., pre-training); 3) both the mappings from X to Y' in regeneration learning and from X' to Y in representation learning are simpler than the direct mapping from X to Y.
WuYun: Exploring hierarchical skeleton-guided melody generation using knowledge-enhanced deep learning
Although deep learning has revolutionized music generation, existing methods for structured melody generation follow an end-to-end left-to-right note-by-note generative paradigm and treat each note equally.
ResGrad: Residual Denoising Diffusion Probabilistic Models for Text to Speech
Denoising Diffusion Probabilistic Models (DDPMs) are emerging in text-to-speech (TTS) synthesis because of their strong capability of generating high-fidelity samples.
Difformer: Empowering Diffusion Models on the Embedding Space for Text Generation
Diffusion models have achieved state-of-the-art synthesis quality on both visual and audio tasks, and recent works further adapt them to textual data by diffusing on the embedding space.
Memories are One-to-Many Mapping Alleviators in Talking Face Generation
More specifically, the implicit memory is employed in the audio-to-expression model to capture high-level semantics in the audio-expression shared space, while the explicit memory is employed in the neural-rendering model to help synthesize pixel-level details.
SoftCorrect: Error Correction with Soft Detection for Automatic Speech Recognition
In this paper, we propose SoftCorrect with a soft error detection mechanism to avoid the limitations of both explicit and implicit error detection.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
VideoDubber: Machine Translation with Speech-Aware Length Control for Video Dubbing
In this paper, we propose a machine translation system tailored for the task of video dubbing, which directly considers the speech duration of each token in translation, to match the length of source and target speech.
Mask the Correct Tokens: An Embarrassingly Simple Approach for Error Correction
Since the error rate of the incorrect sentence is usually low (e. g., 10\%), the correction model can only learn to correct on limited error tokens but trivially copy on most tokens (correct tokens), which harms the effective training of error correction.
PromptTTS: Controllable Text-to-Speech with Text Descriptions
Thus, we develop a text-to-speech (TTS) system (dubbed as PromptTTS) that takes a prompt with both style and content descriptions as input to synthesize the corresponding speech.
Towards Understanding Omission in Dialogue Summarization
By analyzing this dataset, we find that a large improvement in summarization quality can be achieved by providing ground-truth omission labels for the summarization model to recover omission information, which demonstrates the importance of omission detection for omission mitigation in dialogue summarization.
Museformer: Transformer with Fine- and Coarse-Grained Attention for Music Generation
A recent trend is to use Transformer or its variants in music generation, which is, however, suboptimal, because the full attention cannot efficiently model the typically long music sequences (e. g., over 10, 000 tokens), and the existing models have shortcomings in generating musical repetition structures.
MeloForm: Generating Melody with Musical Form based on Expert Systems and Neural Networks
Specifically, 1) we design an expert system to generate a melody by developing musical elements from motifs to phrases then to sections with repetitions and variations according to pre-given musical form; 2) considering the generated melody is lack of musical richness, we design a Transformer based refinement model to improve the melody without changing its musical form.
StableFace: Analyzing and Improving Motion Stability for Talking Face Generation
In this paper, we conduct systematic analyses on the motion jittering problem based on a state-of-the-art pipeline that uses 3D face representations to bridge the input audio and output video, and improve the motion stability with a series of effective designs.
Re-creation of Creations: A New Paradigm for Lyric-to-Melody Generation
These characteristics cannot be well handled by neural generation models that learn lyric-to-melody mapping in an end-to-end way, due to several issues: (1) lack of aligned lyric-melody training data to sufficiently learn lyric-melody feature alignment; (2) lack of controllability in generation to better and explicitly align the lyric-melody features.
A Study of Syntactic Multi-Modality in Non-Autoregressive Machine Translation
Furthermore, we take the best of both and design a new loss function to better handle the complicated syntactic multi-modality in real-world datasets.
BinauralGrad: A Two-Stage Conditional Diffusion Probabilistic Model for Binaural Audio Synthesis
Combining this novel perspective of two-stage synthesis with advanced generative models (i. e., the diffusion models), the proposed BinauralGrad is able to generate accurate and high-fidelity binaural audio samples.
Transcormer: Transformer for Sentence Scoring with Sliding Language Modeling
Previous works on sentence scoring mainly adopted either causal language modeling (CLM) like GPT or masked language modeling (MLM) like BERT, which have some limitations: 1) CLM only utilizes unidirectional information for the probability estimation of a sentence without considering bidirectional context, which affects the scoring quality; 2) MLM can only estimate the probability of partial tokens at a time and thus requires multiple forward passes to estimate the probability of the whole sentence, which incurs large computation and time cost.
NaturalSpeech: End-to-End Text to Speech Synthesis with Human-Level Quality
In this paper, we answer these questions by first defining the human-level quality based on the statistical significance of subjective measure and introducing appropriate guidelines to judge it, and then developing a TTS system called NaturalSpeech that achieves human-level quality on a benchmark dataset.
Ranked #1 on
Text-To-Speech Synthesis
on LJSpeech
AdaSpeech 4: Adaptive Text to Speech in Zero-Shot Scenarios
We model the speaker characteristics systematically to improve the generalization on new speakers.
Mixed-Phoneme BERT: Improving BERT with Mixed Phoneme and Sup-Phoneme Representations for Text to Speech
However, the works apply pre-training with character-based units to enhance the TTS phoneme encoder, which is inconsistent with the TTS fine-tuning that takes phonemes as input.
Revisiting Over-Smoothness in Text to Speech
Then we conduct a comprehensive study on NAR-TTS models that use some advanced modeling methods.
InferGrad: Improving Diffusion Models for Vocoder by Considering Inference in Training
In this paper, we propose InferGrad, a diffusion model for vocoder that incorporates inference process into training, to reduce the inference iterations while maintaining high generation quality.
Speech-T: Transducer for Text to Speech and Beyond
Experiments on LJSpeech datasets demonstrate that Speech-T 1) is more robust than the attention based autoregressive TTS model due to its inherent monotonic alignments between text and speech; 2) naturally supports streaming TTS with good voice quality; and 3) enjoys the benefit of joint modeling TTS and ASR in a single network.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
DelightfulTTS: The Microsoft Speech Synthesis System for Blizzard Challenge 2021
The goal of this challenge is to synthesize natural and high-quality speech from text, and we approach this goal in two perspectives: The first is to directly model and generate waveform in 48 kHz sampling rate, which brings higher perception quality than previous systems with 16 kHz or 24 kHz sampling rate; The second is to model the variation information in speech through a systematic design, which improves the prosody and naturalness.
A study on the efficacy of model pre-training in developing neural text-to-speech system
However, in terms of ultimately achieved system performance for target speaker(s), the actual benefits of model pre-training are uncertain and unstable, depending very much on the quantity and text content of training data.
FastCorrect 2: Fast Error Correction on Multiple Candidates for Automatic Speech Recognition
Although multiple candidates are generated by an ASR system through beam search, current error correction approaches can only correct one sentence at a time, failing to leverage the voting effect from multiple candidates to better detect and correct error tokens.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
TeleMelody: Lyric-to-Melody Generation with a Template-Based Two-Stage Method
In this paper, we develop TeleMelody, a two-stage lyric-to-melody generation system with music template (e. g., tonality, chord progression, rhythm pattern, and cadence) to bridge the gap between lyrics and melodies (i. e., the system consists of a lyric-to-template module and a template-to-melody module).
PDAugment: Data Augmentation by Pitch and Duration Adjustments for Automatic Lyrics Transcription
Considering that there is a large amount of ASR training data, a straightforward method is to leverage ASR data to enhance ALT training.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Analyzing and Mitigating Interference in Neural Architecture Search
In this paper, we investigate the interference issue by sampling different child models and calculating the gradient similarity of shared operators, and observe: 1) the interference on a shared operator between two child models is positively correlated with the number of different operators; 2) the interference is smaller when the inputs and outputs of the shared operator are more similar.
A Survey on Low-Resource Neural Machine Translation
Neural approaches have achieved state-of-the-art accuracy on machine translation but suffer from the high cost of collecting large scale parallel data.
AdaSpeech 3: Adaptive Text to Speech for Spontaneous Style
While recent text to speech (TTS) models perform very well in synthesizing reading-style (e. g., audiobook) speech, it is still challenging to synthesize spontaneous-style speech (e. g., podcast or conversation), mainly because of two reasons: 1) the lack of training data for spontaneous speech; 2) the difficulty in modeling the filled pauses (um and uh) and diverse rhythms in spontaneous speech.
DeepRapper: Neural Rap Generation with Rhyme and Rhythm Modeling
In this paper, we develop DeepRapper, a Transformer-based rap generation system that can model both rhymes and rhythms.
A Survey on Neural Speech Synthesis
Text to speech (TTS), or speech synthesis, which aims to synthesize intelligible and natural speech given text, is a hot research topic in speech, language, and machine learning communities and has broad applications in the industry.
PriorGrad: Improving Conditional Denoising Diffusion Models with Data-Dependent Adaptive Prior
Denoising diffusion probabilistic models have been recently proposed to generate high-quality samples by estimating the gradient of the data density.
MusicBERT: Symbolic Music Understanding with Large-Scale Pre-Training
Inspired by the success of pre-training models in natural language processing, in this paper, we develop MusicBERT, a large-scale pre-trained model for music understanding.
NAS-BERT: Task-Agnostic and Adaptive-Size BERT Compression with Neural Architecture Search
The technical challenge of NAS-BERT is that training a big supernet on the pre-training task is extremely costly.
FastCorrect: Fast Error Correction with Edit Alignment for Automatic Speech Recognition
A straightforward solution to reduce latency, inspired by non-autoregressive (NAR) neural machine translation, is to use an NAR sequence generation model for ASR error correction, which, however, comes at the cost of significantly increased ASR error rate.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
A Rational Inattention Theory of Echo Chamber
Finite players allocate limited attention capacities between biased primary sources and the other players in order to gather information about an uncertain state.
AdaSpeech 2: Adaptive Text to Speech with Untranscribed Data
In adaptation, we use untranscribed speech data for speech reconstruction and only fine-tune the TTS decoder.
Cross-domain Speech Recognition with Unsupervised Character-level Distribution Matching
End-to-end automatic speech recognition (ASR) can achieve promising performance with large-scale training data.
Ranked #1 on
Cross-environment ASR
on Libri-Adapt
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Adaptive Logit Adjustment Loss for Long-Tailed Visual Recognition
The adaptive adjusting term is composed of two complementary factors: 1) quantity factor, which pays more attention to tail classes, and 2) difficulty factor, which adaptively pays more attention to hard instances in the training process.
Libri-adhoc40: A dataset collected from synchronized ad-hoc microphone arrays
We trained several multi-device speech recognition systems on both the Libri-adhoc40 dataset and a simulated dataset.
AdaSpeech: Adaptive Text to Speech for Custom Voice
2) To better trade off the adaptation parameters and voice quality, we introduce conditional layer normalization in the mel-spectrogram decoder of AdaSpeech, and fine-tune this part in addition to speaker embedding for adaptation.
MixSpeech: Data Augmentation for Low-resource Automatic Speech Recognition
In this paper, we propose MixSpeech, a simple yet effective data augmentation method based on mixup for automatic speech recognition (ASR).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstruction
To further employ the power of quantization, the mixed precision technique is incorporated in our framework by approximating the inter-layer and intra-layer sensitivity.
LightSpeech: Lightweight and Fast Text to Speech with Neural Architecture Search
Text to speech (TTS) has been broadly used to synthesize natural and intelligible speech in different scenarios.
Task-Agnostic and Adaptive-Size BERT Compression
NAS-BERT trains a big supernet on a carefully designed search space containing various architectures and outputs multiple compressed models with adaptive sizes and latency.
Denoising Text to Speech with Frame-Level Noise Modeling
In DenoiSpeech, we handle real-world noisy speech by modeling the fine-grained frame-level noise with a noise condition module, which is jointly trained with the TTS model.
SongMASS: Automatic Song Writing with Pre-training and Alignment Constraint
Automatic song writing aims to compose a song (lyric and/or melody) by machine, which is an interesting topic in both academia and industry.
Speech enhancement aided end-to-end multi-task learning for voice activity detection
Recent studies show that speech enhancement is helpful to VAD, but the performance improvement is limited.
HiFiSinger: Towards High-Fidelity Neural Singing Voice Synthesis
To tackle the difficulty of singing modeling caused by high sampling rate (wider frequency band and longer waveform), we introduce multi-scale adversarial training in both the acoustic model and vocoder to improve singing modeling.
PopMAG: Pop Music Accompaniment Generation
To improve harmony, in this paper, we propose a novel MUlti-track MIDI representation (MuMIDI), which enables simultaneous multi-track generation in a single sequence and explicitly models the dependency of the notes from different tracks.
LRSpeech: Extremely Low-Resource Speech Synthesis and Recognition
However, there are more than 6, 000 languages in the world and most languages are lack of speech training data, which poses significant challenges when building TTS and ASR systems for extremely low-resource languages.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Neural Machine Translation with Error Correction
Neural machine translation (NMT) generates the next target token given as input the previous ground truth target tokens during training while the previous generated target tokens during inference, which causes discrepancy between training and inference as well as error propagation, and affects the translation accuracy.
Task-Level Curriculum Learning for Non-Autoregressive Neural Machine Translation
SAT contains a hyperparameter k, and each k value defines a SAT task with different degrees of parallelism.
Accuracy Prediction with Non-neural Model for Neural Architecture Search
Considering that most architectures are represented as sequences of discrete symbols which are more like tabular data and preferred by non-neural predictors, in this paper, we study an alternative approach which uses non-neural model for accuracy prediction.
Ranked #79 on
Neural Architecture Search
on ImageNet
DeepSinger: Singing Voice Synthesis with Data Mined From the Web
DeepSinger has several advantages over previous SVS systems: 1) to the best of our knowledge, it is the first SVS system that directly mines training data from music websites, 2) the lyrics-to-singing alignment model further avoids any human efforts for alignment labeling and greatly reduces labeling cost, 3) the singing model based on a feed-forward Transformer is simple and efficient, by removing the complicated acoustic feature modeling in parametric synthesis and leveraging a reference encoder to capture the timbre of a singer from noisy singing data, and 4) it can synthesize singing voices in multiple languages and multiple singers.
SimulSpeech: End-to-End Simultaneous Speech to Text Translation
In this work, we develop SimulSpeech, an end-to-end simultaneous speech to text translation system which translates speech in source language to text in target language concurrently.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+6
UWSpeech: Speech to Speech Translation for Unwritten Languages
Existing speech to speech translation systems heavily rely on the text of target language: they usually translate source language either to target text and then synthesize target speech from text, or directly to target speech with target text for auxiliary training.
XiaoiceSing: A High-Quality and Integrated Singing Voice Synthesis System
This paper presents XiaoiceSing, a high-quality singing voice synthesis system which employs an integrated network for spectrum, F0 and duration modeling.
MultiSpeech: Multi-Speaker Text to Speech with Transformer
Transformer-based text to speech (TTS) model (e. g., Transformer TTS~\cite{li2019neural}, FastSpeech~\cite{ren2019fastspeech}) has shown the advantages of training and inference efficiency over RNN-based model (e. g., Tacotron~\cite{shen2018natural}) due to its parallel computation in training and/or inference.
FastSpeech 2: Fast and High-Quality End-to-End Text to Speech
In this paper, we propose FastSpeech 2, which addresses the issues in FastSpeech and better solves the one-to-many mapping problem in TTS by 1) directly training the model with ground-truth target instead of the simplified output from teacher, and 2) introducing more variation information of speech (e. g., pitch, energy and more accurate duration) as conditional inputs.
Ranked #6 on
Text-To-Speech Synthesis
on LJSpeech
(using extra training data)
LightPAFF: A Two-Stage Distillation Framework for Pre-training and Fine-tuning
While pre-training and fine-tuning, e. g., BERT~\citep{devlin2018bert}, GPT-2~\citep{radford2019language}, have achieved great success in language understanding and generation tasks, the pre-trained models are usually too big for online deployment in terms of both memory cost and inference speed, which hinders them from practical online usage.
A Study of Non-autoregressive Model for Sequence Generation
In this work, we conduct a study to understand the difficulty of NAR sequence generation and try to answer: (1) Why NAR models can catch up with AR models in some tasks but not all?
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
MPNet: Masked and Permuted Pre-training for Language Understanding
Since BERT neglects dependency among predicted tokens, XLNet introduces permuted language modeling (PLM) for pre-training to address this problem.
VESR-Net: The Winning Solution to Youku Video Enhancement and Super-Resolution Challenge
This paper introduces VESR-Net, a method for video enhancement and super-resolution (VESR).
Semi-Supervised Neural Architecture Search
On ImageNet, it achieves 23. 5% top-1 error rate (under 600M FLOPS constraint) using 4 GPU-days for search.
Ranked #79 on
Neural Architecture Search
on ImageNet
A Study of Multilingual Neural Machine Translation
Multilingual neural machine translation (NMT) has recently been investigated from different aspects (e. g., pivot translation, zero-shot translation, fine-tuning, or training from scratch) and in different settings (e. g., rich resource and low resource, one-to-many, and many-to-one translation).
Fine-Tuning by Curriculum Learning for Non-Autoregressive Neural Machine Translation
Non-autoregressive translation (NAT) models remove the dependence on previous target tokens and generate all target tokens in parallel, resulting in significant inference speedup but at the cost of inferior translation accuracy compared to autoregressive translation (AT) models.
Microsoft Research Asia's Systems for WMT19
We Microsoft Research Asia made submissions to 11 language directions in the WMT19 news translation tasks.
ESPnet-TTS: Unified, Reproducible, and Integratable Open Source End-to-End Text-to-Speech Toolkit
Furthermore, the unified design enables the integration of ASR functions with TTS, e. g., ASR-based objective evaluation and semi-supervised learning with both ASR and TTS models.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Efficient Bidirectional Neural Machine Translation
In this paper, we propose an efficient method to generate a sequence in both left-to-right and right-to-left manners using a single encoder and decoder, combining the advantages of both generation directions.
Multilingual Neural Machine Translation with Language Clustering
We study two methods for language clustering: (1) using prior knowledge, where we cluster languages according to language family, and (2) using language embedding, in which we represent each language by an embedding vector and cluster them in the embedding space.
Language Graph Distillation for Low-Resource Machine Translation
Neural machine translation on low-resource language is challenging due to the lack of bilingual sentence pairs.
Hard but Robust, Easy but Sensitive: How Encoder and Decoder Perform in Neural Machine Translation
Neural machine translation (NMT) typically adopts the encoder-decoder framework.
Representation Degeneration Problem in Training Natural Language Generation Models
We study an interesting problem in training neural network-based models for natural language generation tasks, which we call the \emph{representation degeneration problem}.
Unsupervised Pivot Translation for Distant Languages
In this work, we introduce unsupervised pivot translation for distant languages, which translates a language to a distant language through multiple hops, and the unsupervised translation on each hop is relatively easier than the original direct translation.
FastSpeech: Fast, Robust and Controllable Text to Speech
In this work, we propose a novel feed-forward network based on Transformer to generate mel-spectrogram in parallel for TTS.
Ranked #10 on
Text-To-Speech Synthesis
on LJSpeech
(using extra training data)
FastSpeech: Fast,Robustand Controllable Text-to-Speech
Compared with traditional concatenative and statistical parametric approaches, neural network based end-to-end models suffer from slow inference speed, and the synthesized speech is usually not robust (i. e., some words are skipped or repeated) and lack of controllability (voice speed or prosody control).
Almost Unsupervised Text to Speech and Automatic Speech Recognition
Text to speech (TTS) and automatic speech recognition (ASR) are two dual tasks in speech processing and both achieve impressive performance thanks to the recent advance in deep learning and large amount of aligned speech and text data.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
MASS: Masked Sequence to Sequence Pre-training for Language Generation
Pre-training and fine-tuning, e. g., BERT, have achieved great success in language understanding by transferring knowledge from rich-resource pre-training task to the low/zero-resource downstream tasks.
Token-Level Ensemble Distillation for Grapheme-to-Phoneme Conversion
Recently, G2P conversion is viewed as a sequence to sequence task and modeled by RNN or CNN based encoder-decoder framework.
Ranked #1 on
Text-To-Speech Synthesis
on CMUDict 0.7b
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Multilingual Neural Machine Translation with Knowledge Distillation
Multilingual machine translation, which translates multiple languages with a single model, has attracted much attention due to its efficiency of offline training and online serving.
Non-Autoregressive Neural Machine Translation with Enhanced Decoder Input
Non-autoregressive translation (NAT) models, which remove the dependence on previous target tokens from the inputs of the decoder, achieve significantly inference speedup but at the cost of inferior accuracy compared to autoregressive translation (AT) models.
Sentence-wise Smooth Regularization for Sequence to Sequence Learning
Maximum-likelihood estimation (MLE) is widely used in sequence to sequence tasks for model training.
Layer-Wise Coordination between Encoder and Decoder for Neural Machine Translation
Neural Machine Translation (NMT) has achieved remarkable progress with the quick evolvement of model structures.
Hybrid Self-Attention Network for Machine Translation
The encoder-decoder is the typical framework for Neural Machine Translation (NMT), and different structures have been developed for improving the translation performance.
FRAGE: Frequency-Agnostic Word Representation
Continuous word representation (aka word embedding) is a basic building block in many neural network-based models used in natural language processing tasks.
Ranked #3 on
Machine Translation
on IWSLT2015 German-English
Beyond Error Propagation in Neural Machine Translation: Characteristics of Language Also Matter
Many previous works have discussed the relationship between error propagation and the \emph{accuracy drop} (i. e., the left part of the translated sentence is often better than its right part in left-to-right decoding models) problem.
Model-Level Dual Learning
Many artificial intelligence tasks appear in dual forms like English$\leftrightarrow$French translation and speech$\leftrightarrow$text transformation.
Double Path Networks for Sequence to Sequence Learning
In this work we propose Double Path Networks for Sequence to Sequence learning (DPN-S2S), which leverage the advantages of both models by using double path information fusion.
Dense Information Flow for Neural Machine Translation
Recently, neural machine translation has achieved remarkable progress by introducing well-designed deep neural networks into its encoder-decoder framework.
Ranked #66 on
Machine Translation
on WMT2014 English-German
Achieving Human Parity on Automatic Chinese to English News Translation
Machine translation has made rapid advances in recent years.
Ranked #3 on
Machine Translation
on WMT 2017 English-Chinese
