Search Results for author:
Found 19 papers, 2 papers with code
Lego-Features: Exporting modular encoder features for streaming and deliberation ASR
Overall, they present a modular, powerful and cheap alternative to the standard encoder output, as well as the N-best hypotheses.
Towards Computationally Verifiable Semantic Grounding for Language Models
Compared to a baseline that generates text using greedy search, we demonstrate two techniques that improve the fluency and semantic accuracy of the generated text: The first technique samples multiple candidate text sequences from which the semantic parser chooses.
Scaling Laws for Neural Machine Translation
We show that cross-entropy loss as a function of model size follows a certain scaling law.
Data Troubles in Sentence Level Confidence Estimation for Machine Translation
For an English-Spanish translation model operating at $SACC = 0. 89$ according to a non-expert annotator pool we can derive a confidence estimate that labels 0. 5-0. 6 of the $good$ translations in an "in-domain" test set with 0. 95 Precision.
Multi-Stage Influence Function
With this score, we can identify the pretraining examples in the pretraining task that contribute most to a prediction in the finetuning task.
Practical Perspectives on Quality Estimation for Machine Translation
Sentence level quality estimation (QE) for machine translation (MT) attempts to predict the translation edit rate (TER) cost of post-editing work required to correct MT output.
Faster Transformer Decoding: N-gram Masked Self-Attention
Motivated by the fact that most of the information relevant to the prediction of target tokens is drawn from the source sentence $S=s_1, \ldots, s_S$, we propose truncating the target-side window used for computing self-attention by making an $N$-gram assumption.
Dynamically Composing Domain-Data Selection with Clean-Data Selection by ``Co-Curricular Learning'' for Neural Machine Translation
Noise and domain are important aspects of data quality for neural machine translation.
Tagged Back-Translation
Recent work in Neural Machine Translation (NMT) has shown significant quality gains from noised-beam decoding during back-translation, a method to generate synthetic parallel data.
Dynamically Composing Domain-Data Selection with Clean-Data Selection by "Co-Curricular Learning" for Neural Machine Translation
Noise and domain are important aspects of data quality for neural machine translation.
Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modeling
Lingvo is a Tensorflow framework offering a complete solution for collaborative deep learning research, with a particular focus towards sequence-to-sequence models.
Denoising Neural Machine Translation Training with Trusted Data and Online Data Selection
Measuring domain relevance of data and identifying or selecting well-fit domain data for machine translation (MT) is a well-studied topic, but denoising is not yet.
GroupReduce: Block-Wise Low-Rank Approximation for Neural Language Model Shrinking
Model compression is essential for serving large deep neural nets on devices with limited resources or applications that require real-time responses.
N-gram Language Modeling using Recurrent Neural Network Estimation
Experiments on a small corpus (UPenn Treebank, one million words of training data and 10k vocabulary) have found the LSTM cell with dropout to be the best model for encoding the $n$-gram state when compared with feed-forward and vanilla RNN models.
Sparse Non-negative Matrix Language Modeling
We evaluate SNM language models on two corpora: the One Billion Word Benchmark and a subset of the LDC English Gigaword corpus.
Multinomial Loss on Held-out Data for the Sparse Non-negative Matrix Language Model
In experiments on the one billion words language modeling benchmark, we are able to slightly improve on our previous results which use a different loss function, and employ leave-one-out training on a subset of the main training set.
Skip-gram Language Modeling Using Sparse Non-negative Matrix Probability Estimation
We present a novel family of language model (LM) estimation techniques named Sparse Non-negative Matrix (SNM) estimation.
Ranked #23 on
Language Modelling
on One Billion Word
One Billion Word Benchmark for Measuring Progress in Statistical Language Modeling
We propose a new benchmark corpus to be used for measuring progress in statistical language modeling.
Ranked #22 on
Language Modelling
on One Billion Word
