Search Results for author:
Found 14 papers, 3 papers with code
Unifying Language Learning Paradigms
Our model also achieve strong results at in-context learning, outperforming 175B GPT-3 on zero-shot SuperGLUE and tripling the performance of T5-XXL on one-shot summarization.
Ranked #4 on
Long-range modeling
on SCROLLS
Building Machine Translation Systems for the Next Thousand Languages
In this paper we share findings from our effort to build practical machine translation (MT) systems capable of translating across over one thousand languages.
PaLM: Scaling Language Modeling with Pathways
To further our understanding of the impact of scale on few-shot learning, we trained a 540-billion parameter, densely activated, Transformer language model, which we call Pathways Language Model PaLM.
Ranked #1 on
Question Answering
on BoolQ
Scaling Up Models and Data with $\texttt{t5x}$ and $\texttt{seqio}$
Recent neural network-based language models have benefited greatly from scaling up the size of training datasets and the number of parameters in the models themselves.
Using natural language prompts for machine translation
We explore the use of natural language prompts for controlling various aspects of the outputs generated by machine translation models.
Examining Scaling and Transfer of Language Model Architectures for Machine Translation
Natural language understanding and generation models follow one of the two dominant architectural paradigms: language models (LMs) that process concatenated sequences in a single stack of layers, and encoder-decoder models (EncDec) that utilize separate layer stacks for input and output processing.
Towards the Next 1000 Languages in Multilingual Machine Translation: Exploring the Synergy Between Supervised and Self-Supervised Learning
While recent progress in massively multilingual MT is one step closer to reaching this goal, it is becoming evident that extending a multilingual MT system simply by training on more parallel data is unscalable, since the availability of labeled data for low-resource and non-English-centric language pairs is forbiddingly limited.
Few-shot Controllable Style Transfer for Low-Resource Multilingual Settings
When compared to prior work, our model achieves 2-3x better performance in formality transfer and code-mixing addition across seven languages.
Scaling Laws for Neural Machine Translation
We show that cross-entropy loss as a function of model size follows a certain scaling law.
Towards Universality in Multilingual Text Rewriting
In this work, we take the first steps towards building a universal rewriter: a model capable of rewriting text in any language to exhibit a wide variety of attributes, including styles and languages, while preserving as much of the original semantics as possible.
Towards Continual Learning for Multilingual Machine Translation via Vocabulary Substitution
We propose a straightforward vocabulary adaptation scheme to extend the language capacity of multilingual machine translation models, paving the way towards efficient continual learning for multilingual machine translation.
Harnessing Multilinguality in Unsupervised Machine Translation for Rare Languages
We outperform all current state-of-the-art unsupervised baselines for these languages, achieving gains of up to 14. 4 BLEU.
Machine learning applied in the multi-scale 3D stress modelling
One low-frequency solution is obtained via inexpensive finite element modeling at a coarse scale.
A Multilingual View of Unsupervised Machine Translation
We present a probabilistic framework for multilingual neural machine translation that encompasses supervised and unsupervised setups, focusing on unsupervised translation.
