Search Results for author:
Found 16 papers, 5 papers with code
PaLM 2 Technical Report
Through extensive evaluations on English and multilingual language, and reasoning tasks, we demonstrate that PaLM 2 has significantly improved quality on downstream tasks across different model sizes, while simultaneously exhibiting faster and more efficient inference compared to PaLM.
Ranked #1 on
Question Answering
on TriviaQA
(using extra training data)
The unreasonable effectiveness of few-shot learning for machine translation
We demonstrate the potential of few-shot translation systems, trained with unpaired language data, for both high and low-resource language pairs.
Building Machine Translation Systems for the Next Thousand Languages
In this paper we share findings from our effort to build practical machine translation (MT) systems capable of translating across over one thousand languages.
Data Scaling Laws in NMT: The Effect of Noise and Architecture
In this work, we study the effect of varying the architecture and training data quality on the data scaling properties of Neural Machine Translation (NMT).
LaMDA: Language Models for Dialog Applications
We demonstrate that fine-tuning with annotated data and enabling the model to consult external knowledge sources can lead to significant improvements towards the two key challenges of safety and factual grounding.
Ranked #36 on
Code Generation
on HumanEval
GLaM: Efficient Scaling of Language Models with Mixture-of-Experts
Scaling language models with more data, compute and parameters has driven significant progress in natural language processing.
Ranked #9 on
Question Answering
on TriviaQA
(using extra training data)
Beyond Distillation: Task-level Mixture-of-Experts for Efficient Inference
On WMT, our task-MoE with 32 experts (533M parameters) outperforms the best performing token-level MoE model (token-MoE) by +1. 0 BLEU on average across 30 language pairs.
Scaling Laws for Neural Machine Translation
We show that cross-entropy loss as a function of model size follows a certain scaling law.
GSPMD: General and Scalable Parallelization for ML Computation Graphs
We present GSPMD, an automatic, compiler-based parallelization system for common machine learning computations.
Exploring Routing Strategies for Multilingual Mixture-of-Experts Models
Sparsely-Gated Mixture-of-Experts (MoE) has been a successful approach for scaling multilingual translation models to billions of parameters without a proportional increase in training computation.
GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding
Neural network scaling has been critical for improving the model quality in many real-world machine learning applications with vast amounts of training data and compute.
Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges
We introduce our efforts towards building a universal neural machine translation (NMT) system capable of translating between any language pair.
Reinforcement Learning based Curriculum Optimization for Neural Machine Translation
We consider the problem of making efficient use of heterogeneous training data in neural machine translation (NMT).
Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modeling
Lingvo is a Tensorflow framework offering a complete solution for collaborative deep learning research, with a particular focus towards sequence-to-sequence models.
Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation
In addition to improving the translation quality of language pairs that the model was trained with, our models can also learn to perform implicit bridging between language pairs never seen explicitly during training, showing that transfer learning and zero-shot translation is possible for neural translation.
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
To improve parallelism and therefore decrease training time, our attention mechanism connects the bottom layer of the decoder to the top layer of the encoder.
Ranked #35 on
Machine Translation
on WMT2014 English-French
