Search Results for author:
Found 16 papers, 10 papers with code
PaLM: Scaling Language Modeling with Pathways
To further our understanding of the impact of scale on few-shot learning, we trained a 540-billion parameter, densely activated, Transformer language model, which we call Pathways Language Model PaLM.
Ranked #1 on
Question Answering
on MultiRC
Scaling Up Models and Data with $\texttt{t5x}$ and $\texttt{seqio}$
Recent neural network-based language models have benefited greatly from scaling up the size of training datasets and the number of parameters in the models themselves.
Scaling Language Models: Methods, Analysis & Insights from Training Gopher
Language modelling provides a step towards intelligent communication systems by harnessing large repositories of written human knowledge to better predict and understand the world.
Ranked #1 on
Analogical Similarity
on BIG-bench
Exploring the limits of Concurrency in ML Training on Google TPUs
Recent results in language understanding using neural networks have required training hardware of unprecedentedscale, with thousands of chips cooperating on a single training run.
A High-Quality Multilingual Dataset for Structured Documentation Translation
We build and evaluate translation models for seven target languages from English, with several different copy mechanisms and an XML-constrained beam search.
PyTorch: An Imperative Style, High-Performance Deep Learning Library
Deep learning frameworks have often focused on either usability or speed, but not both.
OpenSpiel: A Framework for Reinforcement Learning in Games
OpenSpiel is a collection of environments and algorithms for research in general reinforcement learning and search/planning in games.
A Flexible Approach to Automated RNN Architecture Generation
The process of designing neural architectures requires expert knowledge and extensive trial and error.
Block-diagonal Hessian-free Optimization for Training Neural Networks
Second-order methods for neural network optimization have several advantages over methods based on first-order gradient descent, including better scaling to large mini-batch sizes and fewer updates needed for convergence.
Non-Autoregressive Neural Machine Translation
Existing approaches to neural machine translation condition each output word on previously generated outputs.
Ranked #3 on
Machine Translation
on IWSLT2015 English-German
Towards Neural Machine Translation with Latent Tree Attention
Building models that take advantage of the hierarchical structure of language without a priori annotation is a longstanding goal in natural language processing.
Learned in Translation: Contextualized Word Vectors
For fine-grained sentiment analysis and entailment, CoVe improves performance of our baseline models to the state of the art.
Quasi-Recurrent Neural Networks
Recurrent neural networks are a powerful tool for modeling sequential data, but the dependence of each timestep's computation on the previous timestep's output limits parallelism and makes RNNs unwieldy for very long sequences.
Ranked #15 on
Machine Translation
on IWSLT2015 German-English
Pointer Sentinel Mixture Models
Recent neural network sequence models with softmax classifiers have achieved their best language modeling performance only with very large hidden states and large vocabularies.
Ask Me Anything: Dynamic Memory Networks for Natural Language Processing
Most tasks in natural language processing can be cast into question answering (QA) problems over language input.
Ranked #54 on
Sentiment Analysis
on SST-2 Binary classification
