Search Results for author:
Found 18 papers, 14 papers with code
Training Verifiers to Solve Math Word Problems
State-of-the-art language models can match human performance on many tasks, but they still struggle to robustly perform multi-step mathematical reasoning.
Evaluating Large Language Models Trained on Code
We introduce Codex, a GPT language model fine-tuned on publicly available code from GitHub, and study its Python code-writing capabilities.
Ranked #1 on
Code Generation
on APPS
Rethinking Attention with Performers
We introduce Performers, Transformer architectures which can estimate regular (softmax) full-rank-attention Transformers with provable accuracy, but using only linear (as opposed to quadratic) space and time complexity, without relying on any priors such as sparsity or low-rankness.
Ranked #14 on
Image Generation
on ImageNet 64x64
Parallel Scheduled Sampling
Experimentally, we find the proposed technique leads to equivalent or better performance on image generation, summarization, dialog generation, and translation compared to teacher-forced training.
Sample Efficient Text Summarization Using a Single Pre-Trained Transformer
Language model (LM) pre-training has resulted in impressive performance and sample efficiency on a variety of language understanding tasks.
Ranked #1 on
Text Summarization
on DUC 2004 Task 1
(ROUGE-2 metric)
Model-Based Reinforcement Learning for Atari
We describe Simulated Policy Learning (SimPLe), a complete model-based deep RL algorithm based on video prediction models and present a comparison of several model architectures, including a novel architecture that yields the best results in our setting.
Ranked #12 on
Atari Games
on Atari games
Area Attention
We propose area attention: a way to attend to areas in the memory, where each area contains a group of items that are structurally adjacent, e. g., spatially for a 2D memory such as images, or temporally for a 1D memory such as natural language sentences.
Generating Wikipedia by Summarizing Long Sequences
We show that generating English Wikipedia articles can be approached as a multi- document summarization of source documents.
Unsupervised Cipher Cracking Using Discrete GANs
This work details CipherGAN, an architecture inspired by CycleGAN used for inferring the underlying cipher mapping given banks of unpaired ciphertext and plaintext.
Large Scale Multi-Domain Multi-Task Learning with MultiModel
We present a single model that yields good results on a number of problems spanning multiple domains.
One Model To Learn Them All
We present a single model that yields good results on a number of problems spanning multiple domains.
Attention Is All You Need
The dominant sequence transduction models are based on complex recurrent or convolutional neural networks in an encoder-decoder configuration.
Ranked #2 on
Multimodal Machine Translation
on Multi30K
(BLUE (DE-EN) metric)
Depthwise Separable Convolutions for Neural Machine Translation
In this work, we study how depthwise separable convolutions can be applied to neural machine translation.
Ranked #47 on
Machine Translation
on WMT2014 English-German
TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
TensorFlow is an interface for expressing machine learning algorithms, and an implementation for executing such algorithms.
Adding Gradient Noise Improves Learning for Very Deep Networks
This success is partially attributed to architectural innovations such as convolutional and long short-term memory networks.
Multi-task Sequence to Sequence Learning
This paper examines three multi-task learning (MTL) settings for sequence to sequence models: (a) the oneto-many setting - where the encoder is shared between several tasks such as machine translation and syntactic parsing, (b) the many-to-one setting - useful when only the decoder can be shared, as in the case of translation and image caption generation, and (c) the many-to-many setting - where multiple encoders and decoders are shared, which is the case with unsupervised objectives and translation.
Grammar as a Foreign Language
Syntactic constituency parsing is a fundamental problem in natural language processing and has been the subject of intensive research and engineering for decades.
Ranked #20 on
Constituency Parsing
on Penn Treebank