Search Results for author:
Found 21 papers, 14 papers with code
Competitive Programming with Large Reasoning Models
We competed live at IOI 2024 with o1-ioi and, using hand-crafted test-time strategies, placed in the 49th percentile.
OpenAI o1 System Card
The o1 model series is trained with large-scale reinforcement learning to reason using chain of thought.
GPT-4o System Card
We also include third-party assessments on dangerous capabilities, as well as discussion of potential societal impacts of GPT-4o's text and vision capabilities.
Ranked #1 on
Visual Question Answering (VQA)
on VLM2-Bench
Evaluating Large Language Models Trained on Code
We introduce Codex, a GPT language model fine-tuned on publicly available code from GitHub, and study its Python code-writing capabilities.
Ranked #1 on
Multi-task Language Understanding
on BBH-alg
Rethinking Attention with Performers
We introduce Performers, Transformer architectures which can estimate regular (softmax) full-rank-attention Transformers with provable accuracy, but using only linear (as opposed to quadratic) space and time complexity, without relying on any priors such as sparsity or low-rankness.
Ranked #9 on
D4RL
on D4RL
Parallel Scheduled Sampling
Experimentally, we find the proposed technique leads to equivalent or better performance on image generation, summarization, dialog generation, and translation compared to teacher-forced training.
Sample Efficient Text Summarization Using a Single Pre-Trained Transformer
Language model (LM) pre-training has resulted in impressive performance and sample efficiency on a variety of language understanding tasks.
Ranked #1 on
Text Summarization
on DUC 2004 Task 1
(ROUGE-2 metric)
Model-Based Reinforcement Learning for Atari
We describe Simulated Policy Learning (SimPLe), a complete model-based deep RL algorithm based on video prediction models and present a comparison of several model architectures, including a novel architecture that yields the best results in our setting.
Ranked #12 on
Atari Games 100k
on Atari 100k
Area Attention
We propose area attention: a way to attend to areas in the memory, where each area contains a group of items that are structurally adjacent, e. g., spatially for a 2D memory such as images, or temporally for a 1D memory such as natural language sentences.
Unsupervised Cipher Cracking Using Discrete GANs
This work details CipherGAN, an architecture inspired by CycleGAN used for inferring the underlying cipher mapping given banks of unpaired ciphertext and plaintext.
Large Scale Multi-Domain Multi-Task Learning with MultiModel
We present a single model that yields good results on a number of problems spanning multiple domains.
One Model To Learn Them All
We present a single model that yields good results on a number of problems spanning multiple domains.
Attention Is All You Need
The dominant sequence transduction models are based on complex recurrent or convolutional neural networks in an encoder-decoder configuration.
Ranked #2 on
Multimodal Machine Translation
on Multi30K
(BLUE (DE-EN) metric)
Depthwise Separable Convolutions for Neural Machine Translation
In this work, we study how depthwise separable convolutions can be applied to neural machine translation.
Ranked #62 on
Machine Translation
on WMT2014 English-German
TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
TensorFlow is an interface for expressing machine learning algorithms, and an implementation for executing such algorithms.
Adding Gradient Noise Improves Learning for Very Deep Networks
This success is partially attributed to architectural innovations such as convolutional and long short-term memory networks.
Multi-task Sequence to Sequence Learning
This paper examines three multi-task learning (MTL) settings for sequence to sequence models: (a) the oneto-many setting - where the encoder is shared between several tasks such as machine translation and syntactic parsing, (b) the many-to-one setting - useful when only the decoder can be shared, as in the case of translation and image caption generation, and (c) the many-to-many setting - where multiple encoders and decoders are shared, which is the case with unsupervised objectives and translation.
Grammar as a Foreign Language
Syntactic constituency parsing is a fundamental problem in natural language processing and has been the subject of intensive research and engineering for decades.
Ranked #25 on
Constituency Parsing
on Penn Treebank
