Search Results for author:
Found 29 papers, 11 papers with code
Multi-Vector Attention Models for Deep Re-ranking
Large-scale document retrieval systems often utilize two styles of neural network models which live at two different ends of the joint computation vs. accuracy spectrum.
QueryForm: A Simple Zero-shot Form Entity Query Framework
Zero-shot transfer learning for document understanding is a crucial yet under-investigated scenario to help reduce the high cost involved in annotating document entities.
Efficiently Scaling Transformer Inference
We study the problem of efficient generative inference for Transformer models, in one of its most challenging settings: large deep models, with tight latency targets and long sequence lengths.
Scaling Instruction-Finetuned Language Models
We find that instruction finetuning with the above aspects dramatically improves performance on a variety of model classes (PaLM, T5, U-PaLM), prompting setups (zero-shot, few-shot, CoT), and evaluation benchmarks (MMLU, BBH, TyDiQA, MGSM, open-ended generation).
Ranked #1 on
Multi-task Language Understanding
on BBH-nlp
Cross-Lingual Question Answering
Multi-task Language Understanding
+1
PaLM: Scaling Language Modeling with Pathways
To further our understanding of the impact of scale on few-shot learning, we trained a 540-billion parameter, densely activated, Transformer language model, which we call Pathways Language Model PaLM.
Ranked #1 on
Auto Debugging
on Big-bench Lite
Zero-Shot Entity Linking by Reading Entity Descriptions
First, we show that strong reading comprehension models pre-trained on large unlabeled data can be used to generalize to unseen entities.
Synthetic QA Corpora Generation with Roundtrip Consistency
We introduce a novel method of generating synthetic question answering corpora by combining models of question generation and answer extraction, and by filtering the results to ensure roundtrip consistency.
Natural Questions: a Benchmark for Question Answering Research
The public release consists of 307, 373 training examples with single annotations, 7, 830 examples with 5-way annotations for development data, and a further 7, 842 examples 5-way annotated sequestered as test data.
Ranked #7 on
Question Answering
on Natural Questions (long)
Language Model Pre-training for Hierarchical Document Representations
Hierarchical neural architectures are often used to capture long-distance dependencies and have been applied to many document-level tasks such as summarization, document segmentation, and sentiment analysis.
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
We introduce a new language representation model called BERT, which stands for Bidirectional Encoder Representations from Transformers.
Ranked #1 on
Question Answering
on CoQA
Leveraging Grammar and Reinforcement Learning for Neural Program Synthesis
Program synthesis is the task of automatically generating a program consistent with a specification.
Universal Neural Machine Translation for Extremely Low Resource Languages
Our proposed approach utilizes a transfer-learning approach to share lexical and sentence level representations across multiple source languages into one target language.
Semantic Code Repair using Neuro-Symbolic Transformation Networks
We study the problem of semantic code repair, which can be broadly defined as automatically fixing non-syntactic bugs in source code.
Neural Program Meta-Induction
In our first proposal, portfolio adaptation, a set of induction models is pretrained on a set of related tasks, and the best model is adapted towards the new task using transfer learning.
Sharp Models on Dull Hardware: Fast and Accurate Neural Machine Translation Decoding on the CPU
By combining these techniques, our best system achieves a very competitive accuracy of 38. 3 BLEU on WMT English-French NewsTest2014, while decoding at 100 words/sec on single-threaded CPU.
RobustFill: Neural Program Learning under Noisy I/O
Recently, two competing approaches for automatic program learning have received significant attention: (1) neural program synthesis, where a neural network is conditioned on input/output (I/O) examples and learns to generate a program, and (2) neural program induction, where a neural network generates new outputs directly using a latent program representation.
Visual Storytelling
We introduce the first dataset for sequential vision-to-language, and explore how this data may be used for the task of visual storytelling.
Generating Natural Questions About an Image
There has been an explosion of work in the vision & language community during the past few years from image captioning to video transcription, and answering questions about images.
Detecting Interrogative Utterances with Recurrent Neural Networks
In this paper, we explore different neural network architectures that can predict if a speaker of a given utterance is asking a question or making a statement.
A Survey of Current Datasets for Vision and Language Research
Integrating vision and language has long been a dream in work on artificial intelligence (AI).
Statistical Machine Translation Features with Multitask Tensor Networks
We present a three-pronged approach to improving Statistical Machine Translation (SMT), building on recent success in the application of neural networks to SMT.
Exploring Nearest Neighbor Approaches for Image Captioning
We explore a variety of nearest neighbor baseline approaches for image captioning.
Language Models for Image Captioning: The Quirks and What Works
Two recent approaches have achieved state-of-the-art results in image captioning.
