Search Results for author:
Found 18 papers, 11 papers with code
Large Language Models Can Be Easily Distracted by Irrelevant Context
We use this benchmark to measure the distractibility of cutting-edge prompting techniques for large language models, and find that the model performance is dramatically decreased when irrelevant information is included.
Solving Quantitative Reasoning Problems with Language Models
Language models have achieved remarkable performance on a wide range of tasks that require natural language understanding.
Ranked #1 on
Multi-task Language Understanding
on MMLU
(STEM metric)
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
BIG-bench focuses on tasks that are believed to be beyond the capabilities of current language models.
Towards Learning Universal Hyperparameter Optimizers with Transformers
Meta-learning hyperparameter optimization (HPO) algorithms from prior experiments is a promising approach to improve optimization efficiency over objective functions from a similar distribution.
PaLM: Scaling Language Modeling with Pathways
To further our understanding of the impact of scale on few-shot learning, we trained a 540-billion parameter, densely activated, Transformer language model, which we call Pathways Language Model PaLM.
Ranked #1 on
Natural Language Inference
on RTE
Show Your Work: Scratchpads for Intermediate Computation with Language Models
Large pre-trained language models perform remarkably well on tasks that can be done "in one pass", such as generating realistic text or synthesizing computer programs.
Program Synthesis with Large Language Models
Our largest models, even without finetuning on a code dataset, can synthesize solutions to 59. 6 percent of the problems from MBPP using few-shot learning with a well-designed prompt.
Latent Programmer: Discrete Latent Codes for Program Synthesis
The latent codes are learned using a self-supervised learning principle, in which first a discrete autoencoder is trained on the output sequences, and then the resulting latent codes are used as intermediate targets for the end-to-end sequence prediction task.
Is Transfer Learning Necessary for Protein Landscape Prediction?
In this paper, we show that CNN models trained solely using supervised learning both compete with and sometimes outperform the best models from TAPE that leverage expensive pretraining on large protein datasets.
Fixed-Length Protein Embeddings using Contextual Lenses
We consider transformer (BERT) protein language models that are pretrained on the TrEMBL data set and learn fixed-length embeddings on top of them with contextual lenses.
Rethinking Attention with Performers
We introduce Performers, Transformer architectures which can estimate regular (softmax) full-rank-attention Transformers with provable accuracy, but using only linear (as opposed to quadratic) space and time complexity, without relying on any priors such as sparsity or low-rankness.
Ranked #15 on
Image Generation
on ImageNet 64x64
(Bits per dim metric)
Population-Based Black-Box Optimization for Biological Sequence Design
The cost and latency of wet-lab experiments requires methods that find good sequences in few experimental rounds of large batches of sequences--a setting that off-the-shelf black-box optimization methods are ill-equipped to handle.
Masked Language Modeling for Proteins via Linearly Scalable Long-Context Transformers
In response, solutions that exploit the structure and sparsity of the learned attention matrix have blossomed.
Model-based reinforcement learning for biological sequence design
In response, we propose using reinforcement learning (RL) based on proximal-policy optimization (PPO) for biological sequence design.
Model-based Reinforcement Learning
reinforcement-learning
+1
QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension
On the SQuAD dataset, our model is 3x to 13x faster in training and 4x to 9x faster in inference, while achieving equivalent accuracy to recurrent models.
Ranked #27 on
Question Answering
on SQuAD1.1 dev
EXPLORING NEURAL ARCHITECTURE SEARCH FOR LANGUAGE TASKS
Neural architecture search (NAS), the task of finding neural architectures automatically, has recently emerged as a promising approach for unveiling better models over human-designed ones.
Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks
Collecting well-annotated image datasets to train modern machine learning algorithms is prohibitively expensive for many tasks.
