Search Results for author:
Found 17 papers, 13 papers with code
Finetuned Language Models Are Zero-Shot Learners
We show that instruction tuning -- finetuning language models on a collection of tasks described via instructions -- substantially improves zero-shot performance on unseen tasks.
Ranked #1 on
Question Answering
on OBQA
Scaling Up Models and Data with $\texttt{t5x}$ and $\texttt{seqio}$
Recent neural network-based language models have benefited greatly from scaling up the size of training datasets and the number of parameters in the models themselves.
The Power of Scale for Parameter-Efficient Prompt Tuning
More remarkably, through ablations on model size using T5, we show that prompt tuning becomes more competitive with scale: as models exceed billions of parameters, our method "closes the gap" and matches the strong performance of model tuning (where all model weights are tuned).
Overcoming Catastrophic Forgetting in Zero-Shot Cross-Lingual Generation
In this paper, we explore the challenging problem of performing a generative task in a target language when labeled data is only available in English, using summarization as a case study.
Baseline: A Library for Rapid Modeling, Experimentation and Development of Deep Learning Algorithms targeting NLP
We introduce Baseline: a library for reproducible deep learning research and fast model development for NLP.
Computationally Efficient NER Taggers with Combined Embeddings and Constrained Decoding
The CRF layer is used to facilitate global coherence between labels, and the contextual embeddings provide a better representation of words in context.
Git-Theta: A Git Extension for Collaborative Development of Machine Learning Models
Currently, most machine learning models are trained by centralized teams and are rarely updated.
iobes: A Library for Span-Level Processing
After a model assigns labels to each token, these prefixes are used to group the tokens into spans.
iobes: Library for Span Level Processing
After a model assigns labels to each token, these prefixes are used to group the tokens into spans.
Leader: Prefixing a Length for Faster Word Vector Serialization
Two competing file formats have become the de facto standards for distributing pre-trained word embeddings.
Constrained Decoding for Computationally Efficient Named Entity Recognition Taggers
Current state-of-the-art models for named entity recognition (NER) are neural models with a conditional random field (CRF) as the final layer.
Multiple Word Embeddings for Increased Diversity of Representation
Most state-of-the-art models in natural language processing (NLP) are neural models built on top of large, pre-trained, contextual language models that generate representations of words in context and are fine-tuned for the task at hand.
An Effective Label Noise Model for DNN Text Classification
In this paper, we propose an approach to training deep networks that is robust to label noise.
Intent Features for Rich Natural Language Understanding
Complex natural language understanding modules in dialog systems have a richer understanding of user utterances, and thus are critical in providing a better user experience.
SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer
Finally, we propose an efficient retrieval approach that interprets task prompts as task embeddings to identify similar tasks and predict the most transferable source tasks for a novel target task.
Reducing Retraining by Recycling Parameter-Efficient Prompts
Parameter-efficient methods are able to use a single frozen pre-trained large language model (LLM) to perform many tasks by learning task-specific soft prompts that modulate model behavior when concatenated to the input text.
Training LLMs over Neurally Compressed Text
In this paper, we explore the idea of training large language models (LLMs) over highly compressed text.
