TASK	DATASET	MODEL	METRIC NAME	METRIC VALUE	GLOBAL RANK
Question Answering	BoolQ	T5-11B	Accuracy	91.2	# 1
Abstractive Text Summarization	CNN / Daily Mail	T5	ROUGE-1	43.52	# 7
Abstractive Text Summarization	CNN / Daily Mail	T5	ROUGE-2	21.55	# 1
Abstractive Text Summarization	CNN / Daily Mail	T5	ROUGE-L	40.69	# 7
Document Summarization	CNN / Daily Mail	T5-11B	ROUGE-1	43.52	# 5
Document Summarization	CNN / Daily Mail	T5-11B	ROUGE-2	21.55	# 1
Document Summarization	CNN / Daily Mail	T5-11B	ROUGE-L	40.69	# 2
Linguistic Acceptability	CoLA	T5-Base	Accuracy	51.1%	# 16
Linguistic Acceptability	CoLA	T5-3B	Accuracy	67.1%	# 8
Linguistic Acceptability	CoLA	T5-Large	Accuracy	61.2%	# 12
Linguistic Acceptability	CoLA	T5-11B	Accuracy	70.8%	# 1
Linguistic Acceptability	CoLA	T5-Small	Accuracy	41.0%	# 20
Natural Language Inference	CommitmentBank	T5-11B	F1	93.9	# 2
Natural Language Inference	CommitmentBank	T5-11B	Accuracy	96.8	# 2
Question Answering	COPA	T5-11B	Accuracy	94.8	# 3
Semantic Textual Similarity	MRPC	T5-11B	Accuracy	90.0%	# 9
Semantic Textual Similarity	MRPC	T5-11B	F1	91.9	# 3
Semantic Textual Similarity	MRPC	T5-Large	Accuracy	89.9%	# 10
Semantic Textual Similarity	MRPC	T5-Large	F1	92.4	# 2
Semantic Textual Similarity	MRPC	T5-3B	Accuracy	89.2%	# 12
Semantic Textual Similarity	MRPC	T5-3B	F1	92.5	# 1
Semantic Textual Similarity	MRPC	T5-Small	Accuracy	86.6%	# 16
Semantic Textual Similarity	MRPC	T5-Small	F1	89.7	# 6
Semantic Textual Similarity	MRPC	T5-Base	Accuracy	87.5%	# 14
Semantic Textual Similarity	MRPC	T5-Base	F1	90.7	# 5
Natural Language Inference	MultiNLI	T5-3B	Matched	91.4	# 2
Natural Language Inference	MultiNLI	T5-3B	Mismatched	91.2	# 2
Natural Language Inference	MultiNLI	T5-Base	Matched	87.1	# 14
Natural Language Inference	MultiNLI	T5-Base	Mismatched	86.2	# 10
Natural Language Inference	MultiNLI	T5-11B	Matched	92.0	# 1
Natural Language Inference	MultiNLI	T5-11B	Mismatched	91.7	# 1
Natural Language Inference	MultiNLI	T5-Small	Matched	82.4	# 19
Natural Language Inference	MultiNLI	T5-Small	Mismatched	82.3	# 14
Natural Language Inference	MultiNLI	T5-Large	Matched	89.9	# 8
Natural Language Inference	MultiNLI	T5-Large	Mismatched	89.6	# 6
Question Answering	MultiRC	T5-11B	F1a	88.1	# 2
Question Answering	MultiRC	T5-11B	EM	63.3	# 2
Natural Language Inference	QNLI	T5-Base	Accuracy	93.7%	# 12
Natural Language Inference	QNLI	T5-Small	Accuracy	90.3%	# 17
Natural Language Inference	QNLI	T5-3B	Accuracy	96.3%	# 4
Natural Language Inference	QNLI	T5-11B	Accuracy	96.7%	# 3
Natural Language Inference	QNLI	T5-Large	Accuracy	94.8%	# 9
Question Answering	Quora Question Pairs	T5-Large	Accuracy	89.9%	# 9
Question Answering	Quora Question Pairs	T5-11B	Accuracy	90.4%	# 4
Question Answering	Quora Question Pairs	T5-Base	Accuracy	89.4%	# 12
Question Answering	Quora Question Pairs	T5-Small	Accuracy	88.0%	# 16
Question Answering	Quora Question Pairs	T5-3B	Accuracy	89.7%	# 11
Common Sense Reasoning	ReCoRD	T5-11B	F1	94.1	# 2
Common Sense Reasoning	ReCoRD	T5-11B	Acc	93.4	# 2
Natural Language Inference	RTE	T5-Base	Accuracy	80.1%	# 10
Natural Language Inference	RTE	T5-Large	Accuracy	87.2%	# 6
Natural Language Inference	RTE	T5-11B	Accuracy	92.5%	# 2
Natural Language Inference	RTE	T5-Small	Accuracy	69.9%	# 16
Natural Language Inference	RTE	T5-3B	Accuracy	91.1%	# 3
Question Answering	SQuAD1.1 dev	T5-3B	EM	88.53	# 6
Question Answering	SQuAD1.1 dev	T5-3B	F1	94.95	# 6
Question Answering	SQuAD1.1 dev	T5-Small	EM	79.1	# 13
Question Answering	SQuAD1.1 dev	T5-Small	F1	87.24	# 15
Question Answering	SQuAD1.1 dev	T5-Base	EM	85.44	# 8
Question Answering	SQuAD1.1 dev	T5-Base	F1	92.08	# 8
Question Answering	SQuAD1.1 dev	T5-Large	EM	86.66	# 7
Question Answering	SQuAD1.1 dev	T5-Large	F1	93.79	# 7
Question Answering	SQuAD1.1 dev	T5-11B	EM	90.06	# 2
Question Answering	SQuAD1.1 dev	T5-11B	F1	95.64	# 2
Sentiment Analysis	SST-2 Binary classification	T5-Large	Accuracy	96.3	# 12
Sentiment Analysis	SST-2 Binary classification	T5-11B	Accuracy	97.1	# 3
Sentiment Analysis	SST-2 Binary classification	T5-Small	Accuracy	91.8	# 26
Sentiment Analysis	SST-2 Binary classification	T5-3B	Accuracy	97.4	# 2
Sentiment Analysis	SST-2 Binary classification	T5-Base	Accuracy	95.2	# 15
Semantic Textual Similarity	STS Benchmark	T5-Large	Pearson Correlation	0.899	# 12
Semantic Textual Similarity	STS Benchmark	T5-Large	Spearman Correlation	0.886	# 4
Semantic Textual Similarity	STS Benchmark	T5-11B	Pearson Correlation	0.925	# 2
Semantic Textual Similarity	STS Benchmark	T5-11B	Spearman Correlation	0.921	# 2
Semantic Textual Similarity	STS Benchmark	T5-Small	Pearson Correlation	0.856	# 16
Semantic Textual Similarity	STS Benchmark	T5-Small	Spearman Correlation	0.85	# 8
Semantic Textual Similarity	STS Benchmark	T5-Base	Pearson Correlation	0.894	# 14
Semantic Textual Similarity	STS Benchmark	T5-3B	Pearson Correlation	0.906	# 10
Semantic Textual Similarity	STS Benchmark	T5-3B	Spearman Correlation	0.898	# 3
Coreference Resolution	Winograd Schema Challenge	T5-11B	Accuracy	93.8	# 2
Machine Translation	WMT2014 English-French	T5	BLEU score	43.4	# 7
Machine Translation	WMT2014 English-German	T5-11B	BLEU score	32.1	# 4
Natural Language Inference	WNLI	T5-11B	Accuracy	93.2%	# 2
Natural Language Inference	WNLI	T5-Small	Accuracy	69.2%	# 10
Natural Language Inference	WNLI	T5-3B	Accuracy	89.7%	# 6
Natural Language Inference	WNLI	T5-Large	Accuracy	85.6%	# 8
Natural Language Inference	WNLI	T5-Base	Accuracy	78.8%	# 9
Word Sense Disambiguation	Words in Context	T5-11B	Accuracy	76.9	# 3

Badge	Markdown
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/exploring-the-limits-of-transfer-learning/question-answering-on-boolq)](https://paperswithcode.com/sota/question-answering-on-boolq?p=exploring-the-limits-of-transfer-learning)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/exploring-the-limits-of-transfer-learning/linguistic-acceptability-on-cola)](https://paperswithcode.com/sota/linguistic-acceptability-on-cola?p=exploring-the-limits-of-transfer-learning)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/exploring-the-limits-of-transfer-learning/natural-language-inference-on-multinli)](https://paperswithcode.com/sota/natural-language-inference-on-multinli?p=exploring-the-limits-of-transfer-learning)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/exploring-the-limits-of-transfer-learning/natural-language-inference-on-commitmentbank)](https://paperswithcode.com/sota/natural-language-inference-on-commitmentbank?p=exploring-the-limits-of-transfer-learning)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/exploring-the-limits-of-transfer-learning/question-answering-on-multirc)](https://paperswithcode.com/sota/question-answering-on-multirc?p=exploring-the-limits-of-transfer-learning)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/exploring-the-limits-of-transfer-learning/common-sense-reasoning-on-record)](https://paperswithcode.com/sota/common-sense-reasoning-on-record?p=exploring-the-limits-of-transfer-learning)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/exploring-the-limits-of-transfer-learning/natural-language-inference-on-rte)](https://paperswithcode.com/sota/natural-language-inference-on-rte?p=exploring-the-limits-of-transfer-learning)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/exploring-the-limits-of-transfer-learning/question-answering-on-squad11-dev)](https://paperswithcode.com/sota/question-answering-on-squad11-dev?p=exploring-the-limits-of-transfer-learning)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/exploring-the-limits-of-transfer-learning/sentiment-analysis-on-sst-2-binary)](https://paperswithcode.com/sota/sentiment-analysis-on-sst-2-binary?p=exploring-the-limits-of-transfer-learning)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/exploring-the-limits-of-transfer-learning/semantic-textual-similarity-on-sts-benchmark)](https://paperswithcode.com/sota/semantic-textual-similarity-on-sts-benchmark?p=exploring-the-limits-of-transfer-learning)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/exploring-the-limits-of-transfer-learning/coreference-resolution-on-winograd-schema)](https://paperswithcode.com/sota/coreference-resolution-on-winograd-schema?p=exploring-the-limits-of-transfer-learning)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/exploring-the-limits-of-transfer-learning/natural-language-inference-on-wnli)](https://paperswithcode.com/sota/natural-language-inference-on-wnli?p=exploring-the-limits-of-transfer-learning)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/exploring-the-limits-of-transfer-learning/question-answering-on-copa)](https://paperswithcode.com/sota/question-answering-on-copa?p=exploring-the-limits-of-transfer-learning)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/exploring-the-limits-of-transfer-learning/natural-language-inference-on-qnli)](https://paperswithcode.com/sota/natural-language-inference-on-qnli?p=exploring-the-limits-of-transfer-learning)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/exploring-the-limits-of-transfer-learning/word-sense-disambiguation-on-words-in-context)](https://paperswithcode.com/sota/word-sense-disambiguation-on-words-in-context?p=exploring-the-limits-of-transfer-learning)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/exploring-the-limits-of-transfer-learning/question-answering-on-quora-question-pairs)](https://paperswithcode.com/sota/question-answering-on-quora-question-pairs?p=exploring-the-limits-of-transfer-learning)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/exploring-the-limits-of-transfer-learning/machine-translation-on-wmt2014-english-german)](https://paperswithcode.com/sota/machine-translation-on-wmt2014-english-german?p=exploring-the-limits-of-transfer-learning)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/exploring-the-limits-of-transfer-learning/document-summarization-on-cnn-daily-mail)](https://paperswithcode.com/sota/document-summarization-on-cnn-daily-mail?p=exploring-the-limits-of-transfer-learning)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/exploring-the-limits-of-transfer-learning/abstractive-text-summarization-on-cnn-daily)](https://paperswithcode.com/sota/abstractive-text-summarization-on-cnn-daily?p=exploring-the-limits-of-transfer-learning)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/exploring-the-limits-of-transfer-learning/machine-translation-on-wmt2014-english-french)](https://paperswithcode.com/sota/machine-translation-on-wmt2014-english-french?p=exploring-the-limits-of-transfer-learning)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/exploring-the-limits-of-transfer-learning/semantic-textual-similarity-on-mrpc)](https://paperswithcode.com/sota/semantic-textual-similarity-on-mrpc?p=exploring-the-limits-of-transfer-learning)`

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

arXiv 2019 • Colin Raffel • Noam Shazeer • Adam Roberts • Katherine Lee • Sharan Narang • Michael Matena • Yanqi Zhou • Wei Li • Peter J. Liu

Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice... (read more)

PDF Abstract

Code

Add Remove Mark official

google-research/text-to-text-transfer-transformer official

↳ Quickstart in

Colab

3,330

huggingface/transformers

44,646

lucidrains/x-transformers

262

abelriboulot/onnxt5

147

Ki6an/fastT5

See all 14 implementations

Tasks

Add Remove

SEMANTIC TEXTUAL SIMILARITY

TRANSFER LEARNING

Datasets

Introduced in the Paper:

Mentioned in the Paper:

ImageNet

SQuAD

GLUE

SST

TriviaQA

MultiNLI

QNLI

NewsQA

CNN/Daily Mail

MRPC

SuperGLUE

WebText

CoLA

Quora

SentEval

WMT 2014

WMT 2016

WSC

COPA

MultiRC

ReCoRD

BoolQ

RealNews

Results from the Paper

Edit

Ranked #1 on Natural Language Inference on MultiNLI

Get a GitHub badge

TASK	DATASET	MODEL	METRIC NAME	METRIC VALUE	GLOBAL RANK	BENCHMARK
Question Answering	BoolQ	T5-11B	Accuracy	91.2	# 1	Compare
Abstractive Text Summarization	CNN / Daily Mail	T5	ROUGE-1	43.52	# 7	Compare
			ROUGE-2	21.55	# 1	Compare
			ROUGE-L	40.69	# 7	Compare
Document Summarization	CNN / Daily Mail	T5-11B	ROUGE-1	43.52	# 5	Compare
			ROUGE-2	21.55	# 1	Compare
			ROUGE-L	40.69	# 2	Compare
Linguistic Acceptability	CoLA	T5-Base	Accuracy	51.1%	# 16	Compare
Linguistic Acceptability	CoLA	T5-3B	Accuracy	67.1%	# 8	Compare
Linguistic Acceptability	CoLA	T5-Large	Accuracy	61.2%	# 12	Compare
Linguistic Acceptability	CoLA	T5-11B	Accuracy	70.8%	# 1	Compare
Linguistic Acceptability	CoLA	T5-Small	Accuracy	41.0%	# 20	Compare
Natural Language Inference	CommitmentBank	T5-11B	F1	93.9	# 2	Compare
Natural Language Inference	CommitmentBank	T5-11B	Accuracy	96.8	# 2	Compare
Question Answering	COPA	T5-11B	Accuracy	94.8	# 3	Compare
Semantic Textual Similarity	MRPC	T5-11B	Accuracy	90.0%	# 9	Compare
Semantic Textual Similarity	MRPC	T5-11B	F1	91.9	# 3	Compare
Semantic Textual Similarity	MRPC	T5-Large	Accuracy	89.9%	# 10	Compare
Semantic Textual Similarity	MRPC	T5-Large	F1	92.4	# 2	Compare
Semantic Textual Similarity	MRPC	T5-3B	Accuracy	89.2%	# 12	Compare
Semantic Textual Similarity	MRPC	T5-3B	F1	92.5	# 1	Compare
Semantic Textual Similarity	MRPC	T5-Small	Accuracy	86.6%	# 16	Compare
Semantic Textual Similarity	MRPC	T5-Small	F1	89.7	# 6	Compare
Semantic Textual Similarity	MRPC	T5-Base	Accuracy	87.5%	# 14	Compare
Semantic Textual Similarity	MRPC	T5-Base	F1	90.7	# 5	Compare
Natural Language Inference	MultiNLI	T5-3B	Matched	91.4	# 2	Compare
Natural Language Inference	MultiNLI	T5-3B	Mismatched	91.2	# 2	Compare
Natural Language Inference	MultiNLI	T5-Base	Matched	87.1	# 14	Compare
Natural Language Inference	MultiNLI	T5-Base	Mismatched	86.2	# 10	Compare
Natural Language Inference	MultiNLI	T5-11B	Matched	92.0	# 1	Compare
Natural Language Inference	MultiNLI	T5-11B	Mismatched	91.7	# 1	Compare
Natural Language Inference	MultiNLI	T5-Small	Matched	82.4	# 19	Compare
Natural Language Inference	MultiNLI	T5-Small	Mismatched	82.3	# 14	Compare
Natural Language Inference	MultiNLI	T5-Large	Matched	89.9	# 8	Compare
Natural Language Inference	MultiNLI	T5-Large	Mismatched	89.6	# 6	Compare
Question Answering	MultiRC	T5-11B	F1a	88.1	# 2	Compare
Question Answering	MultiRC	T5-11B	EM	63.3	# 2	Compare
Natural Language Inference	QNLI	T5-Base	Accuracy	93.7%	# 12	Compare
Natural Language Inference	QNLI	T5-Small	Accuracy	90.3%	# 17	Compare
Natural Language Inference	QNLI	T5-3B	Accuracy	96.3%	# 4	Compare
Natural Language Inference	QNLI	T5-11B	Accuracy	96.7%	# 3	Compare
Natural Language Inference	QNLI	T5-Large	Accuracy	94.8%	# 9	Compare
Question Answering	Quora Question Pairs	T5-Large	Accuracy	89.9%	# 9	Compare
Question Answering	Quora Question Pairs	T5-11B	Accuracy	90.4%	# 4	Compare
Question Answering	Quora Question Pairs	T5-Base	Accuracy	89.4%	# 12	Compare
Question Answering	Quora Question Pairs	T5-Small	Accuracy	88.0%	# 16	Compare
Question Answering	Quora Question Pairs	T5-3B	Accuracy	89.7%	# 11	Compare
Common Sense Reasoning	ReCoRD	T5-11B	F1	94.1	# 2	Compare
Common Sense Reasoning	ReCoRD	T5-11B	Acc	93.4	# 2	Compare
Natural Language Inference	RTE	T5-Base	Accuracy	80.1%	# 10	Compare
Natural Language Inference	RTE	T5-Large	Accuracy	87.2%	# 6	Compare
Natural Language Inference	RTE	T5-11B	Accuracy	92.5%	# 2	Compare
Natural Language Inference	RTE	T5-Small	Accuracy	69.9%	# 16	Compare
Natural Language Inference	RTE	T5-3B	Accuracy	91.1%	# 3	Compare
Question Answering	SQuAD1.1 dev	T5-3B	EM	88.53	# 6	Compare
Question Answering	SQuAD1.1 dev	T5-3B	F1	94.95	# 6	Compare
Question Answering	SQuAD1.1 dev	T5-Small	EM	79.1	# 13	Compare
Question Answering	SQuAD1.1 dev	T5-Small	F1	87.24	# 15	Compare
Question Answering	SQuAD1.1 dev	T5-Base	EM	85.44	# 8	Compare
Question Answering	SQuAD1.1 dev	T5-Base	F1	92.08	# 8	Compare
Question Answering	SQuAD1.1 dev	T5-Large	EM	86.66	# 7	Compare
Question Answering	SQuAD1.1 dev	T5-Large	F1	93.79	# 7	Compare
Question Answering	SQuAD1.1 dev	T5-11B	EM	90.06	# 2	Compare
Question Answering	SQuAD1.1 dev	T5-11B	F1	95.64	# 2	Compare
Sentiment Analysis	SST-2 Binary classification	T5-Large	Accuracy	96.3	# 12	Compare
Sentiment Analysis	SST-2 Binary classification	T5-11B	Accuracy	97.1	# 3	Compare
Sentiment Analysis	SST-2 Binary classification	T5-Small	Accuracy	91.8	# 26	Compare
Sentiment Analysis	SST-2 Binary classification	T5-3B	Accuracy	97.4	# 2	Compare
Sentiment Analysis	SST-2 Binary classification	T5-Base	Accuracy	95.2	# 15	Compare
Semantic Textual Similarity	STS Benchmark	T5-Large	Pearson Correlation	0.899	# 12	Compare
Semantic Textual Similarity	STS Benchmark	T5-Large	Spearman Correlation	0.886	# 4	Compare
Semantic Textual Similarity	STS Benchmark	T5-11B	Pearson Correlation	0.925	# 2	Compare
Semantic Textual Similarity	STS Benchmark	T5-11B	Spearman Correlation	0.921	# 2	Compare
Semantic Textual Similarity	STS Benchmark	T5-Small	Pearson Correlation	0.856	# 16	Compare
Semantic Textual Similarity	STS Benchmark	T5-Small	Spearman Correlation	0.85	# 8	Compare
Semantic Textual Similarity	STS Benchmark	T5-Base	Pearson Correlation	0.894	# 14	Compare
Semantic Textual Similarity	STS Benchmark	T5-3B	Pearson Correlation	0.906	# 10	Compare
Semantic Textual Similarity	STS Benchmark	T5-3B	Spearman Correlation	0.898	# 3	Compare
Coreference Resolution	Winograd Schema Challenge	T5-11B	Accuracy	93.8	# 2	Compare
Machine Translation	WMT2014 English-French	T5	BLEU score	43.4	# 7	Compare
Machine Translation	WMT2014 English-German	T5-11B	BLEU score	32.1	# 4	Compare
Natural Language Inference	WNLI	T5-11B	Accuracy	93.2%	# 2	Compare
Natural Language Inference	WNLI	T5-Small	Accuracy	69.2%	# 10	Compare
Natural Language Inference	WNLI	T5-3B	Accuracy	89.7%	# 6	Compare
Natural Language Inference	WNLI	T5-Large	Accuracy	85.6%	# 8	Compare
Natural Language Inference	WNLI	T5-Base	Accuracy	78.8%	# 9	Compare
Word Sense Disambiguation	Words in Context	T5-11B	Accuracy	76.9	# 3	Compare

Methods used in the Paper

Add Remove

METHOD	TYPE
GLU	Activation Functions
GELU	Activation Functions
BPE	Subword Segmentation
Multi-Head Attention	Attention Modules
Adafactor	Stochastic Optimization
Residual Connection	Skip Connections
Inverse Square Root Schedule	Learning Rate Schedules
Attention Dropout	Regularization
SentencePiece	Tokenizers
Dense Connections	Feedforward Networks
Softmax	Output Functions
Dropout	Regularization
Layer Normalization	Normalization
Scaled Dot-Product Attention	Attention Mechanisms
T5	Transformers

Edit Social Preview

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Edit social preview

Code Edit Add Remove Mark official

Tasks Edit Add Remove

Datasets Edit

Results from the Paper Edit

Methods used in the Paper Edit Add Remove