Multi-task Language Understanding
14 papers with code • 4 benchmarks • 2 datasets
The test covers 57 tasks including elementary mathematics, US history, computer science, law, and more. https://arxiv.org/pdf/2009.03300.pdf
Benchmarks
These leaderboards are used to track progress in Multi-task Language Understanding
| Trend | Dataset | Best Model | Paper | Code | Compare |
|---|---|---|---|---|---|
|
|||||
|
|||||
|
|||||
|
Libraries
Use these libraries to find Multi-task Language Understanding models and implementations
MMLU
Tasksource
Most implemented papers
RoBERTa: A Robustly Optimized BERT Pretraining Approach
pytorch/fairseq
•
•
Language model pretraining has led to significant performance gains but careful comparison between different approaches is challenging.
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
google-research/ALBERT
•
•
ICLR 2020
Increasing model size when pretraining natural language representations often results in improved performance on downstream tasks.
Language Models are Few-Shot Learners
By contrast, humans can generally perform a new language task from only a few examples or from simple instructions - something which current NLP systems still largely struggle to do.
Language Models are Unsupervised Multitask Learners
PaddlePaddle/PaddleNLP
•
•
Preprint 2019
Natural language processing tasks, such as question answering, machine translation, reading comprehension, and summarization, are typically approached with supervised learning on taskspecific datasets.
Evaluating Large Language Models Trained on Code
We introduce Codex, a GPT language model fine-tuned on publicly available code from GitHub, and study its Python code-writing capabilities.
PaLM: Scaling Language Modeling with Pathways
lucidrains/CoCa-pytorch
•
•
Google Research 2022
To further our understanding of the impact of scale on few-shot learning, we trained a 540-billion parameter, densely activated, Transformer language model, which we call Pathways Language Model PaLM.
GLM-130B: An Open Bilingual Pre-trained Model
thudm/glm-130b
•
•
We introduce GLM-130B, a bilingual (English and Chinese) pre-trained language model with 130 billion parameters.
UnifiedQA: Crossing Format Boundaries With a Single QA System
allenai/unifiedqa
•
•
Findings of the Association for Computational Linguistics 2020
As evidence, we use the latest advances in language modeling to build a single pre-trained QA model, UnifiedQA, that performs surprisingly well across 17 QA datasets spanning 4 diverse formats.
Measuring Massive Multitask Language Understanding
By comprehensively evaluating the breadth and depth of a model's academic and professional understanding, our test can be used to analyze models across many tasks and to identify important shortcomings.
LLaMA: Open and Efficient Foundation Language Models
facebookresearch/llama
•
•
arXiv 2023
We introduce LLaMA, a collection of foundation language models ranging from 7B to 65B parameters.
