GPT is a Transformer-based architecture and training procedure for natural language processing tasks. Training follows a two-stage procedure. First, a language modeling objective is used on the unlabeled data to learn the initial parameters of a neural network model. Subsequently, these parameters are adapted to a target task using the corresponding supervised objective.
Source: Improving Language Understanding by Generative Pre-TrainingPaper | Code | Results | Date | Stars |
---|
Task | Papers | Share |
---|---|---|
Language Modelling | 43 | 14.48% |
Text Generation | 16 | 5.39% |
Question Answering | 11 | 3.70% |
Machine Translation | 10 | 3.37% |
Natural Language Understanding | 9 | 3.03% |
Text Classification | 8 | 2.69% |
Sentiment Analysis | 7 | 2.36% |
Speech Recognition | 6 | 2.02% |
Self-Supervised Learning | 5 | 1.68% |