Transformers

GPT is a Transformer-based architecture and training procedure for natural language processing tasks. Training follows a two-stage procedure. First, a language modeling objective is used on the unlabeled data to learn the initial parameters of a neural network model. Subsequently, these parameters are adapted to a target task using the corresponding supervised objective.

Source: Improving Language Understanding by Generative Pre-Training

Papers


Paper Code Results Date Stars

Tasks


Task Papers Share
Language Modelling 56 6.94%
Language Modeling 46 5.70%
Large Language Model 41 5.08%
Retrieval 27 3.35%
Question Answering 24 2.97%
Text Generation 24 2.97%
Prompt Engineering 23 2.85%
RAG 22 2.73%
Fairness 16 1.98%

Categories