Transformers

SC-GPT is a multi-layer Transformer neural language model, trained in three steps: (i) Pre-trained on plain text, similar to GPT-2; (ii) Continuously pretrained on large amounts of dialog-act labeled utterances corpora to acquire the ability of controllable generation; (iii) Fine-tuned for a target domain using very limited amounts of domain labels. Unlike GPT-2, SC-GPT generates semantically controlled responses that are conditioned on the given semantic form, similar to SC-LSTM but requiring much less domain labels to generalize to new domains. It is pre-trained on a large set of annotated NLG corpus to acquire the controllable generation ability, and fine-tuned with only a few domain-specific labels to adapt to new domains.

Source: Few-shot Natural Language Generation for Task-Oriented Dialog

Papers


Paper Code Results Date Stars

Tasks


Task Papers Share
Data-to-Text Generation 1 33.33%
Few-Shot Learning 1 33.33%
Text Generation 1 33.33%

Components


Component Type
🤖 No Components Found You can add them if they exist; e.g. Mask R-CNN uses RoIAlign

Categories