Modern French Poetry Generation with RoBERTa and GPT-2
We present a novel neural model for modern poetry generation in French. The model consists of two pretrained neural models that are fine-tuned for the poem generation task. The encoder of the model is a RoBERTa based one while the decoder is based on GPT-2. This way the model can benefit from the superior natural language understanding performance of RoBERTa and the good natural language generation performance of GPT-2. Our evaluation shows that the model can create French poetry successfully. On a 5 point scale, the lowest score of 3.57 was given by human judges to typicality and emotionality of the output poetry while the best score of 3.79 was given to understandability.
PDF Abstract
Datasets
Add Datasets
introduced or used in this paper
Results from the Paper
Submit
results from this paper
to get state-of-the-art GitHub badges and help the
community compare results to other papers.
Methods
Adam •
Attention Dropout •
BERT •
BPE •
Cosine Annealing •
Dense Connections •
Discriminative Fine-Tuning •
Dropout •
GELU •
GPT-2 •
Layer Normalization •
Linear Layer •
Linear Warmup With Cosine Annealing •
Linear Warmup With Linear Decay •
Multi-Head Attention •
Residual Connection •
RoBERTa •
Scaled Dot-Product Attention •
Softmax •
Weight Decay •
WordPiece
