Investigating Pretrained Language Models for Graph-to-Text Generation

Graph-to-text generation aims to generate fluent texts from graph-based data. In this paper, we investigate two recently proposed pretrained language models (PLMs) and analyze the impact of different task-adaptive pretraining strategies for PLMs in graph-to-text generation. We present a study across three graph domains: meaning representations, Wikipedia knowledge graphs (KGs) and scientific KGs. We show that the PLMs BART and T5 achieve new state-of-the-art results and that task-adaptive pretraining strategies improve their performance even further. In particular, we report new state-of-the-art BLEU scores of 49.72 on LDC2017T10, 59.70 on WebNLG, and 25.66 on AGENDA datasets - a relative improvement of 31.8%, 4.5%, and 42.4%, respectively. In an extensive analysis, we identify possible reasons for the PLMs' success on graph-to-text tasks. We find evidence that their knowledge about true facts helps them perform well even when the input graph representation is reduced to a simple bag of node and edge labels.

PDF Abstract EMNLP (NLP4ConvAI) 2021 PDF EMNLP (NLP4ConvAI) 2021 Abstract

Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Uses Extra
Training Data
Result Benchmark
KG-to-Text Generation AGENDA BART-large BLEU 23.65 # 2
KG-to-Text Generation AGENDA BART-large+ STA BLEU 25.66 # 1
AMR-to-Text Generation LDC2017T10 T5-TSS BLEU 48.85 # 2
AMR-to-Text Generation LDC2017T10 BART BLEU 43.72 # 5
METEOR 41.27 # 2
ChrF++ 71.27 # 2
Data-to-Text Generation WebNLG T5-small BLEU 65.05 # 5
KG-to-Text Generation WebNLG (All) BART_large BLEU 54.72 # 2
METEOR 42.23 # 2
chrF++ 72.29 # 2
KG-to-Text Generation WebNLG (All) T5_large BLEU 59.70 # 1
METEOR 44.18 # 1
chrF++ 75.40 # 1
Data-to-Text Generation WebNLG Full T5-large BLEU 59.70 # 4
KG-to-Text Generation WebNLG (Seen) T5_large BLEU 64.71 # 1
METEOR 45.85 # 1
chrF++ 78.29 # 1
KG-to-Text Generation WebNLG (Seen) BART_large BLEU 63.45 # 2
METEOR 45.49 # 2
chrF++ 77.57 # 2
KG-to-Text Generation WebNLG (Unseen) BART_large BLEU 43.97 # 2
METEOR 38.61 # 2
chrF++ 66.53 # 2
KG-to-Text Generation WebNLG (Unseen) T5_large BLEU 53.67 # 1
METEOR 42.26 # 1
chrF++ 72.25 # 1

Methods