EventNarrative: A large-scale Event-centric Dataset for Knowledge Graph-to-Text Generation

30 Oct 2021  ·  Anthony Colas, Ali Sadeghian, Yue Wang, Daisy Zhe Wang ·

We introduce EventNarrative, a knowledge graph-to-text dataset from publicly available open-world knowledge graphs. Given the recent advances in event-driven Information Extraction (IE), and that prior research on graph-to-text only focused on entity-driven KGs, this paper focuses on event-centric data. However, our data generation system can still be adapted to other other types of KG data. Existing large-scale datasets in the graph-to-text area are non-parallel, meaning there is a large disconnect between the KGs and text. The datasets that have a paired KG and text, are small scale and manually generated or generated without a rich ontology, making the corresponding graphs sparse. Furthermore, these datasets contain many unlinked entities between their KG and text pairs. EventNarrative consists of approximately 230,000 graphs and their corresponding natural language text, 6 times larger than the current largest parallel dataset. It makes use of a rich ontology, all of the KGs entities are linked to the text, and our manual annotations confirm a high data quality. Our aim is two-fold: help break new ground in event-centric research where data is lacking, and to give researchers a well-defined, large-scale dataset in order to better evaluate existing and future knowledge graph-to-text models. We also evaluate two types of baseline on EventNarrative: a graph-to-text specific model and two state-of-the-art language models, which previous work has shown to be adaptable to the knowledge graph-to-text domain.

PDF Abstract

Datasets


Introduced in the Paper:

EventNarrative

Used in the Paper:

AGENDA GenWiki WikiGraphs
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
KG-to-Text Generation EventNarrative BART BLEU 31.38 # 3
METEOR 26.68 # 4
ROUGE 62.65 # 5
BertScore 93.12 # 3
ChrF++ 64.71 # 1
CIDEr 3.31 # 2
KG-to-Text Generation EventNarrative T5 BLEU 12.8 # 6
METEOR 22.77 # 6
ROUGE 52.06 # 6
BertScore 89.59 # 5
ChrF++ 56.76 # 2
CIDEr 3 # 3
KG-to-Text Generation EventNarrative GraphWriter BLEU 30.78 # 5
METEOR 27.72 # 1
ROUGE 71.92 # 1
BertScore 92.12 # 4
ChrF++ 47.91 # 3
CIDEr 4.59 # 1

Methods


No methods listed for this paper. Add relevant methods here