WikiGraphs: A Wikipedia Text - Knowledge Graph Paired Dataset

We present a new dataset of Wikipedia articles each paired with a knowledge graph, to facilitate the research in conditional text generation, graph generation and graph representation learning. Existing graph-text paired datasets typically contain small graphs and short text (1 or few sentences), thus limiting the capabilities of the models that can be learned on the data. Our new dataset WikiGraphs is collected by pairing each Wikipedia article from the established WikiText-103 benchmark (Merity et al., 2016) with a subgraph from the Freebase knowledge graph (Bollacker et al., 2008). This makes it easy to benchmark against other state-of-the-art text generative models that are capable of generating long paragraphs of coherent text. Both the graphs and the text data are of significantly larger scale compared to prior graph-text paired datasets. We present baseline graph neural network and transformer model results on our dataset for 3 tasks: graph -> text generation, graph -> text retrieval and text -> graph retrieval. We show that better conditioning on the graph provides gains in generation and retrieval quality but there is still large room for improvement.

PDF Abstract NAACL (TextGraphs) 2021 PDF NAACL (TextGraphs) 2021 Abstract

Datasets


Introduced in the Paper:

WikiGraphs

Used in the Paper:

WikiText-2 DBpedia WikiText-103 GenWiki

Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
KG-to-Text Generation WikiGraphs Unconditional Test perplexity 25.85 # 1
rBLEU (Valid) 10.97 # 4
rBLEU (Test) 9.98 # 4
rBLEU(w/title)(Valid) 27.98 # 4
rBLEU(w/title)(Test) 24.07 # 4
KG-to-Text Generation WikiGraphs Nodes Test perplexity 27.4 # 4
rBLEU (Valid) 30.51 # 2
rBLEU (Test) 25.31 # 2
rBLEU(w/title)(Valid) 32.60 # 2
rBLEU(w/title)(Test) 27.43 # 2
KG-to-Text Generation WikiGraphs BoW Test perplexity 26.65 # 2
rBLEU (Valid) 29.53 # 3
rBLEU (Test) 24.41 # 3
rBLEU(w/title)(Valid) 32.41 # 3
rBLEU(w/title)(Test) 27.39 # 3
KG-to-Text Generation WikiGraphs GNN Test perplexity 26.93 # 3
rBLEU (Valid) 31.39 # 1
rBLEU (Test) 26.22 # 1
rBLEU(w/title)(Valid) 32.65 # 1
rBLEU(w/title)(Test) 28.35 # 1

Methods


No methods listed for this paper. Add relevant methods here