Fact-based Text Editing

ACL 2020  ยท  Hayate Iso, chao qiao, Hang Li ยท

We propose a novel text editing task, referred to as \textit{fact-based text editing}, in which the goal is to revise a given document to better describe the facts in a knowledge base (e.g., several triples). The task is important in practice because reflecting the truth is a common requirement in text editing. First, we propose a method for automatically generating a dataset for research on fact-based text editing, where each instance consists of a draft text, a revised text, and several facts represented in triples. We apply the method into two public table-to-text datasets, obtaining two new datasets consisting of 233k and 37k instances, respectively. Next, we propose a new neural network architecture for fact-based text editing, called \textsc{FactEditor}, which edits a draft text by referring to given facts using a buffer, a stream, and a memory. A straightforward approach to address the problem would be to employ an encoder-decoder model. Our experimental results on the two datasets show that \textsc{FactEditor} outperforms the encoder-decoder approach in terms of fidelity and fluency. The results also show that \textsc{FactEditor} conducts inference faster than the encoder-decoder approach.

PDF Abstract ACL 2020 PDF ACL 2020 Abstract

Datasets


Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Fact-based Text Editing RotoEdit FactEditor BLEU 84.43 # 1
SARI 74.72 # 1
KEEP 98.41 # 1
ADD 41.5 # 1
DELETE 84.24 # 1
Exact Match 2.65 # 1
Precision 78.84 # 1
Recall 52.3 # 1
F1 63.39 # 1
Fact-based Text Editing WebEdit FactEditor BLEU 75.68 # 1
SARI 72.2 # 1
KEEP 0.9184 # 1
ADD 47.69 # 1
DELETE 0.7707 # 1
Exact Match 24.8 # 1
Precision 96.88 # 3
Recall 89.74 # 1
F1 93.17 # 1
Fact-based Text Editing WebEdit EncDecEditor BLEU 71.03 # 2
SARI 69.59 # 2
KEEP 0.8949 # 2
ADD 43.82 # 2
DELETE 0.7548 # 2
Exact Match 20.96 # 2
Precision 98.06 # 2
Recall 87.56 # 2
F1 92.51 # 2
Fact-based Text Editing WebEdit Text-to-Text BLEU 63.61 # 4
SARI 58.73 # 3
KEEP 0.8262 # 3
ADD 25.77 # 4
DELETE 0.678 # 3
Exact Match 6.22 # 3
Precision 81.93 # 5
Recall 77.16 # 4
F1 79.48 # 5
Fact-based Text Editing WebEdit Table-to-Text BLEU 33.75 # 5
SARI 43.83 # 4
KEEP 0.5144 # 5
ADD 27.86 # 3
DELETE 0.5219 # 4
Exact Match 5.78 # 4
Precision 98.23 # 1
Recall 83.72 # 3
F1 90.4 # 3
Fact-based Text Editing WebEdit No-Editing BLEU 66.67 # 3
SARI 31.51 # 5
KEEP 0.7862 # 4
ADD 3.91 # 5
DELETE 0.1202 # 5
Exact Match 0 # 5
Precision 84.49 # 4
Recall 76.34 # 5
F1 80.21 # 4

Methods


No methods listed for this paper. Add relevant methods here