Have Your Text and Use It Too! End-to-End Neural Data-to-Text Generation with Semantic Fidelity

COLING 2020  ·  Hamza Harkous, Isabel Groves, Amir Saffari ·

End-to-end neural data-to-text (D2T) generation has recently emerged as an alternative to pipeline-based architectures. However, it has faced challenges in generalizing to new domains and generating semantically consistent text. In this work, we present DataTuner, a neural, end-to-end data-to-text generation system that makes minimal assumptions about the data representation and the target domain. We take a two-stage generation-reranking approach, combining a fine-tuned language model with a semantic fidelity classifier. Each of our components is learnt end-to-end without the need for dataset-specific heuristics, entity delexicalization, or post-processing. We show that DataTuner achieves state of the art results on the automated metrics across four major D2T datasets (LDC2017T10, WebNLG, ViGGO, and Cleaned E2E), with a fluency assessed by human annotators nearing or exceeding the human-written reference texts. We further demonstrate that the model-based semantic fidelity scorer in DataTuner is a better assessment tool compared to traditional, heuristic-based measures. Our generated text has a significantly better semantic fidelity than the state of the art across all four datasets

PDF Abstract COLING 2020 PDF COLING 2020 Abstract

Results from the Paper

Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Data-to-Text Generation Cleaned E2E NLG Challenge DataTuner_FC BLEU (Test set) 43.6 # 2
AMR-to-Text Generation LDC2017T10 DataTuner_FC BLEU 37.7 # 6
Data-to-Text Generation ViGGO DataTuner_FC BLEU 53.6 # 1
Data-to-Text Generation WebNLG Full DATATUNER_NO_FC BLEU 52.9 # 7