Grounded Adaptation for Zero-shot Executable Semantic Parsing

We propose Grounded Adaptation for Zero-shot Executable Semantic Parsing (GAZP) to adapt an existing semantic parser to new environments (e.g. new database schemas). GAZP combines a forward semantic parser with a backward utterance generator to synthesize data (e.g. utterances and SQL queries) in the new environment, then selects cycle-consistent examples to adapt the parser. Unlike data-augmentation, which typically synthesizes unverified examples in the training environment, GAZP synthesizes examples in the new environment whose input-output consistency are verified. On the Spider, Sparc, and CoSQL zero-shot semantic parsing tasks, GAZP improves logical form and execution accuracy of the baseline parser. Our analyses show that GAZP outperforms data-augmentation in the training environment, performance increases with the amount of GAZP-synthesized data, and cycle-consistency is central to successful adaptation.

PDF Abstract EMNLP 2020 PDF EMNLP 2020 Abstract


Results from the Paper

Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Dialogue State Tracking CoSQL GAZP+BERT question match accuracy 39.7 # 7
interaction match accuracy 12.8 # 7
Text-To-SQL SParC GAZP + BERT interaction match accuracy 23.5 # 6
question match accuracy 45.9 # 6


No methods listed for this paper. Add relevant methods here