ReTraCk: A Flexible and Efficient Framework for Knowledge Base Question Answering

We present Retriever-Transducer-Checker (ReTraCk), a neural semantic parsing framework for large scale knowledge base question answering (KBQA). ReTraCk is designed as a modular framework to maintain high flexibility. It includes a retriever to retrieve relevant KB items efficiently, a transducer to generate logical form with syntax correctness guarantees and a checker to improve transduction procedure. ReTraCk is ranked at top1 overall performance on the GrailQA leaderboard and obtains highly competitive performance on the typical WebQuestionsSP benchmark. Our system can interact with users timely, demonstrating the efficiency of the proposed framework.

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Benchmark
Knowledge Base Question Answering GrailQA ReTraCk Overall F1 65.3 # 1
Overall EM 58.1 # 1
I.I.D. F1 87.5 # 1
I.I.D. EM 84.4 # 1
Compositional F1 70.9 # 1
Compositional EM 61.5 # 1
Zero-shot F1 52.5 # 1
Zero-shot EM 44.6 # 1
Knowledge Base Question Answering WebQuestionsSP ReTraCk Oracle EL Hits@1 74.6 # 3
F1 74.7 # 1
Knowledge Base Question Answering WebQuestionsSP ReTraCk Hits@1 71.6 # 4
F1 71 # 2

Methods


No methods listed for this paper. Add relevant methods here