ReTraCk: A Flexible and Efficient Framework for Knowledge Base Question Answering
We present Retriever-Transducer-Checker (ReTraCk), a neural semantic parsing framework for large scale knowledge base question answering (KBQA). ReTraCk is designed as a modular framework to maintain high flexibility. It includes a retriever to retrieve relevant KB items efficiently, a transducer to generate logical form with syntax correctness guarantees and a checker to improve transduction procedure. ReTraCk is ranked at top1 overall performance on the GrailQA leaderboard and obtains highly competitive performance on the typical WebQuestionsSP benchmark. Our system can interact with users timely, demonstrating the efficiency of the proposed framework.
PDF AbstractCode
Datasets
| Task | Dataset | Model | Metric Name | Metric Value | Global Rank | Benchmark |
|---|---|---|---|---|---|---|
| Knowledge Base Question Answering | GrailQA | ReTraCk | Overall F1 | 65.3 | # 1 | |
| Overall EM | 58.1 | # 1 | ||||
| I.I.D. F1 | 87.5 | # 1 | ||||
| I.I.D. EM | 84.4 | # 1 | ||||
| Compositional F1 | 70.9 | # 1 | ||||
| Compositional EM | 61.5 | # 1 | ||||
| Zero-shot F1 | 52.5 | # 1 | ||||
| Zero-shot EM | 44.6 | # 1 | ||||
| Knowledge Base Question Answering | WebQuestionsSP | ReTraCk | Hits@1 | 71.6 | # 6 | |
| F1 | 71 | # 3 | ||||
| Knowledge Base Question Answering | WebQuestionsSP | ReTraCk Oracle EL | Hits@1 | 74.6 | # 5 | |
| F1 | 74.7 | # 2 |
Methods
No methods listed for this paper. Add
relevant methods here
