A Unified Pre-training Framework for Conversational AI
In this work, we explore the application of PLATO-2 on various dialogue systems, including open-domain conversation, knowledge grounded dialogue, and task-oriented conversation. PLATO-2 is initially designed as an open-domain chatbot, trained via two-stage curriculum learning. In the first stage, a coarse-grained response generation model is learned to fit the simplified one-to-one mapping relationship. This model is applied to the task-oriented conversation, given that the semantic mappings tend to be deterministic in task completion. In the second stage, another fine-grained generation model and an evaluation model are further learned for diverse response generation and coherence estimation, respectively. With superior capability on capturing one-to-many mapping, such models are suitable for the open-domain conversation and knowledge grounded dialogue. For the comprehensive evaluation of PLATO-2, we have participated in multiple tasks of DSTC9, including interactive evaluation of open-domain conversation (Track3-task2), static evaluation of knowledge grounded dialogue (Track3-task1), and end-to-end task-oriented conversation (Track2-task1). PLATO-2 has obtained the 1st place in all three tasks, verifying its effectiveness as a unified framework for various dialogue systems.
PDF AbstractCode
Datasets
MultiWOZ
Results from the Paper
| Task | Dataset | Model | Metric Name | Metric Value | Global Rank | Benchmark |
|---|---|---|---|---|---|---|
| Interactive Evaluation of Dialog | DSTC9 Track 3 - Task 2 | PLATO-2 | Overall Human Rating | 4.15 | # 1 | |
| Coherent | 2.8017 | # 1 | ||||
| Error Recovery | 2.7518 | # 1 | ||||
| Consistent | 0.9390 | # 1 | ||||
| Diversity | 2.7441 | # 1 | ||||
| Topic Depth | 2.7678 | # 1 | ||||
| Likeable | 2.7878 | # 1 | ||||
| Understanding | 2.8285 | # 1 | ||||
| Flexible | 2.8000 | # 1 | ||||
| Informative | 2.7881 | # 1 | ||||
| Inquisitive | 2.7949 | # 1 |
