Fusing task-oriented and open-domain dialogues in conversational agents

9 Sep 2021  ยท  Tom Young, Frank Xing, Vlad Pandelea, Jinjie Ni, Erik Cambria ยท

The goal of building intelligent dialogue systems has largely been separately pursued under two paradigms: task-oriented dialogue (TOD) systems, which perform goal-oriented functions, and open-domain dialogue (ODD) systems, which focus on non-goal-oriented chitchat. The two dialogue modes can potentially be intertwined together seamlessly in the same conversation, as easily done by a friendly human assistant. Such ability is desirable in conversational agents, as the integration makes them more accessible and useful. Our paper addresses this problem of fusing TODs and ODDs in multi-turn dialogues. Based on the popular TOD dataset MultiWOZ, we build a new dataset FusedChat, by rewriting the existing TOD turns and adding new ODD turns. This procedure constructs conversation sessions containing exchanges from both dialogue modes. It features inter-mode contextual dependency, i.e., the dialogue turns from the two modes depend on each other. Rich dependency patterns including co-reference and ellipsis are features. The new dataset, with 60k new human-written ODD turns and 5k re-written TOD turns, offers a benchmark to test a dialogue model's ability to perform inter-mode conversations. This is a more challenging task since the model has to determine the appropriate dialogue mode and generate the response based on the inter-mode context. But such models would better mimic human-level conversation capabilities. We evaluate baseline models on this task, including classification-based two-stage models and two-in-one fused models. We publicly release FusedChat and the baselines to propel future work on inter-mode dialogue systems https://github.com/tomyoung903/FusedChat.

PDF Abstract

Datasets


Introduced in the Paper:

FusedChat

Used in the Paper:

MultiWOZ

Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Dialogue Generation FusedChat Classification-based model Slot Accuracy 0.973 # 1
Joint SA 0.600 # 1
Inform 75.1 # 1
Inform_mct 90.8 # 1
Success 60.9 # 1
Success_mct 74.4 # 1
BLEU 12.17 # 1
PPL 10.50 # 1
Sensibleness 0.58 # 1
Specificity 0.51 # 1
SSA 0.55 # 1
Dialogue Generation FusedChat Two-in-one model Slot Accuracy 0.972 # 2
Joint SA 0.592 # 2
Inform 70.4 # 2
Inform_mct 90.1 # 2
Success 57.0 # 2
Success_mct 72.7 # 2
BLEU 12.05 # 2
PPL 10.49 # 2
Sensibleness 0.52 # 2
Specificity 0.47 # 2
SSA 0.50 # 2

Methods


No methods listed for this paper. Add relevant methods here