DeliData: A dataset for deliberation in multi-party problem solving

11 Aug 2021  ·  Georgi Karadzhov, Tom Stafford, Andreas Vlachos ·

Dialogue systems research is traditionally focused on dialogues between two interlocutors, largely ignoring group conversations. Moreover, most previous research is focused either on task-oriented dialogue (e.g.\ restaurant bookings) or user engagement (chatbots), while research on systems for collaborative dialogues is an under-explored area. To this end, we introduce the first publicly available dataset containing collaborative conversations on solving a cognitive task, consisting of 500 group dialogues and 14k utterances. Furthermore, we propose a novel annotation schema that captures deliberation cues and release 50 dialogues annotated with it. Finally, we demonstrate the usefulness of the annotated data in training classifiers to predict the constructiveness of a conversation. The data collection platform, dataset and annotated corpus are publicly available at https://delibot.xyz

PDF Abstract
No code implementations yet. Submit your code now

Datasets


Introduced in the Paper:

DeliData
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Problem-Solving Deliberation DeliData Bag-Of-Annotations + Sol. Tracker AUC 0.62 # 1
Problem-Solving Deliberation DeliData Conversation + Sol. Tracker AUC 0.53 # 2
Problem-Solving Deliberation DeliData Majority Class Baseline AUC 0.50 # 3

Methods


No methods listed for this paper. Add relevant methods here