DeliData: A dataset for deliberation in multi-party problem solving

11 Aug 2021  ·  Georgi Karadzhov, Tom Stafford, Andreas Vlachos ·

Group deliberation enables people to collaborate and solve problems, however, it is understudied due to a lack of resources. To this end, we introduce the first publicly available dataset containing collaborative conversations on solving a well-established cognitive task, consisting of 500 group dialogues and 14k utterances. In 64% of these conversations, the group members are able to find a better solution than they had identified individually, and in 43.8% of the groups who had a correct answer as their final solution, none of the participants had solved the task correctly by themselves. Furthermore, we propose a novel annotation schema that captures deliberation cues and release all 14k utterances annotated with it. Finally, we use the proposed dataset to develop and evaluate two methods for generating deliberation utterances. The data collection platform, dataset and annotated corpus are publicly available at https://delibot.xyz.

PDF Abstract
No code implementations yet. Submit your code now

Datasets


Introduced in the Paper:

DeliData
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Problem-Solving Deliberation DeliData Bag-Of-Annotations + Sol. Tracker AUC 0.62 # 1
Problem-Solving Deliberation DeliData Conversation + Sol. Tracker AUC 0.53 # 2
Problem-Solving Deliberation DeliData Majority Class Baseline AUC 0.50 # 3

Methods


No methods listed for this paper. Add relevant methods here