DDRel: A New Dataset for Interpersonal Relation Classification in Dyadic Dialogues

4 Dec 2020  ·  Qi Jia, Hongru Huang, Kenny Q. Zhu ·

Interpersonal language style shifting in dialogues is an interesting and almost instinctive ability of human. Understanding interpersonal relationship from language content is also a crucial step toward further understanding dialogues. Previous work mainly focuses on relation extraction between named entities in texts. In this paper, we propose the task of relation classification of interlocutors based on their dialogues. We crawled movie scripts from IMSDb, and annotated the relation labels for each session according to 13 pre-defined relationships. The annotated dataset DDRel consists of 6300 dyadic dialogue sessions between 694 pair of speakers with 53,126 utterances in total. We also construct session-level and pair-level relation classification tasks with widely-accepted baselines. The experimental results show that this task is challenging for existing models and the dataset will be useful for future research.

PDF Abstract

Datasets


Introduced in the Paper:

DDRel

Used in the Paper:

TACRED FewRel

Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Dialog Relation Extraction DDRel BERT Session-level 4-class Acc 47.1 # 1
Session-level 6-class Acc 41.87 # 1
Session-level 13-class Acc 39.4 # 1
Pair-level 4-class Acc 58.13 # 1
Pair-level 6-class Acc 42.33 # 1
Pair-level 13-class Acc 39.73 # 1

Methods


No methods listed for this paper. Add relevant methods here