CPED: A Large-Scale Chinese Personalized and Emotional Dialogue Dataset for Conversational AI

Human language expression is based on the subjective construal of the situation instead of the objective truth conditions, which means that speakers' personalities and emotions after cognitive processing have an important influence on conversation. However, most existing datasets for conversational AI ignore human personalities and emotions, or only consider part of them. It's difficult for dialogue systems to understand speakers' personalities and emotions although large-scale pre-training language models have been widely used. In order to consider both personalities and emotions in the process of conversation generation, we propose CPED, a large-scale Chinese personalized and emotional dialogue dataset, which consists of multi-source knowledge related to empathy and personal characteristic. These knowledge covers gender, Big Five personality traits, 13 emotions, 19 dialogue acts and 10 scenes. CPED contains more than 12K dialogues of 392 speakers from 40 TV shows. We release the textual dataset with audio features and video features according to the copyright claims, privacy issues, terms of service of video platforms. We provide detailed description of the CPED construction process and introduce three tasks for conversational AI, including personality recognition, emotion recognition in conversations as well as personalized and emotional conversation generation. Finally, we provide baseline systems for these tasks and consider the function of speakers' personalities and emotions on conversation. Our motivation is to propose a dataset to be widely adopted by the NLP community as a new open benchmark for conversational AI research. The full dataset is available at https://github.com/scutcyr/CPED.

PDF Abstract

Datasets


Introduced in the Paper:

CPED
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Emotion Recognition in Conversation CPED BERT+AVG+MLP Accuracy of Sentiment 51.50 # 1
Macro-F1 of Sentiment 48.02 # 1
Personality Recognition in Conversation CPED BERT$_{ssenet}^{c}$ Accuracy (%) 67.25 # 1
Macro-F1 74.08 # 1
Accuracy of Neurotism 53.27 # 3
Accuracy of Extraversion 78.21 # 1
Accuracy of Openness 55.42 # 2
Accuracy of Agreeableness 85.89 # 1
Accuracy of Conscientiousness 63.48 # 2
Personality Recognition in Conversation CPED BERT$_{senet}^{c}$ Accuracy (%) 66.02 # 4
Macro-F1 71.89 # 4
Accuracy of Neurotism 53.4 # 2
Accuracy of Extraversion 77.71 # 4
Accuracy of Openness 55.42 # 2
Accuracy of Agreeableness 81.99 # 3
Accuracy of Conscientiousness 61.59 # 4
Personality Recognition in Conversation CPED BERT$^{c}$ Accuracy (%) 66.32 # 3
Macro-F1 72.69 # 3
Accuracy of Neurotism 55.29 # 1
Accuracy of Extraversion 78.08 # 2
Accuracy of Openness 53.90 # 4
Accuracy of Agreeableness 80.98 # 4
Accuracy of Conscientiousness 63.35 # 3
Personality Recognition in Conversation CPED BERT$^{s}$ Accuracy (%) 67.23 # 2
Macro-F1 72.93 # 2
Accuracy of Neurotism 50.75 # 4
Accuracy of Extraversion 78.08 # 2
Accuracy of Openness 57.93 # 1
Accuracy of Agreeableness 85.76 # 2
Accuracy of Conscientiousness 63.60 # 1
Personalized and Emotional Conversation CPED GPT-{emo} PPL 17.48 # 1
BLEU 0.1342 # 5
Distinct-1 0.0614 # 1
Distinct-2 0.3430 # 1
Greedy Embedding 0.4996 # 5
Average Embedding 0.5588 # 5
bertscore 0.5709 # 4
Personalized and Emotional Conversation CPED GPT-{per+emo+da} PPL 17.80 # 4
BLEU 0.1382 # 2
Distinct-1 0.0601 # 4
Distinct-2 0.3404 # 2
Greedy Embedding 05012 # 1
Average Embedding 0.5608 # 3
bertscore 0.5722 # 1
Personalized and Emotional Conversation CPED GPT-{per+emo} PPL 17.70 # 2
BLEU 0.1403 # 1
Distinct-1 0.0602 # 3
Distinct-2 0.3388 # 4
Greedy Embedding 0.5026 # 2
Average Embedding 0.5617 # 1
bertscore 0.5719 # 2
Personalized and Emotional Conversation CPED GPT-{da} PPL 17.72 # 3
BLEU 0.1372 # 3
Distinct-1 0.0605 # 2
Distinct-2 0.3389 # 3
Greedy Embedding 0.5017 # 3
Average Embedding 0.5610 # 2
bertscore 0.5703 # 5
Personalized and Emotional Conversation CPED GPT-{per} PPL 18.08 # 5
BLEU 0.1372 # 3
Distinct-1 0.0592 # 5
Distinct-2 0.3363 # 5
Greedy Embedding 0.5009 # 4
Average Embedding 0.5606 # 4
bertscore 0.5715 # 3
Personalized and Emotional Conversation CPED {emo+da}-GPT w/o da PPL 22.09 # 8
BLEU 0.1272 # 7
Distinct-1 0.0473 # 8
Distinct-2 0.2790 # 6
Greedy Embedding 0.4962 # 7
Average Embedding 0.5556 # 7
bertscore 0.5669 # 7
Personalized and Emotional Conversation CPED {emo+da}-GPT w/o emo PPL 22.84 # 9
BLEU 0.1252 # 8
Distinct-1 0.0451 # 9
Distinct-2 0.2746 # 8
Greedy Embedding 0.4964 # 6
Average Embedding 0.5564 # 6
bertscore 0.5666 # 8
Personalized and Emotional Conversation CPED {emo+da}-GPT PPL 21.60 # 7
BLEU 0.1304 # 6
Distinct-1 0.0476 # 7
Distinct-2 0.2785 # 7
Greedy Embedding 0.4962 # 7
Average Embedding 0.5552 # 8
bertscore 0.5674 # 6
Personalized and Emotional Conversation CPED GPT PPL 20.07 # 6
BLEU 0.1171 # 9
Distinct-1 0.0482 # 6
Distinct-2 0.2738 # 9
Greedy Embedding 0.4922 # 9
Average Embedding 0.5509 # 9
bertscore 0.5629 # 9

Methods