Douban (Douban Conversation Corpus)
Introduced by Wu et al. in Sequential Matching Network: A New Architecture for Multi-turn Response Selection in Retrieval-based Chatbots
We release Douban Conversation Corpus, comprising a training data set, a development set and a test set for retrieval based chatbot. The statistics of Douban Conversation Corpus are shown in the following table.
| Train | Val | Test | |
|---|---|---|---|
| session-response pairs | 1m | 50k | 10k |
| Avg. positive response per session | 1 | 1 | 1.18 |
| Fless Kappa | N\A | N\A | 0.41 |
| Min turn per session | 3 | 3 | 3 |
| Max ture per session | 98 | 91 | 45 |
| Average turn per session | 6.69 | 6.75 | 5.95 |
| Average Word per utterance | 18.56 | 18.50 | 20.74 |
The test data contains 1000 dialogue context, and for each context we create 10 responses as candidates. We recruited three labelers to judge if a candidate is a proper response to the session. A proper response means the response can naturally reply to the message given the context. Each pair received three labels and the majority of the labels was taken as the final decision.
As far as we known, this is the first human-labeled test set for retrieval-based chatbots. The entire corpus link https://www.dropbox.com/s/90t0qtji9ow20ca/DoubanConversaionCorpus.zip?dl=0
Benchmarks
| Trend | Task | Dataset Variant | Best Model | Paper | Code |
|---|---|---|---|---|---|
|
Douban
|
SEMSOL
|
|||
|
Douban Monti
|
GLocal-K
|
|||
|
Douban
|
I-CFN
|
|||
|
Douban
|
HSRL
|
Papers
| Paper | Code | Results | Date | Stars |
|---|
