Sequential Matching Network: A New Architecture for Multi-turn Response Selection in Retrieval-based Chatbots

ACL 2017  ·  Yu Wu, Wei Wu, Chen Xing, Ming Zhou, Zhoujun Li ·

We study response selection for multi-turn conversation in retrieval-based chatbots. Existing work either concatenates utterances in context or matches a response with a highly abstract context vector finally, which may lose relationships among utterances or important contextual information. We propose a sequential matching network (SMN) to address both problems. SMN first matches a response with each utterance in the context on multiple levels of granularity, and distills important matching information from each pair as a vector with convolution and pooling operations. The vectors are then accumulated in a chronological order through a recurrent neural network (RNN) which models relationships among utterances. The final matching score is calculated with the hidden states of the RNN. An empirical study on two public data sets shows that SMN can significantly outperform state-of-the-art methods for response selection in multi-turn conversation.

PDF Abstract ACL 2017 PDF ACL 2017 Abstract

Datasets


Introduced in the Paper:

Douban Douban Conversation Corpus

Used in the Paper:

UDC E-commerce RRS
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Conversational Response Selection Douban SMN MAP 0.529 # 12
MRR 0.569 # 12
P@1 0.397 # 12
R10@1 0.233 # 12
R10@2 0.396 # 12
R10@5 0.724 # 12
Conversational Response Selection E-commerce SMN R10@1 0.453 # 13
R10@2 0.654 # 13
R10@5 0.886 # 13
Conversational Response Selection RRS SMN R10@1 0.281 # 7
MAP 0.487 # 7
MRR 0.501 # 7
P@1 0.309 # 7
R10@2 0.442 # 7
R10@5 0.723 # 7
Conversational Response Selection Ubuntu Dialogue (v1, Ranking) SMN R10@1 0.726 # 18
R10@2 0.822 # 17
R10@5 0.960 # 17
R2@1 0.926 # 10

Methods