The benchmarks section lists all benchmarks using a given dataset or any of
its variants. We use variants to distinguish between results evaluated on
slightly different versions of the same dataset. For example, ImageNet 32⨉32
and ImageNet 64⨉64 are variants of the ImageNet dataset.
Dialog System Technology Challenges 8 (DSTC) Track 2 builds on the success of DSTC 7 Track 1 (NOESIS: Noetic End-to-End Response Selection Challenge). It proposes an extension of the task, incorporating new elements that are vital for the creation of a deployed task-oriented dialogue system. Specifically, three new dimensions are added to the challenge:
Conversations with more than 2 participants
Predicting whether a dialogue has solved the problem yet,
Handling multiple simultaneous conversations. Each of these adds an exciting new dimension and brings the task closer to the creation of systems able to handle the complexity of real-world conversation.
This challenge is offered with two goal oriented dialog datasets, used in 4 subtasks.