We call this dataset as MobileCS (mobile customer-service) dialog dataset, which differs from existing TOD datasets in both size and nature significantly. To the best of our knowledge, MobileCS is not only the largest publicly available TOD dataset, but also consists of real-life data (namely collected in real-world scenarios). For comparison, the widely used MultiWOZ dataset consists of 10,000 dialogs and is in fact simulated data (namely collected in a Wizard-of-Oz simulated game). See data statistics shown in Table S2T1. S2T1 Table 1: Comparison of our MobileCS corpus to MultiWOZ This metric measures if the system successfully completes the user goal by interacting with the user; Coherency. This metric measures whether the system’s response is logically coherent with the dialogue context; Fluency. The metric measures the fluency of the system’s response. The MobileCS dataset is annotated according to its schema and intent-sets. The schema used in our annotation of the MobileCS dataset is shown in Figure S4F3, which is needed for both information extraction and TOD systems. The intent-sets for users and custom-service staffs are shown in Listing S4F3, which are mainly required for building TOD systems. Entities in the MobileCS dataset are mostly related to telecommunication services, such as “King Package”. Entity attributes: refer to the attributes of entities, such as “业务费用” (service cost). Besides, user’s personal information is also annotated, such as the attribute “用户状态” (user status) for the concept “用户” (user), as shown in Figure S4F4. The dataset is introduced in detail, including the structure of annotations and the guideline for annotations. Finally, the challenge rules is described.

A Challenge on Semi-Supervised and Reinforced Task-Oriented Dialog Systems

Reader Guidelines

Editor Guidelines