IAM: A Comprehensive and Large-Scale Dataset for Integrated Argument Mining Tasks

Traditionally, a debate usually requires a manual preparation process, including reading plenty of articles, selecting the claims, identifying the stances of the claims, seeking the evidence for the claims, etc. As the AI debate attracts more attention these years, it is worth exploring the methods to automate the tedious process involved in the debating system. In this work, we introduce a comprehensive and large dataset named IAM, which can be applied to a series of argument mining tasks, including claim extraction, stance classification, evidence extraction, etc. Our dataset is collected from over 1k articles related to 123 topics. Near 70k sentences in the dataset are fully annotated based on their argument properties (e.g., claims, stances, evidence, etc.). We further propose two new integrated argument mining tasks associated with the debate preparation process: (1) claim extraction with stance classification (CESC) and (2) claim-evidence pair extraction (CEPE). We adopt a pipeline approach and an end-to-end method for each integrated task separately. Promising experimental results are reported to show the values and challenges of our proposed tasks, and motivate future research on argument mining.

PDF Abstract ACL 2022 PDF ACL 2022 Abstract

Datasets


Introduced in the Paper:

IAM Dataset
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Claim Extraction with Stance Classification (CESC) IAM Dataset Multi-label Macro F1 60.25 # 1
Claim-Evidence Pair Extraction (CEPE) IAM Dataset Multi-task F1 35.92 # 1

Methods


No methods listed for this paper. Add relevant methods here