AxCell: Automatic Extraction of Results from Machine Learning Papers

Tracking progress in machine learning has become increasingly difficult with the recent explosion in the number of papers. In this paper, we present AxCell, an automatic machine learning pipeline for extracting results from papers. AxCell uses several novel components, including a table segmentation subtask, to learn relevant structural knowledge that aids extraction. When compared with existing methods, our approach significantly improves the state of the art for results extraction. We also release a structured, annotated dataset for training models for results extraction, and a dataset for evaluating the performance of models on this task. Lastly, we show the viability of our approach enables it to be used for semi-automated results extraction in production, suggesting our improvements make this task practically viable for the first time. Code is available on GitHub.

PDF Abstract EMNLP 2020 PDF EMNLP 2020 Abstract

Datasets


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Scientific Results Extraction NLP-TDMS (Exp, arXiv only) AxCell Micro Precision 27.4 # 1
Micro Recall 24.4 # 1
Micro F1 25.8 # 1
Macro Precision 20.2 # 1
Macro Recall 20.6 # 1
Macro F1 19.7 # 1
Scientific Results Extraction PWC Leaderboards (restricted) AxCell Micro Precision 37.4 # 1
Micro Recall 23.2 # 1
Micro F1 28.7 # 1
Macro Precision 24 # 1
Macro Recall 21.8 # 1
Macro F1 21.1 # 1

Methods


No methods listed for this paper. Add relevant methods here