SpecNFS: A Challenge Dataset Towards Extracting Formal Models from Natural Language Specifications

1 code implementation • LREC 2022 • Sayontan Ghosh, Amanpreet Singh, Alex Merenstein, Wei Su, Scott A. Smolka, Erez Zadok, Niranjan Balasubramanian

Evaluations show that even when using a state-of-the-art language model, there is significant room for improvement, with the best models achieving an F1 score of only 60. 5 and 33. 3 in the named-entity-recognition and dependency-link-prediction sub-tasks, respectively.

Dependency Parsing Domain Adaptation +7