Benchmarking Hierarchical Script Knowledge

NAACL 2019  ·  Yonatan Bisk, Jan Buys, Karl Pichotta, Yejin Choi ·

Understanding procedural language requires reasoning about both hierarchical and temporal relations between events. For example, {``}boiling pasta{''} is a sub-event of {``}making a pasta dish{''}, typically happens before {``}draining pasta,{''} and requires the use of omitted tools (e.g. a strainer, sink...). While people are able to choose when and how to use abstract versus concrete instructions, the NLP community lacks corpora and tasks for evaluating if our models can do the same. In this paper, we introduce KidsCook, a parallel script corpus, as well as a cloze task which matches video captions with missing procedural details. Experimental results show that state-of-the-art models struggle at this task, which requires inducing functional commonsense knowledge not explicitly stated in text.

PDF Abstract


  Add Datasets introduced or used in this paper

Results from the Paper

  Submit results from this paper to get state-of-the-art GitHub badges and help the community compare results to other papers.


No methods listed for this paper. Add relevant methods here