SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inference

EMNLP 2018 Rowan ZellersYonatan BiskRoy SchwartzYejin Choi

Given a partial description like "she opened the hood of the car," humans can reason about the situation and anticipate what might come next ("then, she examined the engine"). In this paper, we introduce the task of grounded commonsense inference, unifying natural language inference and commonsense reasoning... (read more)

PDF Abstract

Evaluation results from the paper


 SOTA for Question Answering on SWAG (using extra training data)

     Get a GitHub badge
Task Dataset Model Metric name Metric value Global rank Uses extra
training data
Compare
Common Sense Reasoning SWAG ESIM + ELMo Dev 59.1 # 3
Common Sense Reasoning SWAG ESIM + ELMo Test 59.2 # 2
Common Sense Reasoning SWAG ESIM + GloVe Dev 51.9 # 4
Common Sense Reasoning SWAG ESIM + GloVe Test 52.7 # 3
Question Answering SWAG BERT Large Accuracy 86.3 # 1