SVAMP (Simple Variations on Arithmetic Math word Problems)

Introduced by Patel et al. in Are NLP Models really able to Solve Simple Math Word Problems?

A challenge set for elementary-level Math Word Problems (MWP). An MWP consists of a short Natural Language narrative that describes a state of the world and poses a question about some unknown quantities.

The examples in SVAMP test a model across different aspects of solving MWPs: 1) Is the model question sensitive? 2) Does the model have robust reasoning ability? 3) Is it invariant to structural alterations?


Paper Code Results Date Stars

Dataset Loaders


Similar Datasets


  • MIT