VALUE: Understanding Dialect Disparity in NLU

1 code implementation • ACL 2022 • Caleb Ziems, Jiaao Chen, Camille Harris, Jessica Anderson, Diyi Yang

To understand disparities in current models and to facilitate more dialect-competent NLU systems, we introduce the VernAcular Language Understanding Evaluation (VALUE) benchmark, a challenging variant of GLUE that we created with a set of lexical and morphosyntactic transformation rules.

Linguistic Acceptability Natural Language Understanding