no code implementations • 17 Aug 2020 • Anastassia Loukina, Keelan Evanini, Matthew Mulholland, Ian Blood, Klaus Zechner
However, these differences do not lead to differences in human or automated scores of English language proficiency.
no code implementations • WS 2019 • Anastassia Loukina, Nitin Madnani, Klaus Zechner
We illustrate that total fairness may not be achievable and that different definitions of fairness may require different solutions.
no code implementations • WS 2019 • Xinhao Wang, Binod Gyawali, James V. Bruno, Hillary R. Molloy, Keelan Evanini, Klaus Zechner
This study aims to model the discourse structure of spontaneous spoken responses within the context of an assessment of English speaking proficiency for non-native speakers.
no code implementations • NAACL 2018 • Su-Youn Yoon, Aoife Cahill, Anastassia Loukina, Klaus Zechner, Brian Riordan, Nitin Madnani
In large-scale educational assessments, the use of automated scoring has recently become quite common.
no code implementations • WS 2018 • Anastassia Loukina, Klaus Zechner, James Bruno, Beata Beigman Klebanov
In this paper we compare the performance of an automated speech scoring engine using two corpora: a corpus of almost 700, 000 randomly sampled spoken responses with scores assigned by one or two raters during operational scoring, and a corpus of 16, 500 exemplar responses with scores reviewed by multiple expert raters.
no code implementations • ACL 2017 • Xinhao Wang, James Bruno, Hillary Molloy, Keelan Evanini, Klaus Zechner
Considering that the measurement of discourse coherence is typically a key metric in human scoring rubrics for assessments of spoken language, we initiated a research effort to obtain RST annotations of a large number of non-native spoken responses from a standardized assessment of academic English proficiency.