Search Results for author:
Found 17 papers, 0 papers with code
User-centered & Robust NLP OSS: Lessons Learned from Developing & Maintaining RSMTool
For the last 5 years, we have developed and maintained RSMTool – an open-source tool for evaluating NLP systems that automatically score written and spoken responses.
Do face masks introduce bias in speech technologies? The case of automated scoring of speaking proficiency
However, these differences do not lead to differences in human or automated scores of English language proficiency.
Using PRMSE to evaluate automated scoring systems in the presence of label noise
The effect of noisy labels on the performance of NLP systems has been studied extensively for system training.
The many dimensions of algorithmic fairness in educational applications
We illustrate that total fairness may not be achievable and that different definitions of fairness may require different solutions.
My Turn To Read: An Interleaved E-book Reading Tool for Developing and Struggling Readers
Literacy is crucial for functioning in modern society.
Using exemplar responses for training and evaluating automated speech scoring systems
In this paper we compare the performance of an automated speech scoring engine using two corpora: a corpus of almost 700, 000 randomly sampled spoken responses with scores assigned by one or two raters during operational scoring, and a corpus of 16, 500 exemplar responses with scores reviewed by multiple expert raters.
Atypical Inputs in Educational Applications
In large-scale educational assessments, the use of automated scoring has recently become quite common.
Towards Understanding Text Factors in Oral Reading
Using a case study, we show that variation in oral reading rate across passages for professional narrators is consistent across readers and much of it can be explained using features of the texts being read.
Speech- and Text-driven Features for Automated Scoring of English Speaking Tasks
We consider the automatic scoring of a task for which both the content of the response as well its spoken fluency are important.
A Large Scale Quantitative Exploration of Modeling Strategies for Content Scoring
We explore various supervised learning strategies for automated scoring of content knowledge for a large corpus of 130 different content-based questions spanning four subject areas (Science, Math, English Language Arts, and Social Studies) and containing over 230, 000 responses scored by human raters.
Continuous fluency tracking and the challenges of varying text complexity
This paper is a preliminary report on using text complexity measurement in the service of a new educational application.
Building Better Open-Source Tools to Support Fairness in Automated Scoring
Automated scoring of written and spoken responses is an NLP application that can significantly impact lives especially when deployed as part of high-stakes tests such as the GRE® and the TOEFL®.
Textual complexity as a predictor of difficulty of listening items in language proficiency tests
In this paper we explore to what extent the difficulty of listening items in an English language proficiency test can be predicted by the textual properties of the prompt.
