Evaluating Concurrent Validity and Classification Accuracy of a Short Web-based Adaptive Assessment Compared to Two Assessments Widely Used in the United States

Summary of Findings:

Valid and reliable assessments of reading proficiency are essential to educators’ ability to make informed instructional decisions. This study evaluated the results of InSight, a web-based, adaptive reading assessment, in relation to the Smarter Balanced (SBAC) and the Partnership for Assessment of Readiness for College and Careers (PARCC) English Language Arts (ELA) assessments used in eighteen US states and other geopolitical entities during the 2018-19 school year. InSight can be administered in a group or individualized setting with minimal teacher oversight. Students typically need 30-40 minutes to complete the assessment and the results are immediately available. SBAC and PARCC ELA assessments require more oversight and time to administer, usually 3-4 hours, and results are provided within a few weeks to a few months after administration.

InSight showed strong concurrent validity (Pearson’s r >.70) and classification accuracy (lower bound of the Area Under the Curve (AUC) confidence interval > .80 and overall classification accuracy rate >75%) with the SBAC and PARCC ELA assessments in each grade measured (3-8) and overall.

“The ability for a short reading assessment (InSIght) to accurately classify students’ reading/ELA proficiency in comparison to widely-used assessments used for high-stakes accountability is critical for many instructional decisions such as tiered interventions for at-risk students.”