Testing Results

This report provides an update on analyses performed to investigate the comparison in student achievement on Smarter Balanced summative tests from 2016 (spring, 2016) to 2017 (spring, 2017). This comparison raised questions among educators about the validity of the 2017 test scores and the possibility that there might have been technical errors associated with changes in the item pool and other components of the administration. Subsequently, Smarter Balanced worked with its Technical Advisory Committee (TAC) to identify analyses that could be performed to address educator’s questions and related technical issues.

Smarter Balanced Assessment Consortium (2018). Answers to questions about smarter balanced 2017 test results March 27, 2018. Santa Clara, CA: Smarter Balanced Assessment Consortium. http://portal.smarterbalanced.org/library/en/answers-to-questions-about-2017-test-results.pdf

Testing Using Pen Versus Keyboard

This study seeks to examine the rollout of computer-based testing in Massachusetts over 2 years to investigate test mode effects. Findings on the effects of online testing in the second year of administration suggest that mode effects for second-time test takers were about one third as large as the first year in math and about half as large in ELA. There is little evidence of systematic variation in mode effects by student demographic groups, although on ELA tests they are larger for students scoring at the bottom of the achievement distribution.

Backes, B., & Cowan, J. (2018). Is the pen mightier than the keyboard? The effect of online testing on measured student achievement. CALDER Working Paper, 190. https://caldercenter.org/sites/default/files/WP%20190.pdf