BREAKING: Setting Cut Scores is Hard

All students should be held to the same learning expectations throughout their K-12 experience. All teachers should meet the same level of practitioner ability before entering the profession. These two statements seem like no brainers. But in practice, determining the cut scores for student and teacher proficiency is complicated. So complicated that even well-meaning reforms have failed at the task.

This week, Motoko Rich of The New York Times highlighted that states using the same assessment to test students’ proficiency on the Common Core State Standards are setting different proficiency labels. This means that while all students in Ohio, Illinois, and Massachusetts took the same assessment this year (PARCC), a 4th grade student in Ohio who is marked as proficient would be considered below proficient in Illinois and Massachusetts due to higher cut scores in the latter two states.* This disconnect is not new in K-12 education. For a long time states had their own homegrown standards and assessments so cross-state comparisons were all but impossible. It’s just that the Common Core was supposed to change all of that.

Teachers face their own challenges in crossing state lines. A new educator assessment, edTPA, was developed to streamline the skills and knowledge teachers should possess before entering the classroom. This year, 17 states used edTPA and six of them attached consequences to the results. Of those six states, the passage cut scores for the assessment varied greatly state to state and were well below edTPA’s recommendations.

edTPA scores_cut scores

(Click on graph for a larger image.)

All this shows that setting a minimum bar for student and teacher performance is hard. Political and practical barriers get in the way. In setting cut scores for students, states like Ohio experienced major political backlash which no doubt impacted their decisions. When it comes to cut scores for teacher licensure and certification, practical supply and demand matters ultimately impact how selective states can be with their minimum threshold bars. And to be sure, for both students and teachers, we still don’t know whether the new assessments will actually predict later life outcomes like student college-readiness or teacher effectiveness.

None of this, however, should stop the pursuit of a level playing field for teachers and students. In our decentralized education system, students and teachers can and do cross state lines. Finding a way to ease those transitions could boost the quality of the education students receive.

*All of these states administered the same PARCC assessment this year (Disclosure: PARCC is a Bellwether client). Their results won’t be directly comparable to states administering other assessments like Smarter Balanced. As Carmel Martin notes, it is possible to compare state-level proficiency rates on PARCC if you know where to look. But states have chosen to send very different messages to parents about their children’s scores and most states are not PARCC states. To make their own cross-state comparisons, parents would need to know how to find various state cut scores and then convert their child’s scale score into proficiency rates in another state.