A Wonky But Important Argument for Annual Statewide Testing

In Saturday’s New York Times, I wrote a defense of annual statewide testing in reading and math. In the piece, I used data from the District of Columbia to illustrate that withdrawing from annual statewide testing would make it nearly impossible to hold schools accountable for the performance of specific groups of students. That’s a problem, because NCLB’s emphasis on historically disadvantaged groups forced schools to pay attention to these groups and led to real achievement gains. Today, 4th and 8th grade reading and math scores for black, Hispanic, and low-income students have never been higher.

To see how a move away from annual testing would affect subgroup accountability in other cities, I pulled data from Providence, Rhode Island and Richmond, Virginia. The results confirm that a move away from annual testing would leave many subgroups and more than 1 million students functionally “invisible” to state accountability systems. 

As a reminder, No Child Left Behind focuses attention on the progress of groups of students within schools. To be confident that the test results aren’t pulled up or down by a few students and to minimize year-to-year variability, states usually imposed minimum group sizes of 30 or 40 students.

Both Rhode Island and Virginia used relatively high group sizes under NCLB–Rhode Island used a group size of 45 and Virginia used 50. As part of the NCLB waiver process, which allowed states to use relative ranking school accountability systems as opposed to more of a relative ranking system and less of a formulaic trigger, both Rhode Island and Virginia lowered their group sizes. Rhode Island lowered its group size all the way down to 20, and Virginia dropped its group size to 30 students. After these changes, both Virginia and Rhode Island estimated that far more students and subgroups would “count” under their new rules.

For example, Virginia calculated that this change would, “increase the approximate number of schools accountable for African American students from 353 to 451, for Hispanic students from 122 to 183, students with disabilities from 105 to 396, English Learners from 104 to 139, and free and reduced price lunch students from 672 to 717.”

This progress would largely disappear without annual testing.

To see the effects in Rhode Island, I applied Rhode Island’s group size of 20 students to the city of Providence. Providence is relatively poor and has a large number of Hispanic students, and even under a grade-span approach where schools were only accountable for the performance of, say, 5th graders, all schools in the district had enough low-income and Hispanic 5th grade students for the groups to count. But only six out of 22 schools would be accountable for black students, only eight would be accountable for English Learners, five for students with disabilities, and only one for white students.

Without annual test results and under Rhode Island’s old group size of 45, 0 Providence schools would have been accountable for black, white, or students with disabilities.

Another way to look at this is in terms of how many total students would become “invisible” under a grade-span approach. The screenshot below shows the enrollment of a representative Providence elementary school (click on the table to make it bigger). The black squares represent all the students who are currently captured under NCLB’s annual testing requirement. Currently, 810 students “count” toward the school’s accountability rating for individual subgroups. (Note that these are “duplicated” counts, because a black student who is also low-income counts under both categories). Under a grade-span approach, represented by the red squares, only 190 students would count. That’s partly a function of fewer grades that are tested, but it’s also a function of more subgroups being invisible.

Rhode Island ex_before and after

Applying this same approach to Virginia, I looked at this year’s enrollment figures for Richmond’s public schools. Using the same minimum group size currently employed for entire schools (30) to the fifth grade only, here’s how many of Richmond’s 27 public elementary schools would be accountable for various subgroups of students:

– Black students: 22/27 schools
– Low-income students: 22/27
– White students: 2/27
– Asian students: 0/27
– Hispanic students: 2/27
– Students with disabilities: 0/27
– English Learners: 2/27

This may sound like a wonky, esoteric reason to keep annual statewide testing, but NCLB’s focus on the academic progress of historically disadvantaged groups of students was the right one. It caused schools to pay attention to particular groups of students within schools, and that added attention paid off. If we retreat on annual statewide testing, we’d go back to an earlier era where the progress of black, Hispanic, and low-income students could be easily over-looked.

3 thoughts on “A Wonky But Important Argument for Annual Statewide Testing

  1. Tom Hoffman

    Rhode Island *publishes* testing data for even smaller sample sizes, and has for years. You can check it out here: https://reporting.measuredprogress.org/necappublicri/select.aspx

    It looks like “n” is 10 for these reports.

    So basically you’re advocating spending a lot of money against the possibility that while Providence as a whole might look ok, equity-wise, individual schools — made up primarily of low-income minority students — might be serving their primary demographic group well, but systematically discriminating against smaller minority groups, AND simply publishing small subgroup data would not be sufficient to address this issue at the school level, BUT only if the federal government forces accountability at the school level at the scale of dozens of individual students.

    Also, I have to point out that in Providence the distinction between “Black or African American” and “Hispanic or Latino” is particularly complex, as many Hispanics are black, and African Americans are divided between um… “native” African Americans and first and second generation immigrants, including, say Cape Verdeans. If you wanted to systematically discriminate against African Americans but not Hispanics, it would take a lot of work to keep track of who was who.

  2. Schimpffy

    I don’t think you can have appropriate school choice without annual testing.

    Sen Alexander is pro school choice. He has mentioned charters frequently. I don’t think it’s consistent to be pro school choice without having the proper accountability mechanisms in place to monitor such choice (i.e. that would really be annual testing).

    School choice left to a perfect “free market” is subject to abuse. We’ve seen this in lax accountability states (look at the Ohio charter school landscape).

    In Tennessee, we now have a default closure law for charters. Without annual testing, that means that charters could be subject to one grade of scores being cause for them to be shut down or not. Concentrating such make or break incentives to one grade in my opinion is really asking for all sorts of corruption and abuse in the school choice landscape.

    (you alluded to this in your point on things being more high stakes without annual testing in the NY Times opinion)

Comments are closed.