11 Atlanta educators were convicted this week in a shameful conspiracy to alter student test scores. The Atlantic and other news outlets have been quick to call this an example of “Campbell’s Law,” the theory that any quantitative measure of accountability will quickly be corrupted by manipulation or outright cheating. Taken to the extreme, Campbell’s Law suggests we shouldn’t use data to make any policy decisions.
The truth is that the Atlanta story is far more complicated than that. Although 11 educators were convicted of cheating on state tests (and it would likely have been 12 if not for the death of former Superintendent Beverly Hall), the city made remarkable improvements on low-stakes measures of educational progress such as the National Assessment of Educational Progress (NAEP).
NAEP is far more difficult to game, and it requires higher-order thinking skills than Georgia’s prior state tests. And yet Atlanta under Hall tied for the biggest NAEP gains in 8th-grade reading (see graph below). Its gains were bigger than those of other celebrated “reform” cities like New York, the average of all large cities, and Georgia as a state. Atlanta’s students were also making far more progress than the nation as a whole. Continue reading