Category Archives: Accountability

NAEP Results Again Show That Biennial National Tests Aren’t Worth It

Once again, new results from the National Assessment of Educational Progress (NAEP) show that administering national math and reading assessments every two years is too frequent to be useful.

The 2017 NAEP scores in math and reading were largely unchanged from 2015, when those subjects were last tested. While there was a small gain in eighth-grade reading in 2017 — a one-point increase on NAEP’s 500-point scale — it was not significantly different than eighth graders’ performance in 2013.

Many acknowledged that NAEP gains have plateaued in recent years after large improvements in earlier decades, and some have even described 2007-2017 as the “lost decade of educational progress.” But this sluggishness also shows that administering NAEP’s math and reading tests (referred to as the “main NAEP”) every two years is not necessary, as it is too little time to meaningfully change trend lines or evaluate the impact of new policies.

Such frequent testing also has other costs: In recent years, the National Assessment Governing Board (NAGB), the body that sets policy for NAEP, has reduced the frequency of the Long-Term Trends (LTT) assessment and limited testing in other important subjects like civics and history in order to cut costs. NAGB cited NAEP budget cuts as the reason for reducing the frequency of other assessments. However, though NAEP’s budget recovered and even increased in the years following, NAGB did not undo the previously scheduled reductions. (The LTT assessment is particularly valuable, as it tracks student achievement dating back to the early 1970s and provides another measure of academic achievement in addition to the main NAEP test.)

Instead, the additional funding was used to support other NAGB priorities, namely the shift to digital assessments. Even still, the release of the 2017 data was delayed by six months due to comparability concerns, and some education leaders are disputing the results because their students are not familiar enough with using tablets.

That is not to say that digital assessments don’t have benefits. For example, the new NAEP results include time lapse visualizations of students’ progress on certain types of questions. In future iterations of the test, these types of metadata could provide useful information about how various groups of students differ in their test-taking activity.

Animated GIF - Find & Share on GIPHY

However, these innovative approaches should not come at the expense of other assessments that are useful in the present. Given the concerns some have with the digital transition, this is especially true of the LTT assessment. Instead, NAGB should consider administering the main NAEP test less frequently — perhaps only every four years — and use the additional capacity to support other assessment types and subjects.

Three Reasons to Expect Little on Innovative Assessments — and Why That’s Not Such a Bad Thing

Photo by Josh Davis via Flickr

Next week is the deadline for states to submit an application for the innovative assessment pilot to the U.S. Department of Education (ED). If you missed this news, don’t worry, you haven’t missed much. The Every Student Succeeds Act (ESSA) allows ED to grant assessment flexibility to up to seven states to do something different from giving traditional end-of-year standardized tests. The best example of an innovative state assessment system is New Hampshire, which allows some districts to give locally designed performance-based assessments. These assessments look more like in-class activities than traditional standardized tests, and are developed and scored by teachers.

Two years ago, Education Week called the innovative assessment pilot “one of the most buzzed-about pieces” of ESSA because it could allow states to respond to testing pushback while still complying with the new federal law. But now only four states have announced they will apply, and expectations are subdued at best.

Why aren’t more states interested an opportunity to get some leeway on testing? Here are three big reasons:

  1. Most states are playing it safe on ESSA and assessments are no exception

When my colleagues at Bellwether convened an independent review of ESSA state plans with 45 education policy experts, they didn’t find much ambition or innovation in state plans — few states went beyond the requirements of the law, and some didn’t even do that. Even Secretary of Education Betsy DeVos, who has approved the majority of state plans, recently criticized states for plans that “only meet the bare minimum” and don’t take full advantage of the flexibility offered in the law.

Several states responded that they were actually doing more than they had indicated in their plans. As my colleague Julie Squire pointed out last year, putting something extra in an ESSA plan could limit a state’s options and bring on more federal monitoring. If most states were fairly conservative and compliance-based with their big ESSA plans, there’s little reason to think they’ll unveil something new and surprising in a small-scale waiver application.

Additionally, the law includes several requirements for an innovative assessment that might be difficult for states to meet. For example, innovative tests have to be comparable across school districts, they have to meet the needs of special education students and English learners, and the pilot programs have to be designed to scale up statewide. If states have any doubts they can meet that bar, they probably won’t apply. Continue reading

What Does it Take to Be a Quality Authorizer?

The autonomy-for-accountability bargain at the heart of the charter movement rests, crucially, on the effectiveness of the entities — known as authorizers — that have the ability to approve charter schools and the responsibility for holding them accountable. If authorizers are lax in their responsibilities — approving weak applications, failing to effectively monitor or assess school performance, or refusing to close low-performing schools — the accountability part of the bargain isn’t held up. But if they overstep their bounds, by limiting the kinds of schools they will approve, being overly prescriptive about requirements for school approval, or trying to micromanage schools they oversee, the autonomy part of the bargain goes missing. Getting the right balance between holding schools accountable and protecting their autonomy is a crucial question, both for authorizers and the charter movement as a whole, and since the start of the charter movement, it’s been the subject of heated debate — one that has intensified in recent years.

Continue reading

Best in Bellwether 2017: Our Most Read Publications and Posts

Below are the most read posts from Ahead of the Heard and our most read publications in 2017! (To read the top posts from our sister site, TeacherPensions.org, click here.)

Top Ten Blog Posts from Ahead of the Heard in 2017

1.) Anything But Equal Pay: How American Teachers Get a Raw Deal
By Kirsten Schmitz

2.) Exciting News
By Mary K. Wells

3.) Some Exciting Hires and Promotions
By Mary K. Wells

4.) Where Are All The Female Superintendents?
By Kirsten Schmitz

5.) An Expanded Federal Role in School Choice? No Thanks.
By Juliet Squire

6.) Teacher Turnover Isn’t Always Negative – Just Look at D.C. Public Schools’ Results
By Kaitlin Pennington

7.) Georgia Addressed Its Teacher Shortages With This One Trick
By Chad Aldeman

8.) A Day in the Life: Bellwether Analyst Andrew Rayner [Andrew’s now over at Promise54!]
By Heather Buchheim & Tanya Paperny

9.) Welcoming Our New Senior Advisers
By Mary K. Wells

10.) How Will States Handle New Title I Powers with Minimal Federal Oversight?
By Bonnie O’Keefe

Top Five Publications & Releases from Bellwether in 2017

1.) An Independent Review of ESSA State Plans
Chad Aldeman, Anne Hyslop, Max Marchitello, Jennifer O’Neal Schiess, & Kaitlin Pennington

2.) Miles to Go: Bringing School Transportation into the 21st Century
Jennifer O’Neal Schiess & Phillip Burgoyne-Allen

3.) Michigan Education Landscape: A Fact Base for the DeVos Debate
Bonnie O’Keefe, Kaitlin Pennington, & Sara Mead

4.) Voices from Rural Oklahoma: Where’s Education Headed on the Plain?
Juliet Squire & Kelly Robson

5.) The Best Teachers for Our Littlest Learners? Lessons from Head Start’s Last Decade
Marnie Kaplan & Sara Mead

To hear more, you can always sign up here to get our newsletter. Thanks for following our work in 2017!

Under ESSA, Subgroup Accountability is like Gotham Without Batman

When something is wrong in Gotham, Commissioner Gordon puts up the bat signal and a powerful spotlight projects Batman’s symbol into the night sky. It’s a simple idea, really: make it crystal clear that there is a problem and alert the person who can do something about it.

This was the basic concept of the No Child Left Behind Act (NCLB). The law required states to shine a spotlight on schools where individual groups of students were struggling — in order to reveal a problem. And with the problem revealed, schools, districts, and states were required to do something to solve it.

The attention to subgroup performance was the most popular aspect of NCLB. When the law was finally reauthorized in late 2015, the Every Student Succeeds Act (ESSA) created an accountability system in which states must report inequities, but in the vast majority of cases, aren’t required to do anything about them. In lieu of NCLB-style accountability, the hope was that states and districts would create systems to act as Batman, and respond with additional support tailored to meet the needs of struggling schools and students. Barring that, since the law only requires states to identify the lowest of the low-performing schools, ESSA relies on transparency to encourage schools to improve.

Unfortunately, however, our new analysis of the accountability plans from all 50 states and Washington, D.C. found systems that were unambitious, uncreative, and, in some instances, unfinished. The worst part is that, in 41 out of the 51 state plans, subgroup performance will have no bearing on the ratings most schools receive.

This would be akin to Gotham police shining a plain spotlight into the night sky in the hopes that someone — anyone — would respond or that the bright light alone would stop crime.

Across the board, state accountability plans follow more or less the same pattern. They promise to report subgroup achievement across their selected indicators; but, when a school’s overall rating is calculated, it will be based solely on school-wide averages. Under this approach, schools can appear to be at least average while hiding massive achievement gaps.

In most cases, subgroups only truly matter in a few schools in which the performance of a group of students is as low as the worst performing 5 percent of schools in the state. And even then, it’s not clear that states have an effective strategy to improve struggling schools. In short, these plans will shine a light on a still-undetermined number of schools, and do little to resolve them.

There are very few exceptions to these retrograde policies.

In Tennessee, for example, 40 percent of a school’s score is based on subgroup achievement. Under this model, subgroup performance actually matters since the state uses these ratings to determine the schools that will receive additional support and resources. Minnesota takes a different tact to ensure that its schools are held accountable for subgroups: Instead of taking a school-wide average, the state first looks at the average performance for each subgroup and then combines that number into an overall average for the school. This model ensures that all subgroups count equally in the accountability system.

In Gotham, the bat signal works because the medium carries the message, leading Batman to show up and take care of the problem. Although imperfect, that’s how NCLB worked: states revealed problems and they then had to take steps to address them. Under ESSA, that’s no longer the case.

A public school serving mostly poor or non-white students is not a dystopian city like Gotham. Nevertheless, it is a place rife with injustice, where kids have been failed habitually by public institutions. These students don’t need a superhero to save them. What they need are state education systems designed to ensure that they are not misrepresented, underserved, under-resourced, or over-policed. They need access to all of the tenets of a high-quality education that they have so long been denied. And that takes real and intentional action, not just a bare spotlight.