Category Archives: Student Data

Building a School Performance Framework for System Management and Accountability? Lessons From Washington, D.C.

By

At its core, a school performance framework (SPF) is a data-based tool to support local decision making. An SPF designed for system management and accountability provides data and information about system-wide goals to district- or city-level leaders overseeing multiple schools, helps leaders hold schools accountable for student outcomes, allows leaders to understand which schools are performing well and which are not, and informs system-wide improvement strategies and the equitable allocation of resources. 

Our recent publication “School Performance Frameworks: Lessons, Cases, and Purposeful Design,” a website and report available at SchoolPerformanceFrameworks.org, identifies system management and accountability as one of three primary “use cases” that can shape SPF design decisions. A “use case” (a concept borrowed from the field of technology and design) helps designers think through their end users’ needs. Our work imagines local leaders as designers and considers how the choices they make can meet the needs of different end users, including parents, school principals, and district leaders. Among the five long-standing SPFs we looked at in detail for our project, four prioritized the use case of system management and accountability in their SFP design. 

We also found that too many SPFs try to fulfill multiple uses at once, without clearly thinking through priorities and potential tradeoffs. This post is the third in a series that looks at SPFs through the lens of each use case to highlight design considerations and relevant examples.

SPFs built for system management and accountability can inform consequential decisions made at the district level about which schools should be rewarded, replicated, or expanded, and which ones require improvement, intervention, and possibly closure. These SPFs get the most attention when the data they produce result in school closures or other highly visible consequences. While closures may grab headlines and garner resentment for SPFs, a well-designed SPF can actually inject transparency, equity, and fairness into even the most challenging decisions and increase opportunities for students and families by highlighting success and supporting the expansion of quality school options. 

An SPF created for system management and accountability should include:

Continue reading →

Kentucky Has a New Governor. We Hope He’s Not a Jerk About Education Policy.

By

Although he took more than a week to concede, Kentucky’s 62nd governor, Republican Matt Bevin, will not serve a second term. Experts agree that his provocative and insulting style, particularly his comments about teachers, attributed to his loss. Most notoriously, Bevin called teachers “thugs” and blamed them for the sexual assault of children and the shooting of a seven-year-old girl, after teachers protested the legislature’s sneaky efforts to reform the state’s pension systems. 

We are both Kentucky-based Bellwarians, and in the short conversation below, we discuss why Governor Bevin failed to advance education reforms in the state — and what Governor-elect and Democrat Andy Beshear might be able to accomplish given Kentucky’s Republican-dominated legislature. 

Katrina: I think you and I have some diverging ideas and perspectives about politics in general, and even about some education policies. But is it safe to say that we both think Matt Bevin is, well, a bit of a jerk?

Alex: I think we definitely have some common ground there, although I’d be careful about calling him a jerk — he might label you with a nickname like “Kooky Katrina.” More seriously though, I think a big part of his legacy will be the policy wins he left on the table, due in large part to his incredibly abrasive approach to governing.

Katrina: You’re not wrong about that. I was a fan of some of his policy positions, especially much-needed pension reform and increased school choice. If he had a bit more goodwill and emotional intelligence, he might have been able to demonstrate how those policies could actually help teachers and students.

Alex: Yep, but because of his style, pension reform and school choice are likely off the table for the next four years. And while some may be satisfied with the status quo on those issues, there are a lot of teachers and thousands of students who could benefit from reform to teacher pensions and school choice policies. 

Katrina: So where do you think Beshear has the opportunity to move the ball forward on education policy? 

Continue reading →

Building a School Performance Framework for Families? Lessons from Chicago.

By

Families and communities need access to reliable, understandable information about school quality to make decisions for their students. One tool district leaders can use to provide this information is a school performance framework (SPF). But SPFs are only useful to families if they are designed with families in mind. If leaders treat the needs of families as an afterthought during the design phase, it should be no surprise when families don’t use the tool.

Chicago Public Schools' School Quality Rating Policy screenshot

In our recent project at SchoolPerformanceFrameworks.org, my co-authors and I identified family and community information as one of three primary “use cases” that could shape SPF design decisions. My colleague Bonnie O’Keefe explains the concept of use cases and offers another example — school continuous improvement — here.

An SPF designed to show families and communities how schools are performing should include:

  • Early, authentic, and ongoing engagement of families and community members in the design process: District leaders should involve families from the beginning to understand what information they need or may already have. This can be accomplished through task forces, roundtables, or listening sessions, or by administering parent surveys. Leaders should be cautious not to engage only the most visible stakeholders, but instead should use various methods to engage families that will be most impacted by the SPF. Inauthentic engagement risks alienating key stakeholders and reinforcing harmful power dynamics.
  • The information families and community members most want to know: Families typically prefer a higher level of detail, a focus on outcomes, and a summative rating, because they are easier to understand. This contrasts significantly from the granular level of detail school leaders might need. If leaders create a tool that primarily serves families, the SPF might be less useful to school leaders or system leaders.
  • Results displayed in an understandable and accessible way: One reason families may struggle to understand school performance frameworks is when they are full of jargon. For example, parent advocacy organization Learning Heroes has found that someone could misread the phrase “School Climate” on a school report card to mean building temperature as opposed to the quality of school life. District leaders should present data to families that is free of jargon and available in high-quality multilingual translations.

Many of the districts profiled in our report have made improvements to their SPF over the years to make them more accessible to parents. For example, the School Quality Rating Policy (SQRP) in Chicago was not originally designed with families in mind, but the growth of school choice options prompted the district to make changes to give families access to more transparent, shared information across schools. SQRP reports now include the size of the school, the names and contact information for school leaders, programmatic offerings, and information about transportation options to each school. Reports are available in multiple languages and families can easily find the definitions of key terms within one click.

To learn more about other use cases for SPF design, and other long-standing local SPFs, visit SchoolPerformanceFrameworks.org.

A School Performance Framework Could Be Huge for Los Angeles. Why Is the District Backtracking?

By

This week, Los Angeles Unified School District (LAUSD) could miss a big opportunity for students, families, and district leaders.

Under the Every Student Succeeds Act, states must create a report card for every single one of their schools. Unfortunately, California’s approach to reporting school data under ESSA is both overly complex and lacking in key information. That’s why the LAUSD board took the first steps last year to create its own school performance framework (SPF), which could provide families, educators, and taxpayers more and better information about how well schools are serving students. Unfortunately the board now appears to be backtracking on that commitment.

An SPF is an action-oriented tool that gathers multiple metrics related to school quality and can be used by system leaders, principals, and/or parents to inform important decisions like how to intervene in a low-performing school, where to invest in improvements, and which school to choose for a child.

As my colleagues wrote in their 2017 review of ESSA plans, California’s complicated system relies on “a color-coded, 25-square performance grid for each indicator” and “lacks a method of measuring individual student growth over time.” In 2018, LAUSD board members tried to improve upon the state’s approach by passing a resolution to create their own SPF. In a statement from the board at that time, members intended that LAUSD’s SPF would serve as “an internal tool to help ensure all schools are continuously improving,” and “share key information with families as to how their schools are performing.”

A local SPF could provide a common framework for district leaders and families to understand performance trends across the district’s 1,100 plus schools in a rigorous, holistic way. Without usable information on school quality, families are left to make sense of complex state websites, third party school ratings, and word of mouth. And unlike the state’s current report card, a local report card could include student growth data, one of the most powerful ways to understand a school’s impact on its students. Student-level growth data tells us how individual students are progressing over time, and can control for demographic changes or differences among students. Continue reading →

Warren Wants to Slash the Program I Evaluate — But I’ve Seen It Work

By

Last week, Sen. Elizabeth Warren released her education plan, where she proposed slashing federal funding for the expansion of public charter schools by ending the Charter School Program (CSP). Warren claims that the CSP is an “abject failure,” citing a report by an anti-charter organization that the federal government has wasted up to $1 billion on charter schools. (The report’s lack of substance and evidence has already been raised by others.)

U.S. Senator and presidential candidate Elizabeth Warren visiting Roosevelt High School in Des Moines, Iowa. She spoke to a group of about 400 students to outline her K-12 education plan, answer questions, and pose for selfies.

Photo of Elizabeth Warren via Flickr user tabor-roeder

I know Warren is wrong firsthand. As part of my work at Bellwether Education Partners, I serve as an external evaluator for three CSP grantees (as my colleague Cara mentioned yesterday). Our unbiased, rigorous, and data-driven evaluations indicate that when implemented well, high-quality charters use their CSP funds to improve their model and successfully serve more students. These charter schools are the reason their students experience gains in achievement beyond their traditional public school peers. 

Let me step back. The CSP began in 1994 as an amendment to the federal Elementary and Secondary Education Act. It has three goals:

  1. To create promising new public charter schools,
  2. To replicate high-quality public charter schools, and
  3. To disseminate information about effective practices within charter schools.

In 25 years, CSP has awarded close to $4 billion to charters serving disproportionately more low-income and diverse students than traditional public schools. The average award is about $500K. In exchange for CSP replication funds, the schools expand to serve more students. CSP replication grantees are required to evaluate their progress, and it is strongly recommended that they use an external evaluator to do so. The evaluation must demonstrate that the grantee is doing what it said it would do and compare the achievement of its students to students attending nearby traditional public schools for evidence of impact. This is how the CSP holds grantees accountable. 

My colleagues and I use advanced statistical modeling techniques to compare charter-attending students to similar students attending nearby traditional public schools on a 1:1 basis. In the model, when all things are considered, the difference in achievement found can be attributed to the school. In fact, an emerging trend in our current data analysis suggests that the longer a student attends a well-implemented, high-quality charter school, the larger the gains in achievement over their traditional public school student peer become. Continue reading →