Tag Archives: data

Seriously, Stop Asking If Head Start “Works”

Last month, yet another study came out examining the effects of Head Start on children’s long-term outcomes. The findings were lackluster: Depending on the cohort of children and outcomes you’re looking at, the effect of Head Start was either negative or non-existent. 

This study is noteworthy for a few reasons. It uses the same analytical approach as a high-profile 2009 study on Head Start, conducted by Harvard economist David Deming, which found Head Start had unquestionably positive results. And in a twist I’m definitely reading too much into, a former Deming student is one of the lead co-authors on this new study. People are also paying attention to this study because the findings go against a truly massive body of evidence on Head Start, which largely shows that Head Start has positive effects on children and families. 

But what snagged my attention is the fact that the research question at the heart of this study is irritatingly useless. It asks, essentially, “Does Head Start work?” That’s a question we answered a long time ago. And the answer is: It depends.

Again, the existing research on Head Start overall is positive. But we also know that there is wide variation in quality between individual Head Start providers. It’s a valuable federal program that can get better.  Continue reading

We Don’t Know What the Superintendency Looks Like, and That’s a Problem.

This post is part of a week-long series about educator and leader pipelines. Read the rest of the series here.

We’ve talked a lot this week about the teacher pipeline. My colleagues have dug into issues like innate inequities in teacher hiring and the retention of high-performing teachers. There’s absolutely work to be done to ensure districts recruit, train, and retain high-quality educators, and we’re able to ground these efforts in demographic data, with insight into teacher and principal demographics from the Department of Education’s National Center on Education Statistics’ Schools and Staffing Survey. As we make efforts to diversify and expand our teacher pipeline, it’s valuable to know what our current teacher workforce looks like, especially on a state-by-state level.

First graders answer questions for a project about bees. Photo by Allison Shelley/The Verbatim Agency for American Education: Images of Teachers and Students in Action

What we don’t have though, is reliable, state-level data on school superintendent demographics. While we look to improve teacher pipelines, we should not ignore leadership pipelines. And if we don’t know what our existing superintendent pool looks like, it can be challenging to determine how or even if that network could be expanded.

The American Association of School Administrators prints an annual Salary and Benefits Study, which includes survey data capturing school leader demographics. Unfortunately, the survey’s 15 percent response rate prevents it from being truly representative. While we can make broad estimates about the country’s 13,674 districts and their respective leaders based on national figures, there is not, to my knowledge, a publicly available data set of state-level superintendent demographics across race and gender. Anyone know of such a set? I’d love to talk: kirsten.schmitz@bellwethereducation.org.

These roles are powerful, and representation matters. If we can’t analyze broad trends in school leadership at the state level, we miss opportunities to highlight states with diverse administrators, as well as those which may benefit from targeted outreach and recommendations. The same questions we ask about educator diversity — like “is our teacher workforce representative of our student population?” — can be applied to superintendents. We could further answer equity questions around wage gaps, mentoring, and access to leadership opportunities. And finally, as several of the nation’s largest school districts scramble to appoint new superintendents from a finite applicant pool, this field landscaping work becomes especially valuable.

We can and should work to improve our teacher pipeline. But we should also strive to know more about our school leaders. Knowing where we stand is the baseline first step, and it shouldn’t be this challenging to get there.

Questions for Betsy DeVos Inspired by Education Outcomes in Michigan

Tonight is Betsy DeVos’ confirmation hearing to become the next Secretary of Education. Because DeVos doesn’t have a track record as a government official or leader within the public school or higher education system, as most of her predecessors do, analysts are looking at her role as a funder, GOP donor, and board member of education organizations to understand what she might do as Secretary. This scrutiny has drawn particular attention to DeVos’ engagement in education advocacy and political causes in Michigan, where her donations and advocacy have touched many major education policy decisions over the past 20 years.

In many ways, the education system in Michigan is a microcosm of the challenges and opportunities facing the broader U.S. education system — and the next Secretary of Education. In both Michigan and the U.S. as a whole, there are large, persistent achievement gaps for disadvantaged student groups; rural, suburban, and urban schools with unique (sometimes competing) needs; and a long history of hotly debated education reforms that have had mixed success. To help analysts, journalists, policymakers, and practitioners make sense of the education landscape in the Wolverine State — and what it suggests about the perspective and positions DeVos would bring to the role of Secretary — Bellwether has compiled a comprehensive fact base about the education policy landscape in Michigan that we will release next week after the Inauguration.

In the meantime, here are a few Michigan fast-facts to know as you watch tonight’s hearing:

Demographics of Michigan K-12 students by race/ethnicity, family income. Source: MISchoolData.org

Demographics of Michigan K-12 students by race/ethnicity and family income. Source: MISchoolData.org

  • There are over 1.5 million students in Michigan and nearly half of them qualify for free and reduced-price lunch; more than 33 percent are students of color.
  • Michigan ranks 41st in 4th grade reading performance in the U.S. and 42nd in 4th grade math.
  • 35 percent of Michigan 11th grade students are college-ready according to the SAT; there are substantial gaps in college-readiness rates among black, Hispanic, English language learner, and low-income students.
  • Michigan has one of the nation’s largest charter sectors, with 10 percent of students enrolled in charter schools, about 300 charter schools, and over 40 charter authorizers.
  • Over 70 percent of Michigan charter schools are operated by for-profit education service providers.
  • Detroit is the lowest performing urban school district in the country.
  • Detroit charter schools generally outperform Detroit Public Schools, but there are still concerns about the overall quality of the sector.

Given the above facts, here are a few questions we’d like DeVos to answer at tonight’s confirmation hearing:

  • What should be the role of the federal government in addressing longstanding achievement gaps for low-income students and students of color, like those that exist in Michigan?
  • As you know, Detroit students have struggled academically and gone through numerous failed reform efforts over several decades. Given your work in Detroit, what turnaround strategies would the Department of Education encourage for chronically low-performing school districts?
  • What did you learn from advocating for expanded school choice measures in Michigan and how might you enact those measures at the federal level as Secretary of Education?
  • The presence of multiple charter school authorizers in Michigan has decentralized charter responsibility in the state. What quality-control and accountability measures are necessary for charter school authorizers? What should be the federal role in setting that bar?
  • What has your experience and observation of school choice and school turnaround efforts in Michigan taught you about potential strategies for improving low-performing schools? How would those lessons be applied to this spring’s review of states’ Every Student Succeeds Act plans?

Betsy DeVos’ hearing begins at 5pm and can be watched here. Check back here tomorrow for a recap of major events (and anything about Michigan education that needs a fact check).

To read our other coverage of Betsy DeVos, click here.

Anecdotes Don’t Make Good Policy: What We (Don’t) Know About Meeting The Needs of Formerly Incarcerated Students

Nationally, it’s estimated that nearly 66% of students who are released from juvenile court schools never return to local high schools. Many states are struggling to find strategies to intervene so that more justice-involved youth return to school after their incarceration. As with most thorny education policy challenges, the first question that smart leaders typically ask is “who’s doing this well?” and they go from there. What should they do when the answer is “no one”?

In 2015 I was appointed to the California Statewide Transitions Work Group, convened under AB2276 to make recommendations to the state legislature for policies to improve the educational experiences of youth returning to local schools after incarceration.  The report hasn’t yet been publicly released so I’m not in a position to share much about the substance of it – but as it winds its way through the state Department of Education’s approval and revision pathway, I’ve been reflecting on the lack of reliable data and information about justice-involved youth and how that impacts our ability to make good decisions. Continue reading

Don’t Ask if Head Start “Works” – That’s Not the Right Question

Head Start is an $8.5 billion federal program, which means everyone loves asking if it “works.” But that’s a useless question.

We know Head Start produces positive outcomes. There’s a substantial body of evidence showing that Head Start improves children’s learning at school entry. Other research shows that Head Start children are more likely to graduate high school and have better adult outcomes than children who did not. And a growing body of research shows that high-quality preschool programs can produce long-lasting gains in children’s school and life outcomes.

But critics of Head Start cite the same studies I just did to make the opposite argument. They have valid points. Not every Head Start program is high quality, for example, so some programs don’t produce these positive gains for students. And the Head Start Impact Study showed that Head Start’s positive effect on test scores fades as children enter the elementary grades.

Both critics and proponents of Head Start are right – which is why the “Does it work?” question is so useless. We already know the answer, and it’s not a clean yes or no. Taken all together, the available evidence shows that Head Start is a valuable program that can get better. Given, instead of asking if Head Start works, we should be asking a better question: How can policymakers and practitioners make Head Start better for children and families?

That’s the question Sara Mead and I – along with Results for America, the Volcker Alliance, and the National Head Start Association – try to answer in our new report, Moneyball for Head Start. We worked with these organizations to develop a vision for improving Head Start outcomes through data, evidence, and evaluation.

Specifically, we call on local grantees, federal policymakers, the research community, and the philanthropic sector to reimagine Head Start’s continuous improvement efforts.

Local grantees: All Head Start grantees need systems of data collection and analysis that support data-informed, evidence-based continuous improvement, leading to better results for children and families.

Federal oversight: The Office of Head Start (OHS), within the Administration for Children and Families of the U.S. Department of Health and Human Services, needs a stronger accountability and performance measurement system. This would allows federal officials to identify and disseminate effective practices of high-performing grantees, identify and intervene in low-performing grantees, and support continuous improvement across Head Start as a whole.

Research and evaluation: Federal policymakers and the philanthropic sector need to support research that builds the knowledge base of what works in Head Start and informs changes in program design and policies. This will require increasing funding for Head Start research, demonstration, and evaluation from less than 0.25 percent of total federal appropriations to 1 percent, and those funds should focus on research that builds knowledge to help grantees improve their quality and outcomes.

Philanthropy and the private sector: The philanthropic sector, universities and other research institutions, and the private sector should help build grantee capacity and support the development, evaluation, and dissemination of promising practices.

Fully realizing this vision will require a multi-year commitment. There are steps, however, that Congress and the administration can take to make progress towards these goals. In the paper, we propose several recommendations for federal policy. Taken together, these actions can support Head Start grantees in using data, evidence, and evaluation to improve results for children and families.