Category Archives: Research

Seriously, Stop Asking If Head Start “Works”

Last month, yet another study came out examining the effects of Head Start on children’s long-term outcomes. The findings were lackluster: Depending on the cohort of children and outcomes you’re looking at, the effect of Head Start was either negative or non-existent. 

This study is noteworthy for a few reasons. It uses the same analytical approach as a high-profile 2009 study on Head Start, conducted by Harvard economist David Deming, which found Head Start had unquestionably positive results. And in a twist I’m definitely reading too much into, a former Deming student is one of the lead co-authors on this new study. People are also paying attention to this study because the findings go against a truly massive body of evidence on Head Start, which largely shows that Head Start has positive effects on children and families. 

But what snagged my attention is the fact that the research question at the heart of this study is irritatingly useless. It asks, essentially, “Does Head Start work?” That’s a question we answered a long time ago. And the answer is: It depends.

Again, the existing research on Head Start overall is positive. But we also know that there is wide variation in quality between individual Head Start providers. It’s a valuable federal program that can get better.  Continue reading

Teacher Residencies Can Translate Into a More Diverse Workforce, But Who Will Bear the Expenses?

A growing number of studies have documented the benefits of teacher diversity, as my colleagues have previously discussed (see here and here). And research drawing on data from the Measures of Effective Teaching project found that all students preferred teachers of color. Yet despite the value of diversifying the teaching workforce, Black teachers remain underrepresented. They made up around 7 to 8 percent of all teachers between school years 1999-2000 and 2015-2016, while Black students accounted for between 16 to 17 percent of all students in the same time period. The proportion of Hispanic public elementary and secondary school teachers appeared to be increasing slightly, but not nearly as fast as the proportion of Hispanic students, as seen in the figure below.

Proportion of Hispanic students and teachers over time

As my colleague Katrina has noted, a number of barriers to diversifying the teacher workforce exist, including the considerable cost to become a teacher. Traditional preparation programs have high out-of-pocket and opportunity costs (i.e., limited income while enrolled in a program). As Ashley LiBetti and Justin Trinidad describe in their recent report, these costs limit the pool of teacher candidates:  

In the traditional model, candidates must invest more than $24,000 and 1,500 hours to become a teacher…This upfront financial and opportunity cost limits the pool of candidates to those who can afford the risk, effectively cutting out nontraditional candidates, low- and lower-middle-income candidates, and career-changers.

Given well-documented racial disparities in wealth, the high cost of becoming a teacher is likely to have a disproportionate impact on the career decisions of people of color. Alternate routes to teaching may be an attractive option for prospective teachers who are sensitive to costs. As seen below, teachers from alternate routes tend to be more racially diverse than teachers from traditional teacher preparation programs. 

Racial/ethnic diversity for tradition route and alternate route teachers

Yet alternate routes have proven controversial, even when evidence suggests that alternatively certified teachers are equally or more effective at increasing student achievement on standardized tests, relative to their counterparts. In Houston, for example, school district trustees recently voted to end the district’s contract with Teach For America, citing concerns about teacher turnover. 

Teacher residency programs have emerged as perhaps a more politically viable alternative certification route than “fast-track” programs, by emphasizing on-the-job training prior to becoming a teacher of record. Because residents are typically paid a stipend during their apprenticeship period, entering teaching through a residency tends to cost less than entering through a traditional route. However, residencies typically pay a stipend that is less than what a teacher of record would earn. As a result, the cost of entering the profession through a residency program is higher than the cost of entering through a fast-track alternative certification program.  Continue reading

3 Things Head Start Programs Can Do Right Now to Improve Their Practice

Research tells us that, overall, Head Start has positive effects on children’s health, education, and economic outcomes. But there is wide variability in quality from program to program — and, as a field, we don’t understand why. 

Earlier this year, Sara Mead and I tried to figure that out. We published an analysis, conducted over three years, of several of the highest performing Head Start programs across the country. We specifically looked at programs that produce significant learning gains for children. Our goal was to understand what made them so effective.

As part of this project, we provided detailed, tactical information about exemplars’ design and practices. We hope to serve as a resource and starting point for other Head Start programs interested in experimenting with something new and, potentially, more effective.

Here are three action steps that Head Start programs can take right now to improve their practice:  Continue reading

Correlation is Not Causation and Other Boring but Important Cautions for Interpreting Education Research

Journalists, as a general rule, use accessible language. Researchers, as a general rule, do not. So journalists who write about academic research and scholarship, like the reporters at Chalkbeat who cover school spending studies, can help disseminate research to education leaders since they write more plainly.

But the danger is that it’s easy for research to get lost in translation. Researchers may use language that appears to imply some practice or policy causes an outcome. Journalists can be misled when terms like “effect size” are used to describe the strength of the association even though they are not always causal effects.

To help journalists make sense of research findings, the Education Writers Association recently put together several excellent resources for journalists exploring education research, including 12 questions to ask about studies. For journalists (as well as practitioners) reading studies that imply that some program or policy causes the outcomes described, I would add one important consideration (a variation on question 3 from this post): if a study compares two groups, how were people assigned to the groups? This question gets at the heart of what makes it possible to say whether a program or policy caused the outcomes examined, as opposed to simply being correlated with those outcomes.

Randomly assigning people creates a strong research design for examining whether a policy or program causes certain outcomes. Random assignment minimizes pre-existing differences among the groups, so that differences in the outcomes can be attributed to the treatment (program or policy) instead of different characteristics of the people in the groups. In the image below, random assignment results in having similar-looking treatment and control groups. Continue reading

What This Washington Post Opinion Piece Got Wrong on Charter Schools

Over the weekend, the Washington Post Outlook section ran a frustrating cover story on charter schools that offered a narrow and biased picture of the charter sector and perpetuated a number of misconceptions.

Jack Schneider’s “School’s out: Charters were supposed to save public education. Why are Americans turning against them?” argues that the charter sector as a whole isn’t living up to its promises, leading public support for the schools to shrink. Schneider is correct that the charter school hasn’t lived up to all of its most enthusiastic boosters’ promises, but his piece flatly misrepresents data about charter quality. For example, Schneider writes that “average charter performance is roughly equivalent to that of traditional public schools.” This is simply inaccurate, as my colleagues indicated in a recent analysis of charter data and research (slide 37 here). The full body of currently available, high-quality research finds that charters outperform traditional public schools on average, with especially positive effects for historically underserved student groups (a recent Post editorial acknowledged this as well).

slide from Bellwether's "State of the Charter Sector" resource, summarizing research on charter sector performance

To be clear, research also shows that charter performance varies widely across schools, cities, and states — and too many schools are low-performing. Yet Schneider cherry picks examples that illustrate low points in the sector. He cites Ohio, whose performance struggles — and the poorly designed policies that led to them — Bellwether has previously written about. He also (inexplicably, given where his piece ran) overlooks Washington, D.C., where charters not only significantly outperform comparable district-run schools, but have also helped spur improvement systemwide. Over the past decade, public schools in D.C. (including both charters and DC Public Schools, DCPS) have improved twice as fast as those in any other state in the country, as measured by the National Assessment of Educational Progress (NAEP). DCPS was the nation’s fastest growing district in 4th grade math and among the fastest in 4th grade reading and 8th grade math. These gains can be partially attributed to the city’s changing demographics, but are also the result of reforms within DCPS — which the growth of charters created the political will to implement. Over the past decade, Washington, DC has also increased the number of high-performing charter schools while systematically slashing the number of students in the lowest-performing charter schools. When I served on the District of Columbia Public Charter School Board from 2009-2017, I had the chance to observe these exciting changes firsthand, so it was particularly disappointing to see a major feature in our city’s paper overlook them.

It’s frustrating that this biased and narrow picture drew prime real estate in one of the nation’s leading papers, because the charter sector does have real weaknesses and areas for improvement that would benefit from thoughtful dialogue. For example, as Schneider notes, transportation issues and lack of good information can prevent many families from accessing high-quality schools. In cities with high concentrations of charters, such as Washington, D.C. and New Orleans, there is a real need to better support parents in navigating what can feel like a very fragmented system. And despite progress in closing down low-performing charter schools, too many remain in operation. Schneider could have referenced the real work charter leaders are undertaking to address these lingering challenges (more on this in slide 112 of our deck).

Schneider is correct that public support for charters has waned in recent years, due in part to some of the challenges he references, but also because of orchestrated political opposition from established interests threatened by charter school growth. Given the increasingly polarized political environment around charter schools, the need for nuanced, balanced, and data-informed analysis and dialogue about them is greater than ever. Bellwether’s recent report on the state of the charter sector, and our past work on charter schools more broadly, seeks to provide that kind of analysis. Unfortunately, Schneider’s piece falls short on that score.