Category Archives: Student Data

Committing to Continuous Improvement in Schools: A Customizable Workbook

Figure 1: Bellwether’s refined continuous improvement cycle

Bellwether’s Academic and Program Strategy team partnered with K-12 schools in more than a dozen district and charter networks across the country in the 2020-21 academic year to adopt continuous improvement (CI) cycles that diagnose and reverse unfinished student learning through an iterative, evidence-based approach. In this final blog post, the team provides a customizable CI workbook for use in any school context. 

Last week, we unpacked the increasing value of continuous improvement (CI) cycles in education settings and included reflections from four partner schools on what Bellwether’s distinct CI process looks like in practice. 

In Bellwether’s refined CI approach, the technical and adaptive components of the cycle (Figure 1, represented by a circled “T” and “A”) are intentionally blended. This approach enables school leadership to ground CI plans in measurable goal-based data metrics within aligned, agile teams and coalitions focused on supporting seamless execution on behalf of students. 

As schools reopen this fall, leaders and educators will need to get more strategic and efficient about diagnosing and reversing unfinished student learning in their unique school settings. Bellwether’s Continuous Improvement in Schools Workbook provides a customized way to do that. 

We hope this workbook will be a useful tool as school leaders assess and respond to unfinished student learning this fall and beyond.

It’s Time for a New, Refined Commitment to Continuous Improvement in Schools

Figure 1: Bellwether’s refined continuous improvement cycle

Bellwether’s Academic and Program Strategy team partnered with K-12 schools in more than a dozen districts and charter networks across the country in the 2020-21 academic year to adopt continuous improvement (CI) cycles that diagnose and reverse unfinished student learning through an iterative, evidence-based approach. In this first of two blog posts, the team unpacks Bellwether’s comprehensive approach to CI and what each step in the process looks like in K-12 school settings. Next week, stay tuned for a customizable CI workbook for use in any school context. 

The pandemic and its disproportionate impact on students from historically marginalized communities underscores the value of continuous improvement (CI) as a framework for understanding the depth of unfinished learning and responding to it in an urgent, data-driven, and adaptive manner. In the past decade, CI has worked its way into the lexicon of educators, largely due to the Carnegie Foundation’s plan-do-study-act cycle that has been applied to diverse education improvement efforts from implementing ESSA plans to closing achievement and opportunity gaps. This growing education application of CI draws on more than 30 years of CI best practices in improving products, services, or processes through successive, rapid, evidence-based cycles in a range of sectors. 

Since fall 2020, Bellwether has supported more than a dozen districts and charter networks in their CI efforts, within virtual and hybrid settings, and has developed a balanced approach to the process attuned to current realities in the field. Bellwether’s CI cycle (Figure 1) follows a familiar four-step cadence (“Envision-Execute-Examine-Enact”), but builds on prior models by adding a high-impact adaptive leadership action to what’s typically been viewed as a predominantly technical process. This modification — based on 21st century change management research from Chip and Dan Heath, Ronald Heifetz and Marty Linsky, and Dr. John Kotter — is grounded in the idea that while CI’s technical elements are critical to understand what needs to happen, the cycle ultimately doesn’t lead to sustained change without careful consideration of how that change will occur. 

In Bellwether’s refined approach to CI, the technical and adaptive components of the cycle (Figure 1, represented by a circled “T” and “A”) are intentionally blended. This approach enables school leadership to ground CI plans in measurable goal-based data metrics within aligned, agile teams and coalitions focused on supporting seamless execution on behalf of students.

What does this look like in practice?

Four of the schools Bellwether supported this year, each with its own unique context and focus, weigh in:

1. Envision

Achievers Early College Prep Charter School, a public charter middle school in Trenton, New Jersey, built and implemented a new, data-informed intervention program to accelerate the academic growth of its most vulnerable students. The technical work of the CI Envision stage consisted of AECP setting a vision to create a data-driven intervention program that would provide the right content to the right students at the right time. AECP then established a clear goal to leverage its intervention program to have 80% of its highest-need students reach 1.75 to 2 years of academic growth, as measured by the NWEA MAP assessment. Finally, AECP built a progress monitoring system to look at grade level aligned daily exit tickets in intervention and core classes to measure the effectiveness of both prerequisite intervention content and grade level aligned content. On the adaptive side, AECP built a coalition by having a strong eighth grade teacher team pilot this approach in its first CI cycle, enabling teachers to better troubleshoot problems in real time and facilitate training for the sixth and seventh grade teams in future CI cycles.

In AECP’s words: “[This CI cycle] improved our reflection on our targeted areas for improvement. We have been more strategic on creating intervention goals and maintaining strong leadership initiatives throughout our pilot.”

2. Execute

Seguin Independent School District, a K-12 traditional public school district outside of San Antonio, Texas, centered its CI work on developing teacher instructional capacity in a virtual academy. The technical work of the CI Execute stage consisted of a team taking action on its plan by hosting biweekly, district-wide Professional Learning Communities on virtual instruction, facilitating grade level planning time aligned to those instructional moves, and conducting 1:1 observations and coaching for virtual teachers. During this process, the SISD team gathered data and monitored progress on teacher and leader attendance, engagement, and perception of transferability of new strategies to the classroom. On the adaptive side, the team remained focused on designing high-quality supports aligned to the See it. Name it. Do it. Framework and the National Institute for Excellence in Teaching’s Virtual Look-Fors. However, SISD also had to remain agile by adjusting programs, processes, and communications as it responded to an historic set of regional ice storms, ongoing staffing shifts related to virtual instruction, and survey feedback from teachers. 

In SISD’s words: “The structures and logistics were set by the project plan and covered by the central office. This meant we had the capacity and brain space to respond to shifting circumstances and teacher needs as they arose.”

3. Examine

LEEP Dual Language Academy, a K-2 public charter school in Brooklyn, New York, focused on evaluating and coaching effective lesson planning and execution for guided reading in a hybrid setting. On the technical side of the CI Examine stage, LEEP measured impact by analyzing both process and efficacy data for its CI strategy. The team examined process data by analyzing the consistency of its strategy implementation, and dug into efficacy data to see how both teacher practice and student achievement outcomes were impacted. In this stage, the team identified the following key takeaways: (1) they were less consistent in implementing coaching and feedback on lesson execution and would need to make this shift in the second cycle of CI to drive impact, and (2) they saw less reading growth from virtual kindergarten students and identified the schedule, reading group size, and content prioritization as opportunities to address in the second cycle. The team’s adaptive work of celebrating small wins focused on noting the increase of consistency in lesson plan submission and feedback to teachers in guided reading. They also celebrated mid-year growth on the STEP assessment in second grade with 49% of students growing two reading levels or more after one month of implementation. 

In LEEP’s words: “After examining our data, I think that we have remained focused and nimble in our implementation and this has been done through careful data analysis to then inform next steps and any modifications needed to the plan.”

4. Enact

Promise Community School at Baker-Ripley, a small public charter school network in Houston, Texas, piloted a “Just In Time” (JIT) intervention model for elementary math instruction in a hybrid setting. The technical work of the team’s Enact stage centered on translating key takeaways from its first cycle of JIT intervention to make measurable shifts for a second cycle. In the first cycle of implementing the continuous JIT intervention strategy, the team saw a 30%-point increase in mastery for virtual students, however students’ proficiency fluctuated between 50 to 70%. In order to increase consistency of virtual student mastery, the Promise team shifted its data analysis to focus on remote learners by (1) analyzing remote student work and misconceptions, and (2) increasing engagement strategies during small-group virtual instruction. From an adaptive standpoint, the Promise team focused on clearly communicating adjustments for cycle 2, reinvesting the pilot team by including a rationale and updated goals for the shift, and inspiring through a reiteration of the bright spots observed in cycle 1. 

In Promise’s words: “It’s never too late to reset expectations (we reset in January). We use data to help zoom in on places for focus and problem solving, and we need to be flexible and innovative with what works for our kids.”


We hope that Bellwether’s CI cycle framework and glimpses into its application in schools help educators begin to think about how this process could live in their unique school settings. For questions or comments, please feel free to
email us, and stay tuned next week for a customizable CI workbook for use in any school context.

Puerto Rican students during the COVID-19 pandemic: Data update and Q&A with Carlos Rodríguez Silvestre

In October 2020, “Missing in the Margins: Estimating the Scale of the COVID-19 Attendance Crisis” estimated that approximately 3 million American students had experienced minimal to no formal education since March 2020 due to the COVID-19 pandemic. The analysis focused on the most educationally marginalized students in the country — English learners and students with disabilities, in foster care, experiencing homelessness, and/or eligible for the Migrant Education Program. But the original 50 state and D.C. estimates left out Puerto Rico, the Bureau of Indian Education, and other U.S. territories. These students have been just as affected by the pandemic, if not moreso, and deserve more attention. 

Missing in the Margins data estimates now include data from Puerto Rico and the Bureau of Indian Education. Other U.S. territories did not have sufficient comparable data on the population groups in our original estimates, but we are hopeful that this expanded version paints a more comprehensive picture. 

The estimates emphasize the scale of the challenge before state and local education agencies as they work to begin the recovery process. As before, estimates are based on student population numbers from educationally marginalized groups, not on 2021-22 enrollment numbers. Those enrollment numbers, where available, track with our earlier estimates: enrollments are down around 2-3% on average, with much larger reductions concentrated in kindergarten. But even this far into the 2020-21 school year, we still do not have a clear enough picture of who these unenrolled students are or where they have gone: private school, home school, work, or elsewhere. 

In 2017, public school enrollment in Puerto Rico was approximately 350,000 students, but economic crises, natural disasters, and the COVID-9 pandemic have all contributed to fast enrollment declines. Local estimates suggest enrollment is now closer to 280,000. We estimate up to 25,000 children on the island may have had little or no access to education due to the pandemic.

To learn more, we sat down virtually with Carlos Rodríguez Silvestre, Executive Director of the Flamboyan Foundation in Puerto Rico, where he oversees all aspects of the Foundation’s mission-driven work and programmatic strategy in Puerto Rico, to ensure students most impacted by inequity are prepared to succeed in school and beyond. This Q&A has been edited for length and clarity.

  • Carlos Rodriguez SilvestreHow are students in Puerto Rico faring during the COVID-19 school closures and virtual learning? How have the effects of COVID-19 interacted with other disaster response efforts, including the ongoing recovery from Hurricane Maria?

COVID-19 compounded trauma for students in Puerto Rico. The end of in-person classes last March 16 due to the COVID-19 pandemic is the latest in a series of interruptions to the education system in Puerto Rico, including Hurricanes Irma and Maria in 2017 and earthquakes in early 2020. We estimate that students in Puerto Rico already lost up to 159 days of school between 2017 and 2020. \Adding on lost instructional and socialization time related to the pandemic further deepens challenges for students, families, and educators. 

Taken together, these disasters have had a profound effect on the physical, social, emotional and academic wellbeing of students. Any resolutions will need to take into account the compounded nature of trauma on students and the uniqueness of Puerto Rico’s context. 

  • Are there experiences or considerations that may be unique to Puerto Rican students and schools? 

Yes. The compounded impacts of multiple disasters present unique challenges for Puerto Rican students and schools. 

A crippling economic crisis had left the island bankrupt in 2006 and 15 years later recovery has not been realized. Schools and island infrastructure were already under-resourced as a result.  Continue reading

How Can Educators Evaluate Virtual Student Engagement During the COVID-19 Pandemic?

Before the COVID-19 pandemic, educators commonly assessed student engagement using student self-report measures, teacher reports, and observational measures. These measures work for in-person learning environments where student participation and connectivity can be easily visible.

Now most teaching and learning occurs in virtual spaces, where instruction is delivered through learning management systems (e.g., Google Classroom) either synchronously or asynchronously. Teachers and school leaders have to engage students — and evaluate their engagement — in a remote environment with little formal or comprehensive training and often a limited ability to even see their students

There has been a great deal of writing recently about improving student engagement, but few resources provide guidance around measuring student engagement during remote learning. 

Based on a close read of the existing research and resources, and our own in-house expertise, Bellwether’s evaluation team is currently helping three clients to design and implement tools to evaluate virtual student engagement. While the measures themselves have not changed much since the pandemic, the uses and evaluation of these measures must adapt to our current realities, like many other approaches in education, to better serve and engage students and families. 

Here are two key considerations for educators and school leaders to keep in mind when evaluating virtual student engagement:

There is no one-size-fits-all model for measurement  Continue reading

The 3 Million Who Never Showed Up for Virtual Schooling

For some students, the devastation of the COVID-19 pandemic on education goes far beyond coping with the difficulties of distance learning. For these students, a day in March or April marks the last day of their formal education altogether. Out of the approximately 53 million K-12 students in the United States who stopped attending in-person school in the spring, an estimated three million may have never showed up online at all.

That figure comes from a new Bellwether analysis that estimates the number of students who, due to various barriers, did not make the transition, remaining disconnected even as schooling continued online. Just as the pandemic has disproportionately affected our most vulnerable and marginalized communities, this “disconnected” population is more likely to be composed of English learners, students in foster care, migrant students, students experiencing homelessness, and students with disabilities. Three million students is approximately 1 out of every 4 of these student populations combined — and also roughly equivalent to the entire school-age population of Florida.

Against the backdrop of the pandemic, wildfires, hurricanes, social unrest, and a general election, it’s easy to miss what’s happening here. But just like the pandemic itself, the defining feature of this crisis is the scale — this is something that every district, in every state, is struggling to address. Once a young person leaves school, it can be very difficult to re-engage them. The long-term implications of even a short period of learning loss are serious, and the outcomes associated with dropping out of high school are even more dire.

We could be witnessing the beginning of an event that has lifelong implications for this generation of students, in much the same way that the Great Recession has hamstrung millennials’ accumulation of wealth relative to previous generations. And among the young people of this new generation, the harshest impacts fall, time after time, on communities of color, students with disabilities, and those living in poverty. 

That’s a gloomy picture to paint, and it’s easy to feel as though we can’t grapple with this problem the way we would want to, given the enormity of everything else happening in the world. But there are concrete steps we can take right now to mitigate the damage being done, from improving attendance data collection — and data sharing — across public agencies to implementing interventions that meet the most vulnerable students where they are. Many teachers and leaders have identified a need for collaboration with social service providers and telecommunications firms to provide Internet connectivity to those in dire need. And states must provide stronger guidance, funding, and resources for schools and social services that can be spent flexibly, effectively, and in a timely manner.

Above all, the number one thing public officials can do to start to repair the damage done — and to prevent the unimaginable harm of a “lost generation” — is to develop and effectively implement the public policies needed to defeat the COVID-19 pandemic by reducing community transmission to levels negligible enough to permit a return to the normal school environment. There is no sustainable education policy workaround that can surmount a public health crisis of this magnitude. Back on March 13, seven months ago, it was difficult to imagine we would find ourselves in this position; our best course of action now is to take aggressive action on behalf of the roughly three million students whose educational futures are in the balance.