Category Archives: Research

The NBA Playoffs, Match-Ups, and Teacher Fit: An Interview with Steve Glazerman

The NBA Finals wrapped up on Friday, but they offered a nice reminder about the importance of how employees fit within an organization and context. Players who looked like All-Stars in one round of the playoffs became unplayable in the next due to match-up problems. In other cases, players who washed out in other contexts were suddenly relevant again.

Victor Oladipo

photo of Victor Oladipo via Flickr user Keith Allison

On a recent podcast, Malcolm Gladwell used the NBA playoffs as an entry point to talk about worker fit. Gladwell gave the example of NBA player Victor Oladipo. Oladipo was the second pick in the 2013 draft, made the All-Rookie team his first year, and then…didn’t quite live up to his perceived potential. He was traded once but didn’t mesh well with his new teammates, and then he was traded again, this time to the Indiana Pacers. Suddenly he looked like a different player. He made the All-Star team this year and almost took down LeBron James and the Cleveland Cavaliers in the first round of the playoffs.

Gladwell’s point is that we should think much more carefully about employee fit. A successful worker somewhere may not be successful everywhere. Individuals are dependent on their teammates and organizational supports; context matters.

Steve Glazerman

How much does fit matter for teachers? Is it as important in schools as it seems to be in basketball? To learn more, we reached out to Steve Glazerman, a Senior Fellow at Mathematica Policy Research. We talked to him at the end of last month, just as the NBA Finals were starting. What follows is an edited transcript of our conversation.  

Chad Aldeman: You helped lead an evaluation of a federal program called the Talent Transfer Initiative. Can you tell us about what the program tried to do, whether it was successful, and what we can learn from it? Continue reading

Early Childhood Educator Profession and Competencies: Our Take on What “Power to the Profession” Gets Right and Wrong

Bellwether’s early childhood team regularly publishes research and analysis on the early childhood workforce and advises foundations and other clients seeking to improve early childhood teaching, strengthen the early childhood workforce, and support early childhood educators. In our work we routinely confront the deep disconnect between what research demonstrates about the importance of and skills required for high-quality early childhood teaching and the inconsistent standards, low compensation, and lack of professional prestige accorded to early educators.

two teachers read to preschool students

Courtesy of Allison Shelley/The Verbatim Agency for American Education: Images of Teachers and Students in Action.

In light of this, we’ve been avidly following the National Association for the Education of Young Children (NAEYC) Power to the Profession process, which seeks to advance the early childhood field by defining a unifying framework for career pathways; knowledge; and competencies, qualifications, standards, and compensation.

This work is being carried out in iterative cycles by a Task Force representing 15 professional associations and organizations related to the early childhood field. In February, the Task Force released draft recommendations for Decision Cycles 3-5, which deal with qualifications requirements for early childhood educators and sources and pathways for acquiring competencies and credentials. The recommendations have sparked lively debate in the field. My overall take, shared with my colleagues Ashley LiBetti and Marnie Kaplan, is that the recommendations would represent progress in setting a baseline of training for early childhood educators in many roles and settings, but could also represent a step backwards in standards for teachers leading publicly funded pre-K classrooms serving 3- and 4-year-olds.

We also believe it’s crucial that any conversation about qualifications for early childhood educators engage seriously with the need to improve quality of early childhood educator preparation programs — as well as the tensions and gaps in knowledge about how best to do so. These conversations also need to provide space for innovative thinking about new models for delivering preparation and training that meet the needs of current and prospective early educators with diverse life and professional experiences and prior education backgrounds.

Marnie, Ashley, and I published our full comments* on these recommendations here:screenshot of first page of Power to the Profession comments

You can learn what others are saying here or comment yourself by visiting NAEYC’s survey here.

Want to know more from Bellwether? Check out our recent research and reports on the Head Start workforce, what we know about coaching as a strategy to improve early childhood teaching quality, the role of community colleges in early childhood preparation, and what it would take to make equitable access to quality higher education a reality for all pre-K teachers.

*Note: The statements contained in this comment reflect the personal views of the authors, and should not be attributed to Bellwether Education Partners or any others within the organization. Bellwether does not take organizational positions except on issues that affect nonprofit organizations as a class.

Education Policy, Meet Human-Centered Design

In a lot of ways, the worlds of education policy and human-centered design couldn’t be more dissimilar. The former relies heavily on large-scale quantitative analysis and involves a long, complex public process. The latter is deeply qualitative, fast moving, creative, and generative. Policy professionals come up through the ranks in public agencies, campaigns, and think tanks. Deep issue expertise and sophisticated deductive reasoning are highly valued. Designers come from an array of backgrounds — the more unorthodox the better. Success for them comes from risk-taking, novel ideas, and synthesizing concepts across time, space, and sectors.

figure from Creating More Effective, Efficient, and Equitable Education Policies with Human-Centered Design comparing policy and design methods

figure from Creating More Effective, Efficient, and Equitable Education Policies with Human-Centered Design

I’m fortunate to have spent some time in both worlds. They each appeal to different parts of my personality. Policy analysis affords me order and confidence in answers based on facts. Design lets me flex my creative muscles, fail fearlessly, and have confidence in answers based on experience.

So when a grant from the Carnegie Corporation of New York gave me the opportunity to write a paper about bringing these two worlds together, I jumped at the chance — I knew that each could benefit from the other.

Creating More Effective, Efficient, and Equitable Education Policies with Human-Centered Design makes the case that policy practitioners can use human-centered methods to create better education policies because they are informed by the people whose lives will be most affected by them.

The underpinning hypothesis is that 1) co-designing policies with constituents can generate more accurate definitions of problems and more relevant solutions, 2) human-centered design can generate a wider variety of potential solutions leading to innovation, and 3) the process can mitigate or reverse constituent disenfranchisement with the lawmaking process.

Human-centered policy design is still a new practice, however, and there are still important questions to work out, like how to make sure the process is inclusive and where exactly human-centered design methods can enhance policy research and design.

Luckily, SXSW EDU, a huge national conference focused on innovation in education, is a perfect place to test new ideas. So I reached out to Maggie Powers, director of STEAM Innovation at Agnes Irwin School and member of IDEO’s Teachers Guild, and Matt Williams, vice president of Education at Goodwill of Central Texas, to explore what it would look like to apply human-centered design to policies that affect high school students whose education suffers because of lost credits when they transfer schools. Our session will pressure test some of the ideas that emerged in the paper. The results will inform the next phase of this work, which will help policy practitioners implement human-centered design methods. Keep an ear to the ground for that!

How an East Coast/West Coast Hip Hop Rivalry Helped Us Find Evaluation’s Middle Ground

Everyone loves a good rivalry. The Hatfields vs. the McCoys. Aaron Burr vs. Alexander Hamilton. Taylor Swift vs. Katy Perry.

As evaluators, we’re partial to Tupac vs. Biggie. For the better part of three decades, these rappers from opposing coasts have remained in the public eye, recently reemerging with the release of a television series about their unsolved murders. Interestingly, their conflict about artistry and record labels mirrors a conflict within evaluation’s own ranks around a controversial question:

Can advocacy be evaluated?

Images via Stanford University, Zennie Abraham, Takeshl, and Harvard University

On the East Coast, Harvard’s Julia Coffman acknowledges that evaluating advocacy can be challenging, thanks to the unique, difficult-to-measure goals that often accompany these efforts. Nevertheless, she contends, these challenges can be mitigated by the use of structured tools. By using a logic model to map activities, strategies, and outcomes, advocates can understand their efforts more deeply, make adjustments when needed, and, overall, reflect upon the advocacy process. This logic model, she claims, can then become the basis of an evaluation, and data collected on the model’s components can be used to evaluate whether the advocacy is effectively achieving its intended impact.

In contrast to the East Coast’s structured take, West Coast academics refer to advocacy as an “elusive craft.” In the Stanford Social Innovation Review, Steven Teles and Mark Schmitt note the ambiguous pace, trajectory, and impact related to the work of changing hearts and minds. Advocacy, they claim, isn’t a linear engagement, and it can’t be pinned down. Logic models, they claim, are “at best, loose guides,” and can even hold advocates back from adapting to the constantly changing landscape of their work. Instead of evaluating an organization’s success in achieving a planned course of action, Teles and Schmitt argue that advocates themselves should be evaluated on their ability to strategize and respond to fluctuating conditions.

Unsurprisingly, the “East Coast” couldn’t stand for this disrespect when the “West Coast” published their work. In the comment section of Teles and Schmitt’s article, the “East Coast” Coffman throws down that “the essay does not cite the wealth of existing work on this topic,” clearly referring to her own work. Teles and Schmitt push back, implying that existing evaluation tools are too complex and inaccessible and “somewhat limited in their acknowledgement of politics.” Them’s fighting words: the rivalry was born.

As that rivalry has festered, organizations in the education sector have been building their advocacy efforts, and their need for evidence about impact is a practical necessity, not an academic exercise. Advocacy organizations have limited resources and rely on funders interested in evidence-based results. Organizations also want data to fuel their own momentum toward achieving large-scale impact, so they need to understand which approaches work best, and why.  

A case in point: In 2015, The Collaborative for Student Success, a national nonprofit committed to high standards for all students, approached Bellwether with a hunch that the teacher advocates in their Teacher Champions fellowship were making a difference, but the Collaborative lacked the data to back this up.

Teacher Champions, with support from the Collaborative, were participating in key education policy conversations playing out in their states. For example, in states with hot debates about the value of high learning standards, several Teacher Champions created “Bring Your Legislator to School” programs, inviting local and state policymakers into their classrooms and into teacher planning meetings to see how high-quality, standards-driven instruction provided for engaging learning opportunities and facilitated collaborative planning.

But neither the Collaborative nor the teachers knew exactly how to capture the impact of this workWith Teacher Champions tailoring their advocacy efforts across 17 states, the fellowship required flexible tools that could be adapted to the varied contexts and approaches. Essentially, they needed an East Coast/West Coast compromise inspired by Tupac and Biggie and anchored by Coffman, Teles, and Schmitt. Continue reading

Two Graphs on Teacher Turnover Rates

I have a new piece up at The 74 this morning arguing that, contrary to popular perception within the education field, we do not have a generic teacher turnover crisis. Why do I say that? Two graphs help illustrate my point.

First, consider this graph from the Bureau of Labor Statistics. It shows job openings rates by industry from 2002 to 2017. I’ve added a red arrow pointing to the line for state local government employees who work in education (this group is predominantly public school teachers). As the graph shows, public education has consistently lower job openings rates than all other industries in our economy.

As I write in my piece today, “public schools have much lower rates of job openings, hire rates, quit rates, and voluntary and involuntary separations than every industry except the federal government. Across all these measures, public schools have employee mobility rates that are roughly half the national averages.”

Instead of having some sort of generic turnover problem that applies to all teachers nationally, we actually have problems that are unique to certain schools, districts, and subject areas. To illustrate this point, take a look at the graph below from the annual “Facts and Figures” report from BEST NC. It maps teacher turnover rates by district in North Carolina. Overall, the state has a teacher turnover rate that’s lower than the national average. But some districts have turnover rates about half of the state average, while others are twice as high as the average.

For more, go read the full piece in The 74 for my thoughts on what this means for the education field.