Category Archives: Student Assessment

NAEP Results Again Show That Biennial National Tests Aren’t Worth It

Once again, new results from the National Assessment of Educational Progress (NAEP) show that administering national math and reading assessments every two years is too frequent to be useful.

The 2017 NAEP scores in math and reading were largely unchanged from 2015, when those subjects were last tested. While there was a small gain in eighth-grade reading in 2017 — a one-point increase on NAEP’s 500-point scale — it was not significantly different than eighth graders’ performance in 2013.

Many acknowledged that NAEP gains have plateaued in recent years after large improvements in earlier decades, and some have even described 2007-2017 as the “lost decade of educational progress.” But this sluggishness also shows that administering NAEP’s math and reading tests (referred to as the “main NAEP”) every two years is not necessary, as it is too little time to meaningfully change trend lines or evaluate the impact of new policies.

Such frequent testing also has other costs: In recent years, the National Assessment Governing Board (NAGB), the body that sets policy for NAEP, has reduced the frequency of the Long-Term Trends (LTT) assessment and limited testing in other important subjects like civics and history in order to cut costs. NAGB cited NAEP budget cuts as the reason for reducing the frequency of other assessments. However, though NAEP’s budget recovered and even increased in the years following, NAGB did not undo the previously scheduled reductions. (The LTT assessment is particularly valuable, as it tracks student achievement dating back to the early 1970s and provides another measure of academic achievement in addition to the main NAEP test.)

Instead, the additional funding was used to support other NAGB priorities, namely the shift to digital assessments. Even still, the release of the 2017 data was delayed by six months due to comparability concerns, and some education leaders are disputing the results because their students are not familiar enough with using tablets.

That is not to say that digital assessments don’t have benefits. For example, the new NAEP results include time lapse visualizations of students’ progress on certain types of questions. In future iterations of the test, these types of metadata could provide useful information about how various groups of students differ in their test-taking activity.

Animated GIF - Find & Share on GIPHY

However, these innovative approaches should not come at the expense of other assessments that are useful in the present. Given the concerns some have with the digital transition, this is especially true of the LTT assessment. Instead, NAGB should consider administering the main NAEP test less frequently — perhaps only every four years — and use the additional capacity to support other assessment types and subjects.

Can You Name the Branches of Government? Most Americans Can’t.

Today is Constitution Day, a holiday commemorating the formation and signing of the U.S. Constitution on September 17, 1787 — 230 years ago. As “a nation of immigrants,” America’s national identity is largely tied to our founding documents, endowing the Constitution with a unique importance in American culture. However, many Americans know little about this document that we are supposed to support and defend.

Last week, the Annenberg Public Policy Center (APPC) of the University of Pennsylvania released its Constitution Day Civics Survey, with dismal results. Only one in four respondents were able to name all three branches of government, a 12-point decline since 2011. Shockingly, 33 percent could not name a single branch.

The survey also asked respondents to identify which rights are guaranteed by the First Amendment. While nearly half (48 percent) were able to name “freedom of speech,” only 15 percent could name “freedom of religion.” Even fewer respondents identified the other rights (freedom of the press, right to petition, and right of assembly). Thirty-seven percent couldn’t name any.

Kathleen Hall Jamieson, director of APPC, expressed her concern: “Protecting the rights guaranteed by the Constitution presupposes that we know what they are. The fact that many don’t is worrisome.”

Perhaps, in prior years, this warning may have seemed overblown. But in the Trump era, amid a seemingly constant slew of anti-democratic rhetoric, it feels right on the nose. For example, when asked whether those who are in the country illegally have any rights under the Constitution, 53 percent of APPC’s respondents disagreed. In this context of widespread ignorance and misinformation, the United States has seen an uptick in hate crimes associated with the rise of President Trump, beginning in 2015, persisting into 2016 and 2017, and culminating in the violence of the “Unite the Right” rally of white nationalists in Charlottesville last month.

Luckily, some states are taking action to bolster the civic knowledge of their students. For example, over the past three years, 17 states have adopted a “citizenship test” requirement for high school students. In eight of those states, students must receive a passing score on the test to receive a high school diploma. The questions are drawn from the the United States Citizenship and Immigration Services (USCIS) naturalization civics test, which immigrants must pass to become legal U.S. citizens.

This is a good first step, but it is far from sufficient. The test is not designed to be a high school civic literacy exam. It sets a low bar, with basic multiple-choice questions that ask test-takers to identify one branch of the government, or know how many amendments have been made to the Constitution. The simplicity is reflected in the initial test results, with very high passage rates and few students failing to pass the test after repeated attempts.

However, such a test is only one tool available to policymakers. They can design and administer higher quality civics assessments; implement robust standards and curricula for civics instruction; and provide real-world, project-based opportunities for students to learn about government and civic engagement. For example, New Hampshire passed legislation in 2016 requiring a civics test. But, rather than simply implementing a citizenship test for high school students, the legislation allows for the creation of locally developed assessments that can include a broader range of questions. Additionally, the state created a recognition for students who pass the required test by authorizing school districts to issue civic competency certificates.

New Hampshire Senator Lou D’Allesandro, a former civics teacher who sponsored some of the state’s legislation, summarized the issue well: “We always complain, ‘people don’t know anything about the system, they don’t get involved, they don’t vote.’ Well, they don’t vote because they don’t understand the importance of voting and how meaningful it is to participate in the process.”

If America wants to protect our constitutional rights and democratic ideals, we must ensure that our next generation of citizens are knowledgeable and engaged. That starts in the classroom.

Donald Trump’s Election is a “Sputnik Moment” for Civics Education

Last week, the American Enterprise Institute hosted an event discussing the failings of civics education in America. The panelists referred to the dismal state of civics literacy as a “Sputnik moment” – a reference to when the Soviet Union successfully launched the world’s first satellite in 1957, stirring the United States to create the National Aeronautics and Space Administration (NASA) and dramatically increase its space exploration efforts.

Nothing illustrates this comparison better than the election of Donald Trump. As Trump has demonstrated time and time again, he knows little about governing or policy – instead relying on divisive rhetoric and petulant Twitter tantrums. His most recent gaffe: at a White House convening of the nation’s governors, Trump said that “nobody knew health care could be so complicated.” As it turns out, many people knew.

However, if Trump can name all three branches of government, that alone would put him ahead of nearly three quarters of Americans. According to a 2016 survey conducted by the Annenberg Public Policy Center, only 26 percent of respondents could name all three branches, and 31 percent could not name a single one.

Data from the National Assessment of Educational Progress (NAEP) also show poor results. In 2014 – the most recent NAEP civics assessment – only 23 percent of eighth grade students scored at or above the proficient level. The same is true of older students getting ready to vote. In 2010, when NAEP last tested high school seniors, only 24 percent scored at or above the proficient level. Neither of these results has changed significantly since 1998.

At the same time, faith in many of America’s institutions are at historic lows – even before Trump’s election. And it’s likely that his constant attacks on various institutions will only serve to worsen these numbers. This crisis of confidence only feeds into the growing level of polarization, making it nearly impossible to govern effectively. It’s no wonder that recent congresses have been arguably some of the least productive ever.

Confidence in Institutions

Despite these difficulties, the American people seem well aware of the problem at hand. According to the 2016 PDK poll of the public’s attitudes toward the public schools, 82 percent of Americans believe preparing students to be good citizens is very or extremely important. At the same time, only 33 percent think the public schools in their communities are doing that job very or extremely well.

So what is to be done? Continue reading

New Bellwether Analysis on Michigan Education Provides Facts for DeVos Debate

When President Donald Trump nominated Betsy DeVos to serve as his Secretary of Education, she was not well known on a national scale: her behind-the-scenes advocacy and philanthropic work has concentrated on her home state of Michigan. But DeVos’ nomination put a national spotlight on education in Michigan, and her critics and boosters alike have been making a variety of claims about Michigan that are confusing and contradictory.

Slide1To address this, Bellwether just released a fact base on education in Michigan to inform the conversation about DeVos’ work there and what it might mean for the Department of Education if she is confirmed.

Our slide deck report addresses a number of key questions: How are Michigan students performing, and what do achievement gaps look like for low-income students and students of color? Do charter schools in Michigan produce better results than district-run public schools, and if so, by how much? Why does Michigan have so many charter schools operated by for-profit companies?

Among the things we found:

  • Michigan typically ranks in the lowest third of states in terms of student proficiency on the National Assessment of Educational Progress (NAEP), and state assessment results show wide achievement gaps by racial/ethnic group and income level.
  • Educational authority in Michigan is highly decentralized, with multiple state governing entities and over 40 charter school authorizers.
  • About 150,000 Michigan students attend public charter schools, making up 10 percent of the student population.
  • Another 200,000 students, or 13 percent, take advantage of inter-district choice options to attend schools outside of their home district.
  • On average, students attending charter schools learn more than comparable students attending district-run schools. However, producing greater learning gains compared to schools serving similar students is a low bar because most Michigan charters are in Detroit, one of the lowest-performing large, urban school districts in the country.
  • Repeated reform efforts to improve Detroit Public Schools (DPS) have not produced academic improvements for students or solved the ongoing financial crisis in the school district. A new law reinstates local control over DPS, limits charter school expansion to nationally accredited authorizers, and creates an A-F accountability system for both charter schools and traditional public schools.

Through data analysis and a deeper dive into the context of the Michigan education landscape, we hope to inform the ongoing debate about DeVos and give new insight into education in Michigan. The state has been a laboratory for school choice and educational reform efforts, and demands a more complete context and deeper analysis than sound bytes can provide. Read the full report here and let us know what you think.

What Good Are Higher Graduation Rates If Students Aren’t Learning More?

On Thursday, the National Assessment of Educational Progress (NAEP) released the results of its 2015 science assessment for America’s 4th, 8th, and 12th grade students. Only 22 percent of 12th graders scored at or above the proficient level, compared to 38 percent of 4th graders and 34 percent of 8th graders. And while 4th and 8th graders both saw a small but significant improvement from 2009, high school seniors stagnated — earning the same average score as the 2009 sample.

This was also true across all subgroups. Among students of colors, students with disabilities, English language learners (ELLs), rural students, and female students, not a single group saw a statistically significant score change from 2009.2015 NAEP Science Assessment Scores

We saw a similar trend in April, when NAEP released the 12th grade results of its 2015 reading and math assessments. Seniors’ average reading score did not significantly change — again across every single subgroup. The average 12th grade math score declined.

And yet, earlier this month, data released by the U.S. Department of Education (ED) showed that America’s high school graduation rate has reached a record high of 83 percent, continuing a five year trend. In stark contrast with this year’s NAEP data, rates among students of color, students with disabilities, ELLs, and low-income students have all improved.

While this is certainly good news, it begs the question: What good are higher graduation rates if students aren’t learning more?

According to ED Secretary John King: “Students who have a high school diploma do better in the 21st Century economy than students who don’t. So having a higher graduation rate is meaningful progress.” While high school graduates do earn more than non-graduates, this answer is still deeply unsatisfying.

States will have the opportunity to seriously address America’s stagnant high schools in the coming years. The Every Student Succeeds Act (ESSA), signed into law last December, provides greater flexibility for states in almost every facet of federal K-12 education policy. The law makes it easier for states to spend Title I money on high school students. It also gives states much greater leeway for using school improvement funds, including an optional set-aside for programs like Advanced Placement, International Baccalaureate, and career and technical education. It remains to be seen exactly how states will implement the law, but luckily we’ll have NAEP along the way to give us a national snapshot of student learning.

We’re doing a better job of shepherding students to high school completion — now we just need to make sure they actually learn something.