Released Tuesday, from over 7.7 million students who took the widely used MAP Growth tests from NWEA doesn’t bode well for teens entering high school this fall. Finishing 4^th grade when the pandemic hit, many students not only lost at least a year of in-person learning, but also transitioned to middle school during a chaotic period of teacher vacancies and rising absenteeism.  

The 2023-24 results reflect the last tests administered before federal COVID relief funds run out. Districts must allocate any remaining funds by the end of September — a cutoff that is expected to cause further disruption as districts eliminate staff and programs aimed at learning recovery.

Older students don’t make gains as quickly as younger kids and will have to work harder to catch up, the researchers said. At the same time, the effects of the pandemic “continue to reverberate” for children in the early elementary grades, many of whom missed out on preschool because of COVID. On average, students need at least four extra months of schooling to catch up.

“It’s not fun to continue to bring this bad news to the education community, and I certainly wish it was a brighter story to tell,” said Karyn Lewis, director of research and policy partnerships for NWEA. “It is pretty frustrating for us, and I’m sure very disheartening for folks on the ground that are still working very hard to help kids recover.”

Thus far, only two states, California and Colorado, have asked officials for extra time to spend the diminishing relief funds that remain, according to the U.S. Department of Education. That means the question for most leaders is how to keep paying for extra tutoring and staffing levels for students still learning below grade level — especially those belonging to groups that weren’t meeting expectations before the pandemic.

Relief money “made a difference, but it certainly did not eliminate the learning loss,” said Dan Goldhaber, director of the . A he authored showed that recovery linked to those dollars was small, in part because the federal government gave districts few restrictions on how to spend it. 

The American Rescue Plan’s requirement that districts devote 20% of the $122 billion toward reversing learning decline was “super loosey goosey in terms of what that actually meant, how it was measured and what programs counted,” he said.

Some districts that hired teaching assistants to give students additional practice in reading and math have now lost those positions. Dothan Preparatory Academy in Alabama, a seventh- and eighth-grade school, had several staff members who gave students “a few extra lessons” throughout the day based on their MAP scores, said Charles Longshore, an assistant principal. Now those positions are gone. 

He hopes a new sixth grade academy opening next month will better prepare kids for grade-level material. Two years ago, when he joined the Dothan City Schools, just north of the Florida Panhandle, he attended a districtwide administrator meeting where every school’s data was posted on the walls.  

He remembers looking at elementary school scores with “really low” student proficiency rates of roughly 20% to 30%; teachers have been trying to fill those gaps ever since. 

“We’re trying to go backwards to go forwards,” he said. “What third, fourth and fifth grade standards did you miss that are essential for your understanding of seventh and eighth grade standards?”

The NWEA results show achievement gaps continuing to widen. For example, Asian students are showing some growth, but made fewer gains in math last year than during the pre-COVID years. White, Black and Hispanic students, however, continue to lose ground. In both elementary and middle school, Hispanic students need the most additional instruction to reach pre-COVID levels, the data shows.

In reading, the gap between pre-pandemic growth and current trends widened by an average of 36%, compared with 18% in math. It’s possible, NWEA’s Lewis added, that districts focused extra recovery efforts on math because initial data on learning loss showed those declines to be the most severe. 

But that’s left many students without the reading skills to tackle harder books and vocabulary as they move into high school, said Rebecca Kockler, who leads Reading Reimagined, a project of the nonprofit . The organization is funding research to find which literacy strategies work with adolescents, who are easily turned off by books intended for young kids.

The pandemic, she said, only exacerbated a longstanding literacy problem for older students.

“About 30% of American high schoolers for 30 years have been proficient readers, and that really hasn’t changed,” said Kockler, a former Louisiana assistant superintendent who oversaw a redesign of the state’s reading program. “It’s always the hardest to move middle school reading results, and even some of the success we would see in fourth grade didn’t always carry up into middle school.”

School closures were especially hard on students with learning disabilities. Both of Tracy Compton’s daughters, who are entering fifth and seventh grade this fall, have dyslexia and didn’t receive services during the pandemic when they were in the Fairfax County Public Schools in Virginia. 

“The time they were learning to read was during the school board’s shutdown of schools,” she said. Under a , the Fairfax district still pays for makeup services with a private tutor. 

But Compton also moved to a Massachusetts district where she feels her girls are getting the support they need, like access to audio books and noise-canceling headphones during tests to help them focus. “They have made progress, but [have] not fully recovered,” she said.

She said too many parents don’t know their children are behind.

“They see the report card with A’s or whatever and think all is fine,” she said. “They don’t know where else to check and how to weigh things like standardized tests.”

That’s likely because different tests often tell different stories. The MAP results, for example, are worse than what many states have reported about student performance on their own assessments, which are used for accountability. 

Several states last year noted at least partial recovery, and a few showed students had even reached or were exceeding 2019 scores. Lewis explained that state tests measure the “blunt designation between proficient or not,” while MAP tests capture the full spectrum of student achievement levels during the school year.

Districts, particularly low-income districts that received the most funding, need to contend with the latest snapshots of students’ learning as they adjust to the end of federal relief funds, Goldhaber said. 

“How districts go about dealing with the fiscal cliff is going to have pretty significant consequences, particularly for the kids that were most impacted by the pandemic,” he said. If districts have to lay off staff — and newer teachers are the first to go — they should limit the impact on the neediest students. “They’ll be shuffling teachers within districts, and that shuffle itself is harmful for student achievement.”

As more time passes since the pandemic, Lewis added that school leaders might be tempted to stop comparing their students’ performance to pre-COVID levels, when states were in closing achievement gaps. 

“What keeps me up at night is this idea that these persistent achievement gaps are inevitable, that this is just how it’s going to be,” she said. “I don’t think that’s the case, but I do think it takes innovation and creative thinking … to get us out of this mess.”

But the social scientists who authored the research argue that federal dollars could have been spent in ways that would have helped scores bounce back faster. The per-dollar returns of ESSER, the Elementary and Secondary School Emergency Relief Fund, measure up poorly in comparison with those of previously studied efforts to boost achievement, from reducing class sizes to implementing more rigorous curricula.

Dan Goldhaber, the lead author of and the director of the , said he believed the crisis conditions of the pandemic made it “hard to spend the ESSER funding in thoughtful, effective ways.”

By his own estimate, 35% of the math recovery achieved during the 2022–23 school year was directly attributable to ESSER funding. Fully 87% of English recovery was credited to ESSER, though he found that gains in that subject were statistically insignificant. Still, he said, that upward movement was limited. 

“Candidly, I think the impact was small, and there are some reasons why it wasn’t larger,” Goldhaber said. “Only 20% of ESSER money was even earmarked for learning loss, and I don’t think there was a lot of oversight of whether that 20% was well spent.”

The findings offer a split verdict on the post-COVID academic recovery, while somewhat strengthening the case that putting more resources into schools can elevate their results. The advances measured in both studies are virtually identical not only to one another, but also to earlier, wide-ranging estimates of the impact of additional money on schools.

ESSER was one of the best-known and longest-lasting pillars of Washington’s pandemic response. Years after stimulus checks and free nasal swabs stopped arriving in the mail, many districts are still spending down the aid they received through the program. The last of the supplemental aid will not expire until this September, four years after schools first began to reopen for in-person instruction.

Notably, however, both papers project that American students will not have returned to their pre-COVID learning trajectories by then, and that the cost of a full restoration could amount to hundreds of billions more. With no sign of any further assistance coming from Congress, that bill will need to be picked up by states — if it is paid at all. 

In the meantime, ESSER’s backers can point to real, if incomplete, progress.

Stanford sociologist Sean Reardon helps lead the , which released a second study on Wednesday. In an interview, he noted that the federal cash injection was the equivalent of of the country’s annual K–12 spending, spread over multiple years. While it might have been used more efficiently to stem further learning loss, he added, both national and state leaders were simultaneously focused on goals like reopening schools and alleviating the severe emotional distress that many children are still facing.

“One can certainly imagine ways to spend the money that would lead to even more learning gains,” Reardon said. “But that wasn’t entirely what was on policymakers’ minds when they sent out the money.”

‘A huge missed opportunity’

To pinpoint the impact of additional money on COVID-era learning, the two studies take advantage of differences in how the federal funding was awarded to individual districts.

The total ESSER expenditure was fueled by three laws setting aside $13 billion in March 2020, $57 billion in December of that year, and a further $122 billion the following March. Because there was no data showing where learning loss was most concentrated at that time, dollars were allocated to school districts based on their pre-pandemic grants from Title I, the Department of Education’s main program benefiting disadvantaged children.  

But not all districts received comparable amounts, even if they served similar numbers of needy students. Instead, governing Title I — including rules that ensure small states receive minimum allotments, as well as larger sums being granted to states with higher per-pupil spending — introduced significant spending gaps between different schools. Those disparities were significantly magnified as each new emergency funding bill was passed, said Harvard economist Thomas Kane, Reardon’s co-author. 

“With the second two ESSER packages, the federal government was essentially pushing $175 billion through pipes that were meant to handle $16 billion in Title I,” Kane said. “So what might have been a $500 or $600 difference per student in Title I dollars became a $5,000 or $6,000 difference in ESSER funding per student.” 

Both Goldhaber and the Education Recovery Scorecard team accessed standardized test results from the Stanford Education Data Archive, which compiles student scores from different local exams to allow for cross-state comparisons. In each of their studies, $1,000 in ESSER spending per student was found to raise math scores by 0.008 of a standard deviation (a scientific measure showing the distance from a statistical mean).

In the world of education research, an improvement of that size is considered small: something like one-tenth of a medium-sized effect. But the average conceals substantial variation across different states, and many school districts received much more than $1,000 per student. 

As an example, Reardon, Kane, and their collaborators identified 704 districts in which over 70% of students were eligible for free and reduced-price lunch — a commonly used proxy for poverty — then compared the results for those that received unusually large ESSER allocations (more than $8,600 per pupil) to those that received much less (less than $4,600 per pupil). 

The differences were striking. The working-class district of Brockton, Massachusetts was awarded $3,224 per student from the second and third ESSER funding bills, and its students’ math achievement improved by the equivalent of .06 grade levels between 2022 and 2023; but in Dayton, Ohio, per-pupil funding increased almost three times as much ($11,444), and math scores jumped by a factor of 10 (.65 grade levels).

Goldhaber argued that figures like those cast considerable doubt on the proposition that the U.S. government’s emergency relief to schools was mostly wasted.

“One of the ideas that’s out there is that we spent $190 billion and got nothing,” he said. “I don’t think that’s the right answer.”

Yet he also voiced disappointment that neither Washington nor states had directly measured what kinds of ESSER spending (tutoring programs or school renovations, improved ventilation or increased staffing) were correlated with higher performance. Despite its huge cost and high stakes, Goldhaber concluded, ESSER was simply “not designed to learn from what districts do.”

“To my mind, that makes it a huge missed opportunity. We can see that there are pretty big differences across states and districts in the degree of catch-up.”

‘Who’s going to pick up the reins?’

While the studies can shed little light on the most successful aspects of ESSER, they will be collectively seen as a major contribution to the research on school finance reforms. This is true both because of the scale of the government’s intervention — perhaps the single greatest natural experiment on the effects of windfall cash on schools that has ever been attempted — and the consistency of the papers’ results. 

Not only do the findings of both studies mirror one another, they also hew closely to those of , published in January, that gathered the results of dozens of previous experiments in increased school funding. That paper also pointed to an average test-score increase of about .032 standard deviations per $1,000 spent over four years, or roughly .008 annually.

Marguerite Roza, head of Georgetown University’s finance-focused Edunomics Lab, called the coinciding findings “reassuring.”

Yet she also noted the “wildly expensive” cost of sending operating aid to states that was not specifically dedicated to learning recovery. According to Goldhaber’s calculations, the government would need to spend an additional $450–$650 billion to fund a full return to levels of academic achievement last seen in 2019; Reardon and Kane tallied a likely cost of just over $904 billion. 

Whether or not those figures represent the true price tag, Roza said, states that intend to replace federal dollars should be more consistent in disbursing them and more stringent about what they pay for.

“Why repeat the same strategy given how unevenly the dollars were distributed and how uneven the effects were on districts and states?” Roza asked. “Given what we know now, any new federal dollars for recovery should probably be structured differently.”

But in Kane’s view, that recommendation may be too optimistic. With just a few months left before the deadline to spend ESSER funds, he observed, too few state authorities had even committed to picking up the torch of learning recovery. 

“In most states, there hasn’t even been a discussion started about what the state role will be now that the federal money is running out,” he said. “Our results are basically saying that there was a positive effect, but it wasn’t enough. Now who’s going to pick up the reins?”

, from at the American Institutes for Research, looked at the academic progress of students who attended summer school in 2022 across eight school districts, most of them urban or suburban, in seven states.

Here are four key takeaways:

1. Finally, a bit of positive news about post-pandemic interventions.

The new paper represents what could be the first encouraging findings coming out of post-pandemic interventions, said CALDER’s . Earlier research , he noted, with few positive effects. “One of my takeaways is that it’s nice to be able to say, ‘Hey, there is something that school systems can do to help kids get back on track,” he said, “even if it’s only making incremental change.”  

2. Summer school works for math … but for reading: not so much.

The researchers found that summer programs had sizable positive effects on students’ math achievement, potentially closing about 2% to 3% of districts’ total learning losses in math, but not in reading. The math gains were “positive and significant,” said Goldhaber, large enough for researchers to suggest that districts consider offering summer math programs to many more students in the future. Reading scores improved in just one of the eight districts.

He noted that research has established that so-called “math effects” due to school interventions are more likely than reading effects. 

“The ‘math but not reading’ is consistent with education research writ large,” he said. Simply put, schools have a more significant impact on kids’ math skills than on reading skills, probably because kids read and write outside of school, but don’t necessarily do math.

3. Post-COVID summer programs are at least as good as those schools operated pre-COVID — and they’re targeting kids who need them most.

Researchers compared impacts of current summer programs to those operating before the pandemic and found that they’re having “about the same kind of impact as summer school programs pre-pandemic,” Goldhaber said. That’s encouraging, since in many districts, summer programs have grown in scale but haven’t suffered in quality, according to the new findings. Any time educators push to scale up interventions, he said, it’s harder to maintain quality. “So it’s encouraging” to see quality stay high. 

He also said the programs they examined typically targeted students who were struggling and actually needed the extra help. 

4. The good news about summer math learning is tempered by the fact that so few students are getting it at the moment.

The CALDER researchers estimated that only 15% of eligible students in 2022 were receiving summer math instruction. That means schools last year were under-utilizing what could have been a powerful, effective intervention.

The positive effects, Goldhaber said, “are kind of dwarfed by the magnitude of the COVID learning losses,” with the small number of students in effect disguising its potential effect on achievement. What could be a game-changer for the moment shows a “pretty small” effect on achievement as a result. 

The new study is part of a larger “Road to COVID Recovery” partnership between researchers at the American Institutes for Research, Harvard University, NWEA and 11 school districts in total. Just eight supplied data to this study.  

Researchers used value-added models to estimate the effect of each of the eight summer programs on MAP Growth test scores, with Spring 2022 as the baseline and Fall 2022 as the outcome. Summer sessions ran from three to six weeks, depending on the program, and daily classes ranged from 45 minutes to two hours.

CALDER’s findings could scarcely come at a better time, with recent NAEP scores suggesting that COVID had a “cataclysmic” impact on K–12 education, coming on the heels of a decade of stagnation. 

Other recent research from Michigan showed that the pandemic slowed students’ math achievement over the three-year period from spring 2019 through spring 2022, with achievement growth “substantially lower” than that of comparable students in the three earlier years.

As with the CALDER findings, the Michigan researchers found that scores for English language arts, which include reading and writing, were small and generally not statistically significant.

In December, researchers from , NWEA and Harvard University, looked at achievement in a dozen mid-to-large sized school districts, enrolling more than 600,000 students across 10 states, and found that between fall 2021 and spring 2022, schools had basically put an end to student achievement declines in math and reading relative to pre-pandemic levels — but that average test score gains during the 2021-22 school year hadn’t moved past pre-pandemic levels.

Students in a few elementary grades improved substantially in math, but beyond a few areas, researchers didn’t find a lot of compelling evidence of recovery in other subjects or grades.

Aaron Dworkin, CEO of the , said he was encouraged by the new CALDER findings, adding that summer programs can often try different strategies “that you might not be able to always utilize” during the school year. 

He noted one successful free program in Detroit called , created by Wayne State University’s math department, that uses an unusual model: College math majors get paid to teach high school students, who get paid to teach middle school students.

The program maintains a fun, playful high-energy atmosphere that catches students’ attention, especially in the summer. “Hundreds of kids and families love it,” Dworkin said. “And they’re so supportive.”

Those conclusions represent the latest and widest-ranging evidence yet of the damage inflicted by the emergence of COVID — both in terms of direct interruptions to schooling and the social and economic turmoil in other spheres of life. They dovetail with the observations of education experts who have also pointed to steep declines in nationwide academic performance, along with the billion-dollar investments made by governments to help schools bounce back.

But they also come as have argued that learning losses may be less harmful than advertised, that a single-minded focus on the pandemic’s toll could hurt teacher morale. With American students now returned to a post-COVID reality in the classroom, a recent speech by Secretary of Education Miguel Cardona largely skirted the subject of learning loss.

By contrast, the review, based on 42 research studies from 15 countries, calls for heightened urgency from both leaders and educators in re-setting the trajectory for student outcomes.  

“The persistence of learning deficits two and a half years into the pandemic highlights the need for well-designed, well-resourced and decisive policy initiatives to recover learning deficits,” the authors write. “Policymakers, schools, and families will need to identify and realize opportunities to complement and expand on regular school-based learning.”

Robin Lake, director of the Center for Reinventing Public Education, said the findings were consistent with those of her own organization’s , released last summer.

“There is no question that learning gaps have become chasms and that while some students are catching up quickly, too many are not,” Lake wrote in an email. “This is a global concern and requires innovative and urgent action. I am deeply concerned that national educational and civic leaders in the U.S. are not taking this learning crisis seriously enough.”

Here are four key takeaways from the study:

1. Inequality in education grew

While the dozens of studies classified socioeconomic status according to different metrics (including parental income, free lunch eligibility, and neighborhood disadvantage, among others), most estimates indicated that achievement gaps between richer and poorer students tended to increase in both math and reading. Most showed widening inequality during the first year of the pandemic, but a sizable number also pointed to the same trend even into its second and third years.

In an email, Dan Goldhaber, director of the Center for Analysis of Longitudinal Data in Education Research, called the results “depressingly consistent about the pandemic exacerbating learning loss along preexisting lines of inequality.”

2. Poorer countries were hit harder too

No low-income countries were included in the analysis, as little evidence exists to identify the academic impact of COVID in the developing world. But the four middle-income countries under study (Brazil, Colombia, Mexico, and South Africa) saw larger average learning deficits than their higher-income counterparts, including the three largest overall estimates of learning loss. 

The authors write that the effects of the last three years are likely to worsen the long-term inequality in international learning outcomes and “undo past progress” in closing gaps across borders.

3. Learning loss is stuck in 2020

After first emerging in the early months of the pandemic, the literature suggests that learning deficits have roughly held steady in the time since. That implies that efforts to adjust to ever-changing disruptions in schools were successful in holding off further losses, the authors write, but also that those efforts “have been unable to reverse them” so far. The pattern, which appears throughout all 15 countries, is also consistent within the three (the United States, United Kingdom, and Netherlands) that have cumulatively generated the most research findings.

4. Losses were greater in math than reading

In keeping with one of the most consistent findings in education research, the review shows that academic reversals in math have been significantly greater than those in literacy. Given the to teach their children reading skills, math has been characterized as a more “school-dependent subject,” and previous data on COVID-era achievement — including results from America’s foremost standardized K–12 exam, the National Assessment of Educational Progress — has already shown math scores absorbing a greater blow over the past few years. Across the study’s included in the review, reading losses at the median were only about half the magnitude of math losses.

New research on post-pandemic student achievement presents a sobering picture, offering a reality check for anyone who might think recovery is proceeding apace.

The study, from at the American Institutes for Research, NWEA and Harvard University, suggests school districts should do more. “We need more kids to get more hours of interventions,” said CALDER’s Dan Goldhaber.  

Taken together, the dozen mid-to-large sized school districts investigated enroll more than 600,000 students across 10 states in every region in the country. And they serve higher proportions of students of color and students attending high-poverty schools than national averages. The districts range from Alexandria City Public Schools in Virginia to Dallas Independent School District, Guilford County Schools in North Carolina, and Tulsa Public Schools in Oklahoma.

Here are five big takeaways from the research:

1. Districts stopped the bleeding in the 2021-22 school year.

Based on achievement tests administered between fall 2021 and spring 2022, the new data show that districts for the most part put an end to student achievement declines in math and reading relative to pre-pandemic levels. 

2. Beyond that, though, results are mixed.

On average, math and reading test score gains during the 2021-22 school year didn’t move past pre-pandemic levels, with “little evidence of systematic catch-up” to where districts were before COVID hit. Students in a few elementary grades improved substantially in math, but beyond a few areas, researchers didn’t find a lot of compelling evidence of recovery in other subjects or grades.

In most cases, the positive effects were small — certainly less than what one might expect from such a large, national effort. Goldhaber told The 74, “I thought that there would be more clear, standout, home-run initiatives. And maybe naively I thought that.” While he and others envisioned the research showing strong evidence that, for instance, districts’ extended learning time or tutoring initiatives “are really moving the needle,” in reality, he said, “the results are much more mixed than that.”  

3. Two big problems: implementation and scale

Researchers found that in many districts, remediation initiatives planned for the fall of the 2021-22 school year were still in the process of starting as late as last spring — and in some cases, they hadn’t even launched. “We as a country have never tried to do anything like this at scale nationwide,” Goldhaber said. “And so that would be challenging no matter what.” But putting such a comprehensive system in place with a low national unemployment rate may be making it even harder, he said. “Districts may be competing with each other, both for talent and to bring in new programs. But they get much, much harder when you’re doing this for many, many students.”

And even when remediations were in place, researchers found, unforeseen events like teacher absences during Delta and Omicron surges — and chronic student absences — reduced the planned “frequency and dosage” of interventions. In a few schools, administrators said, teachers even told them they feared that sending students to pull-out groups “would increase everyone’s risk of infection.” One district leader said challenging student behavior made it harder to deliver remediation, while another said, “If behavior is the thing that students need to get going [in school], maybe behavior should be the intervention.” 

4. Schools share the blame with parents, who may have an inaccurate picture of their children’s post-Covid achievement.

Researchers interviewed district leaders, who said they faced several challenges, including staffing and capacity, as well as scheduling headaches. But a major problem was their ability to consistently engage targeted students and their parents, which will be critical going forward. Goldhaber said “hold-harmless” post-Covid grading policies may give parents a too-rosy picture of their children’s achievement. 

The researchers recommended that districts and states do more to inform families “about how students are doing now, whether they are on track for recovery, and what can be done if recovery does not look like it is happening at an adequate pace.” Goldhaber said grades “are probably the best direct signal that parents get about how their kids are doing. And so if the grades don’t end up lining up well with some objective measures, and parents are taking the grade as the signal, then they might be right to think things are O.K.”

5. Time is running out on closing gaps. 

The researchers noted that at this pace of recovery, according to recent findings, most students in grades 3 to 8 will need a minimum of three school years to fully get back up to speed, with upper elementary and middle school students potentially needing much longer. While a few of the districts studied are on pace to make up for lost progress, researchers said, others “will not reach pre-pandemic levels without significant acceleration” of their efforts.

“It’s not that the programs aren’t working, for the most part,” Goldhaber said. “I think it is that the scale and intensity of the treatments are less than what was planned — and less than what would be needed to see large gains.” Given that federal ESSER aid must be committed by 2024 and spent by 2025, schools are running out of time to offer effective interventions.

More broadly, Goldhaber cautioned that the study shouldn’t leave readers with the impression that “districts are just sort of fumbling the ball. I don’t think that’s the case.”

But he said districts and communities need to step up their game, with more tutoring and parent outreach, among other things.

For their part, parents need to understand that more needs to be done. “There are these national surveys that are suggesting that parents think things are more or less back on track,” Goldhaber said. “And I think there’s good evidence that kids are not yet back on track. So we’ve got what I would say is an ‘urgency gap.’” 

Disclosure: Walton Family Foundation and Carnegie Corporation of New York  provided financial support to this paper from CALDER and The 74.

Dismal releases from the National Assessment of Educational Progress (NAEP) — often referred to as the “nation’s report card” — have become a biannual tradition in recent years as academic progress first stalled, then eroded for both fourth and eighth graders. But today’s publication, tracking long-term academic trends for 9-year-olds from the 1970s to the present, includes the first federal assessment of how learning was affected by COVID-19.

The picture it offers is bleak. In a special data collection combining scores from early 2020, just before schools began to close, with additional results from the winter of 2022, the report shows average long-term math performance falling for the first time ever; in reading, scores saw the biggest drop in 30 years. And in another familiar development, the declines were much larger for students at lower performance levels, widening already-huge learning disparities between the country’s high- and low-achievers. 

The results somewhat mirror last fall’s release of scores for 13-year-olds, which also revealed unprecedented learning reversals on the long-term exam. But that data was only collected through the fall of 2019; the latest evidence shows further harm sustained by younger students in the following years. 

Peggy Carr, commissioner of the National Center for Education Statistics, said on a call with reporters that the “sobering” findings illustrated the learning losses inflicted by prolonged school closures and student dislocation. 

“It’s clear that COVID-19 shocked American education and stunned the academic growth of this age group of students,” Carr said. “We don’t make this statement lightly.”

Average math scores for 9-year-olds sank by a staggering seven points between 2020 and 2022, the only such decline since the long-term test was first administered in 1973. Average reading performance — generally by schooling than math, and therefore theoretically shielded from pandemic shock — fell by five points. 

Inevitably, that means that fewer students hit the test’s benchmark performance levels than two years ago. For math, the percentage of 9-year-olds scoring at 250 or above (defined as “numerical operations and basic problem solving”) fell from 44 percent of test takers to 37 percent this year; those scoring 200 or higher (“beginning skills and understanding”) fell from 86 percent to 80 percent; even the vast majority scoring at the most basic threshold of 150 (“simple arithmetic facts”) shrank slightly, from 98 percent to 97 percent, across the two testing periods.

No demographic subgroup saw gains on the test, but disparities existed in the rates of decline. For instance, math achievement for white 9-year-olds dropped by five points, but for their Hispanic and African American counterparts, the damage was even greater (eight points and 13 points, respectively). As a result, the math achievement gap between whites and African Americans increased by a statistically significant amount. 

In reading, scores for African Americans, Hispanics, and whites were all six points lower, leaving relative gaps unchanged. Scores for Asian students only fell by one point. 

Notably, the long-term trend assessment differs somewhat from the main NAEP test administered every two years. It follows student performance going back a half-century, and it is taken with a paper and pencil instead of digitally. For the most part, testing items are unchanged from the early 1970s, assessing more basic skills of literacy and computation than are generally seen on the main NAEP.

The broad trend-line has been positive over the life of the exam, and even in the most recent release, student scores on both subjects are far higher than when they were first measured. But Dan Goldhaber, a researcher and longtime observer of student performance, said it was striking to see that upward momentum evaporate so quickly.

“A bit of a hidden story in education, when you look at a swath of 40 or 50 years, is the progress that students have made — and the disproportionate progress that historically marginalized students have made,” said Goldhaber, the of the Center for Analysis of Longitudinal Data in Education Research (CALDER) at the American Institutes. “We’re seeing a lot of that very long-term progress completely erased over the course of a couple of years.”

‘Particularly bad’

One of the most consistent, and consistently worrying, findings of previous NAEP rounds has been the sharp disjunction of students at either end of the performance scale. For over a half-decade, high-scoring students have generally performed a point or two better with each iteration of the test — or at least stayed at the same level — while low-scoring students have seen their scores fall.

The phenomenon of growing outcome gaps is again apparent in the post-COVID results, though it takes a slightly different form. At all performance levels across both subjects, 9-year-olds experienced statistically significant declines in their scores; but even with the identical downward trajectory, struggling students lost so much ground that disparities still expanded.

In reading, 9-year-olds scoring at the 90th percentile of all test takers in 2022 lost two points compared with their predecessors in 2020. But students scoring far below the mean, 10th percentile fell by 10 points.

Consequently, the average reading gap between kids at the 90th versus the 10th percentile grew from 103 points to 110 points in just two years. In math,the divergence grew from 95 points to 105 points over the same period.

Goldhaber said that the trends visible in NAEP performance largely dovetailed with those using test scores from the MAP test, administered by the assessment group NWEA. In multiple data sources, he argued, it has become clear that the pandemic’s effects have been disproportionately negative for already struggling and disadvantaged children.

“It’s not just the drops, it’s where we’re seeing the drops in math and reading tests, and they’re disproportionately at the bottom of the test distribution,” he said. “So the pandemic is reversing a long-term trend of narrowing achievement gaps. That’s particularly bad, to my mind.”

The fact that losses are so heavily concentrated among the lowest-scoring segment of students may help explain what Goldhaber termed an “urgency gap”; neither states, school districts, or even families seemed driven to embrace the generational learning interventions — from dramatically lengthening the school year to implementing widespread one-to-one tutoring — that the scale of learning loss demands. As just one indicator, billions of dollars of federal COVID aid to schools remains unspent more than a year after it was first allocated.

That may change in the wake of the NAEP release. While previous studies have pointed to similar, and similarly inequitable, learning loss over the last few years by using data from the MAP and state standardized tests, the Nation’s Report Card is seen as the authoritative performance metric for American K-12 schools. As NCES Commissioner Carr noted, today’s release provides the first nationally representative results measuring achievement before and after the pandemic. Ninety-two percent of schools where the test was administered in 2020 were re-assessed earlier this year.

Tom Kane, an economist at the Harvard Graduate School of Education, agreed that NAEP scores definitively affirmed what prior studies have already demonstrated. More observers needed to study the magnitude of the loss, he added, because the proposed academic remedies in most of the country are “nowhere near enough” to combat it.

Kane analogized classroom learning to an industrial process — the conveyor belt slowed in 2020 and 2021, but has resumed functioning since at roughly the same rate as before the pandemic. But to make up for lost time, he argued, it would need to be sped even further.

“What we learned…is that the conveyor belt is back on, but at about the same old speed,” Kane said. “Somehow, we’ve got to figure out how to help students learn even more per year in the next few years, or these losses will become permanent. And that will be a tragedy.”

She granted a blanket, one-year “accountability waiver.” But in February, with a new administration in place, then-Education Secretary nominee Miguel Cardona said he’d require states to administer the federally mandated tests in the spring, with an asterisk: They had the option of giving shorter, remote, or delayed versions.

Now, as students begin a third year of school under the cloud of COVID-19, a pair of researchers suggest that those two moves, by two administrations, may have made the results of annual testing less valuable — and could harm the delicate political support such testing still enjoys.

Get stories like this delivered straight to your inbox. Sign up for The 74 Newsletter

Using different versions of tests makes the results less comparable across different years and school districts. And shorter tests produce less “actionable” information about individual student achievement in the short term, said Dan Goldhaber of the National Center for Analysis of Longitudinal Data in Education Research at the University of Washington.

“The waivers looked to us like they made state tests less useful for diagnostic purposes, both for parents and for teachers,” Goldhaber said in an interview.

In a new , Goldhaber, along with Paul Bruno of the University of Illinois at Urbana-Champaign, say that as states begin planning for next spring’s tests, they should consider exactly how useful the results are for families, who spent much of the past school year getting an up-close look at just how much their children know.

When these tests return full-force in schools post-pandemic, as they likely will in 2022, they run the risk of being out-of-step with parents’ new, pandemic-fueled understanding of their children’s needs, the authors warn.

If the test results can’t help guide decisions about student placement and skills levels, they could lose what tenuous political support they still have, according to the analysis.

The researchers looked at testing policies nationwide and found that in most states, educators use tests either for diagnostics, for research and evaluation, or as the basis for accountability systems.

But they might also be better used to provide “actionable and timely information” about how to help individual students do better in the subjects tested. If results could be disaggregated more often and in a timely fashion, they say, that would help parents and teachers look more closely at students’ skill levels and academic needs.

As it is, they say, state test results “often take several months to make it into the hands of educators or families, impeding the use of testing to help individual students.”

“We need to make sure, I think, that they are useful for more than just accountability purposes,” said Goldhaber, who is also affiliated with the American Institutes for Research.

He noted, for instance, that in Washington State, many high-achieving students in underrepresented minority groups, who wouldn’t typically be assigned to advanced classes, get that option based on end-of-year assessment results. “They’re used for those kinds of things, but I don’t think it’s well-known,” he said.

Twenty years after No Child Left Behind first mandated widespread spring testing in K-12 schools, the authors say refocusing the tests could also keep them from losing popular support among parents and teachers.

Federal testing requirements are “popular in the abstract,” Bruno and Goldhaber write, but that support appears fragile: 41 percent of respondents in a 2020 Phi Delta Kappa poll said there’s “too much emphasis on achievement testing” in public schools, up from 37 percent in 2008 and just 20 percent in 1997.

They also note that support for testing drops 20 percentage points when respondents are told that test administration takes, on average, eight hours of class time annually.

“I think there’s probably less public support than there was, certainly, when No Child Left Behind passed” in 2001, Goldhaber said.

Because of remote schooling, Goldhaber said, “Many parents have a window into what’s actually going on inside the classroom, in a way that they did not have before the pandemic, because they could sit in with their kids during classes.”

But in many cases, he said, the test results don’t necessarily offer “concrete information that suggests maybe your kids need help with complex fractions — the kind of information that you could at least imagine would inform parent-teacher meeting discussions.”

Jonathan Schweig, a researcher at the RAND Corp. and a professor at Pardee RAND Graduate School who studies education policy and teacher evaluations, among other topics, said he generally agreed with Bruno’s and Goldhaber’s notion that using tests for diagnostic purposes might be a way to increase public support.

Echoing Goldhaber’s point about concrete data, he said state summative tests generally “were not designed to provide diagnostic or instructionally useful information. Even under routine conditions, the tests are administered towards the end of the school year, and score reports are returned to schools and families during the summer, after the school year has ended.”

Schweig also said the scores generated by these assessment systems “are not at a grain size that would be useful to support remediation or other diagnostic uses.”

Summative tests, he said, “are best thought of as providing one piece of information about student learning, but they do not provide the only piece and perhaps not even the most important piece. As such, it is important for school leaders to think comprehensively about assessment and design coherent systems that include a mix of formative, interim and state-wide summative assessments.”

Any broad new federal testing policies will have to wait until Congress approves a reauthorization of the Elementary and Secondary Education Act, which helps fund public schools. That could take years, since lawmakers typically push back the timeline for reauthorization by years. But in today’s political climate, Goldhaber said, “I think that if we were to have a negotiation right now, I don’t know that the tests would survive.”

Across the country each year, thousands of teaching candidates get ready to begin their classroom careers. They finish up their graduate coursework, start scanning excitedly for job openings — and then fail their states’ teacher licensure exams. Dejected and daunted by the prospect of retaking the test, many never become teachers.

It’s a distressing pattern that has been documented for years and increasingly draws the focus of policymakers attempting to diversify the profession. As more experts point to the improved academic performance of students who are assigned to even one instructor of the same racial or ethnic background, some advocates have called for states to modify, or , their licensure tests, which are more likely to be a stumbling block to African American and Hispanic candidates.

by the National Council on Teacher Quality, a reform-oriented think tank based in Washington, D.C., puts the issue in perspective. Data gathered from 38 states and the District of Columbia show that a huge portion of prospective elementary school teachers don’t pass their licensing exam on the first try. Of those that fail, non-white teachers are much less likely to re-take the test than their white classmates. And within states, students at different institutions faced radically different odds of ultimately securing a teaching credential. The conclusions will lead many to wonder whether novice educators are receiving enough training before being hired, and what can be done to assist those who weren’t adequately prepared.

NCTQ president Kate Walsh, a longtime and often critical observer of teacher prep programs, said in an interview that while the results were themselves “terrible,” a more pressing concern was the sheer difficulty of obtaining the evidence. State authorities should be active in publicizing that information, she argued, but many don’t even bother to collect it, and even federal efforts to investigate these questions have been “mired in confusion.”

“What’s interesting to me is that states didn’t have this data,” Walsh said. “We thought they did, but almost all states said that this was the first time they’d ever seen this data.”

The national findings, encompassing program-level exam results between 2015 and 2018, demonstrate clearly that large numbers of graduate students struggle to reach the finish line and become licensed teachers. Across all states that provided data, the average “best-attempt” rate for prep programs — the rate at which test-takers ever pass the test, whether or not they fail on their first try — is 83 percent, meaning that roughly one in six don’t realize their ambitions. And certain programs do much better than others: The average gap between the highest-performing and lowest-performing institutions in each state was 44 percentage points.

What’s more, 29 percent of all prep programs reported that less than half of their teaching students passed the licensing exam on their first try. Six states (Connecticut, Florida, Louisiana, New Jersey, South Carolina, and Virginia) were home to at least one program in which no teaching candidate did so.

Florida, which the fourth-most teachers of any state, offers a revealing example. In the three years studied, two different teacher prep institutions saw no teaching candidates pass the Florida Teacher Certification Examinations their first time, though both are tiny programs serving roughly a dozen students between them. Many others — including Florida International University, Florida Gulf Coast University, University of North Florida, and University of Central Florida — reported first-time pass rates of 41 percent or less. All of those schools, which collectively produced over 2,300 test-takers, were rated by NCTQ as among the most selective in the state.

At the same time, a huge number of Florida’s prospective teachers who failed their licensure tests the first time go on to pass in later attempts, likely with the support and encouragement of their prep programs. And a substantial number of programs enrolling large numbers of low-income students (measured via their eligibility to receive Pell grants) report higher first-time pass rates than the average across the state.

Dan Goldhaber, a professor at the University of Washington who directs the Center for Analysis of Longitudinal Data in Education Research (CALDER), called the data “compelling,” adding that researchers might fruitfully study which institutions are able to work best with teaching candidates who initially stumble with the exam.

The findings “begin to open up the teacher preparation black box a little bit and point to where we should be looking for different supports, curricula, interventions that seem to be able to help teacher candidates while they are in teacher preparation,” Goldhaber said.

Dodging the ‘heat’ of publication

In order to reach their findings, NCTQ first had to receive the data from states. A 2019 study, using national results for the commonly used Praxis exam that were provided by the testing vendor ETS, offered somewhat similar findings, but did not delve into differences by state or institution.

Acquiring results at that more granular level was much harder, Walsh said, because some local authorities “didn’t want the heat of being the ones to publish this data.” In all, seven states did not provide timely data on licensure exam performance, and eight provided only partial data. Of the states that did share their data with NCTQ, some had to be subjected to public records requests.

Meagan Comb, director of the Wheelock Educational Policy Center at Boston University and a one-time NCTQ fellow, said she wasn’t particularly surprised at the challenges posed by a third-party examination of test results. In a two-year stint as the director of educator effectiveness at Massachusetts’s Department of Elementary and Secondary Education, Comb oversaw the state’s policy on both teacher preparation and licensure. She recalled that many in the state — considered a national leader in the collection and dissemination of education data — had ached for more information because “it was really hard to know how our pass rates compared with other states.” This often included program heads themselves, who didn’t always have a clear picture of which student groups were struggling or what aspects of the test gave them trouble.

“This report shows that you have to invest in data infrastructure,” Comb Said. There are a lot of states that can’t even link their teacher workforce with their teacher prep candidates, or there’s a state statute prohibiting them from looking at the efficacy of their teacher candidates. I think there’s a lot of opportunity in states across the country to think a lot about the data infrastructure they provide to teacher preparation programs for continuous improvement.”

Goldhaber noted that bottlenecks on data can arise in different areas. While some state education departments are “more inquisitive than others,” he acknowledged, schools of education would often prefer that low pass rates float under the radar.

“I think that, sometimes, the politics are really hard,” he said. “You have important political constituencies — deans, and everything they bring — who sometimes don’t want these data to be out there.”

Lack of transparency can be damaging not only to state authorities, and prep programs themselves, but also to prospective graduate students, some of whom will enroll unwittingly in a teacher prep program where they stand a significant chance of not gaining a license. The risk of failure is especially great for teaching candidates of color, who are significantly less likely than white candidates to retake the exam if they fail. The stress and expenses associated with the test, often amounting to hundreds of dollars in fees or preparation materials, can act as a roadblock to attracting more diverse teachers.

Walsh said that on grounds of consumer protection alone, that had to change.

“The whole point is, you’re supposed to be preparing candidates for licensure. And nobody points out to the kids going into these programs, ‘Your chances of getting a license at this institution are nil.’ So they take their money, take their time, and that’s it.”

Disclosure: The Carnegie Corporation of New York and the Charles and Lynn Schusterman Family Foundation provide financial support for both the National Council on Teacher Quality and The 74.