Each year, state test scores offer a snapshot of how much New York City students have learned. But they say little about how the city’s schools stack up against other districts’, in part because the raw scores largely reflect student demographics — wealthier districts tend to have higher scores.
Now, a major new analysis of several years of test scores from across the country provides a better way to judge and compare districts: Instead of looking at a single moment, it shows how well school systems help students grow their skills over time.
Based on that measure, New York City falls just below the middle of the pack: In the five years from third to eighth grade, its students collectively make about 4.6 grade levels of progress — landing New York in the 35th percentile of districts nationally. By contrast, Chicago students advance the equivalent of six grades within those five years, giving the district one of the highest growth rates in the country.
Still, New York is slightly above average when compared to other large districts with many students from low-income families. And it trounces the state’s other urban districts — including Yonkers, Syracuse, and Rochester, which have some of the nation’s worst growth rates.
“Among big poor districts, it’s better than average,” said Sean Reardon, the Stanford University researcher who conducted the analysis. “In the grand scheme, it’s pretty middle-of-the-road.”
Reardon’s analysis — based on 300 million standardized tests taken by students across more than 11,000 school districts from 2009 to 2015 — is the largest of its kind. It looks both at student proficiency on third-grade math and English tests (that is, what share of students earned a score deemed “proficient”) and student growth between grades three and eight (how much their scores improved over time). Reardon’s research was supported by several foundations, including the Bill and Melinda Gates Foundation, which also provides funding to Chalkbeat.
The analysis controls for the differences in tests across states and over time by converting scores into a common scale that measures growth in grade levels, making it possible to compare nearly every district in the country to one another. (It excludes New York’s scores from 2015 and some grades in 2014 because of the high number of students who boycotted the state tests those years. However, each district’s five-year growth rates is actually an average of its year-over-year growth, so Reardon was still able to calculate a five-year rate for New York.)
Experts generally prefer growth rates over proficiency as a way to evaluate school quality, since growth measures the progress students make in school rather than where they started. Even if a district enrolls many poor students who are less likely than their affluent peers to hit the “proficiency” benchmark, its schools can still help them advance at a rate comparable to or even better than schools filled with wealthier students.
“Growth is way better than achievement,” said Douglas Ready, an education and public policy professor at Teachers College, Columbia University. “We know low-income students start school behind — the question is what do school districts do with the kids they get?”
New York’s growth rate falls just below the national median of 4.8 grade levels. Among big districts, its students made gains similar to those in Dallas and Detroit, and greater than students in Los Angeles, Miami, and Indianapolis.
By contrast, Rochester ranks rock-bottom nationally. In that high-poverty district, where the median income among families with children in the public schools is $26,000, students advanced about three grade levels in five years. Yonkers’ $48,000 median income is much higher, yet its schools barely do better, with students moving just 3.5 grade levels. (Among New York City public-school parents, the median income is $42,000.)
Reardon emphasized that test scores provide an important but incomplete picture of student learning, and growth rates are an imperfect measure of school effectiveness since factors outside of the classroom also influence how much students learn over time.
Still, he argued that officials who rate schools and parents who choose them would do much better to look at a school’s growth rate over its average test scores. In fact, he said, a focus on growth rates could theoretically drive down socioeconomic segregation since higher-income parents might be willing to enroll their children in schools with many poor students and low overall test scores if the schools nonetheless had outstanding growth rates.
Ready, however, pointed out that even when schools and districts are highly effective at helping students make progress, they are still unlikely to close the yawning achievement gaps that separate most poor and wealthier students from the time they start school. Reardon came to the same conclusion.
“The large gaps in students’ academic skills between low- and higher-[socioeconomic status] districts are so large,” Reardon’s analysis says, “that even the highest growth rate in the country would be insufficient to close even half of the gap by eighth grade.”
In response to the analysis, New York City education department officials pointed to the National Assessment of Educational Progress, a standardized test taken by a representative sample of students in each state and certain districts, including New York. Only one other district among the country’s 10 largest cities performed better in reading and math than New York, which had the highest share of low-income students reach the proficient level on the reading test.
“Our schools are the strongest they’ve ever been, with record-high graduation and college enrollment rates, and improving state test scores,” said the district’s spokesman, Will Mantell.