A widely publicized statistic showing that charter schools do a poor job of retaining their special education students was based on flawed data, the city’s Independent Budget Office has admitted.
The agency reported in January that 80 percent of special education students identified at the start of kindergarten in 2008 left their charter school within three years. But the real figure is likely different, since the IBO only accounted for students receiving full-time special education services—omitting students with less-severe needs.
“If we made a mistake, it was the following: we should have named our finding full-time special education students,” said Raymond Domanico, the IBO’s director of education research.
It’s not clear how much the error affected the results, given that relatively few students are already identified as having special needs at the very start of kindergarten.
Still, it complicates long-simmering debates about how well charter schools serve special education students in which the 80 percent figure had already become a flashpoint. That number was emphasized repeatedly in the IBO report, and prompted a number of headlines (and Chalkbeat coverage).
It also raises questions about how committed the agency is to accuracy, given its role as a nonpartisan education data watchdog. The state legislature specifically charged the IBO with sifting through Department of Education data when it voted to renew mayoral control of the city’s schools in 2009.
The IBO’s numbers leave out a large chunk of students who fall under the city’s special education umbrella, including students who get pulled out of general education classes for some period of the day and students who receive services like speech and occupational therapy. State data shows that 31 percent of the city’s special education students in 2008-9 received services for less than 40 percent of the school day.
Domanico blamed the error on confusing data the agency received from the Department of Education, since students labeled “special education” did not include all students receiving special education services.
And though he acknowledged that the report wasn’t as specific as it could be, Domanico insisted that a correction was not warranted because that finding still reflects special education students, it was one part of a report much larger in scope, and because the city’s data has been inconsistent in the past.
“There was nothing that’s incorrect,” he said. “Our findings still hold; it’s a matter of specifying who we’re talking about.”
Special education advocates see it differently. Ellen McHugh, a member of the Citywide Council on Special Education, said the IBO’s conflation was “misleading,” since many special education students don’t receive services full-time.
In a highly polarized environment, the agency’s independent status also gives its findings additional credibility, McHugh said.
“It’s disappointing for advocates who thought they had this as examples of discrimination [by charter schools]. It’s also reinforcement for charter school advocates who say everyone lies about charter schools,” she said.
The error was first brought to the IBO’s attention by researcher Marcus Winters, who is affiliated with the right-wing Manhattan Institute and had conducted a study using data on the same cohort of students, who were in kindergarten and at a charter school in the 2008-9 school year. (That study found that special education students left district schools and charter schools at about the same rate.)
Winters and the IBO used student data from different points in the school year, making direct comparisons between the two reports difficult. Still, Winters was struck by just how far apart their numbers of special education students were.
Returning to his data, Winters says he accounted for 198 such students in spring 2009; the IBO had counted just 25 that fall.
“Whatever it is, that number they’re using is way too small,” Winters said.
Winters asked the Department of Education to explain the discrepancies. The response from the department was that, in the confusing labyrinth of the city’s data sets, the total number of special education students is not captured by only those students labeled “special education,” but instead by the labels of “IEP” and “disability.”
Winters passed that along to the IBO, and the New York City Charter School Center noted his explanation last month. The IBO now says he’s right, but only acknowledged that publicly after inquiries from a reporter.
“It rang true to us when we heard it,” Domanico said about Winters’ information. “The data that we’re reporting is accurate, but perhaps it could have used the ['full-time'] qualifier.”