In honor of Secretary of Education Arne Duncan offering some qualified support for the DC Opportunity Scholarship program–if only by saying that students currently receiving vouchers should not be pulled out of their current schools–here’s one last skoolboy post on the only federally-funded voucher program enabling students to attend religious and other private schools:

On Monday, following the publication of the Washington Post‘s editorial in favor of the DC Opportunity Scholarship Program, the Post hosted a bizarre but fascinating on-line debate between readers and Jo-Ann Armao, an editorial writer for the Post who probably wrote the editorial.  The Washington Post has some smart readers!  Not quite in the same league as GothamSchools readers, of course, but pretty sharp nevertheless.

At one point in the debate, Ms. Armao wrote:

“Let’s be clear on what researchers have determined about the program. The latest study was released in June of 2008 and it is true that no statisitcally significant difference in test scores was found between students who were offered scholarships and those who were not. At the same time, though, researchers noted a promising trend: 88 per cent of participating students were reading two to four months ahead of children who did not receive a scholarship.”

An astute reader suggested that Ms. Armao was quoting herself, as a Post editorial from June 16, 2008 stated, “But researchers reported an encouraging trend. Specifically, 88 percent of participating students are reading two to four months ahead of children who did not receive a scholarship.”

That language didn’t sound right to skoolboy’s ear, but I was able to track down its source.  In her letter transmitting the federally-mandated evaluation report to George Miller, Chairman of the House Committee on Education and Labor, Secretary of Education Margaret Spellings wrote:

“The program had a positive impact on reading achievement for three large subgroups of students, comprising 88 percent of participating students…The advantage that accrued to these students equates to an additional two to four months of learning.”

Similar language appears in the Department’s 6/16/08 press release titled “Report Reaffirms Academic Gains for DC Opportunity Scholarship Participants” (keep in mind that the report found no statistically significant difference in the academic achievement of voucher recipients and the control group.)

Two to four months of learning in a single year!  That sounds like a lot!  Little wonder that Margaret Spellings was so willing to set aside the fact that the achievement differences between voucher recipients and non-recipients might be due to chance.

There’s just one problem:  The evaluation report never claimed that voucher recipients learned two to four months more than non-recipients.

What the report does state is that the 3.17-point regression-adjusted difference in reading performance between students offered a voucher and the control group which was not offered a voucher represents an effect size of .09-that is, a difference between the two groups equal to 9% of the standard deviation of reading performance in the control group.  How this difference gets translated into “two to four months of learning” is something I’ll explore next week, in a discussion of why it’s best to take claims about academic gains expressed as months or years of growth with a grain of salt.

In the meantime, Madame Secretary’s claims have taken on a life of their own, and have been stretched even further out of shape by journalists and advocates alike.  skoolboy’s favorite rendition is from a 6/16/08 Reuters article:  “More than 88 percent of students who receive D.C. Opportunity Scholarships posted statistically significant increases in reading achievement, according to a federal study released today. To achieve the same results, other students would have needed about two to four months of additional instruction in reading.”

Oy.  It pains skoolboy to see research represented so poorly to the public.  But this distortion originated with Secretary of Education Margaret Spellings.

Margaret Spellings.  The gift that keeps on giving.