Passing the state reading test might be even easier than recent criticism has suggested, a former Department of Education testing analyst is arguing.
Independent statistician Frederick Smith examined the way free-response questions were graded and found that virtually every student received enough points on that section to then pass the test by guessing randomly on the multiple-choice questions.
Smith’s finding expands on an informal study by a city teacher who concluded that arbitrarily filling in a pattern of multiple-choice answers and leaving the open-response section blank could yield scores high enough to promote a student to the next grade in New York City. That amounts to eight correct multiple choice answers on the fifth-grade English Language Arts exam.
Smith found that students who answered only 6 multiple choice questions correctly almost certainly would also pass the Level 2 bar. That’s because an overwhelming majority–99 percent–of fifth-graders who took the exam in 2009 received at least two points on the open-ended essay and free response sections, which would boost them to the eight-point cut-off level.
In addition, more than 97 percent of students received three or more points on that section, meaning that they would have had to answer just five multiple-choice questions correctly to receive a Level 2 score.
“There’s no point in setting a cut-off score that you can guess your way through,” Smith said. By effectively reducing the number of correct multiple choice answers needed, he said, the tests significantly increased the odds that a student could guess randomly and still hop the Level 2 bar.
Columbia University Teacher’s College professor James Corter calculated the likelihood that randomly guessing students would score at these levels. He found that more than 57 percent of students guessing would answer at least six answers correctly. Just over 75 percent of guessing students were likely to receive five or more correct answers.
The statistical calculations apply only to students who are truly guessing randomly, but could apply to a student with no knowledge of test content.
The number of students receiving failing Level 1 scores has declined dramatically over the past four years. On the ELA exams Smith analyzed, the number of students scoring at Level 1 declined by 95 percent between 2006 and 2009.
Corter cautioned that even though almost all students are receiving at least partial credit on the free-response questions, it doesn’t necessarily mean the questions are easy.
But Smith said the low scoring bar harms the tests’ credibility.
“The fact remains that this is the first year you can waltz your way to a level 2,” Smith said. The scoring effectively removes the failing Level 1 as a meaningful tier in the test evaluations, he said.
Smith has been publicly calling for an independent testing ombudsman in New York City since the early 1980s. He said the results of his examination highlight the need for a truly independent auditor of the state testing program.
“It has to have real authority, real teeth, and be able to get data without having to beg for it from the Department of Education,” he said. “This is the era of accountability. It’s time for some real answers here.”