Challenges in assessing the effectiveness of the Core Knowledge Reading Program

Yesterday, Michael Shaughnessy of EdNews interviewed Dr. Matthew Davis, who is leading the implementation of the Core Knowledge Reading Program pilot in New York City. Much of the interview covers basics of the program which we’ve discussed here already, including the two-strand approach to teaching reading and comprehension and the body of research supporting this method. What the interview highlighted for me are the contradictions of researching a program while trying to decide whether to continue using it, especially when real children are the subjects.

Davis says that the pilot will begin this year in kindergarten classes at 10 high-needs schools, then add grade 1 next year and grade 2 in 2010-11. But the continuation of the pilot “will be contingent on success in year one and a continuation of funding,” he says. Sounds fair: a program should prove itself before people (in this case, the Fund for Public Schools) invest further. Davis describes the plan for assessing the program:

Within the next several weeks, students in both sets of schools will be administered nationally standardized reading assessments in order to establish a baseline performance. These same tests will be administered again at the end of the kindergarten. In addition, there will be formal observation of all teachers in the pilot classrooms to ascertain any possible correlation between the level of implementation of the Core Knowledge program and the level of student achievement. In addition, specific case studies will be conducted by the NYCDOE in three pilot schools to provide additional qualitative information. As far as the test are concerned, we hope to see a significant difference in word attack, word reading, decoding skills, and spelling by the end of the kindergarten year — because the program has what we think is a very strong way of teaching the mechanics of reading. Background knowledge and vocabulary take a bit longer to build, and gains don’t start to show up on some tests until later, but, by the end of the three-year period, we hope to see the front end of what we think will eventually be a very significant difference in vocabulary, oral comprehension, and reading comprehension.

So although the survival of the program may rest on a single year’s results, the promised impact of the program — increased vocabulary and content knowledge — may take three years to show up. At least three years:

The effect of implementing the Core Knowledge curriculum is often relatively small in the first year or so, but it tends to be larger after the students have been in a CK school for several years. One study showed large effects in the later elementary grades, but not before. This is exactly what one would expect. It takes a while for background knowledge to build up to the point where it shows up on tests of reading comprehension, but there’s no getting around the connection between background knowledge and reading comprehension. There are no quick fixes.

Davis says that Chancellor Klein chose the program, in part, “in light of the continuing disappointing performance of 8th and 12th graders on reading tests.” If CKRP is designed to address comprehension gaps that don’t show up until late elementary or middle school, when children are asked, increasingly, to “read to learn,” then the program’s effectiveness ought to be studied for the next six to eight years, at least. In the meantime, decisions must be made about what programs to use, based on the best available evidence.

(Via CoreKnowledge Blog)

About our First Person series:

First Person is where Chalkbeat features personal essays by educators, students, parents, and others trying to improve public education. Read our submission guidelines here.