New York State is releasing the results of the 2010 state assessments in reading and math tomorrow. We’re told that the 2010 tests were more difficult than those in previous years, and less predictable, the first steps towards a new assessment system that provides a realistic picture of student proficiency. Testing experts such as Dan Koretz, Jennifer Jennings and Howard Everson presented evidence to the Board of Regents that being judged proficient on the state’s tests in grades three through eight or on the Regents exams did not always predict later success in high school or in college. This evidence strongly suggested that the threshold for proficiency was set too low; students who were classified as proficient in eighth-grade math had only a 30% chance of earning a Regents score of 80, which many colleges in the state judge to be the bare minimum for college readiness, had a high chance of scoring below 500 on the SAT, and were likely to be placed in remedial classes if they entered college. And, based on this chart prepared by the NYC Department of Education, of uncertain provenance, a student who is at the minimum threshold for proficiency on the eighth-grade tests has only about a 55% chance of earning a Regents diploma in high school, the state’s minimum standard for high school graduation for all students who entered 9th grade in 2008 or later.
Last week, the Board of Regents voted to adjust the cut scores that determine proficiency on the state’s readingand math assessments in grades through eight. They didn’t say by how much, but we have a clue from Merryl Tisch’s assertion that the “inflation rate” on the state tests has been about 20% in recent years. Twenty percent of what is not clear. But I’m going to assume that the cut score for Level 3, which represents proficiency in a subject at a particular grade level, is going to rise substantially at all grades for both reading and math. What are the likely consequences?
We’ll see tomorrow, but here’s my prediction, focusing on eighth-grade math. First, I’m assuming that the distribution of scale scores for 2010 will be the same as it was for 2009. If the tests were more difficult in 2010, the average scale score might go down a bit; if students were actually learning more in 2010 than in 2009, the average scale score might go up a bit. For my little prediction exercise, I’m assuming that these two things cancel each other out.
We don’t know where the NYSED will set the cut score for Level 3, but let’s assume that it’s 675. This is a challenging standard, but one that predicts a probability of 80% of scoring 80 or higher on the Math A regents exam and, according to the NYC chart, predicts a probability of obtaining a Regents diploma of 81%.
Here’s how things looked in 2009: Across the state, 80% of students were judged proficient in eighth-grade math. The percentages varied by racial/ethnic group, with the highest percentage (92%) recorded for Asian students, and the lowest (63%) for Black students. 89% of white students were met the standard for proficiency in eighth-grade math, as did 69% of Latino students.
Here’s what I predict: Across the state, 50% of students will be classified as proficient in eighth-grade math. The proficiency rate for Asian students will fall to 73%; for white students to 60%; for Latino students to 35%; and for Black students to 29%.
But perhaps a proficiency cutoff of 675 is too hard for the Regents and the State Education Department to stomach, as they will have to live with the political fallout of plummeting proficiency rates. Perhaps they will increase the cutoff from 650 to 670. This would still be a substantial increase, and proficiency in eighth grade would still mean a lot more than it does today. What would the prediction look like then?
If the cut score for Level 3 for eighth-grade math were to rise to 670, I predict the following: Across the state, 56% of students will be classified as proficient in eighth-grade math. The proficiency rate for Asian students will be 77%; for white students, 66%; for Latino students, 41%; and for Black students, 35%.
Calculating the achievement gap using group differences in average scale scores, which is what Jennifer Jennings and I have argued for in the past, would not be affected by the shift in the proficiency cutoff. But for those who calculate the achievement gap in “points” (I’m talking to you, New York City Department of Education), the increase in the cutoff is destined to increase the achievement gap—even if the score distributions for the groups stay the same.
For New York State as a whole, the 2009 achievement gap in eighth-grade math, calculated as differences in group proficiency rates, was 26 percentage points for the white-Black difference, and 20 percentage points for the white-Latino difference. (And, for completeness, the gap between Asian and Black students was 29 percentage points, and between Asian and Latino students 23 percentage points.) If the state moves the proficiency cutoff to 675, the white-Black difference will rise from 26 percentage points to 31 percentage points, and the white-Latino difference from 20 percentage points to 25 percentage points. The same increase would be observed if the state increases the proficiency threshold to 670.
Tomorrow, we’ll see whether I’m a good prognosticator … or whether I should quit my day job and become a meteorologist instead.
About our First Person series:
First Person is where Chalkbeat features personal essays by educators, students, parents, and others trying to improve public education. Read our submission guidelines here.