testing testing

Oversight of Regents scoring has serious flaws, state audit finds

The New York State Education Department is failing to ensure that Regents tests are properly scored, according to an audit published today by the state comptroller’s office.

The exams are given to high school students, who have to pass five in different subject areas in order to receive a Regents diploma. Teachers normally administer and score the tests under the supervision of each school’s principal, and the school district is responsible for reporting scores to the state.

The audit focused on the review process the state uses to ensure the scoring is accurate and consistent. In these reviews, a group of teachers and NYSED officials re-score a random selection of exams and compare them to how the tests were originally scored to judge the accuracy. The review team then makes recommendations to the state and to schools about how to improve the scoring process.

In the most recent review, completed in 2005, the scores awarded by schools were routinely higher than the scores given by the reviewers, and reviewers reported that school scorers frequently assigned full credit to student answers that were “vague, incomplete, inaccurate or insufficiently detailed.”

But auditors found little to suggest that the state followed up to improve the process, the report says.

“For example, we found no evidence actions were taken to implement the Review team’s recommendations to improve scoring training and enhance quality control during the scoring process. We also found no evidence actions were taken to bring about improvements at particular schools,” the auditors write.

Moreover, auditors found that when school districts don’t submit their scored exams for review, NYSED regularly failed to follow up with the districts to obtain the tests and review them.

“We recommend follow up take place because there is considerable risk that the failure to submit scored exams may be a willful attempt to avoid scrutiny of scoring accuracy,” the report states.

The audit also reported flaws in the state’s investigations of complaints about Regents exam scoring practices. Of 13 complaints logged in recent years, auditors found documentation that an investigation had occurred in only one case. They did find evidence of investigations into complaints not found in the state’s log, raising questions about the education department’s record-keeping.

Auditors reviewed the state’s oversight process between 2006 and 2009 both in the department and in selected school districts, though it did not specify which districts. (I’ve asked the state comptroller’s office for this information and have not heard back; I’ll update when I find out.)

The audit arrives at a time when the reliability of state exam results has come under increasingly intense scrutiny. Most of the attention has been directed toward the third through eighth grade tests, which critics say have become increasingly easy over time and subject to score inflation. But the Regents exams have not escaped criticism.

The state education department has “played with the scoring in a way that has played with the public trust,” said education historian Diane Ravitch. “I think a shadow has been cast over the Regents because they’re not transparent.”

The audit issued 12 recommendations for how the state should improve its oversight of the exam scoring process, including requiring schools to implement a plan to correct their practices if scoring is found to be unreliable and creating a formal way to identify schools that are at high risk for scoring problems.

State education commissioner David Steiner said in a statement today that the department is reviewing its entire accountability system. “We are implementing the recommendations in the [Office of the State Comptroller] audit and have moved to put additional improvements in place for the oversight of the scoring process for Regents examinations,” Steiner said, though he did not specify what those additional measures might be.

In a September letter to auditors, interim education commissioner Carole Huxley agreed with 11 of the proposals and said that the education department would implement them.

The one area that state officials contested was the recommendation that the department require schools with known scoring irregularities to report changes made to exam scores that the reviewers found to have errors. Huxley argued that because students have already graduated by the time scores are reviewed, it’s not possible to go back and revise the scores.

Ravitch said that while it’s impossible to recall the diplomas of individual students, it is important to know which scores may have been compromised for policy and research purposes.

“Basically the state needs to say any research study based on these scores must acknowledge that the scores are compromised,” she said. “You cannot reach any policy decision based on these scores.”

The reliability of state test scores may also come under even more scrutiny because of the federal government’s emphasis on building data tracking systems and using them to evaluate schools and teacher performance over time. The federal Department of Education is tying state initiatives in these areas to Race to the Top grant money.

Dee Alpert, publisher of the advocacy site Special Education Muckraker, said that problems with state test data should hurt the state’s chances at receiving the federal grant money. “Why on earth is anyone going to give New York State money to set up big sophisticated databases when the numbers in it are complete fiction?” she said.

Steiner and Board of Regents Chancellor Merryl Tisch have said that one of their main priorities is to improve the validity and reliability of state exams.

Here’s the audit report from the state comptroller’s office. The education department’s response to the audit begins on page 21:

that was weird

The D.C. school system had a pitch-perfect response after John Oliver made #DCPublicSchools trend on Twitter

Public education got some unexpected attention Sunday night when John Oliver asked viewers watching the Emmys to make #DCPublicSchools trend on Twitter.

Oliver had been inspired by comedian Dave Chappelle, who shouted out the school system he attended before he announced an award winner. Within a minute of Oliver’s request, the hashtag was officially trending.

Most of the tweets had nothing to do with schools in Washington, D.C.

Here are a few that did, starting with this pitch-perfect one from the official D.C. Public Schools account:

Oliver’s surreal challenge was far from the first time that the late-show host has made education a centerpiece of his comedy — over time, he has pilloried standardized testing, school segregation, and charter schools.

Nor was it the first education hashtag to take center stage at an awards show: #PublicSchoolProud, which emerged as a response to new U.S. Education Secretary Betsy DeVos, got a shoutout during the Oscars in February.

And it also is not the first time this year that D.C. schools have gotten a surprise burst of attention. The Oscars were just a week after DeVos drew fire for criticizing the teachers she met during her first school visit as secretary — to a D.C. public school.

Startup Support

Diverse charter schools in New York City to get boost from Walton money

PHOTO: John Bartelstone
Students at Brooklyn Prospect Charter School in 2012. The school is one of several New York City charters that aim to enroll diverse student bodies.

The Walton Family Foundation, the philanthropy governed by the family behind Walmart, pledged Tuesday to invest $2.2 million over the next two years in new charter schools in New York City that aim to be socioeconomically diverse.

Officials from the foundation expect the initiative to support the start of about seven mixed-income charter schools, which will be able to use the money to pay for anything from building space to teachers to technology.

The effort reflects a growing interest in New York and beyond in establishing charter schools that enroll students from a mix of backgrounds, which research suggests can benefit students and is considered one remedy to school segregation.

“We are excited to help educators and leaders on the front lines of solving one of today’s most pressing education challenges,” Marc Sternberg, the foundation’s K-12 education director and a former New York City education department official, said in a statement.

Walton has been a major charter school backer, pouring more than $407 million into hundreds of those schools over the past two decades. In New York, the foundation has helped fund more than 100 new charter schools. (Walton also supports Chalkbeat; read about our funding here.)

Some studies have found that black and Hispanic students in charter schools are more likely to attend predominantly nonwhite schools than their peers in traditional schools, partly because charter schools tend to be located in urban areas and are often established specifically to serve low-income students of color. In New York City, one report found that 90 percent of charter schools in 2010 were “intensely segregated,” meaning fewer than 10 percent of their students were white.

However, more recently, a small but rising number of charter schools has started to take steps to recruit and enroll a more diverse student body. Often, they do this by drawing in applicants from larger geographic areas than traditional schools can and by adjusting their admissions lotteries to reserve seats for particular groups, such as low-income students or residents of nearby housing projects.

Founded in 2014, the national Diverse Charter Schools Coalition now includes more than 100 schools in more than a dozen states. Nine New York City charter groups are part of the coalition, ranging from individual schools like Community Roots Charter School in Brooklyn to larger networks, including six Success Academy schools.

“There’s been a real shift in the charter school movement to think about how they address the issue of segregation,” said Halley Potter, a senior fellow at the Century Foundation, a think tank that promotes socioeconomic diversity.

The Century Foundation and researchers at Teachers College at Columbia University and Temple University will receive additional funding from Walton to study diverse charter schools, with the universities’ researchers conducting what Walton says is the first peer-reviewed study of those schools’ impact on student learning.