Field test

How New York City principals are thinking about the opt-out movement

PHOTO: Stephanie Snyder

For the past year, leaders of the opt-out movement have campaigned to chip away at the role state tests play in New York’s education system. And, to a large degree, it’s a fight they’re winning.

But for some city principals, the boycott poses a set of less ideological questions: Could it affect their ability to get a full view of students’ abilities? Could they be sanctioned if they are suspected of pressing their thumb to the scale in conversations with parents? Could students without test scores struggle to get into competitive high schools?

In interviews with a handful of school leaders, principals offered a range of answers. Principals at schools with high opt-out rates seem largely unconcerned about the consequences, with a couple of notable exceptions. And for many others, the debate is simply a non-issue in their school community.

Mark Federman, echoing several other principals, says he wasn’t worried about high opt-out rates resulting in the loss of valuable data on students’ math and reading skills over time.

“We just view [the state tests] as a small bit of information,” said Federman, principal at Manhattan’s East Side Community School, where roughly 20 percent of students opted out last year. “If you go to the doctor and say, ‘I’m having headaches,’ they’re going to take my blood pressure, they might give me an MRI. Every little bit of information you can get about a kid helps.”

The tests “work really well for people outside of schools,” he said, referring to politicians and policymakers who often think the tests give them useful information. “But for educators, they don’t really give us what we need.”

Arguments that there could be severe consequences for schools that look the other way, or actually encourage the boycotts, also seem thin to principals at schools with high opt-out rates last year. A few of those principals said they didn’t get pushback from the education department.

Part of the reason is likely that the policy winds are shifting: The state math and reading assessments have been detached from teacher evaluations, and despite threats from the federal government to withhold funding from districts with high opt-out rates, many educators see those as empty. The state’s top education official has even signaled her support for the boycotts.

Still, some educators worry that even if there are no external consequences, opting out could hurt students in the high school admissions process.

Stacy Goldstein, who runs the School of the Future in Manhattan, says she is sympathetic to parents who want to opt out, but has encouraged some parents to reconsider, as it can be difficult to understand exactly how selective schools weigh test scores in the admissions process.

“I tell them to feel it out [and] do their due diligence,” she said. “I tell them to be kind of cynical about it.”

“If you’re the parent of a seventh grader and you’re worried about what high school you’re going to, that’s a legitimate issue,” echoed Brooklyn New School’s Anna Allanbrook, a vocal test critic who said the drawbacks of opting out are outweighed by the benefits. Her school had a 95 percent opt-out rate last year.

Goldstein isn’t predicting anything that high at her school. Even though virtually no students opted out at her school last year, she anticipates a jump to a roughly 10 percent opt-out rate.

For principals like Goldstein, the prospect of a big jump in opt-outs also creates new logistical hurdles. How should teachers handle time allocated for test prep so that students who boycott the tests don’t simply check out without consequence?

“I get worried that it splits the class,” Goldstein said, “and then the teacher has to negotiate the kids who are and kids who aren’t [opting out].” She plans to assign reading about the testing debate for the students who opt out and require they write an essay during the exam time.

But others aren’t addressing the issue at all, either to avoid wading into potential controversy or because their families simply aren’t talking about it. Even though a record-breaking 20 percent of students statewide boycotted the exams last year, fewer than 2 percent of students citywide sat out.

“It’s not something we are actually talking about,” said Jazmin Rivera Polanco, principal of Entrada Academy, a Bronx school that is part of Mayor Bill de Blasio’s “Renewal” program. Last year, her school reported virtually no opt-outs.

Polanco said she doesn’t believe that many of her students, a third of whom are English language learners, should have to take the tests at all because of their limited exposure to the language. Polanco is reluctant to discuss that with parents, even though many of them incorrectly believe their children could be held back if they refuse the tests.

“We don’t want to say something that would be seen as jumping the gun, or saying information that those in the DOE might not find that it’s our place to [say],” Polanco added.

Multiple principals interviewed for this story expressed uneasiness talking about how their schools were handling students who boycott. And the education department has cautioned teachers against expressing their opinions about the tests.

“Schools have spent so many years answering to those test scores,” said Julia Zuckerman, principal of the Castle Bridge School, where not a single student sat for the state tests last year. “To think that there’s another way of doing it is a tricky position to take.”

Goldstein at School of the Future put it much more bluntly.

Since she became a principal, “I feel much more strongly that these tests are problematic,” she said. “But I want to make sure that [the opt-out movement] is not feeding into a sense of entitlement or laziness.”

“I still want rigor,” she said. “I still want assessment.”

failing grade

Why one Harvard professor calls American schools’ focus on testing a ‘charade’

PHOTO: Alan Petersime

Harvard professor Daniel Koretz is on a mission: to convince policymakers that standardized tests have been widely misused.

In his new book, “The Testing Charade,” Koretz argues that federal education policy over the last couple of decades — starting with No Child Left Behind, and continuing with the Obama administration’s push to evaluate teachers in part by test scores — has been a barely mitigated disaster.

The focus on testing in particular has hurt schools and students, Koretz argues. Meanwhile, Koretz says the tests are of little help for accurately identifying which schools are struggling because excessive test prep inflates students’ scores.

“Neither good intentions nor the value of well-used tests justifies continuing to ignore the absurdities and failures of the current system and the real harms it is causing,” Koretz writes in the book’s first chapter.

Daniel Koretz, Harvard Graduate School of Education

His skepticism will be welcome to families of students who have opted out of state tests across the country and others who have led a testing backlash in recent years. That sentiment helped shape the new federal education law, ESSA.

Koretz has another set of allies in some conservative charter and voucher advocates, including — to an extent — Secretary of Education Betsy DeVos, who criticized No Child Left Behind in a recent speech. “As states and districts scrambled to avoid the law’s sanctions and maintain their federal funding, some resorted to focusing specifically on math and reading at the expense of other subjects,” she said. “Others simply inflated scores or lowered standards.”

But national civil rights groups and some Democratic politicians have made a different case: That it’s the government’s responsibility to continue to use test scores to hold schools accountable for serving their students, especially students of color, poor students, and students with disabilities. (ESSA continues to require testing in grades three through eight and for states to identify their lowest performing schools, largely by using test scores.)

We talked to Koretz about his book and asked him to explain how he reached his conclusions and what to make of research that paints a more positive picture of tests and No Child Left Behind.

The interview has been edited for clarity and length.

Do you want to walk me through the central thesis of your book?

The reason I wrote the book is really the subtitle: we’re “pretending to make schools better.”

Most of the bad news that’s in this book is old news. We’ve been collecting evidence of various kinds about the impact of the very heavy handed, high-stakes testing that we use in this country for a long time. I lost patience with people pretending that these facts aren’t present. So I decided it would be worth writing a book that summarizes the evidence both good and bad about the effects of test-based accountability. When you do that, you end up with an awful lot on the bad side and not very much on the good side.

Can you talk about some of the bad effects?

There are a few that are particularly important. One is absolutely rampant bad test prep. It’s just everywhere. One of the consequences of that is that test scores are often very badly inflated.

There aren’t all that many studies of this because it’s not really a welcome suggestion. When you go to the superintendent and say, “Gee, I’d like to see whether your scores are inflated,” they rarely say, “Boy, we’ve been waiting for you to show up.” There aren’t that many studies, but they’re very consistent. The inflation that does show up is sometimes absolutely massive. Worse, there is growing evidence that that problem is more severe for disadvantaged kids, creating the illusion of improved equity.

Another is increasingly widespread cheating. We, of course, will never know just how widespread because there aren’t resources to examine the data from 13,000 school districts. Everyone knows about Atlanta, a few people know about El Paso, but that’s just the tip of the iceberg.

There’s obviously also — and perhaps this should be on the same par — enormous amounts of stress for teachers, for kids, and for parents. That’s the bad side.

I want to ask a little more about test score inflation. What is the strongest evidence for inflation? And let me give you two pieces that to me seem like potentially countervailing evidence. One piece is when I’m looking at research on school turnaround — like the most recent School Improvement Grant program and also turnaround efforts in New York City — these schools have been under intensive pressure to raise test scores. And yet their test scores gains on high-stakes tests have been pretty modest at best. The other example is the Smarter Balanced exam. The scores on the Smarter Balanced exam don’t seem to be going up. If anything, they’re going down.

The main issue is that score inflation doesn’t occur in the same amount everywhere. You’ve come up with two examples where there is apparently very little. There are other examples that are much worse than the aggregate data suggest.

In the case of Smarter Balanced, I would wait and see. Score inflation can only occur when people become sufficiently aware of predictable patterns in the test. You can’t game a test when you don’t know what irrelevant things are going to recur, and that just may take some time.

I’m wondering your take on why some of the strongest advocates for test-based accountability have been national civil rights groups.

One of the rationales for some of the most draconian test-based accountability programs we’ve had has been to improve equity. If you got back to the enactment of NCLB, you had [then-Massachusetts Sen.] Teddy Kennedy and [then-California Rep.] George Miller actively lobbying their colleagues in support of a Republican bill. George Miller summed that up in one sentence in a meeting I went to. He said, “It will shed some light in the corners.” He said that schools had been getting away with giving lousy services to disadvantaged kids by showing good performance among advantaged kids, and this would make it in theory impossible to do that.

Even going back before NCLB, I think that’s why there was so much support in the disability community for including disabled kids in test-based accountability in the 1990s — so they couldn’t be hidden away in the basement anymore. I think that’s absolutely laudable. It’s the thing I praise the most strongly about NCLB.

It just didn’t work. That’s really clear from the evidence.

I think the intention was laudable and I think the intention was why high-stakes testing has gotten so much support in the minority community, but it just has failed.

You mention in your book probably the most widely cited study on the achievement effects of No Child Left Behind, showing that there were big gains in fourth grade math and some gains in eighth grade math, but there wasn’t anything good or bad in reading.

Pretty much. There was a little bit of improvement in some years in reading but nothing to write home about.

So the math gains — and that was on the low-stakes federal NAEP test — they’re just not worth it in your view?

I think the gains are real. But there are some reasons not be terribly excited about these. One is that they don’t persist. They decline a little bit by eighth grade, they disappear by the time kids are out of high school. We don’t have good data about kids as they graduate from high school, but what we do have doesn’t show any improvement.

The biggest reason I’m not as excited as some people are about those gains is we’ve had evidence going back to the 1980s that one of the responses that teachers have had to test-based accountability is to take time out of untested subjects and to put it into math and reading. We don’t know how much of that gain in math is because people are teaching math better and how much is because kids aren’t learning about civics.

That’s, in my view, not enough to justify all of the stuff on the other side of the ledger.

When I’ve looked at some studies on the impact of NCLB on students’ social-emotional skills, the impact on teachers’ attitudes in the classrooms, and the impact on voluntary teacher turnover, they haven’t found any negative effects. They also haven’t found positive effects in most cases. But that would seem to at least in one sense undermine the argument that NCLB had big harmful effects on these other outcomes.

I haven’t seen those studies, but I don’t think what you describe does undermine it. What I would like to see is an analysis of long-term trends not just on teacher attrition but on teacher selection. A lot of what I have heard has really been, frankly, anecdotal. I was once a public school teacher and teaching now is utterly unlike what it was when I taught. It seems unlikely that that had no effect on who opts in and who opts out to be a teacher.

I don’t have evidence of this but I suspect that to some extent different types of people are selecting into teaching now than were teaching 30 years ago.

Can you talk about what you see as good versus bad test prep?

Something that Audrey Qualls at the University of Iowa said was, “A student has only mastered something if she can do it when confronted with unfamiliar particulars.”

Think about training pilots — you would never train pilots by putting them in a simulator and then always running exactly the same set of conditions because next time you were in the plane and the conditions were different you’d die. What you want to know is that the pilot has enough understanding and a good enough command of the physical motions and whatnot that he or she can respond to whatever happens to you while you’re up there. That’s not all that distant an analogy from testing.

Bad test prep is test prep that is designed to raise scores on the particular test rather than give kids the underlying knowledge and skills that the test is supposed to capture. It’s absolutely endemic. In fact, districts and states peddle this stuff themselves.

I take it it’s very hard to quantify this test prep phenomenon, though?

It is extremely hard, and there’s a big hole in the research in this area.

Let’s turn from a backward-looking to a forward-looking discussion. What is your take on ESSA? Do you think it’s a step in the right direction?

This may be a little bit simplistic, but I think of ESSA as giving states back a portion of the flexibility they had before No Child Left Behind. It doesn’t give them as much flexibility as they had in 2000.  

It has the potential to substantially reduce pressure, but it doesn’t seem to be changing the basic logic of the system, which is that the thing that will drive school improvement is pushing people to improve test scores. So I’m not optimistic.

One of things that I argue very strongly at the end of the book is that we need to look at a far broader range of, not just outcomes, but aspects of schooling to create an accountability system that will generate more of what we want. ESSA takes one tiny step in that direction: it says you have to have one measure beyond testing and graduation rates. But if you read the statute it almost doesn’t matter what that measure is. The one mandate is that it can’t count as much as test scores — that’s written in the statute. The notion that it means the same thing to monitor the quality of practice or to monitor attendance rates is just absurd

As I’m sure you know, research — including from some of your colleagues at Harvard — has shown that so-called “no-excuses” charter schools in places like Boston, Chicago, and New York City, have led to substantial test score gains and in some cases improvements in four-year college enrollment. Are you skeptical that those gains are the result of genuine learning?

It depends on which test you’re talking about. Some of the no-excuses charter schools drill kids on the state test, so I don’t trust the state test scores for some of those schools. I think it’s entirely plausible that some of those schools are going to affect long-term outcomes because they’re in some cases replacing a very disorderly environment with a very orderly one. In fact, I would say too orderly by quite a margin.

But those reforms are much bigger than just test-based accountability or just the control structure we call charters. It’s a whole host of different things that are going on: different disciplinary policies, different kinds of teacher selection, different kinds of behavioral requirements, all sorts of things.

A lot of the discussion around accountability, including in your book, is about the measures we should be using to identify schools. I’m interested in your take on what happens when a school is identified by whatever system — perhaps by the holistic system you described in the book — as low performing.

The first step is to figure out why is it bad. I would use scores as an opening to a better evaluation of schools. If scores on a good test are low, something is wrong, but we don’t know what. Before we intervene we ought to find out what’s wrong.

This is the Dutch model: school inspections are concentrated on schools that shows signs of having problems, because that’s where the payoff is. I would want to know what’s wrong and then you can design an alternative. In some cases, it may be the teaching staff is too weak. It may be in some cases the teaching staff needs supports they don’t have. It may be like in the case of Baltimore, they need to turn the heat on. Who knows? But I don’t think we can design sensible interventions until we know what the problems are.

Testing reboot

ACT do-overs pay off for 40 percent of Tennessee high school seniors who tried

PHOTO: Alan Petersime

Tennessee’s $2 million investment in helping high school seniors retake the ACT test appears to be paying off for a second year in a row.

Almost three-fourths of the class of 2018 took the national college entrance test last fall for a second time, doubling the participation rate in Tennessee’s ACT Senior Retake Day for public schools. State officials announced Wednesday that 40 percent of the do-overs resulted in a higher overall score.

Of the 52,000 students who participated in the initiative’s second year, 2,333 raised their average composite to a 21 or higher, making them eligible for HOPE Scholarship funds of up to $16,000 for tuition. That’s potentially $37 million in state-funded scholarships.

In addition, Tennessee students are expected to save almost $8 million in remedial course costs — and a lot of time — since more of them hit college-readiness benchmarks that allow direct enrollment into credit-bearing coursework.

But besides the benefits to students, the early results suggest that Tennessee is inching closer to raising its ACT average to the national average of 21 by 2020, one of four goals in Tennessee’s five-year strategic plan.

After years of mostly stagnant scores, the state finally cracked 20 last year when the class of 2017 scored an average of 20.1, buoyed in part by the senior retake strategy.

(The ACT testing organization will release its annual report of state-by-state scores in August, based on the most recent test taken. Tennessee will release its own report based on the highest score, which is what colleges use.)

Tennessee is one of 13 states that require its juniors to take the ACT or SAT and, in an effort to boost scores, became the first to pay for public school seniors to retake their ACTs in 2016. Only a third of that class took advantage of the opportunity, but enough students scored higher to make it worth expanding the voluntary program in its second year.

Last fall, the state worked with local districts to make it easier for seniors to participate. The retake happened during the school day in students’ own schools, instead of on a Saturday morning at an ACT testing site.

Education Commissioner Candice McQueen said the expanded access has paid off tenfold. “Now, more Tennessee students are able to access scholarship funding, gain admission to colleges and universities, and earn credit for their work from day one,” she said.

Of the state’s four urban districts, Metropolitan Nashville Public Schools, which serves Davidson County, increased its average composite score the most (up .5 to 18.4), followed by Hamilton County (up .3 to 19.4), and Shelby County Schools, (up .2 to 17.1). Knox County Schools and the state-run Achievement School District, which operates high schools in Memphis, saw slight drops from their retakes and will retain their higher average scores taken earlier.

Statewide, 10 school systems logged a half point or more of growth from their junior test day to the senior retake:

  • Anderson County, up .6 to 19.3
  • Arlington City, up .6 to 22.5
  • Collierville City, up .6 to 24.3
  • Davidson County, up .5 to 18.4
  • Franklin County, up .6 to 20.1
  • Haywood County, up .5 to 17.5
  • Henderson County, up .5 to 21.2
  • Humboldt City, up .8 to 17.4
  • Maryville City, up .5 to 22.1
  • Williamson County, up .6 to 24.1

Tennessee set aside up to $2.5 million to pay for its 2017 Retake Day, and Gov. Bill Haslam is expected to fund the initiative in the upcoming year as well. The state already pays for the first ACT testing day statewide, which it’s done since 2009.

Correction: January 17, 2018: This story has been corrected to show that, while the state set aside $2.5 million for its ACT retake initiative, it spent only $2 million on the program this fiscal year.