First Person

The Right To Know What?

Each fall, thousands of runners descend on the Big Apple to run the New York City marathon. They’ve trained hard all year, and give their all on the course. Long after the elite runners have finished, they stream across the finish line in clumps, exhausted at the end of their 26.2-mile journey. In the middle of the pack, as many as eight or 10 runners might cross the finish line in a single second, and nearly 400 in a single minute.

The difference between a time of 4:08:00 and 4:09:00, however, isn’t large enough to be important. It’s the difference between a rate of 9:28 per mile and 9:30 per mile. Given the vagaries of marathon running — the wind, the temperature, the features of the course — it would be unwise to conclude that the runner who crossed the finish line in 4:08:00 is a much better marathoner than the one who finished in 4:09:00.

But the runner with a time of 4:08:00 finished several hundred places ahead of the runner who finished in 4:09:00 — surely that counts for something! Not really, I’d say. We can quantify the difference, both in absolute terms and in relative position, but these differences are not large enough to be meaningful.

The same is true of the information in the Teacher Data Reports recently released in New York City. Small differences in the estimated effects of teachers on their students’ achievement can appear to be much larger, because most teachers are about equally successful with the assortment of students they teach in a given year, regardless of whether those students begin the year as low-achievers or high-achievers. A trivial difference can appear much larger than it actually is, because, like the marathoners, many teachers are “crossing the finish line” at about the same time.

Here’s an example drawn from the 2008-09 Teacher Data Reports. (I chose the example because it’s convenient and have no reason to believe it’s unusual.) In 2009, fifth-graders took New York State’s English Language Arts exam, which consisted of 24 multiple-choice test items and three constructed-response items, which were combined to create a raw score ranging from 0 to 31. The raw scores were then converted to scale scores, which were used to classify individual students into four levels of performance, with Level 3 representing grade-level proficiency. The average student in New York City got 25.5 raw score points out of 31, which in New York City’s scheme represented an average proficiency level of 3.29. (Sounds pretty good, right? Of course, this was before the state wised up that being proficient on the test didn’t mean a student was on track to graduate from high school ready for college.)

The logic of the city’s Teacher Data Reports is to estimate Teacher A’s contribution to his or her students’ test-scores by comparing how other students with the same measured characteristics would be expected to do on the state test, based on their prior achievement and individual and classroom characteristics, with how Teacher A’s students actually did on the test. If Teacher A’s students score at the same level as was predicted by a statistical model, Teacher A is claimed to not “add value” to her students. If Teacher B’s students perform better than expected, Teacher B is said to add value. (And poor Teacher C, whose students score lower than they are predicted to do, is subtracting value, I guess. Maybe we should call him Teacher F.) These “value-added” scores are then ranked, and a teacher is assigned a percentile value representing the percentage of other teachers teaching the same grade and subject who scored below he or she did.

An “average” teacher, according to this calculation, is one whose value-added score is 0. Of the 1,751 NYC teachers with three or more years of experience who received a value-added rating in fifth-grade English in 2008-09, 84 got a score that rounded to .00. Their percentile ratings—the number that’s getting all of the attention in the traditional and neo-tabloids—range from 53 to 58. A tiny shift of .01 in either direction yields an additional 152 teachers, and a percentile rating of 48 to 63. What seems to be a small range of value-added scores could be anywhere from the 48th to the 63th percentile, because the value-added scores in this range are clumped together.

But it’s hard to know whether a shift of .01 in either direction is large or small. How can we tell? Here’s an idea. Suppose that Ruiz had 20 students who took the fifth-grade English test in 2009, and they were at the city average of 25.5 out of 31 raw score points on the test. What if half of the students got one more question right on the test? Doesn’t seem like a big stretch, does it? Just like the variation in the conditions on marathon day, half of the students getting one more question correct on a given test on a given day doesn’t seem out of the realm of possibility.

If this were to happen, Ruiz’s value-added score would rise from 0 to .05. And the percentile range associated with a value-added score of .05 is 75 to 77. All of a sudden, an “average” teacher looks pretty good. And this isn’t due to the margin of error! It’s just because many teachers are about equally effective in promoting student achievement, according to the value-added model in use. A relatively small change in student performance shifts a teacher’s location in the value-added distribution by a surprisingly large amount.

To be sure, this example is based on one year of student test-score data, not multiple years. But that’s what New York State is proposing to rely on in its first year of the new Annual Professional Performance Review process, and it’s what other jurisdictions, such as Washington, D.C., use in their teacher-evaluation systems. And, as with the marathon, the clumping together of teachers is more of an issue in the middle of the distribution than among those in the lead or at the back of the pack. But that’s little consolation to the teachers whose percentile rankings will figure into annual evaluations that will determine whether they’re permitted to continue teaching.

Speaking at Coney Island Feb. 28, Mayor Bloomberg defiantly affirmed the public’s right to know the contents of teachers’ performance evaluations. “Parents have a right to know every bit of information that we can possibly collect about the teacher that’s in front of their kids,” he said.

That statement is utterly ridiculous. There’s no legitimate interest in information about teachers’ private lives if it has no bearing on their professional performance. But here’s something parents do have the right to know: just how fragile value-added measures based on the New York State testing system are. The New York State tests were never intended to be used to rate teachers’ contributions to student learning — and so it’s little wonder they do a pretty poor job of it.

This post also appears on Eye on Education, Aaron Pallas’s Hechinger Report blog.

First Person

I was an attorney representing school districts in contract talks. Here’s why I hope the Supreme Court doesn’t weaken teachers unions.

PHOTO: Creative Commons / supermac1961

Many so-called education reformers argue that collective bargaining — and unions — are obstacles to real change in education. It’s common to hear assertions about how “restrictive” contracts and “recalcitrant” unions put adult interests over children’s.

The underlying message: if union power were minimized and collective bargaining rights weakened or eliminated, school leaders would be able to enact sweeping changes that could disrupt public education’s status quo.

Those that subscribe to this view are eagerly awaiting the Supreme Court’s decision in the case of Janus v. American Federation of State, County, and Municipal Employees. At issue is the constitutionality of “agency” or “fair share” fees — employee payroll deductions that go to local unions, meant to cover the costs of negotiating and implementing a bargaining agreement.

In states that permit agency fees (there are about 20), a teacher may decline to be part of a union but must still pay those fees. If the Supreme Court rules that those agency fees are unconstitutional, and many teachers do not voluntarily pay, local unions will be deprived of resources needed to negotiate and enforce bargaining agreements.

Based on my experience as an attorney representing school districts in bargaining and contract issues, I have this to say to those hoping the Court will strike down these fees: be careful what you wish for.

Eliminating fair share fees (and trying to weaken unions) represents a misguided assumption about bargaining — that the process weakens school quality. To the contrary, strong relationships with unions, built through negotiations, can help create the conditions for student and school success. Indeed, in my experience, the best superintendents and school boards seized bargaining as an opportunity to advance their agenda, and engaged unions as partners whenever possible.

Why, and how, can this work? For one, the process of negotiations provides a forum for school leaders and teachers to hear one another’s concerns and goals. In my experience, this is most effective in districts that adopt “interest-based bargaining,” which encourages problem-solving as starting point for discussions as opposed to viewing bargaining as a zero-sum game.

Interest-based bargaining begins with both sides listing their major concerns and brainstorming solutions. The touchstone for a solution to be adopted in a bargaining agreement: Is the proposal in the best interests of children? This important question, if embedded in the process, forces both sides to carefully consider their shared mission.

For example, some districts I worked with paid teachers less than comparable neighboring districts did. It would have been unreasonable for unions to insist that their pay be increased enough to even that difference out, because that would mean reducing investments in other items of importance to children, like technology or infrastructure. At the same time, it would have been untenable for management to play “hard ball” and deny the problem, because to do so would likely lead to a disgruntled workforce.

Instead, both sides were forced to “own” the issue and collaboratively craft plausible solutions. That made unions more agreeable to proposals that demonstrated some commitment by the district to addressing the issue of pay, and districts open to other things that they could provide without breaking the budget (like more early release days for professional development).

To be sure, many school administrators could get frustrated with the process of bargaining or having to consult the negotiated agreement when they want to make a change. Some districts would very much like to adopt an extended school day, for example, but they know that they must first consult and negotiate such an idea with the union.

Yet, in districts where school administrators had built a reservoir of goodwill through collective bargaining, disagreement does not come at the cost of operating schools efficiently. Both sides come to recognize that while they inevitably will disagree on some things, they can also seek agreement — and often do on high-stakes matters, like teacher evaluations.

How does this relate to the Supreme Court’s pending decision? Without fees from some teachers, unions may lack the resources to ensure that contract negotiations and enforcement are robust and done well. This could create a vicious cycle: teachers who voluntarily pay fees for bargaining in a post-Janus world, assuming the court rules against the unions, will view such payments as not delivering any return on investment. In turn, they will stop contributing voluntarily, further degrading the quality of the union’s services.

Even more troubling, if fair share fees are prohibited, resentment and internal strife will arise between those who continue to pay the fees and those who refuse. This would undercut a primary benefit of bargaining — labor peace and a sense of shared purpose.

Speaking as a parent, this raises a serious concern: who wants to send their child to a school where there is an undercurrent of bitterness between teachers and administrators that will certainly carry over into the classroom?

It is easy to see the appeal of those opposing agency fees. No one wants to see more money going out of their paycheck. The union-as-bogeyman mentality is pervasive. Moreover, in my experience, some teachers (especially the newer ones) do not recognize the hidden benefits to bargaining contracts.

But, obvious or not, agency fees help promote a stable workplace that allows teachers to concentrate on their primary responsibility: their students. Removing the key ingredient threatens this balance.

Mark Paige is a former school teacher and school law attorney who represented school districts in New England. He is currently an associate professor of public policy at the University of Massachusetts – Dartmouth.

First Person

I’m a Florida teacher in the era of school shootings. This is the terrifying reality of my classroom during a lockdown drill.

Outside of Marjory Stoneman Douglas High School in Parkland, Florida. (Photo by Mark Wilson/Getty Images)

“Remember,” I tell the children, looking them in the eyes in the darkened classroom. “Remember to keep the scissors open. They’ll stab better that way.”

My students, the target demographic for many a Disney Channel sitcom, laugh nervously at me as they try to go back to their conversations. I stare at the talkative tweens huddling in a corner and sigh.

“Seriously, class,” I say in the tone that teachers use to make goosebumps rise. As they turn back to me with nervous laughter, I hold up that much-maligned classroom tool, the metal scissor that’s completely ineffective at cutting paper. “If a gunman breaks in, I’ll be in the opposite corner with the utility knife.” Said tool is in my hand, and more often used to cut cardboard for projects. All the blood it’s hitherto tasted has been accidental. “If I distract him and you can’t get out, we have to rush him.” I don’t mention that my classroom is basically an inescapable choke point. It is the barrel. We are the fish.

They lapse into silence, sitting between the wires under the corner computer tables. I return to my corner, sidestepping a pile of marbles I’ve poured out as a first line of defense, staring at the classroom door. It’s been two hours of this interminable lockdown. This can’t be a drill, but no information will be forthcoming until it’s all over.

I wonder if I really believe these actions would do anything, or am I just perpetrating upon my students and myself the 21st century version of those old “Duck and Cover” posters.

We wait.

The lockdown eventually ends. I file it away in the back of my head like the others. Scissors are handed back with apathy, as if we were just cutting out paper continents for a plate tectonics lab. The tool and marbles go back into the engineering closet. And then, this Wednesday, the unreal urge to arm myself in my classroom comes back. A live feed on the television shows students streaming out of Marjory Stoneman Douglas, a high school just a short drive away. I wonder whether the teachers in its classrooms have passed out scissors.

*

The weapons. It’s not a subject we teachers enjoy bringing up. You’d have an easier time starting a discussion on religion or politics in the teacher’s lounge then asking how we all prepare for the darkness of the lockdown. Do you try to make everyone cower, maybe rely on prayer? Perhaps you always try to convince yourself it’s a drill. Maybe you just assume that, if a gun comes through the door, your ticket is well and truly up. Whatever token preparation you make, if at all, once belonged only to the secret corners of your own soul.

In the aftermath of Parkland, teachers across the nation are starting to speak. The experience of being isolated, uninformed, and responsible for the lives of dozens of children is now universal to our profession, whether because of actual emergencies or planned drills. You don’t usually learn which is which until at least an hour and sometimes not until afterwards. In both cases, the struggle to control the dread and keep wearing the mask of bravery for your students is the same.

And you need a weapon.

I’ve heard of everything from broken chair legs lying around that never seem to be thrown away to metal baseball bats provided by administration. One teacher from another district dealt with it by always keeping a screwdriver on her desk. “For construction projects,” she told students. She taught English.

There’s always talk, half-jokingly (and less than that, lately) from people who want teachers armed. I have a friend in a position that far outranks my own whose resignation letter is ready for the day teachers are allowed to carry guns in the classroom.

I mean, we’ve all known teachers who’ve had their cell phones stolen by students …

*

Years earlier, I am in the same corner. I am more naïve, the most soul-shaking of American massacres still yet to come. The corner is a mess of cardboard boxes gathered for class projects, and one of them is big enough for several students to crawl inside.

One girl is crying, her friend hugging her as she shakes. She’s a sensitive girl; a religious disagreement between her friends having once brought her to tears. “How can they be so cruel to each other?” She asked me after one had said that Catholics didn’t count as Christians.

I frown. It’s really my fault. An offhand comment on how the kids needed to quiet down because I’m not ready to die pushed her too far. Seriously rolling mortality around in her head, she wanted nothing more than to call her family. None of them are allowed to touch their cell phones, however, and the reasoning makes sense to me. The last thing we need is a mob of terrified parents pouring onto campus if someone’s looking to pad their body count.

She has to go to the bathroom, and there are no good options.

I sit with her, trying to comfort her, wondering what the occasion is. Is there a shooter? Maybe a rumor has circulated online. Possibly there’s just a fleeing criminal with a gun at large and headed into our area. Keeping watch with a room full of potential hostages, I wonder if I can risk letting her crawl through the inner building corridors until she reaches a teacher’s bathroom. We wait together.

It seemed different when I was a teen. In those brighter pre-Columbine times, the idea of a school shooting was unreal to me, just the plot of that one Richard Bachman book that never seemed to show up in used book stores. I hadn’t known back then that Bachman (really Stephen King) had it pulled from circulation after it’d been found in a real school shooter’s locker.

Back then my high school had plenty of bomb threats, but they were a joke. We’d all march out around the flagpole, sitting laughably close to the school, and enjoy the break. Inevitably, we’d all learn that the threat had been called in by a student in the grip of “senioritis,” a seemingly incurable disease that removes the victim’s desire to work. We’d sit and chat and smile and never for a second consider that any of us could be in physical danger. The only threat we faced while waiting was boredom.

*

Today, in our new era of mass shootings, the school districts do what they can, trying to plan comprehensively for a situation too insane to grasp. Law enforcement officials lecture the faculty yearly, giving well-rehearsed speeches on procedures while including a litany of horrors meant to teach by example.

At this level, we can only react to the horrors of the world. The power to alter things is given to legislators and representatives who’ve been entrusted with the responsibility to govern wisely while listening to the will of the people. It’s they who can change the facts on the ground, enact new laws, and examine existing regulations. They can work toward a world where a lockdown is no longer needed for a preteen to grapple with gut-churning fear.

We’re still waiting.

K.T. Katzmann is a teacher in Broward County, Florida. This piece first appeared on The Trace, a nonprofit news site focused on gun violence.