First Person

Randomness is Not a Fluke

“I think there’s nothing wrong with anything.”  So spoke Chancellor Joel Klein at yesterday’s release of the 2009 elementary and middle school progress reports.  As Anna Phillips reported, 84% of the schools received a letter grade of A, and an additional 13% received a B.  Only two schools out of 1,058 received an F, and just five more were awarded a D.

The letter grades were driven by the remarkable/suspicious gains in 2009 on the state’s ELA and math tests.  Schools weren’t actually compared to one another on their performance this year to derive the letter grades.  Rather, they were compared to last year’s peer and citywide benchmarks.  To use a football metaphor, because test scores rose across the board, virtually all schools moved up the field, but the goalposts didn’t move.  I wasn’t sure that the progress report letter grades could actually be less useful this year than last, but Chancellor Klein’s administration has achieved that dubious feat.  When 84% of the schools receive an A—the top grade, which everyone understands to signify excellence—what useful information about the school’s relative performance is being conveyed to parents, students, educators, and others with a stake in our schools?  Not much, in my view.

Last year, my blogging partner Jennifer Jennings (who, for those keeping score at home, is now Dr. J) and I were sharply critical of the 2008 school progress reports.  Writing on Jennifer’s eduwonkette site, we demonstrated that student achievement growth over the past year—which makes up 60% of the overall progress report letter grade—was highly unreliable.  Schools that demonstrated high gains in student achievement from 2006 to 2007 were no more likely to show gains from 2007 to 2008 than schools that showed low gains in 2006 to 2007.  We concluded that the measure of student progress making up 60% of the overall progress report grade was picking up chance fluctuations from year to year.  And if 60% of the score is random, there’s not much genuine information about school performance in the progress report grade.

It wasn’t a fluke.

I’ve now replicated our analysis using the 2009 progress report grades.  Many measures of school performance—its environment, based on parent and teacher surveys, its attendance, even the percentage of students achieving Level 3 or Level 4 on the state ELA and math tests—are fairly stable from year to year.  But once again, the student progress measure is wholly unpredictable.  The schools that showed big gains in student achievement from 2007 to 2008 were no more likely to show big gains from 2008 to 2009 than those schools that had small gains from 2007 to 2008. 

Here are some simple graphical representations, using the roughly 600 elementary schools in New York City that received progress report grades in 2008 and 2009.  The first graph plots a school’s academic expectations in 2009, based on surveys, with that school’s academic expectations in 2008.  There is a strong positive correlation of .75, which indicates a great deal of stability from 2008 to 2009 in the relative position of a school compared to other schools.

acad-expec

The second graph plots a school’s attendance rate in 2009 with that school’s attendance rate in 2008.  Attendance is highly stable from year to year:  the correlation between 2008 attendance and 2009 attendance is .95.  There’s almost as much consistency in the percentage of students scoring at levels 3 or 4 on the state ELA test over these two years.  The correlation between the 2008 percentage and the 2009 percentage is .94, which is a very strong association indicating that schools that are relatively high in 2008 are likely to be relatively high in 2009.    

attendance
levels-3-4

As you look at the three graphs above, you can see a clear pattern in the data:  elementary schools that are relatively low compared to other schools in 2008 are also relatively low compared to their peers in 2009.  Similarly, elementary schools that are relatively high compared to other schools in 2008 are likely to be relatively high in comparison to other schools in 2009.  Now, look at the graph below, which plots each school’s student progress from 2008 to 2009 by that school’s progress from 2007 to 2008.  Do you see a pattern?

ela-growth

 If you do, there’s probably a psychologist somewhere salivating at the idea of administering a Rohrschach inkblot test to you.  The correlation between student progress from 2007 to 2008 and student progress from 2008 to 2009 is -.02, which is indistinguishable from zero.  This means that there really is no pattern to the results, and certainly not a pattern that demonstrates consistency or stability from one year to the next.

It’s no surprise to see this, since it follows from what we know about the instability of test score gains from one year to the next in schools across the country.  As was true last year, there’s a bit more consistency in progress in math, but the state ELA and math tests on which these scores are based are coming under increasing scrutiny for their predictability, easiness, and poor content coverage.  The New York City Department of Education has hitched its accountability wagon to a runaway horse.

When it comes to annual test score gains, randomness is not a fluke.

First Person

I was an attorney representing school districts in contract talks. Here’s why I hope the Supreme Court doesn’t weaken teachers unions.

PHOTO: Creative Commons / supermac1961

Many so-called education reformers argue that collective bargaining — and unions — are obstacles to real change in education. It’s common to hear assertions about how “restrictive” contracts and “recalcitrant” unions put adult interests over children’s.

The underlying message: if union power were minimized and collective bargaining rights weakened or eliminated, school leaders would be able to enact sweeping changes that could disrupt public education’s status quo.

Those that subscribe to this view are eagerly awaiting the Supreme Court’s decision in the case of Janus v. American Federation of State, County, and Municipal Employees. At issue is the constitutionality of “agency” or “fair share” fees — employee payroll deductions that go to local unions, meant to cover the costs of negotiating and implementing a bargaining agreement.

In states that permit agency fees (there are about 20), a teacher may decline to be part of a union but must still pay those fees. If the Supreme Court rules that those agency fees are unconstitutional, and many teachers do not voluntarily pay, local unions will be deprived of resources needed to negotiate and enforce bargaining agreements.

Based on my experience as an attorney representing school districts in bargaining and contract issues, I have this to say to those hoping the Court will strike down these fees: be careful what you wish for.

Eliminating fair share fees (and trying to weaken unions) represents a misguided assumption about bargaining — that the process weakens school quality. To the contrary, strong relationships with unions, built through negotiations, can help create the conditions for student and school success. Indeed, in my experience, the best superintendents and school boards seized bargaining as an opportunity to advance their agenda, and engaged unions as partners whenever possible.

Why, and how, can this work? For one, the process of negotiations provides a forum for school leaders and teachers to hear one another’s concerns and goals. In my experience, this is most effective in districts that adopt “interest-based bargaining,” which encourages problem-solving as starting point for discussions as opposed to viewing bargaining as a zero-sum game.

Interest-based bargaining begins with both sides listing their major concerns and brainstorming solutions. The touchstone for a solution to be adopted in a bargaining agreement: Is the proposal in the best interests of children? This important question, if embedded in the process, forces both sides to carefully consider their shared mission.

For example, some districts I worked with paid teachers less than comparable neighboring districts did. It would have been unreasonable for unions to insist that their pay be increased enough to even that difference out, because that would mean reducing investments in other items of importance to children, like technology or infrastructure. At the same time, it would have been untenable for management to play “hard ball” and deny the problem, because to do so would likely lead to a disgruntled workforce.

Instead, both sides were forced to “own” the issue and collaboratively craft plausible solutions. That made unions more agreeable to proposals that demonstrated some commitment by the district to addressing the issue of pay, and districts open to other things that they could provide without breaking the budget (like more early release days for professional development).

To be sure, many school administrators could get frustrated with the process of bargaining or having to consult the negotiated agreement when they want to make a change. Some districts would very much like to adopt an extended school day, for example, but they know that they must first consult and negotiate such an idea with the union.

Yet, in districts where school administrators had built a reservoir of goodwill through collective bargaining, disagreement does not come at the cost of operating schools efficiently. Both sides come to recognize that while they inevitably will disagree on some things, they can also seek agreement — and often do on high-stakes matters, like teacher evaluations.

How does this relate to the Supreme Court’s pending decision? Without fees from some teachers, unions may lack the resources to ensure that contract negotiations and enforcement are robust and done well. This could create a vicious cycle: teachers who voluntarily pay fees for bargaining in a post-Janus world, assuming the court rules against the unions, will view such payments as not delivering any return on investment. In turn, they will stop contributing voluntarily, further degrading the quality of the union’s services.

Even more troubling, if fair share fees are prohibited, resentment and internal strife will arise between those who continue to pay the fees and those who refuse. This would undercut a primary benefit of bargaining — labor peace and a sense of shared purpose.

Speaking as a parent, this raises a serious concern: who wants to send their child to a school where there is an undercurrent of bitterness between teachers and administrators that will certainly carry over into the classroom?

It is easy to see the appeal of those opposing agency fees. No one wants to see more money going out of their paycheck. The union-as-bogeyman mentality is pervasive. Moreover, in my experience, some teachers (especially the newer ones) do not recognize the hidden benefits to bargaining contracts.

But, obvious or not, agency fees help promote a stable workplace that allows teachers to concentrate on their primary responsibility: their students. Removing the key ingredient threatens this balance.

Mark Paige is a former school teacher and school law attorney who represented school districts in New England. He is currently an associate professor of public policy at the University of Massachusetts – Dartmouth.

First Person

I’m a Florida teacher in the era of school shootings. This is the terrifying reality of my classroom during a lockdown drill.

Outside of Marjory Stoneman Douglas High School in Parkland, Florida. (Photo by Mark Wilson/Getty Images)

“Remember,” I tell the children, looking them in the eyes in the darkened classroom. “Remember to keep the scissors open. They’ll stab better that way.”

My students, the target demographic for many a Disney Channel sitcom, laugh nervously at me as they try to go back to their conversations. I stare at the talkative tweens huddling in a corner and sigh.

“Seriously, class,” I say in the tone that teachers use to make goosebumps rise. As they turn back to me with nervous laughter, I hold up that much-maligned classroom tool, the metal scissor that’s completely ineffective at cutting paper. “If a gunman breaks in, I’ll be in the opposite corner with the utility knife.” Said tool is in my hand, and more often used to cut cardboard for projects. All the blood it’s hitherto tasted has been accidental. “If I distract him and you can’t get out, we have to rush him.” I don’t mention that my classroom is basically an inescapable choke point. It is the barrel. We are the fish.

They lapse into silence, sitting between the wires under the corner computer tables. I return to my corner, sidestepping a pile of marbles I’ve poured out as a first line of defense, staring at the classroom door. It’s been two hours of this interminable lockdown. This can’t be a drill, but no information will be forthcoming until it’s all over.

I wonder if I really believe these actions would do anything, or am I just perpetrating upon my students and myself the 21st century version of those old “Duck and Cover” posters.

We wait.

The lockdown eventually ends. I file it away in the back of my head like the others. Scissors are handed back with apathy, as if we were just cutting out paper continents for a plate tectonics lab. The tool and marbles go back into the engineering closet. And then, this Wednesday, the unreal urge to arm myself in my classroom comes back. A live feed on the television shows students streaming out of Marjory Stoneman Douglas, a high school just a short drive away. I wonder whether the teachers in its classrooms have passed out scissors.

*

The weapons. It’s not a subject we teachers enjoy bringing up. You’d have an easier time starting a discussion on religion or politics in the teacher’s lounge then asking how we all prepare for the darkness of the lockdown. Do you try to make everyone cower, maybe rely on prayer? Perhaps you always try to convince yourself it’s a drill. Maybe you just assume that, if a gun comes through the door, your ticket is well and truly up. Whatever token preparation you make, if at all, once belonged only to the secret corners of your own soul.

In the aftermath of Parkland, teachers across the nation are starting to speak. The experience of being isolated, uninformed, and responsible for the lives of dozens of children is now universal to our profession, whether because of actual emergencies or planned drills. You don’t usually learn which is which until at least an hour and sometimes not until afterwards. In both cases, the struggle to control the dread and keep wearing the mask of bravery for your students is the same.

And you need a weapon.

I’ve heard of everything from broken chair legs lying around that never seem to be thrown away to metal baseball bats provided by administration. One teacher from another district dealt with it by always keeping a screwdriver on her desk. “For construction projects,” she told students. She taught English.

There’s always talk, half-jokingly (and less than that, lately) from people who want teachers armed. I have a friend in a position that far outranks my own whose resignation letter is ready for the day teachers are allowed to carry guns in the classroom.

I mean, we’ve all known teachers who’ve had their cell phones stolen by students …

*

Years earlier, I am in the same corner. I am more naïve, the most soul-shaking of American massacres still yet to come. The corner is a mess of cardboard boxes gathered for class projects, and one of them is big enough for several students to crawl inside.

One girl is crying, her friend hugging her as she shakes. She’s a sensitive girl; a religious disagreement between her friends having once brought her to tears. “How can they be so cruel to each other?” She asked me after one had said that Catholics didn’t count as Christians.

I frown. It’s really my fault. An offhand comment on how the kids needed to quiet down because I’m not ready to die pushed her too far. Seriously rolling mortality around in her head, she wanted nothing more than to call her family. None of them are allowed to touch their cell phones, however, and the reasoning makes sense to me. The last thing we need is a mob of terrified parents pouring onto campus if someone’s looking to pad their body count.

She has to go to the bathroom, and there are no good options.

I sit with her, trying to comfort her, wondering what the occasion is. Is there a shooter? Maybe a rumor has circulated online. Possibly there’s just a fleeing criminal with a gun at large and headed into our area. Keeping watch with a room full of potential hostages, I wonder if I can risk letting her crawl through the inner building corridors until she reaches a teacher’s bathroom. We wait together.

It seemed different when I was a teen. In those brighter pre-Columbine times, the idea of a school shooting was unreal to me, just the plot of that one Richard Bachman book that never seemed to show up in used book stores. I hadn’t known back then that Bachman (really Stephen King) had it pulled from circulation after it’d been found in a real school shooter’s locker.

Back then my high school had plenty of bomb threats, but they were a joke. We’d all march out around the flagpole, sitting laughably close to the school, and enjoy the break. Inevitably, we’d all learn that the threat had been called in by a student in the grip of “senioritis,” a seemingly incurable disease that removes the victim’s desire to work. We’d sit and chat and smile and never for a second consider that any of us could be in physical danger. The only threat we faced while waiting was boredom.

*

Today, in our new era of mass shootings, the school districts do what they can, trying to plan comprehensively for a situation too insane to grasp. Law enforcement officials lecture the faculty yearly, giving well-rehearsed speeches on procedures while including a litany of horrors meant to teach by example.

At this level, we can only react to the horrors of the world. The power to alter things is given to legislators and representatives who’ve been entrusted with the responsibility to govern wisely while listening to the will of the people. It’s they who can change the facts on the ground, enact new laws, and examine existing regulations. They can work toward a world where a lockdown is no longer needed for a preteen to grapple with gut-churning fear.

We’re still waiting.

K.T. Katzmann is a teacher in Broward County, Florida. This piece first appeared on The Trace, a nonprofit news site focused on gun violence.