Evaluating Evaluations

Appeal process in new evaluation plan shifts weight from student scores, for some

PHOTO: Sarah Darville

Imagine this: You’re a gym teacher at a struggling high school. Because of the way teacher evaluations were implemented in your school, 40 percent of your rating is based on test scores in a subject you don’t teach and from students who aren’t yours. And the students do poorly. Even when your principal says you’re doing well, or exceptionally well, in classroom observations, state law says that an “ineffective” for student learning means you’re stamped with an overall “ineffective” rating.

City and union officials said they had thousands of teachers like this one in mind when they agreed to a new appeal process as part of their contract agreement. Beginning this year, the process makes it easy for an “ineffective” rating to be boosted if it’s based on test scores of subjects and students that the teacher does not teach—as long a supervisor gives the teacher higher marks.

The new process isn’t protection for the sake of protection, local officials say. “It’s an appeal to address a particular problem that exists as a result of a lack of assessments,” said UFT Special Representative Amy Arundell.

But the new appeal process will also benefit thousands of math, English and other core subject teachers who receive “ineffective” ratings based on their own students’ academic growth. And shifting the weight of some teacher evaluations away from student performance metrics could make it controversial with state officials, who must approve the evaluation proposal.

core tenet of the state’s evaluation law is flunking teachers whose students show no academic growth. That’s an idea Gov. Andrew Cuomo and State Education Commissioner John King have fiercely defended over the years.

“If we’re serious about supporting excellence in teaching, we can’t have an evaluation system that permits a teacher who scores a zero on student achievement to receive a positive rating,” King said in 2011 in response to a legal challenge from the state teachers union to modify the role of state tests in evaluations.

The use of standardized test scores in teacher evaluations has garnered plenty of criticism, but the scores are preferred by officials because they are seen as a tougher standard than principal observations. Last year, for instance, just 1 percent of teachers across the state received an overall “ineffective” rating, though 6 percent of a subgroup of teachers were rated “ineffective” based solely on standardized test scores.

Research into the reliability of any one measure for teacher evaluations is mixed, with new studies casting doubt on both observations and value-added measures.

The deal has provided more ammunition for the union’s critics, who say it’s another way to weaken teacher accountability.

“Removing objective measures of student achievement undermines the intent of teacher evaluations, which we know was Mike Mulgrew’s goal in the first place,” said StudentsFirstNY Jenny Sedlis said. “Any professional needs an honest assessment of what’s working and what’s not in order to develop and improve.”

The problem with that critique, teachers say, is that the city hasn’t offered an “honest assessment” for many types of educators, including 5,000 physical education and arts teachers, librarians, and others who teach foreign language, health and career education—even though they’re evaluated like the rest of their colleagues.

“I’m being measured largely on kids I don’t even teach this year,” said Tara Brancato, a high school music teacher in the Bronx who, along with the rest of the school’s art department, will be evaluated based on how all students in the school do on their English Regents exams.

The city and union plan to improve assessment options in non-tested subjects, but Arundell said a safeguard is needed until fewer schools use learning measures that don’t relate to the work of some teachers.

“There’s an acknowledgement that there is a lack of authenticity to a person’s rating if you’re going to evaluate both on students you do not teach and subjects you do not teach,” Arundell said.

Brancato and other teachers who could be affected said the change was a welcome step toward addressing what they say has been a neglected part of teacher evaluations.

“They really are taking into consideration all teachers in the system,” said Jason Zanitsch, a high school drama teacher at the High School for Public Service in Brooklyn. “It’s far from perfect, but it’s definitely a step forward.”

Still, there are assessments available to evaluate many of the core subject teachers who will also qualify for appeals. Partially to prevent an onslaught of new tests, many schools opted to rate all teachers using a “group” measure, which gives an entire school’s staff—including math and English teachers—the same rating for the piece of the evaluation based on student performance on state tests. Having a “group” measure factor in to your evaluation qualifies a teacher for the new appeal process.

City officials said they were unable to calculate the number teachers who are being rated on “group” measures this year. But they said they anticipated that only a small percentage would use the appeal process, because the combination of receiving the lowest rating for student learning and an “effective” or “highly effective” rating by a supervisor is rare.

For those who do end up with that combination, the odds of a negative rating being boosted are in a teacher’s favor. The appeal process includes the presumption of a higher rating, meaning the burden of proof needed to uphold the original “ineffective” rating would fall on the city.

A separate appeal process would also be set up for teachers rated “highly effective” on student learning measures, but who earn an overall “ineffective” because of a principal’s observation. In that case, the UFT can choose which cases to appeal to an arbitration panel of three teachers.

Now, the deal must get a final sign-off from Commissioner John King. The city is supposed to submit a draft of its proposal to King by Thursday. A spokesman for the State Education Department declined to comment on the plan.

Department of Education officials said they believe the city’s proposed plan complied with state evaluation laws and regulations. In a statement, a spokeswoman said the agreement “focuses on raising student achievement in ways we’ve never been able to do before.”

Want the latest in New York City education news? Follow Chalkbeat on Facebook or @ChalkbeatNY on Twitter.

teacher prep

Tennessee’s mediocre teacher training programs prompt ‘interventions’ with university presidents

PHOTO: Austin Peay
Austin Peay State University in Clarksville is among four Tennessee schools that have undergone "interventions" with state officials over the quality of their teacher training programs.

Armed with sobering data about the performance of teacher training programs in Tennessee, state officials are holding meetings with top brass at universities where they say programs have grown out of touch with the needs of K-12 classrooms.

About 40 programs in Tennessee feed the state’s teacher pipeline with about 4,000 new teachers annually. The largest are based at colleges and universities.

But those same traditional programs generally aren’t attracting enough high-quality candidates or producing enough effective or diverse teachers. Not a single public university in Tennessee scored in the top fifth of teacher training programs under a state report card issued in 2016. And the outlook isn’t expected to improve much under the 2017 report card being released early next month, officials say.

“This data is sobering. It tells us that higher education must do better,” said Mike Krause, executive director of the Tennessee Higher Education Commission. “I worry our higher education faculty in colleges of education get disconnected from what a K-12 classroom looks like.”

Krause outlined the challenges to state lawmakers during a presentation on Tuesday with Sara Heyburn Morrison, executive director of the Tennessee State Board of Education.

Their first “intervention meetings” were with the presidents and education deans at four universities: Austin Peay, Tennessee-Chattanooga, Tennessee-Martin, and Tennessee Tech. Similar meetings are scheduled this spring with leadership of private colleges and universities across the state.

Krause described the first meetings as “very productive” — and illuminating. “In many cases, the presidents just didn’t know” about their programs’ shortcomings, he said.

Teacher quality is considered a driving factor in students’ success, making the quality of teacher preparation programs a front-burner issue in Tennessee.  A 2016 report said only a handful of the state’s programs are consistently preparing teachers to improve student achievement based on Tennessee’s TVAAS measure. The State Board’s new grading system also highlighted weaknesses based on racial diversity, candidates’ ACT scores, and whether they are producing teachers for high-need areas such as special education.

Reading instruction is another big challenge. In a state where only a third of students are considered proficient in reading, new teachers are arriving in classrooms ill-prepared to instruct students on Tennessee’s new reading standards. The state is working with higher education institutions so their faculty can take the same professional development on literacy that working teachers are taking.

But for the most part, the State Board has limited levers for improving the quality of teacher prep. The biggest hammer comes every seven years when each program undergoes a comprehensive review for licensure. (In 2014, the state raised its standards and revised its measures for effectiveness to include data such as placement, retention and employer satisfaction.)

Chancellor Keith Carver

Tennessee-Martin Chancellor Keith Carver said his school took its last state report card to heart. As a result of its overall score of 2 out of a possible 4, the university hired an assessment coordinator to help guide decisions based on data. “It’s a really good baseline for improving,” he said of the report card. “We’ve got some work to do in our diversity profile.”

Tennessee’s teacher candidates are overwhelmingly white and female. Of those who completed Tennessee’s programs in 2016, only 14 percent identified themselves as non-white, compared with 36 percent of the state’s student population.

“Colleges of education will not stumble into diversity. There has to be a very intentional effort,” Krause said.

View the full presentation from Tuesday’s legislative hearing below.

call out

Our readers had a lot to say in 2017. Make your voice heard in 2018.

PHOTO: Chris Hill/Whitney Achievement School
Teacher Carl Schneider walks children home in 2015 as part of the after-school walking program at Whitney Achievement Elementary School in Memphis. This photograph went viral and inspired a First Person reflection from Schneider in 2017.

Last year, some of our most popular pieces came from readers who told their stories in a series that we call First Person.

For instance, Carl Schneider wrote about the 2015 viral photograph that showed him walking his students home from school in a low-income neighborhood of Memphis. His perspective on what got lost in the shuffle continues to draw thousands of readers.

First Person is also a platform to influence policy. Recent high school graduate Anisah Karim described the pressure she felt to apply to 100 colleges in the quest for millions of dollars in scholarships. Because of her piece, the school board in Memphis is reviewing the so-called “million-dollar scholar” culture at some high schools.

Do you have a story to tell or a point to make? In 2018, we want to give an even greater voice to students, parents, teachers, administrators, advocates and others who are trying to improve public education in Tennessee. We’re looking for essays of 500 to 750 words grounded in personal experience.

Whether your piece is finished or you just have an idea to discuss, drop a line to Community Editor Caroline Bauman at [email protected]

But first, check out these top First Person pieces from Tennesseans in 2017:

My high school told me to apply to 100 colleges — and I almost lost myself in the process

“A counselor never tried to determine what the absolute best school for me would be. I wasted a lot of time, money and resources trying to figure that out. And I almost lost myself in the process.” —Anisah Karim     

Why I’m not anxious about where my kids go to school — but do worry about the segregation that surrounds us

“In fact, it will be a good thing for my boys to learn alongside children who are different from them in many ways — that is one advantage they will have that I did not, attending parochial schools in a lily-white suburb.” —Mary Jo Cramb

I covered Tennessee’s ed beat for Chalkbeat. Here’s what I learned.

“Apathy is often cited as a major problem facing education. That’s not the case in Tennessee.” —Grace Tatter

I went viral for walking my students home from school in Memphis. Here’s what got lost in the shuffle.

“When #blacklivesmatter is a controversial statement; when our black male students have a one in three chance of facing jail time; when kids in Memphis raised in the bottom fifth of the socioeconomic bracket have a 2.6 percent chance of climbing to the top fifth — our walking students home does not fix that, either.” —Carl Schneider

I think traditional public schools are the backbone of democracy. My child attends a charter school. Let’s talk.

“It was a complicated choice to make. The dialogue around school choice in Nashville, though, doesn’t often include much nuance — or many voices of parents like me.” —Aidan Hoyal

I grew up near Charlottesville and got a misleading education about Civil War history. Students deserve better.

“In my classroom discussions, the impetus for the Civil War was resigned to a debate over the balance of power between federal and state governments. Slavery was taught as a footnote to the cause of the war.” —Laura Faith Kebede