Three months into the start of the school year, the Department of Education is just figuring out how to rate more than a thousand itinerant teachers.

Under the current teacher evaluation system in place in nearly all schools, principals rate teachers once a year as either “satisfactory,” or “unsatisfactory.” They are also supposed to offer advice to help teachers improve.

But when the city and UFT struck a deal this summer to avert layoffs, they agreed to move members of the  Absent Teacher Reserve, the pool of teachers who do not have permanent positions, to a different school—with a different principal—each week. The agreement left open the question of who would observe and rate those teachers.

In a year when the city and union are fighting fiercely over the particulars of new teacher evaluations, officials from the United Federation of Teachers told me they have left the decision of how ATRs will be rated up to the DOE.

Now the city has decided that ATRs will receiving ratings from their district superintendent, officials said, with input from the principals of schools where they were sent to work over the course of the year. The city is also testing out other options.

Teachers have raised concerns about the fairness of the process. At union meetings early in the school year, ATRs questioned how their performance could be evaluated when there is little consistency in their jobs: They typically teach a different set of students in a different classroom each week or receive non-teaching assignments, and they spend as little as one day in each school. Some members of the ATR pool said they have been asked to teach subjects outside of their licensed content areas.

The city is piloting one alternative rating process in Brooklyn. Through that program, newly hired “field supervisors” will help some Brooklyn superintendents rate ATR teachers this winter. The supervisors, who were drawn from a pool of assistant principals and principals, will evaluate the teachers’ instructional practices and offer them professional development, according to DOE officials.

Union officials said that ideally the field supervisor would act as both rating agent and career coach, guiding the ATRs toward open, permanent positions at district schools. But DOE officials did not list that role among the supervisors’ job tasks.

A letter the DOE sent to ATRs in Brooklyn last week said the field supervisors would observe the ATRs in the classroom “periodically” over the next two months.

DOE officials said they would analyze the results of the field supervisor initiative at the end of the school year before deciding whether to expand the position to other boroughs.

A Bronx technology teacher in the ATR pool told me last week that no one has observed him in a classroom so far this year. Having a supervisor could be useful, he said, but he would rather the city offer him financial support to get new training to make him a more desirable hire.