DIY Accountability

Frustrated with city's data system, teachers build their own

picture-5
Created by teachers at the High School for Telecommunication, DataCation collects and analyzes student data, rivaling the city's own database.

When he began teaching at a Bronx high school, Jesse Olsen found the school had a large blind spot when it came to taking attendance.

If a student came to class for the first half of the school day and then skipped out, she’d go down in the official record as being present for the full day. The information holes made it impossible for teachers to know what their students’ true attendance was like, Olsen said.

A new, sophisticated database known as ARIS, for Achievement Reporting and Innovation System, might have been just the thing to solve the problem. But the system only let schools see how many days a student had missed, not how many classes they were skipping.

So Olsen took matters into his own hands, drawing on his computer science training to build an attendance system for his school, Validus Preparatory Academy.  In doing so he joined a growing number of teachers who don’t rely on the city’s data tools to track student information.

Brought into the city’s public schools in 2008 as a major initiative of Chancellor Joel Klein, ARIS cost $80 million to make. It debuted at the same time that Klein began to ask teachers to keep close track of student data and use it to adjust their instruction.

To do that, teachers would need more data. But even after recovering from some of its early glitches, ARIS continues to disappoint. Teachers complain that it offers them too little information and parents say it’s hard to access.

To meet the demand for data, some teachers and schools have created their own content management systems and are selling the products to other public schools.

Olsen’s program, called Impact, has an online attendance system that updates instantly and allows teachers to add comments on students’ behavior. Seeing that ARIS only includes students’ final course grades, he added an online gradebook that shows how students did on individual assignments, how well they’ve learned certain skills, and what work they still need to complete.

Impact is now in 21 New York City schools, which pay between $10 to $25 per student for a year of service. Teach for America recently began using it to track how some of its members’ students’ perform.

“I think when tools are made for districts, New York being the superlative example of a big district, they can only be so useful because they have to generalize,” Olsen said. “They have to make it work for the young and the old, the new and traditional.”

“What you emerge with is a tool that works for everybody but it barely does anything,” he said. “Schools should have a choice. The DOE should say here’s a number of recommended partners, we just need the data, you pick the tool that works in your way.”

Olsen’s suggestion comes at exactly the same time that the city is rethinking how schools use ARIS.

Deputy Chancellor for accountability Shael Suransky said the city will begin piloting a new version of the program called ARIS Local in some schools next spring. Teachers will be able to enter data on students’ progress on reading assessments and chapter tests that the current database doesn’t include.

“What we want in the long run is for ARIS to be a platform like the iPhone is a platform, where people can develop applications and they can draw the data from our central system and format it into easy to use ways,” he said. “ARIS is the first step on that path.”

On candidate for app creation might be DataCation, which emerged several years ago from teachers at the High School of Telecommunication Arts and Technology in Brooklyn. Created with a focus on the No Child Left Behind law’s requirements, the program allows schools to track students’ progress toward graduation, their schedules, and their grades.

The Telecommunication teachers sold DataCation to a company called CaseNEX, which also bought a scheduling program called Skedula that was developed by a former programmer at Herbert Lehman High School.

Last year, about 30 city schools purchased DataCation, a sleek program that lets schools do everything from scheduling classes and tracking credit accumulation to predicting their results on the federal government’s accountability system. The full suite can cost $8,500, but even in the midst of budget cuts, schools are finding ways to cover the expense.

Most DataCation clients are high schools, and many are struggling schools that the city or state could close if their graduation rates don’t rise. For them, being able to single out a group of low-performing students and focus on them is a matter of survival.

“It’s designed to really catch kids that are not identified using any other tools and to monitor their progress and make sure that info is available in a timely manner, not three semesters later,” said CaseNEX CEO Marsha Gartland. “It’s a pretty simple concept, but it can bring a whole new level of order to a school that’s been lacking it.”

One of DataCation’s most popular features allows parents to log in and see their children’s recent grades, attendance, and missing work. Parents can also do this on the ARIS website through the Parent Link, but there’s less information and it’s older.

In another case, a group of staff members at Leon Goldstein High School in Brooklyn formed the LMG Data Group to sell data management software to other schools. Their clients buy FileMaker, an Apple software product, and then the group sets up a customized data aggregation and display program based on what the school wants. This year, nine schools will use the software.

Goldstein Principal Joseph Zaza said the program began in 2006 as an experiment and a way for the school to know more about its students than the DOE’s software would permit.

‘We’ve done a lot more than just track student data,” Zaza said. “We use it to track student behavior. Deans put in behavioral problems and when a student doesn’t behave — doesn’t have a photo ID or is cutting class — then immediately the system emails that information to the guidance counselors and myself so that everybody is informed.”

The school also uses FileMaker to track how many hours of community service its students have done and the software has cut down on the number of lost books by linking students’ ID numbers to the books’ bar codes.

Schools are charged based on the complexity of their data demands, with one-time prices ranging from $5,000 to $40,000.

study says...

In new study of school-district effectiveness, New York City falls just below national average

PHOTO: Stephanie Snyder

Each year, state test scores offer a snapshot of how much New York City students have learned. But they say little about how the city’s schools stack up against other districts’, in part because the raw scores largely reflect student demographics — wealthier districts tend to have higher scores.

Now, a major new analysis of several years of test scores from across the country provides a better way to judge and compare districts: Instead of looking at a single moment, it shows how well school systems help students grow their skills over time.

Based on that measure, New York City falls just below the middle of the pack: In the five years from third to eighth grade, its students collectively make about 4.6 grade levels of progress — landing New York in the 35th percentile of districts nationally. By contrast, Chicago students advance the equivalent of six grades within those five years, giving the district one of the highest growth rates in the country.

Still, New York is slightly above average when compared to other large districts with many students from low-income families. And it trounces the state’s other urban districts — including Yonkers, Syracuse, and Rochester, which have some of the nation’s worst growth rates.

“Among big poor districts, it’s better than average,” said Sean Reardon, the Stanford University researcher who conducted the analysis. “In the grand scheme, it’s pretty middle-of-the-road.”

Reardon’s analysis — based on 300 million standardized tests taken by students across more than 11,000 school districts from 2009 to 2015 — is the largest of its kind. It looks both at student proficiency on third-grade math and English tests (that is, what share of students earned a score deemed “proficient”) and student growth between grades three and eight (how much their scores improved over time). Reardon’s research was supported by several foundations, including the Bill and Melinda Gates Foundation, which also provides funding to Chalkbeat.

The analysis controls for the differences in tests across states and over time by converting scores into a common scale that measures growth in grade levels, making it possible to compare nearly every district in the country to one another. (It excludes New York’s scores from 2015 and some grades in 2014 because of the high number of students who boycotted the state tests those years. However, each district’s five-year growth rates is actually an average of its year-over-year growth, so Reardon was still able to calculate a five-year rate for New York.)

Experts generally prefer growth rates over proficiency as a way to evaluate school quality, since growth measures the progress students make in school rather than where they started. Even if a district enrolls many poor students who are less likely than their affluent peers to hit the “proficiency” benchmark, its schools can still help them advance at a rate comparable to or even better than schools filled with wealthier students.

“Growth is way better than achievement,” said Douglas Ready, an education and public policy professor at Teachers College, Columbia University. “We know low-income students start school behind — the question is what do school districts do with the kids they get?”

New York’s growth rate falls just below the national median of 4.8 grade levels. Among big districts, its students made gains similar to those in Dallas and Detroit, and greater than students in Los Angeles, Miami, and Indianapolis.

By contrast, Rochester ranks rock-bottom nationally. In that high-poverty district, where the median income among families with children in the public schools is $26,000, students advanced about three grade levels in five years. Yonkers’ $48,000 median income is much higher, yet its schools barely do better, with students moving just 3.5 grade levels. (Among New York City public-school parents, the median income is $42,000.)

Reardon emphasized that test scores provide an important but incomplete picture of student learning, and growth rates are an imperfect measure of school effectiveness since factors outside of the classroom also influence how much students learn over time.

Still, he argued that officials who rate schools and parents who choose them would do much better to look at a school’s growth rate over its average test scores. In fact, he said, a focus on growth rates could theoretically drive down socioeconomic segregation since higher-income parents might be willing to enroll their children in schools with many poor students and low overall test scores if the schools nonetheless had outstanding growth rates.

Ready, however, pointed out that even when schools and districts are highly effective at helping students make progress, they are still unlikely to close the yawning achievement gaps that separate most poor and wealthier students from the time they start school. Reardon came to the same conclusion.

“The large gaps in students’ academic skills between low- and higher-[socioeconomic status] districts are so large,” Reardon’s analysis says, “that even the highest growth rate in the country would be insufficient to close even half of the gap by eighth grade.”

In response to the analysis, New York City education department officials pointed to the National Assessment of Educational Progress, a standardized test taken by a representative sample of students in each state and certain districts, including New York. Only one other district among the country’s 10 largest cities performed better in reading and math than New York, which had the highest share of low-income students reach the proficient level on the reading test.

“Our schools are the strongest they’ve ever been, with record-high graduation and college enrollment rates, and improving state test scores,” said the district’s spokesman, Will Mantell.

change up

Just as Lower East Side integration plan takes off, superintendent who helped craft it steps down

PHOTO: Christina Veiga
Carry Chan, left, will become acting superintendent in District 1 when Daniella Phillips, right, leaves this month to join the central education department.

The longtime superintendent of the Manhattan community district where parents pushed for a plan to desegregate the local schools is stepping down just as the plan gets underway.

After a decade at the helm of District 1, which includes the Lower East Side and East Village, Superintendent Daniella Phillips is leaving to join the central education department, Chalkbeat has learned. During the yearslong campaign for an integration plan, Phillips acted as a liaison between parents and the education department, which finally approved a new admissions system for the district’s elementary schools this fall.

She will be replaced by Carry Chan, who has also played a role in the district’s diversity efforts as the interim head of a new Family Resource Center, an information hub to help district parents sort through their school options. Chan takes over as acting superintendent on Dec. 18.

The leadership change comes at a crucial time for the district, which also includes a portion of Chinatown. Parents are currently applying to elementary schools, marking the first admissions cycle under the new enrollment system. Under the system, schools give certain students admissions priority based on their economic status and other factors, with the goal of every elementary school enrolling share of disadvantaged students similar to the district average.

It will be up to the new superintendent to help schools recruit and welcome a greater mix of families, and to help steer parents towards a wider range of schools. Advocates hope the district can become a model for the city.

“There is a torch that needs to be carried in order to really, fully execute,” said Naomi Peña, president of the district’s parent council. “The next superintendent has to be a champion for the mission and the cause.”

During heated public meetings, Phillips tried to keep the peace while serving as a go-between for frustrated integration advocates and reluctant education department officials. The tensions sometimes boiled over, with advocates directing their anger at Phillips — though they were eventually won-over and endorsed the final integration plan.

In her new role, she will oversee school consolidations as part of the education department’s Office of School Design and Charter Partnerships. In District 1, Phillips helped steer three such mergers, which often involve combining small, low-performing schools with ones that are higher achieving.

“It has been such a joy and privilege to be District 1 superintendent for over 10 years, and I’m excited for this next chapter in the district and my career,” Phillips said in an emailed statement.

Chan is a former principal who launched the School for Global Leaders, a middle school that focuses on community service projects and offers Mandarin classes. Last year, she joined the education department’s Manhattan support center, where she helped schools form partnerships in order to learn from one another.

Since October, Chan has served as the interim director of District 1’s Family Resource Center, which is seen as an integral part of making the new diversity plan work. Families must apply for seats in the district’s elementary schools, which do not have attendance zones like other districts. The family center aims to arm families with more information about their options, in the hopes that they will consider schools they may not have previously.

“I think we’re all really passionate about this plan and we really want this to work,” Chan said. “Communication is the key, and being transparent with how we’re progressing with this work.”