A chart from the report showing how students with very different high school trajectories can end up in the same place academically—at least on paper.

Not all high school graduates are created equally: Some had to make up ground after falling behind along the path to graduation day. Identifying those future graduates early could be key to getting them to succeed in college later, according to a new report.

The report, authored by researchers with the education nonprofit New Visions for Public Schools, tracked students in 75 New Visions-supported city schools through high school and into college. The report finds that students who graduate with a Regents diploma after years of struggling are much less likely to succeed in college than those students who have a history of good performance.

Schools tend to pay special attention to students with obvious obstacles to overcome, such as a disability or status as an English language learner. But students who have a couple of bad semesters in tenth grade and then earn passing grades in their junior year don’t always register as being “at risk” to their schools, the report concludes.

The report advocates for schools to expand the definition of an “at-risk” student to include any student who has experienced ups and downs—which are marked and reviewed according to a metric system detailed in the study that New Visions schools will continue to use. It also argues that school districts like New York City are pushing schools in this direction by emphasizing schools’ graduation rate as the main benchmark of success.

“We’re trying to take the conversation and say, every kid, whether high or low performing, is vulnerable but in a different way,” said Susan Fairchild, one of the report’s lead authors. “Our accountability structures don’t necessarily support schools. We’re moving in those direction, but our systems are really based on accumulation, not flow, not how kids actually come into the system.”

The system categorizes students into four groups—those “on track to college readiness,” “on track to graduation,” “almost on track,” and “off track”— and re-categorizes them at the end of each semester of high school. By senior year, a student could have scored highly early on, but later fallen to “almost on track.” His data would therefore look different than a student who has been “almost on track,” for the past three years.

Tracking students over time and cataloguing when and where they are on track and off track, as the study does, can help predict a student’s risk of dropping out of college, Fairchild said.

Kirsten Larson, the principal of Marble Hill School for International Studies, a small high school opened in the Bronx in 2002, said she has been tracking student performance over time from early on. Marble Hill is one of the New Visions schools now using this methodology.

Larson said she offers teachers professional development at least once a week to give them tools to address struggling students’ needs, particularly for those not certified to teach English as a second language. Close to a third of her students are English language learners. She also expects teachers to meet with each student to go over grades every quarter.

“We have students coming in without a lick of English, students who may be fluent but haven’t passed the NYSESLAT. We have every issue imaginable here. We have to really make sure that we individualize a program for them.”

For example, Marble Hill teachers assess each student’s math performance during a summer “bridge” orientation program, and will sometimes recommend a student who has already passed algebra in eighth grade take it again in ninth. And at the end of each grading period, the school holds “town hall” style meetings with each grade level to review grades and course requirements.

“We go over exactly what they need from freshman year on,” she said. “Even if they heard it once, we know they didn’t hear everything. Being able to ask questions and see examples and look at their own data makes it that much more relevant to them.”

The study’s recommendations join a growing fervor in policy circles over how to boost college readiness rates across the school system, which were dismally low when the city released the results of its first citywide metric system last year. Fairchild said more schools should adopt these strategies after taking a closer look at how their students perform over time.

“For every single student we look at their grades, and we know which students we need to target right away,” she said, referring to the state test that students take to show English proficiency. ”They might have five people coming to talk to them, as opposed to be ignored or no one noticed.”