Data disagreements muddy takeover debate

PHOTO: G. Tatter
Board member Amy Frogge (left) and an ASD official discuss data on the sidelines of the meeting.

Data is objective. Isn’t it?

In front of a packed auditorium at Madison Middle Prep last week,  Metropolitan Nashville Public Schools board member Jill Speering presented data to prove a point: the middle school should not be taken over by the Achievement School District.

Her voice trembling with emotion, she directed the numbers at Chris Barbic, the superintendent of the statewide school district. She started her speech by comparing Brick Church College Prep, a current ASD school, to Metro Schools overall.

She said that Brick Church wasn’t as extraordinary as Barbic asserted, according to state growth data, called the Tennessee Value Added Assessment System, or TVAAS.

“I don’t know what you’re talking about,” Barbic replied. “They were level five TVAAS [the highest score for growth].”

He then chastised her, causing the crowd to boo: “You’re a school board member. You should know how TVAAS works.”

In the wake of the ASD’s announcement that it will take over one of two middle schools in a Nashville community, district officials and their opponents have continually cited data points that seem contradictory. Both sides have accused the other of playing fast and loose with numbers, and even of lying.

But in fact, representatives from both sides of the school takeover debate were using different yardsticks to measure the same things: the growth of student test scores at the Nashville middle schools up for take over; the schools run by LEAD Public Schools (a charter operator that will manage the taken over school); and the ASD as a whole.

The ASD was created to transform Tennessee’s bottom 5 percent of schools into the top 25 percent by 2020. For the first time, both Madison Middle Prep and Neely’s Bend Middle Prep in Nashville were in the bottom 5 percent of schools this year, making them eligible for state intervention.

Opponents of intervention, led by school board members Speering and Amy Frogge, say that in fact, Madison and Neely’s Bend’s students are improving at comparable rates to LEAD’s schools and other schools in the ASD. Therefore, they say, takeover isn’t necessary, and might even be counterproductive.

ASD officials say that LEAD’s schools are improving at much higher rates than Madison or Neely’s Bend.

So what explains this glaring discrepancy?

To make decisions about a school, including whether it is eligible for ASD takeover, the Tennessee Department of Education takes into account schoolwide passing rates on the state’s standardized test, and TVAAS ratings.

TVAAS is a  way to measure if students at a school are making more or less growth than their peers across the state. (You can read more about the controversy around TVAAS here).  In order to compute it, the state Department of Education calculates a number that shows the growth of students at a school compared to students statewide.

Let’s call that the raw growth score. It’s the first of two steps the state uses in calculating TVAAS ratings.

But those were the numbers that Frogge and Speering cited. According to composite raw growth scores, that took into account test scores in math, reading, science and social studies, both Neely’s Bend and Madison had slight negative two-year a growth scores of -1, and -2.5 respectively.  The composite growth score for Brick Church College Prep, a LEAD school in the ASD that ASD officials tout as a model for their next Nashville take over, was slightly positive at .4.

Those raw numbers don’t suggest that Brick Church is improving its students scores much more than the traditional district schools are. If you break out raw scores for individual subjects, Brick Church sometimes fares worse than the schools up for take over. For example, in reading, its raw growth score for two years was -3.7, while Madison and Neely’s Bend had raw growth scores of -2.8 and -1.5, respectively.

But to get a final TVAAS rating for a school, there’s another step: the state divides that raw growth score by a standard error. Paul Changas, the director of research for Metro Schools, says the standard error takes a variety of factors into account, but that the smaller the student population at a school, the larger the standard error calculation.

“Random factors such as student focus, distractability, energy level, emotional state, or text complexity can be fairly significant when only a few students are involved,” he said. “But as the size of the group gets larger, these random factors tend to balance out – for every student having a bad day taking the typing test there is more likely to be a student having a good day.”

Source: TN.gov

Standard error also takes into account the variability in scores at a school, said Ashley Ball, a spokesperson for the Department of Education. Decision-makers on the state level want the data to tell them a clear story about what’s going on in a school, she said.

“If most of the students are exceeding expectations it tells us, wow, this is a really clear story,” she said. “If students are all across the board, you’re going to have a bigger standard error to account for the fact that the story isn’t as clear.”The number schools get after their raw score is divided by the standard error is called a “growth index.” From that, the state assigns a school a grade from levels one to five, five being the best.

Source: Paul Changas, TN.gov

Since Brick Church is a phase-in school, it has expanded one grade at a time, and the student body is still considerably smaller than other middle schools. Therefore, the school has a higher standard of error, and its growth index looks pretty different from its raw score.

The ASD has built its case for takeover on a variety of factors, among them the growth index. Brick Church’s growth index is higher than Neely’s Bend or Madison’s. Brick Church’s most recent overall growth index is a Level 5, and the Madison and Neely’s Bend are Level 1s — putting them on the hot seat for take over.

So which measurement of a schools’ success is more valid? Opinions differ. Metropolitan Nashville Public Schools officials typically go with the raw score.

“While the growth index [the numbers the ASD are citing] is important to determining how confident we are that growth exceeds a target, it is not the most meaningful scale for measurement [of] the amount of growth that has occurred,” Changas said.

But the state Department of Education disagrees, as does the ASD.

“You’re going to want to factor in standard error, because it is a buffer and a protective measure,” state education department spokeswoman Ball said. “We want to have the most accurate data possible.”

You can see Paul Changas’s data here, and check out the state’s interpretation of the same numbers at the TVAAS website.

Clarification:  The growth index is one of many factors the ASD considers when taking over schools.  An earlier version of the story described the growth index as the predominant factor.

School safety

Hiring more security officers in Memphis after school shootings could have unintended consequences

PHOTO: Jahi Chikwendiu/The Washington Post/Getty Images

Tennessee’s largest district, Shelby County Schools, is slated to add more school resource officers under the proposed budget for next school year.

Superintendent Dorsey Hopson earmarked $2 million to hire 30 school resource officers in addition to the 98 already in some of its 150-plus schools. The school board is scheduled to vote on the budget Tuesday.

But an increase in law enforcement officers could have unintended consequences.

A new state law that bans local governments from refusing to cooperate with federal immigration officials could put school resource officers in an awkward position.

Tennessee Education Commissioner Candice McQueen recently reminded school personnel they are not obligated to release student information regarding immigration status. School resource officers employed by police or sheriff’s departments, however, do not answer to school districts. Shelby County Schools is still reviewing the law, but school board members have previously gone on the record emphasizing their commitment to protecting undocumented students.

“Right now we are just trying to get a better understanding of the law and the impact that it may have,” said Natalia Powers, a district spokeswoman.

Also, incidents of excessive force and racial bias toward black students have cropped up in recent years. Two white Memphis officers were fired in 2013 after hitting a black student and wrestling her to the ground because she was “yelling and cussing” on school grounds. And mothers of four elementary school students recently filed a lawsuit against a Murfreesboro officer who arrested them at school in 2016 for failing to break up a fight that occurred off-campus.

Just how common those incidents are in Memphis is unclear. In response to Chalkbeat’s query for the number and type of complaints in the last two school years, Shelby County Schools said it “does not have any documents responsive to this request.”

Currently, 38 school resource officers are sheriff’s deputies, and the rest are security officers hired by Shelby County Schools. The officers respond and work to prevent criminal activity in all high schools and middle schools, Hopson said. The 30 additional officers would augment staffing at some schools and for the first time, branch out to some elementary schools. Hopson said those decisions will be based on crime rates in surrounding neighborhoods and school incidents.

Hopson’s initial recommendation for more school resource officers was in response to the school shooting in Parkland, Florida, that killed 17 people and sparked a wave of student activism on school safety, including in Memphis.

Gov. Bill Haslam’s recent $30 million budget boost would allow school districts across Tennessee to hire more law enforcement officers or improve building security. Measures to arm some teachers with guns or outlaw certain types of guns have fallen flat.

For more on the role and history of school resource officers in Tennessee, read our five things to know.

Sheriff’s deputies and district security officers meet weekly, said Capt. Dallas Lavergne of the Shelby County Sheriff’s Office. When the Memphis Police Department pulled their officers out of school buildings following the merger of city and county school systems, the county Sheriff’s Office replaced them with deputies.

All deputy recruits go through school resource officer training, and those who are assigned to schools get additional annual training. In a 2013 review of police academies across the nation, Tennessee was cited as the only state that had specific training for officers deployed to schools.

call out

Our readers had a lot to say in 2017. Make your voice heard in 2018.

PHOTO: Chris Hill/Whitney Achievement School
Teacher Carl Schneider walks children home in 2015 as part of the after-school walking program at Whitney Achievement Elementary School in Memphis. This photograph went viral and inspired a First Person reflection from Schneider in 2017.

Last year, some of our most popular pieces came from readers who told their stories in a series that we call First Person.

For instance, Carl Schneider wrote about the 2015 viral photograph that showed him walking his students home from school in a low-income neighborhood of Memphis. His perspective on what got lost in the shuffle continues to draw thousands of readers.

First Person is also a platform to influence policy. Recent high school graduate Anisah Karim described the pressure she felt to apply to 100 colleges in the quest for millions of dollars in scholarships. Because of her piece, the school board in Memphis is reviewing the so-called “million-dollar scholar” culture at some high schools.

Do you have a story to tell or a point to make? In 2018, we want to give an even greater voice to students, parents, teachers, administrators, advocates and others who are trying to improve public education in Tennessee. We’re looking for essays of 500 to 750 words grounded in personal experience.

Whether your piece is finished or you just have an idea to discuss, drop a line to Community Editor Caroline Bauman at cbauman@chalkbeat.org.

But first, check out these top First Person pieces from Tennesseans in 2017:

My high school told me to apply to 100 colleges — and I almost lost myself in the process

“A counselor never tried to determine what the absolute best school for me would be. I wasted a lot of time, money and resources trying to figure that out. And I almost lost myself in the process.” —Anisah Karim     

Why I’m not anxious about where my kids go to school — but do worry about the segregation that surrounds us

“In fact, it will be a good thing for my boys to learn alongside children who are different from them in many ways — that is one advantage they will have that I did not, attending parochial schools in a lily-white suburb.” —Mary Jo Cramb

I covered Tennessee’s ed beat for Chalkbeat. Here’s what I learned.

“Apathy is often cited as a major problem facing education. That’s not the case in Tennessee.” —Grace Tatter

I went viral for walking my students home from school in Memphis. Here’s what got lost in the shuffle.

“When #blacklivesmatter is a controversial statement; when our black male students have a one in three chance of facing jail time; when kids in Memphis raised in the bottom fifth of the socioeconomic bracket have a 2.6 percent chance of climbing to the top fifth — our walking students home does not fix that, either.” —Carl Schneider

I think traditional public schools are the backbone of democracy. My child attends a charter school. Let’s talk.

“It was a complicated choice to make. The dialogue around school choice in Nashville, though, doesn’t often include much nuance — or many voices of parents like me.” —Aidan Hoyal

I grew up near Charlottesville and got a misleading education about Civil War history. Students deserve better.

“In my classroom discussions, the impetus for the Civil War was resigned to a debate over the balance of power between federal and state governments. Slavery was taught as a footnote to the cause of the war.” —Laura Faith Kebede