To our readers

Hey, we heard you. You had a lot of questions about TNReady. We found answers.

The news that Tennessee’s testing company scored some high school tests incorrectly this year uncorked a flood of questions about the validity of the state’s new standardized assessment.

Here are five things to know about the latest brouhaha over TNReady

We wanted to know how the brouhaha was impacting classrooms, so we asked our readers on Facebook.

You responded in droves.

We took your top concerns directly to the state Department of Education and asked for answers. Here’s what you wanted to know — and what we have learned:

Several readers asked why they should trust TNReady results, given the series of setbacks in the test’s first two years.

  • “I do not trust the results. We have had so many problems in the last few years, that I am suspicious of any results we do get. It bothers me greatly that the state uses these numbers to hold students and teachers and districts accountable, but they seem to be unable to deliver scores they believe are accurate in a timely manner.” —Rebecca Dickenson
  • “I no longer trust the accountability of the state nor its methods. My concern is if there is a teacher who has only one year of test data, how is it the same teacher shown multi-year growth when he or she had only last year of testing? This poses a huge concern.” —Mildred Williams  

Tennessee Department of Education: “TNReady is fully aligned to Tennessee’s academic standards, and every question has been reviewed, edited, and approved by Tennessee teachers through a rigorous review process. We also have quantitative checks and processes after a test is over to ensure student responses are reliable. While more than 99.9% of TNReady tests were scored accurately this year, we want to improve on that next year, and our vendor (Questar) is taking new quality assurance steps to make sure their programming is error-free. Also, this year, as soon as the scoring error on some of the English I, II and Integrated Math II EOCs was identified, scores were updated and all TNReady tests were re-reviewed and verified for full accuracy.”

Some teachers told us that, given the delay in score deliveries this spring, many students don’t think the results will arrive in time to affect their final grades next spring. Those teachers are struggling to get their students to buy in.

  • “After two years of TNReady, it still hasn’t counted for my students. Going into year three, I will once again tell them with a hopeful, straight face that it will count as part of their report card grades and implore them to try their best. I quietly wonder what reason they have to believe me, given recent history.” —Mike Stein
  • “I struggle to get students to buy in to the importance of trying their best on state tests because the students are confident that the scores won’t come back in time to affect their grades (which has been the situation for several years now). The students see zero incentive for doing well.” —Nicole Mayfield

TDOE: “We believe that if districts and schools set the tone that performing your best on TNReady is important, then students will take the test seriously, regardless of whether TNReady factors into their grade. We should be able to expect our students will try and do their best at any academic exercise, whether or not it is graded. This is a value that is established through local communication from educators and leaders, and it will always be key to our test administration. We believe that when we share these messages and values celebrating the variety of accomplishments our students have made, taking advantage of TNReady’s scheduling flexibility to minimize disruption, focusing on strong standards-based instruction every day, sending positive messages around the importance of the variety of tests that students take, and sharing that students should always do their best then students will buy-in and TNReady will be successful.”

Other teachers asked what happens to writing scores for tests in English language arts.

  • “I can tell you that two years ago — when we first piloted the new writing test online — districts received not only every student’s scores (broken down by each of the four indicators) but also the actual student responses to each prompt. In my former district our supervisor shared them, and we analyzed them as a department. If you check with your principal, VP, or supervisors, there are some published “anchor papers” with scores available on edtools from this past year. It’s not a lot, but it’s more than we’ve had in the past. My hope is that if online continues, we’ll keep seeing the student responses in the future.” —Wj Gillespie II

TDOE: “The question appears to be referencing the process we had through the 2014-15 school year, when our writing assessment was separate. Since 2015-16, students’ writing responses on TNReady have been incorporated as part of their overall ELA score. Responses are scored based on our writing rubrics, and for educators, we have provided access to the “anchor papers” from the 2016-17 year, so they can see how students’ responses were scored based on the writing rubric, which can help them inform the feedback they give their students.”

On that same issue of writing scores, one teacher referenced the hiring of scorers off of Craigslist. We asked the state if that’s true.

  • “I continue to be curious about our ELA writing scores. Each year we are required to use state writing rubrics, attend PD related to the state’s four types of writing, etc etc…and yet our scores never come back. Students spend hours taking the writing portion of the test, scorers are hired off Craig’s list…, and yet we never actually get the scores back. It seems like every year this is swept under the rug. Where do these writing tests go?” —Elizabeth Faison Clifton

TDOE: “Questar does not use Craigslist. Several years ago, another assessment company supposedly posted advertisements on Craigslist, but Questar does not. We provide opportunities for our educators to be involved in developing our test, and we also encourage Tennessee teachers to apply to hand-score TNReady. To be eligible, each applicant must provide proof of a four-year college degree, and preference is given to classroom teachers. As part of the interview process, an applicant would have to hand-score several items for review and evaluation. Once hired, each scorer is trained based on materials that Tennessee teachers and the department approve — and which are assembled from responses given by Tennessee students on the exam — and scorers are regularly refreshed and “recalibrated” on scoring guidelines. Each writing response is scored at least twice; if those two responses differ significantly, they are sent to a third scorer. Each day, the department reads behind a sample of essays to ensure hand-scorers are adhering to the criteria set by our teachers. Any scores that do not align are thrown out, and those scorers are retrained. Any scorer who does not meet our and Questar’s standards is released from scoring TNReady.”

Finally, readers expressed a lot of concern about the complexity behind growth scores known as TVAAS, which are based on TNReady results and which go into teachers’ evaluations. We asked the state for a simple explanation.

  • “What formula is used in calculating the overall score for TVAAS when fallacies were determined as a result? My performance is weighed heavily on the state TVAAS score which is why this type of error has occurred before. This is quite disturbing. Teachers work tirelessly to ensure student achievement is a success; however, testing to measure performance seems to not be working.” —Mildred Williams  
  • “No one can give me the formula for how my students’ scores are calculated to create my score in TVAAS. How is (t)hat transparency? Yet, I’m required, constantly, to “prove” myself with documentation of education, observations, professional development and the like; all in originals, of course, to numerous overseeing bodies.” —Rachel Bernstein Kannady
  • “I find it ludicrous that data from these tests are used to evaluate MY performance when I get little to no control over most of the variables regarding the test. How could a miscalculated, misinformed, and (for all I know) incomprehensible test demonstrate what my students have learned!? And don’t even get me started on that fact that the rigor of the tests was increased ten-fold, yet not scaffolded in.” —Nicole Mayfield

TDOE: “TVAAS is statistically valid and reliable, and we follow the recommendations outlined by the American Educational Research Association (AERA) on value-added measures. Conceptually, TVAAS looks at how students have performed historically on TCAP and TNReady and compares their performance to their peers who have had similar past performance. If students tended to grow at about the same rate as their peers across the state — the expected amount of growth — they would earn a 3. If students tended to grow faster than their peers, they would earn a 4 or a 5, depending on the amount of progress they showed. If they tended to not show as much growth as their peers, they would earn a 1 or a 2. The model itself is sophisticated and complex to be as fair and nuanced as possible for each teacher’s situation, and we are working with our educator preparation providers as well as district leaders to provide more training on specifically how the model calculates scores. Tennessee educators also have access to a TVAAS user support team that can answer any specific questions about their TVAAS data, including how the data was analyzed.

Because TVAAS always looks at relative growth from year to year, not absolute test scores, it can be stable through transitions — and that is what we saw this year. Students can still grow, even if their overall proficiency level is now different. You can think about it like a running race. If you used to finish a 5K at about the same time as 10 other students, and all 10 students made the same shift to a new race at the same time with the same amount of time to prepare, you should finish the new race at about the same time. If you finished ahead of the group’s average time, you grew faster than your peers. If you lagged behind everyone, that would indicate you did not grow as much as was expected.  Because students’ performance will be compared to the performance of their peers and because their peers are making the transition at the same time, drops in statewide proficiency rates resulting from increased rigor of the new assessments had no impact on the ability of teachers, schools, and districts to earn strong TVAAS scores. Transitions to higher standards and expectations do not change the fact that we still want all students in a district to make a full year’s worth of growth, relative to their peers who are all experiencing the same transition.”

Reporter Laura Faith Kebede contributed to this report.

History alive

Inspired by Hamilton, Colorado students perform their own raps and poems on the big stage

PHOTO: Photo by Hyoung Chang/The Denver Post
From left, West Leadership Academy's Alexandra Andazola Chavez, Jose Torres Andazola, Rossy Martinez Sanchez, and Zehydi Chaparro Rojas perform "The Story of Peggy."

The plush red seats at the Wednesday matinee of Hamilton in Denver were filled with 2,700 teenagers who’d spent weeks studying a special curriculum about the hip-hop musical’s namesake, Alexander Hamilton, and the other Founding Fathers. Even though the show’s four-week Denver run had been sold out for months, the teenagers were seeing it for free.

Some of them had dressed for the occasion in high-heeled boots and three-piece suits. Others wore jeans and Converse. They represented 38 Colorado high schools that serve high proportions of students from low-income families, and many of them were students of color.

That’s notable because most of the cast of Hamilton are actors of color. Hamilton, George Washington, and Thomas Jefferson are played by black and Latino actors, a decision creator Lin-Manuel Miranda has said reflects America’s racial makeup and is meant to pull the audience into the story of an immigrant, Hamilton, who played an important role in the nation’s founding.

Before the show, 23 students took the stage to perform their own spoken word poems, raps, monologues, and scenes inspired by what they’d learned from the Hamilton Education Program curriculum, which was devised in part by Miranda and has its own hashtag: #EduHam.

“My body felt electrified,” said Josiah Blackbear, a 15-year-old sophomore at West Early College in Denver, who performed a rap he’d written about Alexander Hamilton. “The words I was speaking brought power and truth to the rest of the venue.”

Here is video of six of the student performances, including one entirely in Spanish.


During Memphis visit, former Newark schools chief touts ways to change student discipline

PHOTO: Governor's Office/Tim Larsen
Cami Anderson when she was superintendent of schools in Newark, New Jersey.

As the top schools chief in Newark, Cami Anderson was horrified at the strict discipline policy she saw in one of her high schools. Since then, she has left the New Jersey district and taken her ideas on the road about reducing suspensions and moving away from exclusionary discipline practices.

This week, Anderson came to Memphis as part of her Discipline Revolution Project at the invitation of Stand for Children Tennessee, The Community Foundation of Greater Memphis, School Seed, and Shelby County Schools. The New Teacher Project is partnering with her on the national tour.

Anderson has been meeting with Shelby County Schools administrators and board members as well as charter school leaders, philanthropists, education advocates, and students. Her time will culminate in a public event hosted by Stand for Children on Thursday at the National Civil Rights Museum.

Chalkbeat sat down with Anderson after she explained to a group of about 40 charter leaders her six focus areas to reduce classroom disruptions while also preventing sending students home when they’re in trouble. (This interview was edited for clarity and brevity.)

Related story: Tennessee students more likely to be suspended if they’re black boys — or live in Memphis

Question: How did you land on student discipline as an area you wanted to focus on?

Answer: If there’s actually a thread in my career, it’s this. I essentially ran the system of supports for the kids in New York City who are on their last stop on the train, so to speak. I’ve always worked with kids who are marginalized, the ones who really struggled in school. So, I’ve spent a lot of time thinking about what we need to get better at collectively to serve all kids, to really embrace the “all means all.” That’s been my lifelong question.

The three areas to me where inequities are most obvious are: enrollment policies, how we handle discipline, and mobility and how a kid stays connected to school. Discipline is where it comes to a head. It’s both a place where our collective inability to reach all kids shows up and it’s also an opportunity if we actually figure out how to prevent young people from misstepping in the first place, but then respond in healthy ways when they do Then we’d actually start to solve the broader equity issues.

Q. School leaders say they don’t want to have a lot of suspensions because students miss out on class. But they’re also not sure what to replace suspensions with to manage student behavior well. What would you say to them?

"You can’t just be against something. You have to be for something."Cami Anderson

A. That’s one of the main reasons we started Discipline Revolution Project. We don’t want you to do X, whatever X is: suspend kids, use corporal punishment. But educators are saying, rightfully so, then what are we doing? Our whole framework is trying to answer that question and give them tools to get to the “why” behind finding alternative responses.

Most people who use punitive or exclusionary discipline don’t actually think it works that well. They just don’t have a lot of other tools. So, when you give folks a lot of other tools and they find that it works, it’s a very powerful thing. When people try out a restorative conference, they say “Oh, I feel better. The kid feels better. And we actually got back to the lesson faster.” You can’t just be against something. You have to be for something.

I’ve heard a lot of demand for basics of restorative practices (conflict resolution between students and students and teachers), though I don’t think they should stop there. They want training for student support teams. And overwhelmingly, the places I’ve been want to talk about how teacher bias plays into who gets disciplined, but they don’t know how to start the conversation and for it to be productive.

Q. Memphis’ two school districts have emphasized a bottom-up approach on discipline reform: adding behavior specialists, school counselors, soliciting support from principals and teachers. How have you seen other districts do it?

"Policy-level change is critical but insufficient if you only do that."Cami Anderson

A. I’ve seen districts lead with policy and only make statements declaring they will cut suspensions in half or put a moratorium on suspensions or rewrite their policy. Policy-level change is critical but insufficient if you only do that. What you see is folks who are actually on the ground working with students may not have the strategies to replace it with something productive. That causes people to be more entrenched in their views that discipline reform wouldn’t work, some schools subtly pushing kids out, underreporting discipline data, all that.

I’ve also seen the opposite where it’s all about professional development and capacity but at no point is there is any accountability for those same schools, for example, that suspend 90 percent of the kids. People watch what you do, not what you say. If you don’t align your policies and your actions with your values, then you also have limits to the impacts you have for kids.

Frustrated with high suspension rates, Memphis schools shift to restorative justice

I’ve come to believe you need all of it and you need everyone working together. Stop admiring the problem and get on to the solutions.

Q. What pushed student discipline practices more widely into the national conversation? What have you observed from the conversation here in Memphis?

A. People are looking at data, which is a good thing, and seeing patterns like everyone else. Another thing is I believe a lot of people who got into education reform are completely dedicated to equity. And now they’re seeing this side of it, and like someone said in the training today, they feel a sense of “healthy guilt.” I think it’s great they’re having the courage to be honest. And then a lot of folks had kids. You start thinking, “Do I want any of that happening to my own kid?” I’m personally heartened and encouraged and motivated to see a collective sense of responsibility and focus on this.

There’s a lot of energy and candor in Memphis about this issue. Some other cities I’ve been in think they have it figured out when they don’t. When there’s that much energy, I think anywhere — including in Memphis — people can be tempted to devolve into the blame game, no matter what district or charter hat you wear. That energy can be the greatest asset or greatest liability.

Study: When Chicago cut down on suspensions, students saw test scores and attendance rise

Q. The school shooting in Parkland has been a catalyst for more conversations about the trauma students bring into the classroom — conversations that were already happening about violence in low-income communities of color. What would you say to school leaders on how to address that?

A. I’m most interested to know what adults can do to mitigate those risk factors for young people who experience trauma. I feel like it could take us down a very bad path to just observe that there are things called “adverse childhood experiences.” To me, that’s not enough. The question then is what are the environments and strategies that we can put in place as educators and adults to mitigate the impact of those traumatic experiences. Things like relationships, trust, consistency, high expectations, high supports, and support healthy identity development especially in times of conflict. We know from research that young people who face long odds who ultimately prevail, they are exposed to environments that really embody those things.

You can both be aware of and acknowledge those experiences that make it harder for them to succeed in school. But if you stop there, I don’t think you’re doing justice to young people. There are things we can do in schools to help create the environment to help them succeed.