To our readers

Hey, we heard you. You had a lot of questions about TNReady. We found answers.

The news that Tennessee’s testing company scored some high school tests incorrectly this year uncorked a flood of questions about the validity of the state’s new standardized assessment.


Here are five things to know about the latest brouhaha over TNReady


We wanted to know how the brouhaha was impacting classrooms, so we asked our readers on Facebook.

You responded in droves.

We took your top concerns directly to the state Department of Education and asked for answers. Here’s what you wanted to know — and what we have learned:

Several readers asked why they should trust TNReady results, given the series of setbacks in the test’s first two years.

  • “I do not trust the results. We have had so many problems in the last few years, that I am suspicious of any results we do get. It bothers me greatly that the state uses these numbers to hold students and teachers and districts accountable, but they seem to be unable to deliver scores they believe are accurate in a timely manner.” —Rebecca Dickenson
  • “I no longer trust the accountability of the state nor its methods. My concern is if there is a teacher who has only one year of test data, how is it the same teacher shown multi-year growth when he or she had only last year of testing? This poses a huge concern.” —Mildred Williams  

Tennessee Department of Education: “TNReady is fully aligned to Tennessee’s academic standards, and every question has been reviewed, edited, and approved by Tennessee teachers through a rigorous review process. We also have quantitative checks and processes after a test is over to ensure student responses are reliable. While more than 99.9% of TNReady tests were scored accurately this year, we want to improve on that next year, and our vendor (Questar) is taking new quality assurance steps to make sure their programming is error-free. Also, this year, as soon as the scoring error on some of the English I, II and Integrated Math II EOCs was identified, scores were updated and all TNReady tests were re-reviewed and verified for full accuracy.”

Some teachers told us that, given the delay in score deliveries this spring, many students don’t think the results will arrive in time to affect their final grades next spring. Those teachers are struggling to get their students to buy in.

  • “After two years of TNReady, it still hasn’t counted for my students. Going into year three, I will once again tell them with a hopeful, straight face that it will count as part of their report card grades and implore them to try their best. I quietly wonder what reason they have to believe me, given recent history.” —Mike Stein
  • “I struggle to get students to buy in to the importance of trying their best on state tests because the students are confident that the scores won’t come back in time to affect their grades (which has been the situation for several years now). The students see zero incentive for doing well.” —Nicole Mayfield

TDOE: “We believe that if districts and schools set the tone that performing your best on TNReady is important, then students will take the test seriously, regardless of whether TNReady factors into their grade. We should be able to expect our students will try and do their best at any academic exercise, whether or not it is graded. This is a value that is established through local communication from educators and leaders, and it will always be key to our test administration. We believe that when we share these messages and values celebrating the variety of accomplishments our students have made, taking advantage of TNReady’s scheduling flexibility to minimize disruption, focusing on strong standards-based instruction every day, sending positive messages around the importance of the variety of tests that students take, and sharing that students should always do their best then students will buy-in and TNReady will be successful.”

Other teachers asked what happens to writing scores for tests in English language arts.

  • “I can tell you that two years ago — when we first piloted the new writing test online — districts received not only every student’s scores (broken down by each of the four indicators) but also the actual student responses to each prompt. In my former district our supervisor shared them, and we analyzed them as a department. If you check with your principal, VP, or supervisors, there are some published “anchor papers” with scores available on edtools from this past year. It’s not a lot, but it’s more than we’ve had in the past. My hope is that if online continues, we’ll keep seeing the student responses in the future.” —Wj Gillespie II

TDOE: “The question appears to be referencing the process we had through the 2014-15 school year, when our writing assessment was separate. Since 2015-16, students’ writing responses on TNReady have been incorporated as part of their overall ELA score. Responses are scored based on our writing rubrics, and for educators, we have provided access to the “anchor papers” from the 2016-17 year, so they can see how students’ responses were scored based on the writing rubric, which can help them inform the feedback they give their students.”

On that same issue of writing scores, one teacher referenced the hiring of scorers off of Craigslist. We asked the state if that’s true.

  • “I continue to be curious about our ELA writing scores. Each year we are required to use state writing rubrics, attend PD related to the state’s four types of writing, etc etc…and yet our scores never come back. Students spend hours taking the writing portion of the test, scorers are hired off Craig’s list…, and yet we never actually get the scores back. It seems like every year this is swept under the rug. Where do these writing tests go?” —Elizabeth Faison Clifton

TDOE: “Questar does not use Craigslist. Several years ago, another assessment company supposedly posted advertisements on Craigslist, but Questar does not. We provide opportunities for our educators to be involved in developing our test, and we also encourage Tennessee teachers to apply to hand-score TNReady. To be eligible, each applicant must provide proof of a four-year college degree, and preference is given to classroom teachers. As part of the interview process, an applicant would have to hand-score several items for review and evaluation. Once hired, each scorer is trained based on materials that Tennessee teachers and the department approve — and which are assembled from responses given by Tennessee students on the exam — and scorers are regularly refreshed and “recalibrated” on scoring guidelines. Each writing response is scored at least twice; if those two responses differ significantly, they are sent to a third scorer. Each day, the department reads behind a sample of essays to ensure hand-scorers are adhering to the criteria set by our teachers. Any scores that do not align are thrown out, and those scorers are retrained. Any scorer who does not meet our and Questar’s standards is released from scoring TNReady.”

Finally, readers expressed a lot of concern about the complexity behind growth scores known as TVAAS, which are based on TNReady results and which go into teachers’ evaluations. We asked the state for a simple explanation.

  • “What formula is used in calculating the overall score for TVAAS when fallacies were determined as a result? My performance is weighed heavily on the state TVAAS score which is why this type of error has occurred before. This is quite disturbing. Teachers work tirelessly to ensure student achievement is a success; however, testing to measure performance seems to not be working.” —Mildred Williams  
  • “No one can give me the formula for how my students’ scores are calculated to create my score in TVAAS. How is (t)hat transparency? Yet, I’m required, constantly, to “prove” myself with documentation of education, observations, professional development and the like; all in originals, of course, to numerous overseeing bodies.” —Rachel Bernstein Kannady
  • “I find it ludicrous that data from these tests are used to evaluate MY performance when I get little to no control over most of the variables regarding the test. How could a miscalculated, misinformed, and (for all I know) incomprehensible test demonstrate what my students have learned!? And don’t even get me started on that fact that the rigor of the tests was increased ten-fold, yet not scaffolded in.” —Nicole Mayfield

TDOE: “TVAAS is statistically valid and reliable, and we follow the recommendations outlined by the American Educational Research Association (AERA) on value-added measures. Conceptually, TVAAS looks at how students have performed historically on TCAP and TNReady and compares their performance to their peers who have had similar past performance. If students tended to grow at about the same rate as their peers across the state — the expected amount of growth — they would earn a 3. If students tended to grow faster than their peers, they would earn a 4 or a 5, depending on the amount of progress they showed. If they tended to not show as much growth as their peers, they would earn a 1 or a 2. The model itself is sophisticated and complex to be as fair and nuanced as possible for each teacher’s situation, and we are working with our educator preparation providers as well as district leaders to provide more training on specifically how the model calculates scores. Tennessee educators also have access to a TVAAS user support team that can answer any specific questions about their TVAAS data, including how the data was analyzed.

Because TVAAS always looks at relative growth from year to year, not absolute test scores, it can be stable through transitions — and that is what we saw this year. Students can still grow, even if their overall proficiency level is now different. You can think about it like a running race. If you used to finish a 5K at about the same time as 10 other students, and all 10 students made the same shift to a new race at the same time with the same amount of time to prepare, you should finish the new race at about the same time. If you finished ahead of the group’s average time, you grew faster than your peers. If you lagged behind everyone, that would indicate you did not grow as much as was expected.  Because students’ performance will be compared to the performance of their peers and because their peers are making the transition at the same time, drops in statewide proficiency rates resulting from increased rigor of the new assessments had no impact on the ability of teachers, schools, and districts to earn strong TVAAS scores. Transitions to higher standards and expectations do not change the fact that we still want all students in a district to make a full year’s worth of growth, relative to their peers who are all experiencing the same transition.”

Reporter Laura Faith Kebede contributed to this report.

Academic Accountability

Coming soon: Not one, but two ratings for every Chicago school

Starting this month, Chicago schools will have to juggle two ratings — one from the school district, and another from the state.

The Illinois State Board of Education is scheduled to release on October 31 its annual report cards for schools across the state. This year, for the first time, each school will receive one of four quality stamps from the state: an “exemplary” or “commendable” rating signal the school is meeting standards while an “underperforming” or “lowest performing” designation could trigger intervention, according to state board of education spokeswoman Jackie Matthews.

A federal accountability law, the Every Student Succeeds Act, requires these new ratings.

To complicate matters, the city and state ratings are each based on different underlying metrics and even a different set of standardized tests. The state ratings, for example, are based on a modified version of the PARCC assessment, while Chicago ratings are based largely on the NWEA. The new state ratings, like those the school district issues, can be given out without observers ever having visited a classroom, which is why critics argue that the approach lacks the qualitative metrics necessary to assess the learning, teaching, and leadership at individual schools.

Patricia Brekke, principal at Back of the Yards College Preparatory High School, said she’s still waiting to see how the ratings will be used, “and how that matters for us,” but that parents at her school aren’t necessarily focused on what the state says.

“What our parents usually want to know is what [Chicago Public Schools] says about us, and how we’re doing in comparison to other schools nearby that their children are interested in,” she said.

Educators at Chicago Public Schools understand the power of school quality ratings.  The district already has its own five-tiered rating system: Level 1+ and Level 1 designate the highest performing schools, Level 2+ and Level 2 describe for average and below average performing schools, respectively, and Level 3, the lowest performance rating, is for schools in need of “intensive intervention.” The ratings help parents decide where to enroll their children, and are supposed to signal to the district that the school needs more support. But the ratings are also the source of angst — used to justify replacing school leaders, closing schools, or opening new schools in neighborhoods where options are deemed inadequate.

In contrast, the state’s school quality designations actually target underperforming and lowest-performing schools with additional federal funding and support with the goal of improving student outcomes. Matthews said schools will work with “school support managers” from the state to do a self-inquiry and identify areas for improvement. She described Chicago’s school quality rating system as “a local dashboard that they have developed to communicate with their communities.”

Staff from the Illinois State Board of Education will be traveling around the state next week to meet with district leaders and principals to discuss the new accountability system, including the ratings. They’ll be in Bloomington, Marion, O’Fallon, Chicago, and Melrose Park. The Chicago meeting is Wednesday, Oct. 24, at 5 p.m. at Chicago Public Schools headquarters.

Rae Clementz, director of assessment and accountability at the state board said that a second set of ratings reveals “that there are multiple valid ways to look at school quality and success; it’s a richer picture.”

Under auspices of the Every Student Succeeds Act, the state school report cards released at the end of the month for elementary schools are 75 percent based on academics, including English language arts and math test scores, English learner progress as measured by the ACCESS test, and academic growth. The other 25 percent reflects the school climate and success, such as attendance and chronic absenteeism.

Other measures are slated to be phased in over the next several years, including academic indicators like science proficiency and school quality indicators, such as school climate surveys of staff, students and parents

High school designations take a similar approach with English and math test scores but will take into account graduation rates, instead of academic growth, and also includes the percentage of  9th graders on track to graduate — that is freshmen who earn 10 semester credits, and no more than one semester F in a core course.

Critics of Chicago’s school rating system argue that the ratings correlate more with socioeconomic status and race than they do school quality, and say little about what’s happening in classrooms and how kids are learning. Chicago does try to mitigate these issues with a greater emphasis on growth in test scores rather than absolute attainment, school climate surveys, and including academic growth by priority groups, like African-American, Latino, ELL, and students in special education.

Cory Koedel, a professor of economics and public policy at the University of Missouri, said that many rating systems basically capture poverty status with a focus on how high or low students score on tests. Chicago’s approach is fairer than that of many other school systems.

“What I like about this is it does seem to have a high weight on growth and lower weight on attainment levels,” he said.

Morgan Polikoff, a professor at University of Southern California’s school of education, said that Chicago’s emphasis on student growth is a good thing “if the purpose of the system is to identify schools doing a good job educating kids.”

Chicago weights 50 percent of the rating on growth, but he’s seen 35 to as low as 15 percent at other districts. But he said the school district’s reliance on the NWEA test rather than the PARCC test used in the state school ratings was atypical.

“It’s not a state test, and though they say it aligns with standards, I know from talking to educators that a lot of them feel the tests are not well aligned with what they are supposed to be teaching,” he said. “It’s just a little odd to me they would have state assessment data, which is what they are held accountable for with the state, but use the other data.”

He’s skeptical about school systems relying too heavily on standardized test scores, whether the SAT, PARCC or NWEA, because “You worry that now you’re just turning the curriculum to test prep, and that’s an incentive you don’t want to create for educators.”

He said the high school measures in particular include a wide array of measures, including measures that follow students into college, “so I love that.”

“I really like the idea of broadening the set of indicators on which we evaluate schools and encouraging schools to really pay attention to how well they prepare students for what comes next,” he said.

First Person

How football prepared me for my first year of teaching (but maybe not the second)

Football brought me to Memphis, and Memphis brought me to teaching.

That’s how, last August, I found myself the solo teacher for seventh grade science at a KIPP middle school in North Memphis that hadn’t had a teacher in that role make it to May in four years.

I completed and even enjoyed that year of teaching, despite its challenges. And while I don’t think my years of high school and college football gave me every tool or personality trait I needed to do that, the experience helped.

First, football taught me to perform when I was not at 100 percent. One of my former coaches used to ask ailing players, “Are you hurt, or are you injured?” in an attempt to parse the words of high schoolers. Hurt was a bruise; injured was a break. I learned to play with bruises.

I found myself asking the hurt or injured question one early morning in February, when I woke up with a throbbing headache. I was hurt, not injured. I made it in.

But physical ailments aren’t the only ones that can sideline a teacher. Teachers have bad days. Frankly, teachers can have bad weeks or months. The same can go for football players. All-star quarterbacks throw interceptions, and gutsy linebackers miss tackles.

The same coach used to tell me, “The only play that matters is the next play.” I found that true last year, too. I couldn’t go back and change the way I unduly reprimanded a student any more than a wide receiver can get another shot at catching a dropped pass.

Some days, though, you “learn” more than you bargained for. In football, those days may be when you feel like you probably should have never tried to play. Those days you drop every ball that comes your way, you forget where you’re supposed to be on every play, and you wonder if the knitting club has any openings.

Football taught me how to drown out these thoughts of inadequacy with positive visualization and by staying focused on concrete goals. As my coach used to tell us after a particularly good play, or a particularly bad one: “Never too high, never too low.” Just as the bad days will soon be washed away in the unrelenting tide of the school year, so will the good ones.

Retaining any sense of perspective on the school year was hard, and there’s no easy fix to an extended period of self-pity or frustration at a string of bad days. My goals were to help kids learn to appreciate science, and to be an adult that students felt they could go to for support. Keeping them at the front of my mind was the best help I could find.

On that note, I have a confession to make. Before my first year of teaching, I was one of those people who didn’t truly understand how difficult teaching was. The reality of how many hours teachers spend outside of school putting their lessons together never crossed my mind. The fact that planning units ahead for my students felt like scouting out my opponents didn’t make the long hours any easier. That first month of teaching was a shock to my system, and the only solution was to put my head down and go, the way I had been taught to do.

Football also left me with some loose ends. The sport taught me next to nothing about patience or about the virtues of benevolence; it never pays to be gentle on the gridiron. Football also didn’t teach me anything about working with people you don’t agree with. On a football team, everyone is united under the same cause: winning.

The parallels I discovered also raise a few uncomfortable questions. I decided to pursue an advanced degree instead of continuing to teach a second year. Does football truly inform teaching as a career, then, or just that first year? A main tenet of football is to never quit. Did I violate that by switching career paths?

Pushing past pain, and centering most hours of one’s life around one goal, can be difficult principles to build a life around. They were also valuable to me when I needed them most.

And regardless of whether football continues to be popular among young people, I hope that parents still find ways to give their kids a chance to compete — a chance to win, and more importantly, to lose.

Having to do that time and time again made me able to accept struggle in life, and it made me a better learner. I think it made me a better teacher, too.

Evan Tucker is a former teacher at KIPP Memphis Academy Middle. He is now pursuing a master’s degree in ecology.