To our readers

Hey, we heard you. You had a lot of questions about TNReady. We found answers.

The news that Tennessee’s testing company scored some high school tests incorrectly this year uncorked a flood of questions about the validity of the state’s new standardized assessment.


Here are five things to know about the latest brouhaha over TNReady


We wanted to know how the brouhaha was impacting classrooms, so we asked our readers on Facebook.

You responded in droves.

We took your top concerns directly to the state Department of Education and asked for answers. Here’s what you wanted to know — and what we have learned:

Several readers asked why they should trust TNReady results, given the series of setbacks in the test’s first two years.

  • “I do not trust the results. We have had so many problems in the last few years, that I am suspicious of any results we do get. It bothers me greatly that the state uses these numbers to hold students and teachers and districts accountable, but they seem to be unable to deliver scores they believe are accurate in a timely manner.” —Rebecca Dickenson
  • “I no longer trust the accountability of the state nor its methods. My concern is if there is a teacher who has only one year of test data, how is it the same teacher shown multi-year growth when he or she had only last year of testing? This poses a huge concern.” —Mildred Williams  

Tennessee Department of Education: “TNReady is fully aligned to Tennessee’s academic standards, and every question has been reviewed, edited, and approved by Tennessee teachers through a rigorous review process. We also have quantitative checks and processes after a test is over to ensure student responses are reliable. While more than 99.9% of TNReady tests were scored accurately this year, we want to improve on that next year, and our vendor (Questar) is taking new quality assurance steps to make sure their programming is error-free. Also, this year, as soon as the scoring error on some of the English I, II and Integrated Math II EOCs was identified, scores were updated and all TNReady tests were re-reviewed and verified for full accuracy.”

Some teachers told us that, given the delay in score deliveries this spring, many students don’t think the results will arrive in time to affect their final grades next spring. Those teachers are struggling to get their students to buy in.

  • “After two years of TNReady, it still hasn’t counted for my students. Going into year three, I will once again tell them with a hopeful, straight face that it will count as part of their report card grades and implore them to try their best. I quietly wonder what reason they have to believe me, given recent history.” —Mike Stein
  • “I struggle to get students to buy in to the importance of trying their best on state tests because the students are confident that the scores won’t come back in time to affect their grades (which has been the situation for several years now). The students see zero incentive for doing well.” —Nicole Mayfield

TDOE: “We believe that if districts and schools set the tone that performing your best on TNReady is important, then students will take the test seriously, regardless of whether TNReady factors into their grade. We should be able to expect our students will try and do their best at any academic exercise, whether or not it is graded. This is a value that is established through local communication from educators and leaders, and it will always be key to our test administration. We believe that when we share these messages and values celebrating the variety of accomplishments our students have made, taking advantage of TNReady’s scheduling flexibility to minimize disruption, focusing on strong standards-based instruction every day, sending positive messages around the importance of the variety of tests that students take, and sharing that students should always do their best then students will buy-in and TNReady will be successful.”

Other teachers asked what happens to writing scores for tests in English language arts.

  • “I can tell you that two years ago — when we first piloted the new writing test online — districts received not only every student’s scores (broken down by each of the four indicators) but also the actual student responses to each prompt. In my former district our supervisor shared them, and we analyzed them as a department. If you check with your principal, VP, or supervisors, there are some published “anchor papers” with scores available on edtools from this past year. It’s not a lot, but it’s more than we’ve had in the past. My hope is that if online continues, we’ll keep seeing the student responses in the future.” —Wj Gillespie II

TDOE: “The question appears to be referencing the process we had through the 2014-15 school year, when our writing assessment was separate. Since 2015-16, students’ writing responses on TNReady have been incorporated as part of their overall ELA score. Responses are scored based on our writing rubrics, and for educators, we have provided access to the “anchor papers” from the 2016-17 year, so they can see how students’ responses were scored based on the writing rubric, which can help them inform the feedback they give their students.”

On that same issue of writing scores, one teacher referenced the hiring of scorers off of Craigslist. We asked the state if that’s true.

  • “I continue to be curious about our ELA writing scores. Each year we are required to use state writing rubrics, attend PD related to the state’s four types of writing, etc etc…and yet our scores never come back. Students spend hours taking the writing portion of the test, scorers are hired off Craig’s list…, and yet we never actually get the scores back. It seems like every year this is swept under the rug. Where do these writing tests go?” —Elizabeth Faison Clifton

TDOE: “Questar does not use Craigslist. Several years ago, another assessment company supposedly posted advertisements on Craigslist, but Questar does not. We provide opportunities for our educators to be involved in developing our test, and we also encourage Tennessee teachers to apply to hand-score TNReady. To be eligible, each applicant must provide proof of a four-year college degree, and preference is given to classroom teachers. As part of the interview process, an applicant would have to hand-score several items for review and evaluation. Once hired, each scorer is trained based on materials that Tennessee teachers and the department approve — and which are assembled from responses given by Tennessee students on the exam — and scorers are regularly refreshed and “recalibrated” on scoring guidelines. Each writing response is scored at least twice; if those two responses differ significantly, they are sent to a third scorer. Each day, the department reads behind a sample of essays to ensure hand-scorers are adhering to the criteria set by our teachers. Any scores that do not align are thrown out, and those scorers are retrained. Any scorer who does not meet our and Questar’s standards is released from scoring TNReady.”

Finally, readers expressed a lot of concern about the complexity behind growth scores known as TVAAS, which are based on TNReady results and which go into teachers’ evaluations. We asked the state for a simple explanation.

  • “What formula is used in calculating the overall score for TVAAS when fallacies were determined as a result? My performance is weighed heavily on the state TVAAS score which is why this type of error has occurred before. This is quite disturbing. Teachers work tirelessly to ensure student achievement is a success; however, testing to measure performance seems to not be working.” —Mildred Williams  
  • “No one can give me the formula for how my students’ scores are calculated to create my score in TVAAS. How is (t)hat transparency? Yet, I’m required, constantly, to “prove” myself with documentation of education, observations, professional development and the like; all in originals, of course, to numerous overseeing bodies.” —Rachel Bernstein Kannady
  • “I find it ludicrous that data from these tests are used to evaluate MY performance when I get little to no control over most of the variables regarding the test. How could a miscalculated, misinformed, and (for all I know) incomprehensible test demonstrate what my students have learned!? And don’t even get me started on that fact that the rigor of the tests was increased ten-fold, yet not scaffolded in.” —Nicole Mayfield

TDOE: “TVAAS is statistically valid and reliable, and we follow the recommendations outlined by the American Educational Research Association (AERA) on value-added measures. Conceptually, TVAAS looks at how students have performed historically on TCAP and TNReady and compares their performance to their peers who have had similar past performance. If students tended to grow at about the same rate as their peers across the state — the expected amount of growth — they would earn a 3. If students tended to grow faster than their peers, they would earn a 4 or a 5, depending on the amount of progress they showed. If they tended to not show as much growth as their peers, they would earn a 1 or a 2. The model itself is sophisticated and complex to be as fair and nuanced as possible for each teacher’s situation, and we are working with our educator preparation providers as well as district leaders to provide more training on specifically how the model calculates scores. Tennessee educators also have access to a TVAAS user support team that can answer any specific questions about their TVAAS data, including how the data was analyzed.

Because TVAAS always looks at relative growth from year to year, not absolute test scores, it can be stable through transitions — and that is what we saw this year. Students can still grow, even if their overall proficiency level is now different. You can think about it like a running race. If you used to finish a 5K at about the same time as 10 other students, and all 10 students made the same shift to a new race at the same time with the same amount of time to prepare, you should finish the new race at about the same time. If you finished ahead of the group’s average time, you grew faster than your peers. If you lagged behind everyone, that would indicate you did not grow as much as was expected.  Because students’ performance will be compared to the performance of their peers and because their peers are making the transition at the same time, drops in statewide proficiency rates resulting from increased rigor of the new assessments had no impact on the ability of teachers, schools, and districts to earn strong TVAAS scores. Transitions to higher standards and expectations do not change the fact that we still want all students in a district to make a full year’s worth of growth, relative to their peers who are all experiencing the same transition.”

Reporter Laura Faith Kebede contributed to this report.

First Person

Why the phrase ‘with fidelity’ is an affront to good teaching

PHOTO: Alan Petersime

“With fidelity” are some of the most damaging words in education.

Districts spend a ton of money paying people to pick out massively expensive, packaged curriculums, as if every one of a thousand classrooms needs the exact same things. Then officials say, over and over again, that they must be implemented “with fidelity.” What they mean is that teachers better not do anything that would serve their students’ specific needs.

When that curriculum does nothing to increase student achievement, it is not blamed. The district person who found it and purchased it is never blamed. Nope. They say, “Well, the teachers must not have been implementing it with fidelity.”

It keeps happening because admitting that schools are messy and students are human and teaching is both creative and artistic would also mean you have to trust teachers and let them have some power. Also, there are some really crappy teachers out there, and programs for everyone are often meant to push that worst-case-scenario line a little higher.

And if everyone’s doing just what they’re supposed to, we’ll get such good, clean numbers, and isn’t that worth a few thousand more dollars?

I was talking with a friend recently, a teacher at an urban school on the East Coast. He had been called to task by his principal for splitting his kids into groups to offer differentiated math instruction based on students’ needs. “But,” the principal said, “did the pacing guide say to differentiate? You need to trust the system.”

I understand the desire to find out if a curriculum “works.” But I don’t trust anyone who can say “trust the system” without vomiting. Not when the system is so much worse than anything teachers would put together.

Last year, my old district implemented Reading Plus, an online reading program that forces students to read at a pace determined by their scores. The trainers promised, literally promised us, that there wasn’t a single reading selection anywhere in the program that could be considered offensive to anyone. God knows I never learned anything from a book that made me feel uncomfortable!

Oh, and students were supposed to use this program — forced-paced reading of benign material followed by multiple-choice questions and more forced-pace reading — for 90 minutes a week. We heard a lot about fidelity when the program did almost nothing for students (and, I believe quite strongly, did far worse than encouraging independent reading of high-interest books for 90 minutes a week would have done).

At the end of that year, I was handed copies of next year’s great adventure in fidelity. I’m not in that district any longer, but the whole district was all switching over to SpringBoard, another curriculum, in language arts classes. On came the emails about implementing with fidelity and getting everyone on the same page. We were promised flexibility, you know, so long as we also stuck to the pacing guide of the workbook.

I gave it a look, I did, because only idiots turn down potential tools. But man, it seemed custom-built to keep thinking — especially any creative, critical thought from either students or teachers — to a bare minimum.

I just got an email from two students from last year. They said hi, told me they missed creative writing class, and said they hated SpringBoard, the “evil twin of Reading Plus.”

That district ran out of money and had to cut teachers (including me) at the end of the year. But if they hadn’t, I don’t think I would have lasted long if forced to teach from a pacing guide. I’m a good teacher. Good teachers love to be challenged and supported. They take feedback well, but man do we hate mandates for stuff we know isn’t best for the kids in our room.

Because, from inside a classroom full of dynamic, chaotic brilliance;

from a classroom where that kid just shared that thing that broke all of our hearts;

from a classroom where that other kid figured out that idea they’ve been working on for weeks;

from that classroom where that other kid, who doesn’t know enough of the language, hides how hard he works to keep up and still misses things;

and from that classroom where one kid isn’t sure if they trust you yet, and that other kid trusts you too much, too easily, because their bar had been set too low after years of teachers that didn’t care enough;

from inside that classroom, it’s impossible to trust that anyone else has a better idea than I do about what my students need to do for our next 50 minutes.

Tom Rademacher is a teacher living in Minneapolis who was named Minnesota’s Teacher of the Year in 2014. His book, “It Won’t Be Easy: An Exceedingly Honest (and Slightly Unprofessional) Love Letter to Teaching,” was published in April. He can be found on Twitter @mrtomrad and writes on misterrad.tumblr.com, where this post first appeared.

student teaching

Building a teacher pipeline: How one Aurora school has become a training ground for aspiring teachers

Paraprofessional Sonia Guzman, a student of a teaching program, works with students at Elkhart Elementary School in Aurora. (Photo by Yesenia Robles, Chalkbeat)

Students at Aurora’s Elkhart Elementary School are getting assistance from three aspiring teachers helping out in classrooms this year, part of a new partnership aimed at building a bigger and more diverse teacher pipeline.

The teachers-to-be, students at the University of Northern Colorado’s Center for Urban Education, get training and a paid job while they’re in college. Elkhart principal Ron Schumacher gets paraprofessionals with long-term goals and a possibility that they’ll be better prepared to be Aurora teachers.

For Schumacher, it’s part of a plan to not only help his school, but also others in Aurora Public Schools increase teacher retention.

“Because of the nature of our school demographics, it’s a coin flip with a new teacher,” Schumacher said. “If I lose 50 percent of my teachers over time, I’m being highly inefficient. If these ladies know what they’re getting into and I can have them prepared to be a more effective first-year teacher, there’s more likelihood that I’ll keep them in my school in the long term.”

Elkhart has about 590 students enrolled this year. According to state data from last year, more than 95 percent of the students who attend the school qualify for subsidized lunches, a measure of poverty. The school, which operates with an International Baccalaureate program, has outperformed the district average on some state tests.

The three paraprofessionals hired by the school this year are part of the teaching program at UNC’s Lowry campus, which has long required students to work in a school for the four years they work on their degree.

Students get paid for their work in schools, allowing them to earn some money while going to college. Students from the program had worked in Aurora schools in the past, but not usually three students at once at the same school, and not as part of a formal partnership.

The teaching program has a high number of students of color and first-generation college students, which Rosanne Fulton, the program director, said is another draw for partnering with schools in the metro area.

Schumacher said every principal and education leader has the responsibility to help expose students to more teachers who can relate to them.

One of this year’s paraprofessionals is Andy Washington, an 18-year-old who attended Elkhart for a few years when she was a child.

“Getting to know the kids on a personal level, I thought I was going to be scared, but they’re cool,” Washington said.

Another paraprofessional, 20-year-old Sonia Guzman, said kids are opening up to them.

“They ask you what college is like,” Guzman said.

Schumacher said there are challenges to hiring the students, including figuring out how to make use of the students during the morning or early afternoon while being able to release them before school is done for the day so they can make it to their college classes.

Schumacher said he and his district director are working to figure out the best ways to work around those problems so they can share lessons learned with other Aurora principals.

“We’re using some people differently and tapping into volunteers a little differently, but if it’s a priority for you, there are ways of accommodating their schedules,” he said.

At Elkhart, full-time interventionists work with students in kindergarten through third grade who need extra help learning to read.

But the school doesn’t have the budget to hire the same professionals to work with older students. The three student paraprofessionals are helping bridge that gap, learning from the interventionists so they can work with fourth and fifth grade students.

Recently, the three started getting groups of students that they pull out during class to give them extra work on reading skills.

One exercise they worked on with fourth grade students recently was helping them identify if words had an “oi” or “oy” spelling based on their sounds. Students sounded out their syllables and used flashcards to group similar words.

Districts across the country have looked at similar approaches to help attract and prepare teachers for their own schools. In Denver, bond money voters approved last year is helping pay to expand a program this year where paraprofessionals can apply for a one-year program to become teachers while they continue working.

In the partnership at Elkhart, students paraprofessionals take longer than that, but in their first and second year are already learning how to write lessons during their afternoon classes and then working with teachers at the school to deliver the lessons and then reflect on how well they worked. Students say the model helps them feel supported.

“It’s really helping me to become more confident,” said Stephanie Richards, 26, the third paraprofessional. “I know I’m a lot more prepared.”

Schumacher said the model could also work in the future with students from other teaching schools or programs. It’s a small but important part, he said, toward helping larger efforts to attract and retain teachers, and also diversify the ranks.

“You’re doing something for the next generation of folks coming in,” he said.