Are Children Learning

Is Indiana's state ISTEP exam too easy?

PHOTO: Shannan Muskopf via Flickr
State officials are closing as many 38 Michigan schools with low rankings due to test scores but they might have trouble finding higher scoring schools nearby

Indiana’s standardized tests might have gotten so much harder last year that test scores plunged across the state, but two national testing experts say the exam might technically still be too easy according to an earlier study.

“The Indiana test was relatively low-level,” said Ed Roeber, a former Michigan testing director asked to consult on ISTEP.

The 2015 ISTEP exam might have seemed tougher to students and teachers. The state saw a 20 percentage point drop in students passing both English and math last year.

But when Roeber and another testing expert, Derek Briggs, were asked by the Indiana State Board of Education to conduct a review of that exam in January, they discovered that exam questions were not as rigorous as they should have been. The state adopted new academic standards in 2014 that were designed to better prepare kids for college and careers. The new standards were said to be much tougher.

Try it out: 18 practice questions to prepare for Indiana’s 2016 ISTEP test.

Indiana officials brought on Roeber and Briggs on to advise them on the ISTEP last year in the wake of scoring glitches, delays and accuracy questions that raised concerns about the validity and reliability of exam. After examining an earlier study of the test by the education research group WestEd, the pair noted that Indiana’s test questions do accurately reflect the standards, but suggested that they might still be too easy.

The analysis from Roeber and Briggs reached no conclusions on whether the standards themselves are meeting the state’s goals, but the pair found that too many questions on last year’s ISTEP test were too easy.

Specifically, the experts cited the WestEd study that found a majority of the questions on the 2015 exam were fact-based questions that asked students to merely to recall information rather than do higher-level thinking. More than 80 percent of questions on the English exam and every single question on the math test were found to be low-level basic questions.

The test had very few more difficult questions that would have required students to describe or explain a scenario, use evidence to back up answers or test a hypothesis and make connections beyond the facts at hand.

“The fact that (a test question) aligns with the standards is good, but it doesn’t imply that it’s measuring the standards to the same depth the standard is written at,” Roeber said.

What remains unclear is why the questions were too easy. The fault could rest with the test, which was written hastily after lawmakers voted to drop the Common Core and write Indiana-specific standards in 2014. Test questions weren’t tried out on students until the test was actually given last spring — a highly unusual practice in a testing industry in which test questions are typically piloted over several years before being used on an exam, Roeber said.

The Indiana Department of Education decided against using questions from previous tests that could have saved some time, said Cynthia Roach, the state board’s testing director.

Another possible reason for the easy test questions is the state’s standards themselves. It’s not clear just how rigorous Indiana’s standards really are. Until the standards are properly evaluated using the same metrics as the test, Roeber said, the state’s policymakers can’t really know if ISTEP is measuring what kids are expected to learn better than past tests.

It might seem technical, but the mismatch between the standards and the questions could be important. Indiana still must submit its test for review by the U.S. Department of Education for its waiver from the No Child Left Behind law, still in effect until August. Any misalignment between the test and standards could be problematic.

“That’s one of the key things that the peer reviewer will look for is the measure of the rigor of the test, and if this misstates that, then you definitely have the right to challenge the vendor,” Roeber said.

The state’s new testing advisory committee is taking the suggestions from the analysis to try to move forward, Roach said. Nothing can be addressed in time for the 2016 test, which students have already started taking, but questions should be reassessed and Indiana should look into deeper measurement of its standards, Roeber said.

The Indiana Department of Education, which administers the test, declined to make its testing director available for an interview to discuss Roeber and Briggs’ analysis, but Department spokeswoman Samantha Hart issued a statement saying the test was hard enough.

“As with any assessment, there are going to be questions that are more and less rigorous than others — The new ISTEP+ exam is no different,” Hart wrote. “Anyone who thinks that last year’s ISTEP wasn’t hard enough should go talk to a student or a teacher or a parent. This test was clearly more rigorous, just like our standards.”

more digging

Kingsbury High added to list of Memphis schools under investigation for grade changing

PHOTO: Shelby County Schools
Kingsbury High School was added to a list of schools being investigated by an outside firm for improper grade changes. Here, Principal Terry Ross was featured in a Shelby County Schools video about a new school budget tool.

Another Memphis high school has been added to the list of schools being investigated to determine if they made improper changes to student grades.

Adding Kingsbury High School to seven others in Shelby County Schools will further delay the report initially expected to be released in mid-June.

But from what school board Chairwoman Shante Avant has heard so far, “there haven’t been any huge irregularities.”

“Nothing has surfaced that gives me pause at this point,” Avant told Chalkbeat on Thursday.

The accounting firm Dixon Hughes Goodman is conducting the investigation.

This comes about three weeks after a former Kingsbury teacher, Alesia Harris, told school board members that Principal Terry Ross instructed someone to change 17 student exam grades to 100 percent — against her wishes.

Shelby County Schools said the allegations were “inaccurate” and that the grade changes were a mistake that was self-reported by an employee.

“The school administration immediately reported, and the central office team took the necessary actions and promptly corrected the errors,” the district said in a statement.

Chalkbeat requested a copy of the district’s own initial investigation the day after Harris spoke at the board’s June meeting, but district officials said they likely would not have a response for Chalkbeat until July 27.

Harris said that no one from Dixon Hughes Goodman has contacted her regarding the investigation as of Thursday.

The firm’s investigation initially included seven schools. Kingsbury was not among them. Those seven schools are:

  • Kirby High
  • Raleigh-Egypt High
  • Bolton High
  • Westwood High
  • White Station High
  • Trezevant High
  • Memphis Virtual School

The firm’s first report found as many as 2,900 failing grades changed during four years at nine Memphis-area schools. At the request of the board, two schools were eliminated: one a charter managed by a nonprofit, and a school outside the district. The firm said at the time that further investigation was warranted to determine if the grade changes were legitimate.

The $145,000 investigation includes interviews with teachers and administrators, comparing teachers’ paper grade books to electronic versions, accompanying grade change forms, and inspecting policies and procedures for how school employees track and submit grades.

Since the controversy started last year, the district has restricted the number of employees authorized to make changes to a student’s report card or transcript, and also requires a monthly report from principals detailing any grade changes.

Silver Lining Playbook

Memphis’ youngest students show reading gains on 2018 state tests — and that’s a big deal

PHOTO: Caroline Bauman
A student works on reading comprehension skills at Lucie E Campbell Elementary School in Memphis and Shelby County Schools.

Those working to improve early literacy rates in Shelby County Schools got a small morale boost Thursday as newly released scores show the district’s elementary school students improved their reading on 2018 state tests.

The percentage of Memphis elementary-age students considered proficient in reading rose by 3 points to almost one-fourth of the district’s children in grades 3 through 5. That’s still well below the state average, and Superintendent Dorsey Hopson said “we obviously have a long way to go.”

PHOTO: Caroline Bauman
Superintendent Dorsey Hopson has overseen Tennessee’s largest public school district since 2013.

Strengthening early literacy has been a priority for the Memphis district, which views better reading skills as crucial to predicting high school graduation and career success. To that end, Shelby County Schools has expanded access to pre-K programs, adjusted reading curriculum, and made investments in literacy training for teachers.

Hopson said the payoff on this year’s TNReady scores was a jump of almost 5 percentage points in third-grade reading proficiency.

“It was about five years ago when we really, really, really started pushing pre-K, and those pre-K kids are now in the third grade. I think that’s something that’s really positive,” Hopson said of the gains, adding that third-grade reading levels are an important indicator of future school performance.

TNReady scores for Shelby County Schools, which has a high concentration of low-performing schools and students living in poverty, were a mixed bag, as they were statewide.

Math scores went up in elementary, middle, and high schools in Tennessee’s largest district. But science scores went down across the board, and the percentage of high school students who scored proficient in reading dropped by 4 percentage points.

The three charts below illustrate, by subject, the percentages of students who performed on track or better in elementary, middle, and high schools within Shelby County Schools. The blue bars reflect the district’s most recent scores, the black bars show last year’s scores, and the yellow bars depict this year’s statewide averages.

Hopson said he was unsure how much the scores of older students — all of whom tested online — were affected by technical problems that hampered Tennessee’s return this year to computerized testing.

“From what people tell me, kids either didn’t try as hard in some instances or didn’t take it seriously,” Hopson told reporters. “We’ll never know what the real impact is, but we have to accept the data that came from these tests.”

But students in two of the district’s school improvement initiatives — the Innovation Zone and the Empowerment Zone — showed progress. “We’re going to double down on these strategies,” Hopson said of the extra investments and classroom supports.

In the state-run Achievement School District, or ASD, which oversees 30 low-performing schools in Memphis, grades 3 through 8 saw an uptick in scores in both reading and math. But high schoolers scored more than 3 percentage points lower in reading and also took a step back in science.

The ASD takes over schools in the state’s bottom 5 percent and assigns them to charter operators to improve. But in the five years that the ASD has been in Memphis, its scores have been mostly stagnant.

Tennessee Education Commissioner Candice McQueen said she and new ASD Superintendent Sharon Griffin are reviewing the new data to determine next steps.

“We are seeing some encouraging momentum shifts,” McQueen said.

Chalkbeat illustrator Sam Park contributed to this story.