Are Children Learning

Is Indiana's state ISTEP exam too easy?

PHOTO: Shannan Muskopf via Flickr
State officials are closing as many 38 Michigan schools with low rankings due to test scores but they might have trouble finding higher scoring schools nearby

Indiana’s standardized tests might have gotten so much harder last year that test scores plunged across the state, but two national testing experts say the exam might technically still be too easy according to an earlier study.

“The Indiana test was relatively low-level,” said Ed Roeber, a former Michigan testing director asked to consult on ISTEP.

The 2015 ISTEP exam might have seemed tougher to students and teachers. The state saw a 20 percentage point drop in students passing both English and math last year.

But when Roeber and another testing expert, Derek Briggs, were asked by the Indiana State Board of Education to conduct a review of that exam in January, they discovered that exam questions were not as rigorous as they should have been. The state adopted new academic standards in 2014 that were designed to better prepare kids for college and careers. The new standards were said to be much tougher.

Try it out: 18 practice questions to prepare for Indiana’s 2016 ISTEP test.

Indiana officials brought on Roeber and Briggs on to advise them on the ISTEP last year in the wake of scoring glitches, delays and accuracy questions that raised concerns about the validity and reliability of exam. After examining an earlier study of the test by the education research group WestEd, the pair noted that Indiana’s test questions do accurately reflect the standards, but suggested that they might still be too easy.

The analysis from Roeber and Briggs reached no conclusions on whether the standards themselves are meeting the state’s goals, but the pair found that too many questions on last year’s ISTEP test were too easy.

Specifically, the experts cited the WestEd study that found a majority of the questions on the 2015 exam were fact-based questions that asked students to merely to recall information rather than do higher-level thinking. More than 80 percent of questions on the English exam and every single question on the math test were found to be low-level basic questions.

The test had very few more difficult questions that would have required students to describe or explain a scenario, use evidence to back up answers or test a hypothesis and make connections beyond the facts at hand.

“The fact that (a test question) aligns with the standards is good, but it doesn’t imply that it’s measuring the standards to the same depth the standard is written at,” Roeber said.

What remains unclear is why the questions were too easy. The fault could rest with the test, which was written hastily after lawmakers voted to drop the Common Core and write Indiana-specific standards in 2014. Test questions weren’t tried out on students until the test was actually given last spring — a highly unusual practice in a testing industry in which test questions are typically piloted over several years before being used on an exam, Roeber said.

The Indiana Department of Education decided against using questions from previous tests that could have saved some time, said Cynthia Roach, the state board’s testing director.

Another possible reason for the easy test questions is the state’s standards themselves. It’s not clear just how rigorous Indiana’s standards really are. Until the standards are properly evaluated using the same metrics as the test, Roeber said, the state’s policymakers can’t really know if ISTEP is measuring what kids are expected to learn better than past tests.

It might seem technical, but the mismatch between the standards and the questions could be important. Indiana still must submit its test for review by the U.S. Department of Education for its waiver from the No Child Left Behind law, still in effect until August. Any misalignment between the test and standards could be problematic.

“That’s one of the key things that the peer reviewer will look for is the measure of the rigor of the test, and if this misstates that, then you definitely have the right to challenge the vendor,” Roeber said.

The state’s new testing advisory committee is taking the suggestions from the analysis to try to move forward, Roach said. Nothing can be addressed in time for the 2016 test, which students have already started taking, but questions should be reassessed and Indiana should look into deeper measurement of its standards, Roeber said.

The Indiana Department of Education, which administers the test, declined to make its testing director available for an interview to discuss Roeber and Briggs’ analysis, but Department spokeswoman Samantha Hart issued a statement saying the test was hard enough.

“As with any assessment, there are going to be questions that are more and less rigorous than others — The new ISTEP+ exam is no different,” Hart wrote. “Anyone who thinks that last year’s ISTEP wasn’t hard enough should go talk to a student or a teacher or a parent. This test was clearly more rigorous, just like our standards.”

Academic Accountability

Coming soon: Not one, but two ratings for every Chicago school

Starting this month, Chicago schools will have to juggle two ratings — one from the school district, and another from the state.

The Illinois State Board of Education is scheduled to release on October 31 its annual report cards for schools across the state. This year, for the first time, each school will receive one of four quality stamps from the state: an “exemplary” or “commendable” rating signal the school is meeting standards while an “underperforming” or “lowest performing” designation could trigger intervention, according to state board of education spokeswoman Jackie Matthews.

A federal accountability law, the Every Student Succeeds Act, requires these new ratings.

To complicate matters, the city and state ratings are each based on different underlying metrics and even a different set of standardized tests. The state ratings, for example, are based on a modified version of the PARCC assessment, while Chicago ratings are based largely on the NWEA. The new state ratings, like those the school district issues, can be given out without observers ever having visited a classroom, which is why critics argue that the approach lacks the qualitative metrics necessary to assess the learning, teaching, and leadership at individual schools.

Patricia Brekke, principal at Back of the Yards College Preparatory High School, said she’s still waiting to see how the ratings will be used, “and how that matters for us,” but that parents at her school aren’t necessarily focused on what the state says.

“What our parents usually want to know is what [Chicago Public Schools] says about us, and how we’re doing in comparison to other schools nearby that their children are interested in,” she said.

Educators at Chicago Public Schools understand the power of school quality ratings.  The district already has its own five-tiered rating system: Level 1+ and Level 1 designate the highest performing schools, Level 2+ and Level 2 describe for average and below average performing schools, respectively, and Level 3, the lowest performance rating, is for schools in need of “intensive intervention.” The ratings help parents decide where to enroll their children, and are supposed to signal to the district that the school needs more support. But the ratings are also the source of angst — used to justify replacing school leaders, closing schools, or opening new schools in neighborhoods where options are deemed inadequate.

In contrast, the state’s school quality designations actually target underperforming and lowest-performing schools with additional federal funding and support with the goal of improving student outcomes. Matthews said schools will work with “school support managers” from the state to do a self-inquiry and identify areas for improvement. She described Chicago’s school quality rating system as “a local dashboard that they have developed to communicate with their communities.”

Staff from the Illinois State Board of Education will be traveling around the state next week to meet with district leaders and principals to discuss the new accountability system, including the ratings. They’ll be in Bloomington, Marion, O’Fallon, Chicago, and Melrose Park. The Chicago meeting is Wednesday, Oct. 24, at 5 p.m. at Chicago Public Schools headquarters.

Rae Clementz, director of assessment and accountability at the state board said that a second set of ratings reveals “that there are multiple valid ways to look at school quality and success; it’s a richer picture.”

Under auspices of the Every Student Succeeds Act, the state school report cards released at the end of the month for elementary schools are 75 percent based on academics, including English language arts and math test scores, English learner progress as measured by the ACCESS test, and academic growth. The other 25 percent reflects the school climate and success, such as attendance and chronic absenteeism.

Other measures are slated to be phased in over the next several years, including academic indicators like science proficiency and school quality indicators, such as school climate surveys of staff, students and parents

High school designations take a similar approach with English and math test scores but will take into account graduation rates, instead of academic growth, and also includes the percentage of  9th graders on track to graduate — that is freshmen who earn 10 semester credits, and no more than one semester F in a core course.

Critics of Chicago’s school rating system argue that the ratings correlate more with socioeconomic status and race than they do school quality, and say little about what’s happening in classrooms and how kids are learning. Chicago does try to mitigate these issues with a greater emphasis on growth in test scores rather than absolute attainment, school climate surveys, and including academic growth by priority groups, like African-American, Latino, ELL, and students in special education.

Cory Koedel, a professor of economics and public policy at the University of Missouri, said that many rating systems basically capture poverty status with a focus on how high or low students score on tests. Chicago’s approach is fairer than that of many other school systems.

“What I like about this is it does seem to have a high weight on growth and lower weight on attainment levels,” he said.

Morgan Polikoff, a professor at University of Southern California’s school of education, said that Chicago’s emphasis on student growth is a good thing “if the purpose of the system is to identify schools doing a good job educating kids.”

Chicago weights 50 percent of the rating on growth, but he’s seen 35 to as low as 15 percent at other districts. But he said the school district’s reliance on the NWEA test rather than the PARCC test used in the state school ratings was atypical.

“It’s not a state test, and though they say it aligns with standards, I know from talking to educators that a lot of them feel the tests are not well aligned with what they are supposed to be teaching,” he said. “It’s just a little odd to me they would have state assessment data, which is what they are held accountable for with the state, but use the other data.”

He’s skeptical about school systems relying too heavily on standardized test scores, whether the SAT, PARCC or NWEA, because “You worry that now you’re just turning the curriculum to test prep, and that’s an incentive you don’t want to create for educators.”

He said the high school measures in particular include a wide array of measures, including measures that follow students into college, “so I love that.”

“I really like the idea of broadening the set of indicators on which we evaluate schools and encouraging schools to really pay attention to how well they prepare students for what comes next,” he said.

Miseducation

In Colorado’s high-poverty schools, many teachers are just starting their careers

PHOTO: Yesenia Robles/Chalkbeat
A first-grade student reads in Spanish in a biliteracy classroom at Dupont Elementary in Adams 14.

This story is part of a partnership between Chalkbeat and the nonprofit investigative news organization ProPublica. Using federal data from Miseducation, an interactive database built by ProPublica, we are publishing a series of stories exploring inequities in education at the local level.

Koli Jamerson’s residency program gave her tools that she uses every day as a teacher, ideas for developing engaging lessons and for working with English language learners.

But it didn’t teach her how to help a student who explodes in anger because the police were at her house the night before on a domestic violence call or who cries all day because she doesn’t know where she’ll sleep that night.

Jamerson, now in her third year of teaching first grade at Altura Elementary in the Aurora school district, is still standing. She remains committed to her profession in large part due to the help of veteran teachers on her team, who provided advice as she found her footing those first couple of years.

“It helps keep things in perspective,” Jamerson said of her conversations with more experienced educators. “Otherwise, I would have been talking to a bunch of other teachers who were also drowning, and we would have drowned together.”

It’s getting hard for new teachers in Colorado to find those support systems, since the percentage of Colorado’s teachers in their first or second year in the classroom is among the highest in the nation. In 2015-16, the most recent year for which federal data is available, 17 percent of Colorado teachers were new to the classroom, compared with 12 percent nationally. Only Tennessee, Arizona, and Washington, D.C., rank higher. As recently as 2011, less than 11 percent of Colorado’s teachers were new to the classroom.

This information comes from a new interactive database from the investigative news organization ProPublica. It draws on data collected by the U.S. Department of Education’s Office of Civil Rights and for the first time allows parents to easily search their school and district and compare it with others in the region. 

The rates of inexperienced teachers are even higher in certain rural districts and in districts where lots of students come from low-income families and face more challenges. Those districts also often have high numbers of students of color. In contrast, whiter, more affluent suburban districts tend to have low rates of inexperienced teachers.

And those numbers have significant ramifications for Colorado’s students: New teachers can bring energy and innovation to the classroom, and some, especially those with support and guidance, manage to thrive early on. But students with inexperienced teachers tend to have lower test scores on average, according to numerous studies, and new teachers often get lower scores in classroom management from their principals.

Most teachers will readily admit it takes several years to get your bearings in a profession for which no amount of classroom learning can fully prepare you.

“In reality, you get second grade one time, you get third grade one time, as a kid,” said David Singer, founder of Denver’s University Prep charter network, which has shown impressive test results even with plenty of relatively inexperienced teachers. “You deserve an excellent educator.”

Chalkbeat reviewed more recent state data that follows the typical federal definition of “inexperienced teachers” — teachers with less than three full years of classroom experience — and found that the broad trends remain true and in many cases are even more severe than they appear in the federal data. Statewide, one in four Colorado teachers was classified as inexperienced. Last school year, 31 percent of Denver Public Schools teachers were in their first three years on the job, compared with just 7 percent of teachers in the more affluent Boulder Valley School District.

The Adams 14 district, based in the working-class suburb of Commerce City, is one of the lowest-performing in the state. Last year, 45 percent of teachers there were considered inexperienced, compared with 8 percent in the south suburban Littleton district. 

In districts with so many new teachers, it becomes inevitable that students there will encounter educators who haven’t yet reached their prime.

“When a teacher is new to the profession, as with any profession, they’re not as effective,” said Allison Atteberry, an assistant professor in the research and evaluation methods program at the University of Colorado’s School of Education. “There’s a really steep learning curve in those first years. That can’t really be avoided. But if there are more of those teachers, then more students will be exposed to those teachers. And if you have districts with more at-risk students, that has major equity implications.”

Atteberry said the numbers don’t surprise her, and they reflect a perfect storm in the state’s teacher corps. Colorado has experienced rapid population growth, increasing the demand for teachers, at the same time that experienced teachers are retiring or changing careers. That means more new teachers in Colorado classrooms, even as fewer students are entering teacher preparation programs.

Colorado’s low teacher pay exacerbates retention problems. Colorado ranks 30th for teacher pay, and when those salaries are adjusted for cost of living, it falls to 44th. The competitiveness of its teacher salaries is the lowest in the nation, meaning that people who go into teaching take a bigger salary hit compared to their peers with similar levels of education. Nationally, 1 in 10 teachers will leave the profession after their first year, and many more never reach the five-year mark.

Districts around the state are asking voters to raise taxes this November in part to raise teacher pay. Better pay for educators is also a major part of the campaign for Amendment 73, a $1.6 billion statewide tax increase for schools that appears on the ballot. But Colorado voters have so far been reluctant to raise statewide taxes for schools, and critics say there is no guarantee the money will make it into teachers’ paychecks.

Atteberry said raising pay would help mitigate these trends. Beyond that, there isn’t a lot of solid research on the best ways to keep teachers in the classroom, she said, but coaching and support from other teachers can make a difference. Denver is trying a new program to ease the transition for novice teachers with more time spent observing and learning from veterans before getting sole responsibility for a classroom. But just as with teacher salaries, providing adequate coaching is expensive. And the more newcomers there are, the harder it is provide meaningful support to novices.

Denver metro area inexperienced teachers

DISTRICT Teachers with less than three years experience Students receiving subsidized lunches
Adams 14 45 percent 87.3 percent
Dougco 39 percent 12.4 percent
Sheridan 33 percent 90.4 percent
27J (Brighton) 33 percent 37.7 percent
Denver 31 percent 67.2 percent
Jeffco 31 percent 31.7 percent
Aurora 29 percent 68.7 percent
Englewood 29 percent 66 percent
Westminster 24 percent 81.4 percent
Mapleton 21 percent 60.6 percent
St. Vrain 15 percent 30.6 percent
Adams 12 11 percent 39.9 percent
Cherry Creek 9 percent 30.0 percent
Littleton 8 percent 16.8 percent
Boulder Valley 7 percent 19 percent

Source: Colorado Department of Education, 2017-18 school year

This year, for the first time, Annalee Peterson has her own fifth-grade classroom in Columbia Elementary in Colorado Springs, where a large portion of the students are homeless or face other challenges. Before starting an alternative certification process, she ran reading groups as a paraprofessional in the same building for four years. And years before that, she dropped out of a Teach for America placement in a Newark high school where she felt alone and unsupported.

Peterson said her certification program includes intensive classroom observation and feedback that has been invaluable as she made the transition from para to teacher. She also has a trusting relationship with her building principal, who encouraged her to become a teacher.

“I think every new teacher should have a mentor,” she said. “I see other teachers come in, and they don’t have it.”

Peterson said she also benefits from her school’s skilled full-time counselor, something many Colorado schools don’t have.

“If we have a kid with a fair amount of trauma, and they get triggered, they have someone they can go talk to,” she said. “And that’s a huge help. They are getting their emotional needs met, and when they come back to the classroom, they’re ready to work and ready to focus.”

The Adams 14 school district, which has spent eight years on a state watchlist due to its low-performing schools, has the highest rate of inexperienced teachers in the Denver metro area. The 7,000-student district has experienced a lot of turnover not just at the classroom level, but at the highest tiers of leadership.

With an urgent need to improve school performance, Mark Langston, the district’s new manager of educator effectiveness, tries to put a positive face on the large number of new educators that arrive each year.

“I’d rather have a phenomenal teacher for one year, than a bad teacher for many years,” Langston said. “Strong systems have a nice blend of experience.”

At the same time, he’s trying to improve the support those new teachers receive by making changes to the district’s five-day induction program to better meet their individual needs. The thinking is that a 40-year-old switching careers after running a business for 20 years might need different training from a 22-year-old recent college graduate. He’s also trying to match new teachers with mentors earlier in the school year.

But sometimes there aren’t enough mentors or he’s had to make exceptions to allow less experienced teachers to become mentors.

“They are mentoring each other,” said Barb McDowell, president of the Adams 14 teachers union, who says the churn takes a toll on teacher and student morale. “There are no veteran teachers there to help.”

Kevin Clark, a senior at Adams City High, said he always felt supported by his teachers in the district, but very few of them are still there as he enters his final year.

“For the seniors, it’s been rough,” he said. “We really value our support systems. The new teachers are trying to adjust and get their footing, but just because you send in a batch of new teachers, doesn’t mean everything is fine.”

The Denver schools with the highest percentages of inexperienced teachers in 2015-16 include a number of alternative high schools, high-poverty district-run schools, and charter schools. Some of the charter schools are part of high-performing networks whose students do well on state tests.

One of them is University Prep. The homegrown Denver network has two elementary schools, one of which posted the most academic progress in Colorado on state math tests in 2017. But in 2015, the network had just one school — and 42 percent of the teachers there were in their first or second year of teaching, according to the federal data.

At University Prep, some first-year teachers have taken part in a teacher residency program or in a program that has college students work as paraprofessionals while earning their degrees.

“When you think about that individual exiting their undergraduate [education] having spent four years in a building with master teachers, getting all the supports they need to grow, they’re ready to teach on Day 1,” said Singer, the network’s founder.

PHOTO: Eric Gorski/Chalkbeat
Students at University Prep Elementary’s flagship school in Denver

Even so, the network provides its first-year teachers with extra support, he said, such as real-time coaching in the classroom, opportunities to observe more experienced teachers, and help with how to plan a lesson or conduct a parent-teacher conference.

Atteberry said successful charter schools with high rates of inexperienced teachers may be doing something different in the hiring process, looking for “spark teachers who really want to make a difference.”

The high rates of new teachers at some charter schools raise questions, though, about how sustainable the work environment is, and some of these same “spark” teachers may never intend to make a lifelong career of it and instead move on to other challenges. Asked about turnover, Singer said some University Prep teachers have left to pursue careers in medicine and law.

Denver metro area data show another exception to the trend in Douglas County. It’s an affluent and sprawling district southwest of Denver where just 12 percent of students get subsidized lunches, a proxy for poverty. But in 2015-16, 31 percent of teachers were in their first or second year in the classroom, and in 2017-18, 39 percent had less than three years experience.

Kallie Leyba, president of the Douglas County Federation, the teachers union there, said Douglas used to be a “destination district” that teachers aspired to work for. But political upheaval, the election of a conservative school board that has since been replaced, and a “market rate” pay structure that remains have caused experienced teachers to leave in droves — some for much higher salaries in nearby Cherry Creek schools.

The Douglas County pay scale means that teachers with the same amount of experience might make very different salaries. Leyba herself faced the prospect of a lower ceiling on her salary when her building principal asked her to switch from a first grade to a second grade classroom because first-grade teachers are more in demand.

“Even though I knew this was a crazy system, it really hurt to feel like my value had gone down in the eyes of my principal,” she said.

What could Colorado do to get more of today’s inexperienced teachers to become tomorrow’s veteran educators?

Money is a big part of the answer. As it stands, Colorado teachers can earn significantly more money by moving to another state, and with teacher salaries less competitive here than elsewhere, teachers also look to other professions that offer less stress along with better pay.

“The No. 1 thing we should do is increase the prestige and value of teachers in society, and the way we signal that in our society is through salary and compensation,” Atteberry said. “That has a huge influence on who goes into the profession and on who stays.

“This is not an easy change because it costs a lot of money, and it also requires us to change how we think about teachers, but it is the policy that would be most effective.”

Chalkbeat reporters Melanie Asmar and Yesenia Robles contributed reporting to this story.