research shows

Gates Foundation study paints bleak picture of teaching quality

The study measured teachers against the criteria in Charlotte Danielson's Framework for Effective Teaching rubric, which is used in New York as a tool for observing teachers. Teachers scored better at classroom management than they did on measures of higher-order instructional challenges, such as asking productive questions.

A historic look inside the nation’s classrooms, including some in New York City, painted a bleak picture, according to a report released by the Bill and Melinda Gates Foundation today.

The second installment of the foundation’s ambitious Measures of Effective Teaching study, the report focuses on the picture of teaching yielded by five different classroom observation tools. It also scrutinizes those tools themselves, concluding that they are valuable as a way to help teachers improve but only useful as evaluation tools when combined with measures of student learning known as value-added scores.

The conclusion is a strong endorsement of the Obama administration’s approach to improving teaching by implementing new evaluations of teachers that draw on both observations and value-added measures. New York State took this approach to overhauling its evaluation system when it applied for federal Race to the Top funding.

Among the group of five observation tools the foundation studied is the rubric now being piloted in New York City classrooms as part of stalled efforts to implement the changes to teacher evaluation, Charlotte Danielson’s Framework for Effective Teaching.

Through all five lenses, instruction looked mediocre in an overwhelming majority of more than 1,000 classrooms studied, the report concludes. There were some bright spots. Many teachers were scored relatively well for the aspect of teaching known as “classroom management” — keeping students well-behaved, making sure they are engaged.

But teachers often fell short when it came to other elements of teaching, such as facilitating discussions, speaking precisely about concepts, and carefully modeling skills that students need to master. These higher-order skill sets, the report notes, are crucial in order for students to meet the raised standards outlined in the Common Core.

The study is the most expansive known examination of instruction in the U.S., reviewing more than 1,000 teachers for this report and nearly 3,000 for the study. Its lead authors are the economists Thomas Kane, of Harvard, and Douglas Staiger of Dartmouth, although more than a dozens researchers contributed to the study.

The evaluations were conducted by trained evaluators, who watched clips from videotape of more than 1,000 teachers around the country and then judged whether the teaching exhibited certain traits outlined in the observation tools.

One complicated aspect of the study is that it doesn’t just ask what the observation tools have to say about teaching; it also asks whether those observation tools are good ways to measure teaching at all. The question is crucial to the contentious teacher quality debate.

Motivated by the Obama administration’s focus on improving teaching by improving the way teachers are evaluated, the teacher quality debate has been dominated by a search for a better evaluation tool. The idea is that if school districts could have a better way to sort teachers, then they could increase quality by rewarding those who are most effective and improving or removing those who are less effective.

The study offers a qualified endorsement of the five observation tools it studied, saying that they should be one of multiple evaluation measures but that no one observation tool should be a sole measure. While the study found that all five observation tools had a positive association with student achievement as measured by value-added scores, the associations were not perfect.

And the tools’ reliability was relatively low — lower, in some cases, than the famously volatile judgments of value-added measures. When different observers used the same tool to evaluate the same teacher, they sometimes gave very different scores.

But the report does endorse using the observation tools in combination with value-added measures, as New York’s new evaluation system does. When researchers combined multiple observation tools’ judgments of teachers together — and then combined those with the teachers’ value-added scores, the result was a view of a teacher that was more able to predict future student achievement, the report says.

A final complication worth noting is that the study’s ultimate arbiter of what makes a good evaluation tool is itself under heavy scrutiny. That arbiter is a teacher’s value-added score, an estimate that attempts to extrapolate the amount of student learning for which a teacher can be held responsible, excluding other factors such as a student’s family income level.

A study that was the subject of a story in today’s New York Times found that value-added scores indeed are useful predictors not only of student achievement, but other measures of life success. Researchers have cast doubts on value-added measures’ validity, citing a host of concerns from the measures’ volatility to whether a high value-added score reflects true student learning or simply effective test prep.

Though an overhaul of teacher evaluation in New York has been stalled by the failure of teachers unions and school districts to agree on how to conduct it, both the New York City teachers union and the Department of Education agreed to participate in the Gates Foundation study when it launched in 2009. The union helped recruit teachers to join, and ultimately, teachers from about 100 schools signed up to have their lessons videotaped and analyzed.

“It takes the politics out of what’s being measured,” UFT president Michael Mulgrew said when the union first agreed to participate. “Teachers are very frustrated with the political debate. They are always saying, ‘why don’t you just come into the classroom?’ That’s what this is doing.”

Since then, the politics over teacher quality has grown even more heated.

Last summer, a GothamSchools reader who had worked in a school piloting the Danielson evaluation said it was very hard for teachers to be rated “effective.”

Struggling Detroit schools

The list of promises is long: Arts, music, robotics, gifted programs and more. Will Detroit schools be able to deliver?

PHOTO: Detroit Public Television
Detroit schools Superintendent Nikolai Vitti answers questions at a community meeting in Detroit.

Arts. Music. Robotics. Programs for gifted kids. New computers. New textbooks. Dual enrollment programs that let high school students take college classes. International Baccalaureate. Advanced Placement.

They’re all on the list of things that Detroit schools Superintendent Nikolai Vitti told a group of community members assembled in a Brightmoor neighborhood church that he would introduce or expand as soon as next school year.

Vitti didn’t get into the specifics of how the main Detroit district would find the money or partnerships needed to deliver on all of those promises, but they’re part of the plan for the future, he said.

The comments came in a question and answer session last month with students, parents and community members following Vitti’s appearance on Detroit Public Television’s American Black Journal/One Detroit Roadshow. The discussion was recorded at City Covenant Church. DPTV is one of Chalkbeat’s partners in the Detroit Journalism Cooperative.

Vitti has been appearing at community events since taking over the Detroit schools last spring. He is scheduled next week to join officials from two of the city’s major charter school authorizers, Central Michigan University and Grand Valley State University, at a State of the Schools address on October 25.

 

Watch the full Q&A with Vitti below.

Betsy DeVos

To promote virtual schools, Betsy DeVos cites a graduate who’s far from the norm

U.S. Education Secretary Betsy DeVos spoke to the National Alliance for Public Charter Schools in June.

If Betsy Devos is paying any attention to unfolding critiques of virtual charter schools, she didn’t let it show last week when she spoke to free-market policy advocates in Bellevue, Washington.

Just days after Politico published a scathing story about virtual charters’ track record in Pennsylvania, DeVos, the U.S. education secretary, was touting their successes at the Washington Policy Center’s annual dinner.

DeVos’s speech was largely identical in its main points to one she gave at Harvard University last month. But she customized the stories of students who struggled in traditional schools with local examples, and in doing so provided an especially clear example of why she believes in virtual schools.

From the speech:

I also think of Sandeep Thomas. Sandeep grew up impoverished in Bangalore, India and experienced terrible trauma in his youth. He was adopted by a loving couple from New Jersey, but continued to suffer from the unspeakable horrors he witnessed in his early years. He was not able to focus in school, and it took him hours to complete even the simplest assignment.

This changed when his family moved to Washington, where Sandeep was able to enroll in a virtual public school. This option gave him the flexibility to learn in the quiet of his own home and pursue his learning at a pace that was right for him. He ended up graduating high school with a 3.7 GPA, along with having earned well over a year of college credit. Today, he’s working in finance and he is a vocal advocate for expanding options that allow students like him a chance to succeed.

But Thomas — who spoke at a conference of a group DeVos used to chair, Advocates for Children, in 2013 as part of ongoing work lobbying for virtual charters — is hardly representative of online school students.

In Pennsylvania, Politico reported last week, 30,000 students are enrolled in virtual charters with an average 48 percent graduation rate. In Indiana, an online charter school that had gotten a stunning six straight F grades from the state — one of just three schools in that positionis closing. And an Education Week investigation into Colorado’s largest virtual charter school found that not even a quarter of the 4,000 students even log on to do work every day.

The fact that in many states with online charters, large numbers of often needy students have enrolled without advancing has not held DeVos back from supporting the model. (A 2015 study found that students who enrolled in virtual charters in Michigan, Illinois, and Wisconsin did just as well as similar students who stayed in brick-and-mortar schools.) In fact, she appeared to ignore their track records during the confirmation process in January, citing graduation rates provided by a leading charter operator that were far higher — nearly 40 points in one case — than the rates recorded by the schools’ states.

She has long backed the schools, and her former organization has close ties to major virtual school operators, including K12, the one that generated the inflated graduation numbers. In her first week as education secretary, DeVos said, “I expect there will be more virtual schools.”

Correction: An earlier version of this article misstated the location of the dinner.