Gates' Foundation's MET Study Fails to Solve Teacher Evaluation Challenge
EAST LANSING, Mich. (Jan. 31, 2013) – Amidst much fanfare, the Bill and Melinda Gates Foundation recently released its third and final set of findings for the Measures of Effective Teaching (MET) project. The project generated enormous attention and valuable data while it attempted to provide clarity on how to best evaluate teachers.
However, a careful look at the MET research – an ambitious, multi-year study of thousands of teachers in six school districts – finds the study's results were inconclusive and provide little usable guidance about how to design teacher evaluation systems.
Jesse Rothstein, an economist at the University of California Berkeley, and William J. Mathis, managing director of the National Education Policy Center, completed a thorough review of two of the final MET research papers, examining how the assignment of students affects teacher evaluations and how multiple teacher evaluation measures are best combined.
The review was produced by the National Education Policy Center with funding from the Great Lakes Center for Education Research and Practice.
"The MET research does little to settle longstanding debates over how best to evaluate teachers comprehensively," said Rothstein and Mathis.
The study utilized three types of measures (student test scores, classroom observations, and student surveys); the project concluded that the three should be given roughly equal weight in teacher evaluations. Rothstein and Mathis found that the data do not support that conclusion.
Instead, the data indicate that each measure reflects a distinct dimension of teaching. "Any evaluation system needs to be founded on a judgment about what constitutes effective teaching, and that judgment will drive the choice of measures," says Rothstein. Based on the review, there is little reason to believe that an evaluation system based on any of the measures considered in the MET project will do a good job of identifying teachers who are effective (or ineffective) at raising students' performance on more conceptually demanding assessments.
Mathis adds, "While we commend The Bill and Melinda Gates Foundation for investing millions of dollars in tackling critical education issues, the conclusions in this case do not jibe with the data."
At the center of the MET project was an experiment that randomly assigned students to teachers. This approach was meant to determine irretrievably whether value-added (VA) scores are biased by student assignments. That is, do teachers who are assigned more successful students benefit in terms of their VA scores?
Unfortunately, the reviewers found that the group of teachers who participated in the MET experiments was not representative of the teaching force, and many participating schools failed to comply with their experimental assignments. As a result, the experiment did little to resolve the question.
Rothstein and Mathis conclude, "While the Gates MET study has brought an unprecedented vigor and intensity to teacher evaluation research, even its masses of data do not settle disagreements about what makes an effective teacher."
Find the Think Twice Review on the Great Lakes Center website:
Think Twice, a project of the National Education Policy Center, provides the public, policymakers and the press with timely, academically sound reviews of selected publications. The project is made possible by funding from the Great Lakes Center for Education Research and Practice.
The review is also available on the National Education Policy Center website: http://nepc.colorado.edu
The mission of the Great Lakes Center for Education Research and Practice is to support and disseminate high quality research and reviews of research for the purpose of informing education policy and to develop research-based resources for use by those who advocate for education reform.
Find us on Facebook at: https://www.facebook.com/GreatLakesCenter.