A review finds LA Times 2011 teacher effectiveness rankings remain unreliable, risk being misused
High-stakes standardized tests only a snapshot of student's learning
EAST LANSING, Mich. (June 20, 2012) – The Los Angeles Times' second attempt to rate the performance of teachers in the Los Angeles Unified School District (LAUSD) still provides readers with unreliable data and potential misuse, according to an academic review released today.
Catherine Durso of the University of Denver prepared the review titled An Analysis of the Use and Validity of Test-Based Teacher Evaluations Reported by the Los Angeles Times: 2011. The review was produced by the National Education Policy Center with funding from the Great Lakes Center for Education Research and Practice.
In her analysis, Durso addressed the variances of teacher ratings, specifically the imprecision and inconsistency of teacher scores. She found limitations on the use of teacher-linked effects in producing a single classification scale.
The newspaper used a value-added model (VAM) and also rated teachers in one of five categories (least effective, less effective, average, more effective, and most effective). The series was based on a study carried out independently by Dr. Richard Buddin, a senior economist at the RAND Corporation. LAUSD uses the term Average Growth over Time (AGT) to represent VAM.
Though demographics, classroom size and other outside factors were taken into account, the data produced broad "error bands" that essentially created a range score for each teacher, only the midpoint of which was used to rate teachers in a single effectiveness category. The single effectiveness rating was published with limited caution to readers.
The single-scale the newspaper used to rate the effectiveness of teachers drew the greatest concern, "particularly the effectiveness classification categories published in the Teacher Ratings that the lay reader will be drawn to, and these are unreliable," Durso wrote in her review.
She found that the potential for misuse of effective classification categories could prove to be harmful.
Although the newspaper's data efforts were an improvement over a similar series (also based on the value-added model) that ran in 2010, isolating the causal effect of a teacher on a student from the influence of peers and situations outside of school remains challenging.
Additionally, the majority of teachers in the Times' rankings were represented by data for three years or less and 26 percent of the teachers were in their first year. Using estimates from this smaller data sample yields less precise results.
"Even when multiple years of data are employed, the lack of predictive power limits the usefulness of the estimates to a parent considering whether an assigned class will improve a child's test-based performance," Durso stated.
Durso recommends, "Comparison of teacher-linked effects must be understood as comparing teachers and their work environments, not just teachers."
Find the brief on the Great Lakes Center website at:
Find a link to the LA Times series here:
The National Education Policy Center provides the public, policy makers and the press with timely, academically sound reviews of selected publications. The project was made possible in part by funding from the Great Lakes Center for Education Research and Practice.
The reviews are also available on the National Education Policy Center website at:
The mission of the Great Lakes Center for Education Research and Practice is to support and disseminate high quality research and reviews of research for the purpose of informing education policy and to develop research-based resources for use by those who advocate for education reform.
Find us on Facebook at: https://www.facebook.com/GreatLakesCenter.