April 18, 2019

Contact:
William J. Mathis: (802) 383-0058, wmathis@sover.net
John T. Yun: (517) 884-7822, jyun@msu.edu

Multiple Shortcomings Mar Report Ranking Michigan High Schools

An NEPC Review funded by the Great Lakes Center

Key Takeaway: Report does a disservice by introducing questionable information not substantiated by any credible analysis.

EAST LANSING, MI (April 18, 2019) - A recent report from Mackinac Center for Public Policy seeks to measure and publicize high school performance by ranking schools according to their test scores after attempting controlling for students’ “economic status.”
 
Associate Professor John T. Yun of Michigan State University reviewed The Michigan Context and Performance Report Card: High Schools 2018. He concluded that while the stated goal of the report is laudable, the reality falls far short due to several shortcomings.

The report touts its decision to take free-lunch status into account as its major contribution, in comparison to past school rankings (although this type of calculation has been done previously in many contexts). Beyond this choice, however, the study lacks both justification and explanation of its methodological decisions. The validity and reliability of combining disparate tests across different years without proper equating invalidates the findings – particularly for the high-stakes applications presented in the report.

Additionally, the free-lunch percentage measures used in the study have a great deal of measurement error, which argues against this sort of ordinal ranking. And the use of a single predictor with unacceptably low correlations for this type of usage grossly oversimplifies and biases the estimates.

Given these shortcomings, the rankings presented in this report should be given no weight in any discussions of policy or practice. In fact, this report does a disservice by introducing questionable information in an easily readable form that is not substantiated by any credible analysis.

Find the review, by John T. Yun, at:
http://greatlakescenter.org/docs/Think_Twice/TT-Yun-Rankings.pdf

Find The Michigan Context and Performance Report Card: High Schools 2018, written by Ben DeGrow and Ronald Klinger and published by Mackinac Center for Public Policy, at:
https://www.mackinac.org/archives/2019/s2019-01.pdf

NEPC Reviews (http://thinktankreview.org) provide the public, policymakers, and the press with timely, academically sound reviews of selected publications. NEPC Reviews are made possible in part by support provided by the Great Lakes Center for Education Research and Practice: http://www.greatlakescenter.org

The National Education Policy Center (NEPC), housed at the University of Colorado Boulder School of Education, produces and disseminates high-quality, peer-reviewed research to inform education policy discussions. Visit us at: http://nepc.colorado.edu/

About The Great Lakes Center
The mission of the Great Lakes Center for Education Research and Practice is to support and disseminate high quality research and reviews of research for the purpose of informing education policy and to develop research-based resources for use by those who advocate for education reform. Visit the Great Lakes Center Web Site at: http://www.greatlakescenter.org. Follow us on Twitter at: http://twitter.com/greatlakescent. Find us on Facebook at: https://www.facebook.com/GreatLakesCenter.

- ### -

The mission of the Great Lakes Center for Education Research & Practice is to support and disseminate high quality research and reviews of research for the purpose of informing education policy and to develop research-based resources for use by those who advocate for education reform.

Visit the Great Lakes Center website at http://www.greatlakescenter.org/