November 27, 2018

Contacts

William J. Mathis: (802) 383-0058, wmathis@sover.net
Bruce D. Baker: (732) 932-7496, x8232, bruce.baker@gse.rutgers.edu
Great Lakes Center: (517) 203-2940, greatlakescenter@greatlakescenter.org

School Rankings Based on Spending and Outcomes Suffers from Severe Methodological Flaws and Insufficient Research

BOULDER, CO (November 27, 2018) – The Reason Foundation recently published a policy brief that offers an alternative ranking of states' education systems. The brief, which was based on a working paper from the Department of Finance and Managerial Economics at the University of Texas at Dallas, purports to offer needed adjustments and nuance, but makes its own serious mistakes, according to a new review.

Rutgers professor Bruce D. Baker reviewed Everything You Know About State Education Rankings Is Wrong and the underlying working paper, Fixing the Currently Biased State K-12 Education Rankings. He found the analyses provided did little or nothing to advance the conversation about the effectiveness of state education systems.

The twin reports begin with the presumption that high average test scores combined with lower school spending should be the basis for state rankings, which are reasonable premises, depending upon how the analyses are approached. But the reports then head off the rails, Professor Baker explains.

Offering a 'corrected' representation of student outcomes and a crude analysis asserting that spending has no relation to those outcomes, the reports declare states such as New Jersey and Vermont to be poor-performing, highly inefficient systems by comparison to many states. The reports then estimate a regression model and assert that the higher performing states are those with (a) weaker teachers' unions and (b) more children in charter schools.

However, Baker's review details how the reports' so-called corrections involved unreasonable and illogical assumptions and adjustments. For example, the reports re-weight racial and ethnic subgroups so that it inappropriately places equal weight in states like Vermont or Wyoming on students comprising 1 to 2% of the population as the other 98 to 99%. Other problems concern a decision to ignore economic status entirely and a poorly executed adjustment for cost of living.

Regressing multiple, highly related, interdependent measures against a specious outcome measure leads to even more suspect findings and, Baker concludes, would only mislead policymakers.

Find the review, by Bruce D. Baker, at: http://greatlakescenter.org/

Find Everything You Know About State Education Rankings Is Wrong, written by Stan J. Liebowitz and Matthew L. Kelly and published by the Reason Foundation, at: https://reason.com/archives/2018/10/07/everything-you-know-about-stat

Find Fixing the Currently Biased State K-12 Education Rankings, written by Stan J. Liebowitz and Matthew L. Kelly and published by the Department of Finance and Managerial Economics at the University of Texas at Dallas at: https://ssrn.com/abstract=3185152

About The Great Lakes Center
The mission of the Great Lakes Center for Education Research and Practice is to support and disseminate high quality research and reviews of research for the purpose of informing education policy and to develop research-based resources for use by those who advocate for education reform.  Visit the Great Lakes Center Web Site at: http://www.greatlakescenter.org.   Follow us on Twitter at: http://twitter.com/greatlakescent.  Find us on Facebook at: https://www.facebook.com/GreatLakesCenter.

- ### -

Friend on Facebook

Follow on Twitter

The mission of the Great Lakes Center for Education Research & Practice is to support and disseminate high quality research and reviews of research for the purpose of informing education policy and to develop research-based resources for use by those who advocate for education reform.

Visit the Great Lakes Center website at http://www.greatlakescenter.org/