GLC Logo

Contact:
Teri Battaglieri, 517-203-2940, tbattaglieri@greatlakescenter.org
Jaekyung Lee, 716-645-1132, JL224@buffalo.edu

Low Marks for Fordham's 'High Flyers' Report

Study claiming low-performing students gain at the expense of high-achievers uses faulty methods and reaches a false conclusion, new review finds

EAST LANSING, MI (October 13, 2011) –A report released by the Fordham Institute and Northwest Evaluation Association claims that the academic performance of high-achieving students is being undermined by a policy focus on lower-achieving students. According to an academic review released today, the report's conclusions rest on biased methodology and misleading arguments.

The report, Do High Flyers Maintain Their Altitude?, was reviewed for the Think Twice think tank review project by University at Buffalo, SUNY professor Jaekyung Lee, a nationally recognized expert on accountability and equity issues in education.

The review was produced by the National Education Policy Center (NEPC) with funding from the Great Lakes Center for Education Research and Practice.

High Flyers tracks reading and math achievement trends for students who scored extremely well on the Measures of Academic Progress, and concludes that while three in five such students maintain high-achiever status, substantial numbers fall behind compared with their original performance, ultimately costing them access to such benefits as greater college choice and greater merit-based financial aid. It suggests that a possible reason for some high achievers slowing down is the focus on low-performing schools and students, prompted by the federal No Child Left Behind act, or on struggling readers on the part of the Reading First literacy strategy.

In his review, Lee describes a number of flaws in the report, including "its black-box approach that assumes a link between its findings and NCLB-related policies." Lee also points to two key elements of the report that are unclear and inconsistent: the study's approach to defining high achievers and its tracking of academic progress over time. At one point, the report uses percentile ranks in each grade, which is guaranteed to produce "winners" and "losers," and elsewhere it uses developmental scale scores, which as Lee observes "allow for continuing growth regardless of change in relative status; thus all students can be winners or losers." As Lee points out, it is difficult for readers to make sense of these decisions, because the report fails to provide complete technical information on its methods, such as the construction of the samples of public school students on which the report's findings are based.

To test the report's findings, Lee constructed a similar study of his own using a different national dataset, and he found results varied widely depending on which of the report's two metrics was used. Lee also found that the study didn't adequately account for regression to the mean, a statistical artifact that occurs when researchers examine the difference between two imperfectly correlated measures—in this case, achievement tests taken by students at different grades. Due to this measurement phenomenon, "lower-performing students tend to improve their performance status more than higher-performing students," Lee notes. This occurs even if there is no actual change in the achievement being measured.

Lee concludes that the report is of little of value to policymakers. Indeed, he notes, "the report's arguments about the loss of potential human capital and about a purported trade-off between excellence and equity can be more harmful than helpful."

Find  Jaekyung Lee's review and a link to the High Flyers report on the Great Lakes Center website at:
http://www.greatlakescenter.org.
 
The Think Twice think tank review project, a project of the National Education Policy Center, provides the public, policy makers and the press with timely, academically sound reviews of selected publications. The project is made possible in part by the support of the Great Lakes Center for Education Research and Practice.

The review is also available on the National Education Policy Center website at:
http://nepc.colorado.edu/thinktank/review-do-high-flyers-maintain-their-altitude.

--###--



The mission of the Great Lakes Center is to improve public education for all students in the Great Lakes region through the support and dissemination of high quality, academically sound research on education policy and practices.

Visit the Great Lakes Center Web Site at: http://www.greatlakescenter.org.

Follow us on Twitter at: http://twitter.com/greatlakescent.