Special Offer
Save 20%

Collateral Damage: How High-Stakes Testing Corrupts America’s Schools
by David C. Berliner and Sharon L. Nichols

 

Logo

INTRO | PRESS RELEASES | REPORTS & REVIEWS | WEEKLY REPORTS | RELATED ARTICLES

Think Twice
Think Tank Review Project

Think Twice is one of the nation’s first efforts to serve as a watchdog to review think tank research on public education issues and policies, ensuring that published work meets the quality and standards of university scholarship. As think tank research becomes increasingly important reference sources in public policy debates, media and other critics have called for increased scrutiny to ensure validity and objectivity (click here to see related stories).

The goal of the Think Twice project is to provide the public, policy makers and the press with timely academically sound reviews of selected think tank publications. It is a collaboration of the Education Policy Studies Laboratory at Arizona State University and the Education and the Public Interest Center at the University of Colorado at Boulder and is funded by the Great Lakes Center for Education Research and practice. [ Click Here For Full Description ]

Click Here To Read Our Legal Statement

New Recently Released New

May 22, 2008

Press Release:

Florida Voucher Report Too Flawed to be Valuable
Review criticizes report’s conclusion that vouchers have positive impact on special education outcomes

John T. Yun's Review of The Effect of Special Education Vouchers on Public School Achievement: Evidence from Florida’s McKay Scholarship Program

Manhattan Institute for Policy Research Report




Reports & Reviews for 2008

Report Reviewed:

The Effect of Special Education Vouchers on Public School Achievement: Evidence from Florida’s McKay Scholarship Program

Think Tank: Manhattan Institute for Policy Research

This report claims that Florida’s McKay program of providing private-school vouchers to special education students has resulted in improved educational outcomes for special education students who chose to stay in public schools. The theory behind this conclusion is that increased competition to enroll these students has led public schools to improve services and programs for special education students not choosing to leave.

 
Think Twice Review Date: May 22, 2008
Reviewers: John T. Yun, University of California at Santa Barbara

Yun concludes that any possible contribution made by this report is outweighed by research design problems, failure to take into account alternative explanations, and possible selection bias. He cautions against using this report for any decision-making or policy evaluation purposes.

Press Release Link name=Review Link

separator

 

Report Reviewed:

Choice and Education across the States

Think Tank: The Heartland Institute

This report assigns letter grades to states based on how extensive the state’s school choice system is.

 
Think Twice Review Date: April 29, 2008
Reviewers: Wendy Chi, University of Colorado at Boulder

Chi concludes that this report offers little or no useful information for policy makers. The report asserts, based on a faulty use of past research, that an increase in school choice will strengthen accountability and improve student achievement. It awards most states low grades, reflecting a desire for more school choice throughout the nation.

Press Release Link name=Review Link

separator

Report Reviewed:

The Turnaround Challenge: Why America’s Best Opportunity to Dramatically Improve Students Achievement Lies in Our Worst-Performing Schools

Think Tank:

Mass Insight Education & Research Institute

This report puts forth an ambitious proposal aimed at getting chronically underachieving schools to improve academic achievement.

 
Think Twice Review Date: April 24, 2008
Reviewers: Patrick McQuillen, Boston College

In his review, McQuillen commends the report for making a “major contribution” to the debate over school reform, but warns that it has an overly optimistic timeline, relies too much on punitive sanctions, offers key recommendations beyond what research can support and pays little attention to the role students might play in the reform process.

Press Release Link name=Review Link

separator

Report Reviewed:

Fund the Child: Bringing Equity, Autonomy and Portability to Ohio School Finance

Think Tank:

Thomas B. Fordham Institute

This report is the latest in a series which argues that state funding should follow children to the public schools (rather than the districts) of their choice; that per-pupil funding amounts should be weighted according to children’s individual needs and circumstances; and that schools should have flexibility in how they spend the funds they receive. A major goal of these proposals is to end funding disparities within and between districts.

 
Think Twice Review Date: April 7, 2008
Reviewers: Bruce Baker, University of Kansas

In his review, Baker writes that this report is of higher quality than most past reports on the topic and praises it for avoiding overstatement. However, Baker points out that the report’s primary weakness is its general failure to use relevant, empirical research that might provide policy makers with useful insights.

Press Release Link name=Review Link

separator

Report Reviewed:

How Sound an Investment? Analysis of Federal Prekindergarten Proposals

Think Tank:

Lexington Institute

This report considers current proposals for federal involvement in prekindergarten (pre-K). It concludes that federally funded preschool program for all children would be unnecessarily expensive, provide little benefit to most children, decrease options for parents and could likely impair the social development of many children.

 
Think Twice Review Date: March 24, 2008
Reviewers: W. Steven Barnett, Director of the National Institute for Early Education Research at Rutgers University

Barnett’s review of this report finds that it is misleading, poorly grounded, biased and inaccurate. Barnett concludes that the report oversimplifies the debate, ignores relevant research and falls short of thoughtful policy discussion.

Press Release Link name=Review Link

separator

Report Reviewed:

A School Privatization Primer for Michigan School Officials, Media and Residents

Think Tank:

Mackinac Center

This report is billed as a guide for Michigan school officials and other interested in privatizing school services.

 
Think Twice Review Date: February 19, 2008
Reviewers: Clive Belfield, Queens College, City University of New York

Belfield’s review finds that the report offers little evidence to support its premise that privatizing school services saves money or is otherwise beneficial. Belfield credits the report with offering practical steps for issuing and monitoring contracts, but notes that the report’s usefulness stops there.

Press Release Link name=Review Link

separator

Report Reviewed:

Feeling the Florida Heat? How Low-Performing Schools Respond to Voucher and Accountability Pressure

Think Tank:

Urban Institute’s National Center for Analysis of Longitudinal Data in Education Research

This study examines whether Florida’s system of sanctions and incentives for its poorest performing schools has led to improved student achievement. It concludes that the system, including the threat of vouchers, appears to have spurred schools to improve practices and thereby improve performance.

 
Think Twice Review Date: January 15, 2008
Reviewers: Damian Betebenner, National Center for Improvement of Educational Assessment

Thought Betebenner’s review commends the report for its thorough analysis of the data, he observes that a very prominent shortcoming is the report’s overstatement regarding the relationship between the pressure of accountability sanctions and improvements in school achievement. He notes that the report’s title and some of the statements in the body of the report suggest that vouchers and other accountability measures are the cause of the achievement gains despite the fact that there is no supporting evidence that such a causal connection exists.

Press Release Link name=Review Link

separator

Report Reviewed:

The High Cost of Failing to Reform Public Education in Missouri
The High Cost of Failing to Reform Public Education in Indiana
The High Cost of Failing to Reform Public Education in Texas
The High Cost of South Carolina’s Low Graduation Rates
The High Cost of Low Graduation Rates in North Carolina

Think Tank:

The Milton and Rose D. Friedman Foundation

These five sister reports have been published by the Friedman Foundation over the past two years. They assert that private-school voucher programs can reduce the social costs of dropping out while increasing graduation rates.

 
Think Twice Review Date: January 9, 2008
Reviewers: Sherman Dorn, University of South Florida

In his review of these reports, Dorn finds that the reports are of little value because they ignore the abundance of relevant research and offer no means by which to gauge the alleged benefits of vouchers against other alternatives. Dorn advises policymakers who are interested in increasing graduation rates to bypass these reports and seek out “the available, well-researched scholarship on the topic,” much of which he identifies in his review.

Press Release Link name=Review Link

Reports & Reviews for 2007

Report Reviewed:

1) Are Private High Schools Better Academically than Public High Schools?

2) Monopoly Versus Markets: The Empirical Evidence on Private Schools and School Choice

Think Tank:

1) Center on Education Policy (CEP)

2) Milton and Rose D. Friedman Foundation

These two reports come to different conclusions about whether private schools are better than public ones at educating students. The CEP report found no advantage for either public or private schools using the National Education Longitudinal Study database. The Friedman report used the Educational Longitudinal Study database and presents evidence that private schools produce more benefit.

 
Think Twice Review Date: December 12, 2007
Reviewers: Jaekyung Lee, State University of New York at Buffalo

Lee’s review finds that there is little actual difference between the findings of the two reports – and little difference between public and private schools. The  most that can be concluded from the two reports taken together is that “students generally learn in public high schools about as well as in private high schools.” Lee also presents his own, independent cross-examination of the two data sources used in the reports.

Press Release Link name=Review Link

separator

Report Reviewed:

Shortchanging Disadvantaged Students: An analysis of intra-district spending patterns in Ohio

Think Tank: Buckeye Institute

This report argues that funding disparities in high-poverty Ohio schools are the fault of school districts, not the state. Districts, the report contends, have adequate funds but don’t allocate them properly to high-poverty schools under their jurisdiction.  The report’s central claim is that district-level policies are responsible for continued spending and achievement gaps.

 
Think Twice Review Date: October 17, 2007
Reviewers: Bruce Baker, University of Kansas

Baker’s review finds that the report’s authors fail to provide evidence to support their central claim. Further, the report contains weak methodology and a scant review of existing relevant research. Baker concludes that the report “is built on the weakest of foundations” and suggests that if taken seriously, “the Buckeye Institute report will only misguide policymaking.

The report’s claims are further undermined by a technical appendix presenting Baker’s analysis of Ohio data.
Press Release Link name=Review Link

separator

Report Reviewed:

End It, Don’t Mend It: What to Do with No Child Left Behind

Think Tank: Cato Institute

This report urges scrapping the No Child Left Behind law and replacing it with a national voucher or tax credit system.

 
Think Twice Review Date: October 8, 2007
Reviewers: Bruce Fuller, University of California-Berkeley

Fuller’s review offers praise for the report’s analysis of how NCLB has failed to deliver on its promise of raising student achievement and has resulted in a number of negative consequences for schools including narrowing curriculum and a lowering of the achievement bar. Fuller criticizes the report for its use of “highly selective, even misleading” research to make some of its points. He also points out that the report ignores or downplays research raising serious questions about the benefits of market proposals such as school vouchers.

Press Release Link name=Review Link

separator

Report Reviewed:

The ABC’s of School Choice

Think Tank: Milton and Rose Friedman Foundation

This report, released in the form of a booklet, is designed as a resource for persuading people as to the merits of school choice. The report provides updated information on states that have choice policies in place and begins and ends with a “Frequently Asked Questions” section where the attempt is made to interpret the research on school choice issues for the lay reader.

 
Think Twice Review Date: September 24, 2007
Reviewers: Christopher Lubienski, University of Illinois

Lubienski’s review finds that this report is based on selective and shoddy evidence and makes misleading and false claims.
The reviewer considers the merits of each key factual assertion in the report’s “Frequently Asked Questions” section and points out that there is an overwhelming reliance on a very selective sample of studies from other advocacy organizations that are not peer-reviewed and are highly biased and of questionable quality. Lubienski concludes that the report, as a misleading work of advocacy, offers no useful guidance to policymakers.

Press Release Link name=Review Link

separator

Report Reviewed:

Portfolios – A Backward Step in School Accountability

Think Tank: Lexington Institute

This report argues that the current standardized testing system should be retained and criticizes the use of multiple measures, particularly portfolios, to assess school performance.

 
Think Twice Review Date: September 19, 2007
Reviewers: William Mathis, University of Vermont

Mathis’s review concludes that the Lexington report is ill-founded and of little value as research or policy development. Mathis notes that the report more closely resembles political propaganda than a research report.  According to Mathis, It provides no new data, examines only two studies done and includes only results favorable to the report’s conclusions. In addition, the report is problematic because its author presents portfolios as representative of all non-test measures of student performance – which they are not – and concludes that non-test based measures of performance are ineffective.

Press Release Link name=Review Link

separator

Report Reviewed:

Michigan Higher Education: Facts and Fiction

Think Tank: Mackinac Center for Public Policy
This report argues that Michigan ’s colleges and universities did financially fine during a period of sharp cutbacks to higher education between 2000-2004. Further, based on an analysis of a national set of data, the report finds that states with greater appropriations for higher education are more likely to have lower economic growth.
 
Think Twice Review Date: July 18, 2007
Reviewers: Jose Luis Santos, UCLA
In his review, Santos indicates that the report’s findings and conclusions are poorly grounded and misleading. Though Santos does credit the report with raising a number of important issues, he concludes that the report is basically an advocacy document designed to push policy-makers toward the position that increased state appropriations for higher education are wasteful. Santos’s indicates that the report’s authors “grossly overstate their findings,” and he cautions policy-makers to view the report with great caution.
Press Release Link name=Review Link

Report Reviewed:

Answering the Question That Matters Most: Has Student Achievement Increased Since No Child Left Behind?

Think Tank: Center on Education Policy
This report concludes that since the implementation of NCLB in 2002, on average, student test scores have increased, the achievement gap has narrowed, and achievement gains post-NCLB have increased faster than before NCLB.
 
Think Twice Review Date: July 2, 2007
Reviewers: John T. Yun, University of California at Santa Barbara
Yun's review of this report finds that there are important weaknesses in the methodology used and that the wording of the findings and key conclusions in the report imply a much stronger connection between the implementation of NCLB than can be substantiated by the data.
Press Release Link name=Review Link

separator

Report Reviewed:

School Choice by the Numbers: The Fiscal Effect of School Choice Programs

Think Tank: Milton and Rose Friedman Foundation
This report concludes that school choice programs which subsidize private school attendance with public funds have saved taxpayers nearly half a billion dollars cumulatively over a 15 year period.
 
Think Twice Review Date: May 24, 2007
Reviewers: Bruce Baker, University of Kansas
In his review, Baker points out that the conclusions drawn from the report’s analysis are misleading and oversimplified. In addition, even if accurate, the amount of the savings is insignificant as it represents less than 1/100th of one percent of annual public school spending – about 60 cents per child per year.
Press Release Link name=Review Link

Report Reviewed:

State Takeover, School Restructuring, Private Management, and Students Achievement in Philadelphia

Think Tank: RAND Corporation in partnership with Research for Action (RFA)
Report Reviewed:

School Reform in Philadelphia: A Comparison of Student Achievement at Privately-Managed Schools with Student Achievement in Other District Schools

Think Tank: Program on Education Policy and Governance at Harvard University (PEPG)
These two reports examine the impact of private and district management on student achievement in Philadelphia. In 2002, Philadelphia restructured 86 of its lowest-achieving schools and shifted the management of some of them to private companies. The RAND-RFA report concludes that private management has had no impact on math or reading achievement, while district management had a positive effect on math achievement but none on ready. The PEPG report reached virtually the opposite conclusion. It found that private management has had a positive effect on math and reading achievement, while district management has had no measurable effect.
 
Think Twice Review Date: May 7, 2007
Reviewers: Derek Briggs, University of Colorado at Boulder
Briggs’s review of both reports finds that both have flaws and that neither is yet sufficient to make definitive claims about the effects of Philadelphia’s school restructuring reform. He indicates that the different conclusions reached in the reports can largely be explained by the fact that PEPG did not have the same access to data as did RAND-RFA. He also identifies and describes the methodological weaknesses in both reports. Briggs commends the reports for contributing to the understanding of Philadelphia’s experience, but recommends that neither report be used as a basis for decision making about school restructuring.
Press Release Link name=Review Link

Report Reviewed:

How Much Are Teachers Paid?

Think Tank: Manhattan Institute
This report asserts that teachers are paid at a higher rate and work fewer hours than most white-collar professionals and puts forward an analysis that finds no correlation between their measure of teacher pay and student achievement.
 
Think Twice Review Date: February 19, 2007
Reviewers: Sean Corcoran, New York University
Lawrence Mishel, Economic Policy Institute
The review shows that the report’s analysis of the Bureau of Labor Statistics’ (BLS) data is fundamentally flawed. Corcoran and Mishel point out that the BLS had explicitly warned against using the data in the manner in which it is used and indicate that the defects in the report render it virtually useless.
Press Release Link name=Review Link

Report Reviewed:

Whole Language High Jinks: How to tell when ‘scientifically-based reading Instruction’ isn’t

Think Tank: Thomas B. Fordham Institute
This report asserts that scientific research supports systematic phonics instruction and contends that ideologically biased supporters of whole-language instruction have thwarted the advance of scientifically based reading instruction.
 
Think Twice Review Date: February 14, 2007
Reviewer: Richard Allington, University of Tennessee
In his review of the report, Allington indicates that the report’s author, Louisa Moats, fails to provide convincing research to support her position and goes on to say that Moats exaggerates the findings of the research she does cite and incorrectly dismisses approaches that are well-supported by research. Further, Allington points out that Moats’ report is more a promotion of her own reading products than an unbiased report on reading instruction.
Press Release Link name=Review Link

Report Reviewed:

Report Card on Education, 1983-84 to 2004-05

Think Tank: American Legislative Exchange Council (ALEC)
This report, which promotes ALEC’s policy agenda, asserts that student achievement has not been improved by increased spending on education, smaller class sizes or improved teacher salaries. It concludes that improvement will occur when “accountability, choice and competition are injected into our current educational system.”
 
Think Twice Review Date: January 8, 2007
Reviewer: Gene V Glass, Arizona State University
Glass’s review of this report finds that it falls far short of being valid or useful research. He points out that data in the report are not analyzed and contradictory studies are ignored. In fact, Glass points out that the report fails to cite any research studies at all. He indicates that the report lacks firmly established facts and credible findings on which policy decisions could be based and concludes, “In spite of being clad with myriad numbers and statistics, the Report Card on American Education is rhetoric, not research.”
Press Release Link name=Review Link

Think Twice Reviews Released In 2006







Think Twice - Review Panel



Related Information

New Examining the Funding and Activities of Free Market Education Think Tanks New

Truthiness in Education

Think Tanks At Risk For Corruption

Holding Think Tanks To High Standards

US Think-Tanks: Casualties In The War of Ideas

Far-Right "Think Tanks" (Propaganda Mills) – Who Are Those Guys?

Let The Buyer Beware [ Executive Summary | Full Report ]

These articles and/or reports are copyrighted material, the use of which has not been specifically authorized by the copyright owner. We are making such material available in our efforts to advance understanding of educational issues, etc. We believe this constitutes a 'fair use' of any such copyrighted material as provided for in section 107 of the U.S. Copyright Law. In accordance with Title 17 U.S.C. Section 107, the material on this site is distributed without profit to those who have expressed a prior interest in receiving the included information for research and educational purposes. For more information go to: http://www.law.cornell.edu/uscode/17/107.shtml If you wish to use copyrighted material from this site for purposes of your own that go beyond fair use, you must obtain permission from the copyright owner.