ABOUT   DONATE   SUBSCRIBE

ABOUT   DONATE   SUBSCRIBE

Facebook Twitter LinkedIn rss
Great Lakes Center Logo

2009 Think Twice Reviews

Think Twice is one of the nation's first efforts to serve as a watchdog to review think tank research on public education issues and policies, ensuring that published work meets the quality and standards of university scholarship. As think tank research becomes increasingly important reference sources in public policy debates, media and other critics have called for increased scrutiny to ensure validity and objectivity (click here to see related stories).

The Think Twice project is funded by the Great Lakes Center which commissions the National Education Policy Center (NEPC) to conduct expert third party reviews of think tank publications for the project. NEPC is directed by University of Colorado at Boulder professor Kevin Welner. Its publications director is Arizona State University professor Alex Molnar.

Click Here To Read Our Legal Statement

Reports & Reviews for 2009

Report Reviewed: Everyone Wins: How Charter Schools Benefit All New York City Public School Students
Publisher/Think Tank: Manhattan Institute
This report looks at the competition effects of New York City's charter schools and concludes that "students benefit academically when their public school is exposed to competition from a charter."
 
Think Twice Review Date: November 17, 2009
Reviewer: Patrick McEwan, Wellesley College
McEwan's review of the Manhattan Institute's report finds that the report's findings are minimal and may be explained by factors other than charter-school competition.
Press Release Link name=Review Link

 

Report Reviewed: How New York City's Charter Schools Affect Achievement
Publisher/Think Tank: The New York City Charter Schools Evaluation Project
This report estimates the effects on student achievement of attending a New York City charter school rather than a traditional public school. A key finding compares the cumulative effect of attending a New York City charter school for 9 years to the magnitude of average test score differences between students in Harlem and the wealthy New York community of Scarsdale.
 
Think Twice Review Date: November 12, 2009
Reviewer: Sean Reardon, Stanford University
Reardon's review finds that the report "has the potential to add usefully to the growing body of evidence regarding the effectiveness of charter schools." However, it also notes that the report's key findings are grounded in an unsound analysis—an inappropriate set of statistical models—and that the report's authors never provide crucial information that would allow readers to more thoroughly evaluate "its methods, results, or generalizability." He warns that "Policymakers, educators, and parents should therefore not rely on these estimates until the bias issues have been fully investigated and the analysis has undergone rigorous peer review."
Press Release Link name=Review Link

 

Report Reviewed: The Widget Effect: Our National Failure to Acknowledge and Act on Teacher Differences
Publisher/Think Tank: The New Teacher Project
This report describes serious problems with teacher evaluation systems based on surveys conducted in 12 school districts in four states. The report urges changes to establish more effective evaluation systems that will be able to discern the highest and lowest quality teachers.
 
Think Twice Review Date: August 5, 2009
Reviewer: Raymond Pecheone and Ruth Chung Wei, Stanford University
The reviewers find that the Widget report is well grounded and praises the report's policy recommendations as sound and consistent with prior research. However, the review also notes that the report lacks important details about its own methods and fails to consider or to draw upon any promising teacher evaluation strategies currently in use.
Press Release Link name=Review Link

 

Report Reviewed: Multiple Choice: Charter School Performance in 16 States
Publisher/Think Tank: Center for Research on Education Outcomes (CREDO), Stanford University
This report involved a large scale analysis of 65-70% of the nation’s charter schools and found that average charter performance is equal to, or perhaps lower than, the performance of traditional public schools.
 
Think Twice Review Date: June 24, 2009
Reviewer: Gary Miron and Brooks Applegate, Western Michigan University
Miron’s and Applegate’s review of the report finds that the solid analytic approach employed and the comprehensive data set used make the report a valuable addition to the body of charter school research.
Press Release Link name=Review Link

 

Report Reviewed: The Fiscal Impact of Tax-Credit Scholarships in Montana

The Fiscal Impact of Tax-Credit Scholarships in Georgia

The Fiscal Impact of a Corporate & Individual Tax-Credit Scholarship Program on the State of Indiana

Publisher/Think Tank: Friedman Foundation for Educational Choice
The Friedman reports which focus on Georgia, Montana and Indiana purport to demonstrate that programs awarding tax-credits to donors funding private school vouchers will reduce government expenditures and make the finance system more efficient.
 
Think Twice Review Date: June 22, 2009
Reviewer: Luis Huerta, Teachers College, Columbia University.
In his review, Huerta finds the reports' conclusions highly suspect and sharply criticizes them for presenting unsubstantiated claims and failing to adequately consider short- and long-term costs of such tax-credit programs.
Press Release Link name=Review Link

 

Report Reviewed: The MPCP Longitudinal Educational Growth Study Second Year Report
Publisher/Think Tank: School Choice Demonstration Project; Department of Education Reform, University of Arkansas
This report on the second year of a five-year evaluation of the Milwaukee school voucher program finds no achievement growth differences between voucher schools and the Milwaukee Public Schools over a one-year period.
 
Think Twice Review Date: May 28, 2009
Reviewer: Casey Cobb
Cobb's review finds that the report's design and methods are sound, but its usefulness is limited by several factors, including a lack of clarity about the representativeness of the included voucher schools.

Correction On June 24, 2009, the authors of the report under review brought to our attention an error in the original May 28, 2009 review of their work by Casey Cobb. Professor Cobb has revised his review accordingly. As revised, the review includes an important correction. In the initial review, Professor Cobb made an incorrect assumption about the sampling procedures used in the study. As a result, he drew the incorrect conclusion that the external validity of the findings was tempered by a non-random sample of MPCP students, and hence a non-representative sample of MPCP schools. As the study authors have pointed out, the MPCP sample was in fact drawn randomly.
Press Release Link name=Review Link

 

Report Reviewed: Charter Schools in Eight States: Effects on Achievement, Attainment, Integration and Competition
Publisher/Think Tank: RAND Corporation
This report on charter schools in eight states generally finds that charters have had only modest or insignificant effects on such things as student achievement, student attainment, integration and public school competition.
 
Think Twice Review Date: May 27, 2009
Reviewer: Derek C. Briggs
Briggs’s review finds some weaknesses in the data regarding some of the findings, but overall concludes that the report is of high quality and makes an important contribution to the empirical literature on charter school effectiveness.
Press Release Link name=Review Link

 

Report Reviewed: Evaluation of the DC Opportunity Scholarship Program: Impacts After Three Years
Publisher/Think Tank: School Choice Demonstration Project; Institute of Educational Sciences, National Center for Educational Evaluation and Regional Assistance
The report finds that after three years, students in the D.C. voucher program showed modestly higher reading scores, but no significant difference in math scores when compared with a control group of students who didn’t get vouchers.
 
Think Twice Review Date: May 21, 2009
Reviewer: Martin Carnoy, Stanford University
Carnoy’s review points out that any change in scores is modest and provides little support for vouchers as a solution to D.C.’s educational problems. He suggests that the report could have done far better in analyzing the results and presenting them in a more nuanced fashion.
Press Release Link name=Review Link

 

Report Reviewed: The Effect of Milwaukee's Parental Choice Program on Student Achievement in Milwaukee Public Schools
Publisher/Think Tank: Choice Demonstration Project; Department of Education Reform, University of Arkansas
The report contends that competition from Milwaukee's private school voucher program has benefited Milwaukee public schools.
 
Think Twice Review Date: May 18, 2009
Reviewer: Gregory Camilli, Rutgers University
Camilli's review raises a number of questions about the statistical methods used in the report and concludes that any positive effect of competition is very small, if it exists at all.
Press Release Link name=Review Link

 

Report Reviewed: Weighted Student Formula Yearbook 2009
Think Tank: Reason Foundation
The report examines 14 city school systems and one statewide one - Hawaii- that the report presents as reflecting "best practices" in implementing what it calls Weighted Student Funding reforms. The report is based on two underlying premises: (1) school budgets should be allocated directly to schools within a district, with the amount based on each child's needs; and (2) school principals should have full discretion on how to allocate those funds.
 
Think Twice Review Date: May 13, 2009
Reviewers: Bruce Baker, Rutgers University
The Baker review of the report finds that it cherry-picks evidence, lumps many different strategies under a single reform umbrella, ignores contradictory findings, and in one-third of its examples credits the reforms for outcomes that actually preceded the reforms themselves. Baker concludes that the report is reckless and irresponsible.
Press Release Link name=Review Link

 

Report Reviewed: The Impact of Milwaukee Charter Schools on Student Achievement
Think Tank: Brookings Institution
This report finds that there are no math achievement gains and small gains in reading in students who attend Milwaukee charter schools when compared to traditional public schools. The report also found that there is no evidence that the presence of charter schools influences that performance in traditional public schools.
 
Think Twice Review Date: May 5, 2009
Reviewer: Robert Bifulco, Syracuse University
Though Bifulco finds that the methods used in the study have important strengths, he identifies several questions about the internal and eternal validity of the reported estimates. He indicates that this report is most informative when considered in the context of the larger body of research on charter schools. He finds the report's conclusion that charter schools "should not be expected to be the silver bullet that some reformers seek" to be reasonable.
Press Release Link name=Review Link

 

Report Reviewed: The Fiscal Impact of the Milwaukee Parental Choice Program: 2009 Update
Think Tank: School Choice Demonstration Project;
Department of Education Reform University of Arkansas
This report examines issues concerning the funding formula used for the Milwaukee Parental Choice Program and finds that it generates a net savings to taxpayers in Wisconsin but imposes a large fiscal burden on taxpayers in Milwaukee.
 
Think Twice Review Date: April 27, 2009
Reviewer: Christopher Lubienski, University of Illinois
Belfield finds that the report offers valuable insights on the fiscal impact of voucher programs. However, he questions the benefits and notes that the report omits some important factors which are essential in order for a more complete analysis to be done.
Press Release Link name=Review Link

 

Report Reviewed: A Win-Win Solution: The Empirical Evidence on How Vouchers Affect Public Schools
Think Tank: Friedman Foundation for Educational Choice
This report purports to present a summary of previous research about the competitive effect of vouchers on public schools. The report concludes that such competition improves public schools.
 
Think Twice Review Date: April 27, 2009
Reviewer: Christopher Lubienski, University of Illinois
Lubienski finds that the report misrepresents the research and selectively reads the evidence in some of the studies. He points out that the  majority of the studies cited were produced by voucher advocacy organizations and almost none were peer reviewed. Lubienski concludes that the report is actually “an overview that seems designed to build a pro-voucher argument rather than an evenhanded presentation of research.”
Press Release Link name=Review Link

 

Report Reviewed: When Private Schools Take Public Dollars: What’s the Place of Accountability in School Voucher Programs?
Think Tank: Thomas B. Fordham Institute
This report examines the opinions of experts who are school choice advocates regarding accountability issues for private schools that accept public monies (vouchers).
 
Think Twice Review Date: April 15, 2009
Reviewer: Ernest House, University of Colorado at Boulder
House praises the report and says that it offers some useful insights. He credits the Fordham researchers with making an honest attempt to tackle the difficulties of holding private schools accountable.
Press Release Link name=Review Link

 

Report Reviewed: An Evaluation of Teachers Trained Through Different Routes to Certification
Think Tank: Mathematica Policy Research, Inc.
This report finds that students who were assigned an alternatively certified teacher did no worse on achievement tests than students whose teacher came through the traditional teacher-education route. Moreover, the report concludes that there is no association between greater amounts of teacher training coursework and effectiveness in the classroom.
 
Think Twice Review Date: March 10, 2009
Reviewer: Sean P. Corocoran, New York University and Jennifer Jennings
In their review of this report, the authors find that the conclusions drawn by the report are not supported by the report’s actual data. They contend that few, if any, valid conclusions about teacher certification policy can be drawn from the Mathematica report and that any conclusions that are drawn tend to favor traditional certification routes.
Press Release Link name=Review Link

 

Report Reviewed: The High Cost of High School Dropouts in Ohio
Think Tank: The Buckeye Institute for Public Policy Solutions
This report offers up “dropout recovery” charter schools as a solution to reduce Ohio’s dropout rates.
 
Think Twice Review Date: March 4, 2009
Reviewer: Sherman Dorn, University of South Florida
In his review of this report, Dorn finds that the data the report relies upon is grossly exaggerated when compared to Ohio state data regarding the number of students graduating from charter schools. In addition, Dorn points out that the report ignores a large body of useful charter school research. He recommends that policymakers seek out well-researched scholarship on this topic and not rely upon this report.
Press Release Link name=Review Link

These articles and/or reports are copyrighted material, the use of which has not been specifically authorized by the copyright owner. We are making such material available in our efforts to advance understanding of educational issues, etc. We believe this constitutes a 'fair use' of any such copyrighted material as provided for in section 107 of the U.S. Copyright Law. In accordance with Title 17 U.S.C. Section 107, the material on this site is distributed without profit to those who have expressed a prior interest in receiving the included information for research and educational purposes. For more information go to: http://www.law.cornell.edu/uscode/17/107.shtml. If you wish to use copyrighted material from this site for purposes of your own that go beyond fair use, you must obtain permission from the copyright owner.