Facebook Twitter LinkedIn rss
Great Lakes Center Logo

2014 Think Twice Reviews

Think Twice is one of the nation's first efforts to serve as a watchdog to review think tank research on public education issues and policies, ensuring that published work meets the quality and standards of university scholarship. As think tank research becomes increasingly important reference sources in public policy debates, media and other critics have called for increased scrutiny to ensure validity and objectivity (click here to see related stories).

The goal of the Think Twice project is to provide the public, policy makers and the press with timely academically sound reviews of selected think tank publications.

Click Here To Read Our Legal Statement

NEPC: 2013 Bunkum Awards

Think Tank Research Quality

Think Tank Research Quality:
Lessons for Policy Makers, the Media, and the Public

Think Tank Research Quality, edited by Kevin Welner of the University of Colorado at Boulder, Patricia H. Hinchey of Pennsylvania State University, Alex Molnar of Arizona State University, and independent researcher Don Weitzman, offers clearly written, jargon-free expert reviews of studies on topics such as vouchers, charter schools, and alternative teacher certification. Friends of The Great Lakes Center receive 20% off phone, fax or mail-in orders...not valid online.

Reports & Reviews for 2014

Report Reviewed: The Effect of Co-locations on Student Achievement in NYC Public Schools
Publisher/Think Tank: The Manhattan Institute

Marcus A. Winters

The report from the Manhattan Institute investigated the effect of colocations of charter schools and traditional public schools on a narrow range of standardized test scores. It focused on students’ test scores and looked at fourth through eighth grade ELA and math standardized assessment gains. Colocation was not found to have a significant impact on students’ test scores.
Think Twice Review Date: April 15, 2014
Reviewer: Tina Trujillo and Marialena Rivera, UC-Berkeley
An academic review of the report finds that the report omits important details about its analysis. Additionally, the report does not build on existing research or background knowledge on colocations or related topics. Furthermore, the report expressly neglects to consider other important outcomes related to students’ socio-emotional development, safety, health, and broader academic experiences. The report does little to help policymakers and practitioners evaluate the effects of colocation on students’ educational experiences and outcomes.

Report Reviewed: Measuring the Impacts of Teachers I (NBER No. 19423) & II (NBER No. 19424)
Publisher/Think Tank: National Bureau of Economic Research (NBER)
Author: Raj Chetty, John N. Friedman, and Jonah E. Rockoff (both papers)
This NBER working paper - split in two parts - studied the impact of teachers on long-term outcomes using value-added scores.  The reports addressed a key question: Are teachers' impacts on students' test scores ("value added") a good measure of their quality? Economists Raj Chetty, John Friedman, & Jonah Rockoff claim in a two-part paper that higher value-added scores for teachers lead to greater economic success for their students later in life. The paper drew widespread media and politicians’ attention, including the president’s.
Think Twice Review Date: April 10, 2014
Reviewer: Moshe Adler, Columbia University & Empire State College, SUNY
Moshe Adler of the department of Urban Planning at Columbia University and the Harry Van Arsdale, Jr. Center for Labor Studies at Empire State College, SUNY reviewed the reports for the Think Twice think tank review project. In his review Adler found five problems that invalidate the paper’s main claim: (1) the paper fails to mention the existence of a crucial conflicting result that was reported in an earlier version of the same paper; (2) the paper claims that there was insufficient data to investigate whether teacher value-added has an effect on income at any age after the age of 28.  This claim is untrue; (3) the method used to calculate the effect of teacher value-added on income at age 28 is biased and misleading; (4) the paper makes assumptions that inflate its main result, but the evidence contradicts these assumptions; and (5) the studies that the paper cites in support of its methodology don’t actually provide that support.

Report Reviewed: The Economic Benefits of New York City’s Public School Reforms, 2002-2013
Publisher/Think Tank: Sonecon, Inc.

Robert J. Shapiro, Sonecon, Inc. &
Kevin A. Hassett, American Enterprise Institute (AEI)

An estimated $74 billion impact was attributed to the reforms (based on earnings of students who graduated under the reforms – who might not have otherwise – and on property values). The report assumes that higher graduation rates and charter school availability increased residential property values in the city.
Think Twice Review Date: March 31, 2014
Reviewer: Sean P. Corcoran, New York University
Professor Sean Corcoran writes, “While such estimates are always an exercise in some level of speculation, this report relies on highly inappropriate assumptions to reach its conclusions.” Breaking down the math of the authors, Corcoran calculates that the impact on property values attributed to the Bloomberg-era educational reforms is comparable to “two-thirds of the entire increase in residential property values between 2007 and 2013.” Corcoran found that many NYC public school students did experience “ real educational and economic gains” during Bloomberg’s time in office, but the estimates that the Sonecon report makes, he concludes, “are pure fantasy.”


Report Reviewed: Updating Career and Technical Education for the 21st Century
Publisher/Think Tank: The Lexington Institute
Author: Kristen Nye Larson
A recent publication from the Lexington Institute makes the claim that schools are failing to unleash the potential of Career and Technical Education (CTE). The report claims that too many CTE programs offered by today’s high schools are outdated and fail to produce 21st century workplace skills. The report showcased effective CTE models underway in the U.S. and suggests elements for replication in other schools. It identified three strategies for replication: (1) partnerships with businesses; (2) improved accountability measures; and (3) innovative curriculum options.
Think Twice Review Date: March 18, 2014
Reviewer: Marisa Saunders and Jaime L. Del Razo, Annenberg Institute for School Reform at Brown University
A review of the report found that it does not identify or produce evidence on the proposed practices. Furthermore, it fails to provide rationale for why the recommended practices are key to improving CTE experiences or opportunities for students. More important, the review finds that the report focuses on workforce readiness and does not identify postsecondary readiness as a critical component. It reinforces the belief that CTE and a college prep curriculum are mutually exclusive. The reviewers note: “The report is limited in its usefulness, both in practice and policy. It both over-reaches and under-reaches.” The report makes broad claims that are “superficial, inconsistent, and lack a coherency.”

Report Reviewed: A New Frontier: Utilizing Charter Schooling to Strengthen Rural Education
Publisher/Think Tank: Bellwether Education Partners; J. A. and Kathryn Albertson Foundation
Author: Andrew Smarick
A recent document from Bellwether Education Partners argues for expanding charter schools into rurally populated states. It recommends opening the rural charter market and removing barriers to expansion. It highlights several state-level policy recommendations and offers five directives to policymakers: (1) replace barriers to growth with smart, flexible policies; (2) provide flexibility from teacher certification rules; (3) provide fair funding; (4) make facilities accessible; and (5) leverage technology.
Think Twice Review Date: March 13, 2014
Reviewer: Craig Howley, Ohio University
In a review, Ohio University’s Craig Howley finds that the document provides no articulated rationale for its recommendations. Citing missing research and slanted representations, Howley declares it “useless as a source of objective information.” The document employed very little peer-reviewed research and no reference was found to any peer-reviewed empirical research in rural education. Howley notes that the methods employed by New Frontier are entirely rhetorical, based on unstated assumptions that charter schools provide positive outcomes in all regards. Also, Howley finds the document “obscures rather than illuminates consideration of rural education, making it more harmful than helpful.” In summarizing the usefulness of New Frontier, Howley says, “The document is useful only to those who seek to expand charter schools into rural areas.”


Report Reviewed: Pluck & Tenacity: How Five Private Schools in Ohio Have Adapted to Vouchers
Publisher/Think Tank: The Thomas B. Fordham Institute
Author: Ellen Belcher
This report highlights perceived challenges facing private schools brought on by the introduction of vouchers and calls for policymakers to strengthen Ohio’s voucher programs. It praises the private schools profiled and cautions against state accountability programs that may “trample” on private schools’ ability to provide a unique learning experience.
Think Twice Review Date: March 6, 2013
Reviewer: Christopher Lubienski and T. Jameson Brewer, University of Illinois
Overall, the reviewers found the report to be a case-study in how to engage in a slanted selection and interpretation of research evidence. The reviewers note that the report is supported by a total of three endnote references “cherry-picked” to support a contested policy agenda. “Thus, the report is grounded in a twice-skewed and intellectually dishonest view of the research on vouchers and their academic outcomes.”

Report Reviewed: A Legal Lever for Enhancing Productivity
Publisher/Think Tank: George W. Bush Institute
Author: Elizabeth Ettema, Krishanu Sengupta, and Sandy Kress
This report examines the extent to which the Texas education system is efficient. Using a five-factor definition of efficiency, the authors argue that in key areas – teacher training, teacher evaluation, teacher pay-setting, and use of instructional materials – the Texas education system is unlikely to be efficient or cannot demonstrate efficiency.
Think Twice Review Date: February 27, 2014
Reviewer: Clive Belfield, Queens College, City University of New York
Dr. Clive Belfield found that the authors failed to prove that the Texas education system is inefficient.  The review noted positively on the theoretical framework (x-efficiency) used to expand the concept of efficiency beyond the common economic definition.  However, the report suffers from a lack of evidence and sufficient analysis to draw the conclusions made.  The report is of limited use for policymakers and education professionals.

Report Reviewed: Should Charter Schools Pay Rent? Implications for Staffing and Growth
Publisher/Think Tank: Manhattan Institute
Authors: Stephen Eide
This report from the Manhattan Institute examined the potential impact of requiring co-located charter schools to pay rent in NYC. The report reflects concerns of charter advocates and operators regarding potential policy changes under Bill de Blasio, New York's new mayor. The report claims that charging rent to co-located charters in NYC would have triggered an average budget deficit of 10.7 percent at those schools. The report also proposes that paying rent could cripple the co-located charters' growth in NYC.
Think Twice Review Date: February 20, 2014
Reviewer: Bruce Baker, Rutgers University
Bruce Baker finds that the report only presents a handful of poorly documented tables and graphs listing potential budget deficits, speculative layoffs, and average proficiency rates of co-located and non-co-located charter schools. Baker says that the report's greatest weakness is in its assumption that there is no possible downside when resources are transferred from city schools to charter schools. The report assumes that subsidies benefit charter schools and halting these subsidies harms charters and benefits no one. More importantly, Baker finds that the report ignores the broader and more complex policy questions of what it takes to manage a balanced and equitable system of schooling options.


Report Reviewed: Transfer Incentives for High-Performing Teachers: Final Results from a Multisite Randomized Experiment
Publisher/Think Tank: Mathematica Policy Research (MPR)
Authors: Steven Glazerman, Ali Protik, Bing-ru Teh, Julie Bruch, and Jeffery Max
The Teacher Transfer Initiative (TTI) actively encouraged 1,500 “high-performing” elementary teachers in 10 urban school districts to transfer to low-performing schools in return for a total stipend of $20,000 distributed over two years. The teachers were all identified as having students who were in the top 20 percent in their districts, based on a value-added measure of standardized test scores. The assumption was that better teachers would get better results on standardized tests wherever they teach.
Think Twice Review Date: January 28, 2014
Reviewer: Donald Gratz, Curry College
The results of this extensive study are not encouraging, writes reviewer Donald Gratz, who found several flaws that limit the report’s usefulness for policy consideration. In his review, Gratz identifies several flaws. Among these: (1) the study does not consider the potential impact of other relevant school factors that affect teacher success, such as school leadership; and (2) while it considers the overall cost of the program, it only compares that cost to class size reduction, a generally expensive program.  The most significant problem, Gratz says, is that this two-year study was too short to draw conclusions either about teacher retention or student achievement, limiting its impact and usefulness


Report Reviewed: Fixing Classroom Observations: How Common Core Will Change the Way We Look at Teaching
Publisher/Think Tank: TNTP, Student Achievement Partners
This report suggests that many states and school districts are rolling out new teacher evaluation systems that will not succeed in improving instruction because they have not been updated to reflect the Common Core. The report asserts that new evaluation systems have not remedied practical challenges of classroom observations. It calls for increased emphasis on lesson content and a paring down of observation rubrics – to make them more focused and clear.
Think Twice Review Date: January 21, 2014
Reviewer: Jennie Whitcomb, University of Colorado Boulder
Dr. Jennie Whitcomb, associate dean for teacher education at the University of Colorado Boulder, concludes that simply streamlining the instruments and paying greater attention to what content is taught are unlikely to address the core problems surrounding teacher evaluations. Rather than solving implementation problems facing schools, Whitcomb says, “The report appears to be a sales-pitch for TNTP’s soon-to-be-launched observation tools focused on Common Core lesson content.”


These articles and/or reports are copyrighted material, the use of which has not been specifically authorized by the copyright owner. We are making such material available in our efforts to advance understanding of educational issues, etc. We believe this constitutes a 'fair use' of any such copyrighted material as provided for in section 107 of the U.S. Copyright Law. In accordance with Title 17 U.S.C. Section 107, the material on this site is distributed without profit to those who have expressed a prior interest in receiving the included information for research and educational purposes. For more information go to: http://www.law.cornell.edu/uscode/17/107.shtml. If you wish to use copyrighted material from this site for purposes of your own that go beyond fair use, you must obtain permission from the copyright owner.



Worth A Read

A weekly selection of thought-provoking research and commentary focused on
education reform.


Delivered by FeedBurner


Stay Informed

Enter your email address to receive the latest news and information from The Great Lakes Center. We value your privacy and do not share email addresses with any other organizations.

A value is required.
Invalid format.