ABOUT   DONATE   SUBSCRIBE

ABOUT   DONATE   SUBSCRIBE

Facebook Twitter LinkedIn rss
Great Lakes Center Logo

2015 Think Twice Reviews

Think Twice is one of the nation's first efforts to serve as a watchdog to review think tank research on public education issues and policies, ensuring that published work meets the quality and standards of university scholarship. As think tank research becomes increasingly important reference sources in public policy debates, media and other critics have called for increased scrutiny to ensure validity and objectivity (click here to see related stories).

The goal of the Think Twice project is to provide the public, policy makers and the press with timely academically sound reviews of selected think tank publications.

Click Here To Read Our Legal Statement

NEPC: 2014 Bunkum Awards



Think Tank Research Quality

Think Tank Research Quality:
Lessons for Policy Makers, the Media, and the Public

Think Tank Research Quality, edited by Kevin Welner of the University of Colorado at Boulder, Patricia H. Hinchey of Pennsylvania State University, Alex Molnar of Arizona State University, and independent researcher Don Weitzman, offers clearly written, jargon-free expert reviews of studies on topics such as vouchers, charter schools, and alternative teacher certification. Friends of The Great Lakes Center receive 20% off phone, fax or mail-in orders...not valid online.

Reports & Reviews for 2015

Report Reviewed: Increasing Education: What it Will and Will Not Do for Earnings and Earnings Inequality
Publisher/Think Tank: The Hamilton Project
Author: Brad Hershbein, Melissa S. Kearney, and Lawrence H. Summers
A recent report from the Hamilton Project sought to study how public investment in education will further long-term prosperity, economic growth, and individual economic security. The inquiry focused on whether or not a bachelor’s degree or higher education would increase economic prosperity and reduce economic inequality.
 
Think Twice Review Date: May 18, 2015
Reviewer: Marvin Lazerson, Central European University and University of Pennsylvania; and Ryan Pfleger, University of Colorado Boulder
In the review, the authors find that the report and its assertions are straightforward and use an empirically-based simulation for future projections. However, the report has several limitations. Specifically, Marvin Lazerson and Ryan Pfleger find the following insufficiencies: (1) there is little direct evidence in the report to show that increasing educational attainment is the most “efficient and effective,” as the report describes, way to improve prosperity; (2) the data are drawn are only from males, with no attention paid to gender, race, field of study, labor-market conditions, or institutional reputation; and (3) no data were analyzed to evaluate other ways to address economic problems. The reviewers conclude, “Claiming that the primary solution to a wide array of economic problems is to improve ‘human capital,’ the report perpetuates a problematic myth that undervalues alternative ways to address poverty and economic insecurity.”


Report Reviewed: Dramatic Action, Dramatic Improvement: the Research on School Turnaround
Publisher/Think Tank: Center for American Progress (CAP)
Author: Tiffany D. Miller and Catherine Brown
The Center for American Progress (CAP) recently released a report advocating for implementation of evidence-based best practices for turning around low-performing schools through the federal School Improvement Grant (SIG) program. The report asserts that it offers five lessons about the most effective, research-based methods for turning around low-performing schools through the federal SIG program.
 
Think Twice Review Date: May 11, 2015
Reviewer: Tina Trujillo, University of California Berkeley
Tina Trujillo, in her review, finds that the report essentially ignores large bodies of research on high-stakes accountability, school improvement, and emerging evidence on school closures and federally funded turnarounds. Specifically, Trujillo cautions that a more inclusive review of extant research “reveals that the federal SIG program’s turnaround policies are based on unwarranted claims and are contradicted by the empirical evidence.” Trujillo’s review identifies that the report failed to meet standards for evidence and analytic transparency, and is of little use to policymakers or practitioners.


Report Reviewed: Pushed Out? Low-Performing Students and New York City Charter Schools
Publisher/Think Tank: Manhattan Institute
Author: Marcus A. Winters
A Manhattan Institute report by Marcus A. Winters recently investigated claims that New York City charter schools are pushing out low-performing students as a means of inflating academic achievement scores. The report suggested that charter school exit rates were similar to traditional public schools (TPS). In short, the report determines that there is no charter push-out effect for low-performing students in charter schools.
 
Think Twice Review Date: May 4, 2015
Reviewer: Erica Frankenberg, Penn State University
An academic review of the report finds, despite a rich dataset available for the analysis, that the report has little detail and fails to provide guidance to policymakers. Frankenberg, in her review, discovers that: (1) the research design does not address its primary push-out question; (2) the report has little detail; (3) the report does not examine other relevant factors; and (4) there is a substantial gap between the findings and the conclusions of the report. Specifically, Frankenberg notes that the report’s findings show attrition rates for low-achieving students are higher than attrition of higher-performing students. The report’s conclusions that NYC charter schools are not pushing out students are unsupported.


Report Reviewed: Urban Charter School Study
Publisher/Think Tank: Center for Research on Education Outcomes (CREDO) at Stanford University
A recent report from the Center for Research on Education Outcomes (CREDO) at Stanford University attempted to investigate whether charter schools generate better outcomes than traditional public schools (TPS) in urban environments. The report, part of a series of reports on the performance of charter schools relative to TPS, asserts charter schools in urban environments provide a slightly greater test score advantage than those in non-urban environments. The report utilizes a methodological approach similar to previous reports from CREDO, and finds that students in urban charter schools were estimated to score approximately 0.055 standard deviations higher on math tests and 0.039 standard deviations higher on reading tests than their peers in urban TPSs.
 
Think Twice Review Date: April 27, 2015
Reviewer: Andrew Maul, University of California Santa Barbara
An academic review issues concerns with the methodology and reporting of the CREDO study. In a review, Andrew Maul cites the following concerns: (1) the study’s “virtual twin” technique is insufficiently documented; (2) the report’s estimation of growth using “days of learning” requires accepting untested assumptions; and (3) the study includes a number of arbitrary and unexplained analytic choices. All the same, Maul states, “Even setting aside such concerns over analytic methods, the actual effect sizes reported are very small, explaining well under a tenth of one percent the variance in test scores.” In his conclusion, Maul says “The findings of this report cannot be regarded as compelling evidence of the greater effectiveness of charter school compared to traditional public schools, either overall or specifically within urban districts.”


Report Reviewed: Measuring and Understanding Education Advocacy
Publisher/Think Tank: Brown Center on Education Policy at Brookings
Author: Grover J. “Russ” Whitehurst, David Stuit, Claire Graves, and Lauren Shaw
The report looked at the causal influence of organizations for and against education reform in Louisiana, Tennessee, and North Carolina. According to the report, findings indicated that coordination of advocacy groups strengthens their impact on the introduction of policy into the legislative arena, content of legislation, and the votes of members of the legislature. The report also found that the groups’ perceived influence closely tracked outcomes. The report utilized two methodological innovations to measure the impact of advocacy groups on education reform policy: (1) Surveys with Placebo (SwP); and (2) Critical Path Analysis (CPA).
 
Think Twice Review Date: April 16, 2015
Reviewer: Robin Rogers, Queens College, CUNY
Sara Goldrick-Rab, University of Wisconsin-Madison
An academic review of the report finds there is a gap between the evidence and the conclusions presented, however, the methods used in the report may be useful in education policy research. Regarding the overall merits of the report, Rogers and Goldrick-Rab state, “this report is useful primarily for understanding the perceptions of education advocacy groups’ influence and tactics in the three cases studied.” In their review, Rogers and Goldrick-Rab, find that SwP and CPA may be useful in education policy research, but the methods have limitations that are not acknowledged in the report. Additionally, the reviewers suggest the report is further limited because the research is a small case study, included a low response rate, and were based on advocacy groups’ self-reported tactics.


Report Reviewed: Paying the Best Teachers More to Teach More Students
Publisher/Think Tank: Edunomics Lab at Georgetown University
Authors: Marguerite Roza and Amanda Warco
A recent report from the Edunomics Lab at Georgetown University proposed that school districts pay top performing teachers a bonus for accepting additional students into their existing classes. The report claimed that larger classes and reductions to the teaching force would create significant savings. According to the report, districts should pay teachers in the top quartile a bonus for increasing their class size by up to three students. The report considers that students working with more effective teachers would offset any potential sacrifice in student learning.
 
Think Twice Review Date: April 13, 2015
Reviewer: Patricia H. Hinchey, Penn State University
An academic review of the report finds that the report is largely unsubstantiated, ignores what is known about teacher pay, and fails to offer guidance for policymaking. In her review, Penn State’s Patricia Hinchey finds that the report: (1) ignores the technical problem of how the best teachers might be reliably identified; (2) neglects a strong research base that has established a link between class size and student learning; and (3) misrepresents what is known about teacher pay, teacher attitudes, and teacher job satisfaction.

 

Report Reviewed: Separating Fact and Fiction: What You Need to Know about Charter Schools
Publisher/Think Tank: National Alliance for Public Charter Schools (NAPCS)
The report addressed 21 "myths" regarding charter schools, which were rejected. Succinctly, the original report addressed various claims about charter schools in such areas as financial equality of charter schools, lower teacher qualifications, student selection demographics, academic outcomes, segregation, and innovation.
 
Think Twice Review Date: February 23, 2015
Reviewer: Gary Miron, Western Michigan University
Kevin G. Welner, University of Colorado Boulder
and William J. Mathis, University of Colorado Boulder
The reviewers found that the report’s main purpose appears to be the "repetition or 'spinning' of claims voiced by advocacy groups and think tanks that promote privatization and school choice." Furthermore, the reviewers found that it relied almost exclusively on advocacy documents rather than more careful and balanced empirical research, and provides only a superficial examination of any "criticisms" regarding charter schools. The review is organized in a format that lists each of the criticisms identified, and then provides a short commentary based on the extant research literature. Where the original document overlooked research evidence, the reviewers provide readers with a valuable tool to examine charter school criticisms.


Report Reviewed: The Texas Economy and School Choice
Publisher/Think Tank: Texas Public Policy Foundation & Texas Association of Business
Authors: Arthur Laffer, Laffer Associates Investment Research
The Texas Association of Business and the Texas Public Policy Foundation commissioned Laffer Associates to perform an analysis of the Taxpayer Savings Grant Program (TGSP), which is a universal voucher program for Texas. The report theorized that by raising graduation rates, improving education achievement, and thus increasing human capital, the TSGP would create economic growth in Texas.
 
Think Twice Review Date: February 17, 2015
Reviewer: Chris Lubienski & Ee-Seul Yoon, University of Illinois
Chris Lubienski and Ee-Seul Yoon, in their review, highlight two key problems with the report: (1) Arthur Laffer’s assertions about the educational benefits of choice represent a severe overreach with and misapplication of the available research; and (2) the economic estimations are overgeneralized and heavily biased towards those families who already have the wealth to choose and relocate. The reviewers also note that the TSGP could result in further inequities for Texas schoolchildren, as higher income families would be able to supplement their children’s education even further, while devoting fewer resources to low-income families. In their conclusion, Lubienski and Yoon state: “While this report is clearly written to recommend the TSGP to the State of Texas, the lack of comprehensiveness and transparency, as well as the problems in its methodology, literature review and analysis, make it unsuitable as a basis for public policy decisions.”

 

Report Reviewed: Turning Lightning into Electricity: Organizing Parents for Education Reform
Publisher/Think Tank: American Enterprise Institute (AEI)
Author: Andrew P. Kelly
Turning Lightning into Electricity, written by Andrew P. Kelly, presents an inside look at several Education Reform Advocacy Organizations (ERAO) to examine the issue of parent participation and community organizing. The report offers strategy for advocacy groups who support the ERAO agenda to compete at the grassroots level.
 
Think Twice Review Date: February 2, 2015
Reviewer: Mark R. Warren, University of Massachusetts Boston
A review of the report released today finds limitations to the research methods and that the report cannot be considered a reliable study of parent organizing in ERAOs. Mark Warren, in his review, finds that the research methods employed are not adequately explained. Moreover, the research presented does not appear to be systematic or representative of ERAOs. He says: “the sample of ERAO groups is biased toward groups that more highly value long-lasting forms of parent engagement.” According to Warren, the report reduces organizing to a set of tactics to engage parents around an issue or agenda, rather than a democratic practice. “This approach undermines an understanding of community organizing as a profoundly democratic practice rooted in deeply held values of equity and social justice.”


Report Reviewed: No Excuses Charter Schools: A Meta-Analysis of the Experimental Evidence
Publisher/Think Tank: Department of Education Reform (DER) at the University of Arkansas
Authors: Albert Cheng, Collin Hitt, Brian Kasida, & Jonathan N. Mill
The authors of this working paper declared that students attending No Excuses charter schools had significantly improved math and reading scores. No Excuses schools feature high academic standards, strict disciplinary codes, extended instructional time, and targeted supports for low-performing students. The hope is that by having high standards for academics and discipline, No Excuses schools will help close the achievement gap between minority students and their White peers. The authors used findings from 10 studies that utilized experimental methods to estimate achievement outcomes.
 
Think Twice Review Date: January 20, 2015
Reviewer: Jeanne M. Powers, Arizona State University
Jeanne Powers, Arizona State University, found that the working paper is of limited value to policymakers or anyone seeking to understand the advantages or disadvantages of the No Excuses model. Powers, in her review, finds several flaws in the paper: (1) students who apply to charter school lotteries are not representative of all charter school students; (2) the authors did not address student attrition from charter schools; and (3) the results are based on a small sample of schools concentrated in East Coast cities. She says, “As a result, the author’s claim that No Excuses schools can close the achievement gap substantially overstates their findings.” Furthermore, Powers cautions “the current research base is too limited to make conclusions about the effectiveness of No Excuses charter schools.”

 

Report Reviewed: Proposed 2015 Federal Teacher Preparation Regulations
The U.S. Department of Education has released proposed Teacher Preparation Regulations under Title II of the Higher Education Act with a call for public comments through the Federal Register. The comment period closes February 2. The Department of Education claims that the proposed regulations will help ensure teacher-training programs are better preparing educators to succeed in the classroom by requiring states to measure outcomes of how graduates are doing in the classroom. The proposal would require states to assess and rate every teacher preparation program every year with four Performance Assessment Levels (exceptional, effective, at-risk, and low-performing), and states would be required to provide technical assistance to “low-performing” programs. Additionally, programs that do not show improvement could lose state approval, state funding, and federal student financial aid.
 
Think Twice Review Date: January 12, 2015
Reviewer: Kevin K. Kumashiro, University of San Francisco
An academic review of the proposed regulations considers the evidentiary support and identifies concerns. In his review Kevin Kumashiro has identified seven concerns with the proposed regulations. The proposed regulations: (1) will likely burden institutions with costs that are higher than estimated; (2) inaccurately conceptualize the impact and preparedness of teachers apart from systems; (3) mandate an improperly narrow definition of teacher classroom readiness; (4) require a reliance on scientifically discredited test-based accountability and value-added measures for data analysis; (5) disincentivize teachers to work in high-needs schools; (6) could restrict federal funding for students in financial need, and restrict access to the teaching profession for underrepresented groups; and (7) create a narrow view of the purposes of education.


Report Reviewed: Better Data, Better Decisions: Informing School Choosers to Improve Education Markets
Publisher/Think Tank: American Enterprise Institute (AEI)
Authors: John Valant
This report, published by AEI, asserts that choice is seen as a mechanism, and at times a panacea, for better educational quality. The report provides an overview of what families want from schools, where they get information on schools, and how they use information to make decisions. The report concludes that informed choice and the power of the market will produce better educational outcomes.
 
Report Reviewed: A Crisis We Can Solve: Connecticut’s Failing Schools and Their Impact
Publisher/Think Tank: Connecticut Coalition for Achievement Now (ConnCAN)
Authors: No author named
The ConnCAN report makes a case that there is a desperate need to improve school quality in Connecticut’s neediest neighborhoods and promotes charter schools as the best and perhaps only method for increasing the number of high-quality educational seats in Connecticut.
 
Think Twice Review Date: January 5, 2015
Reviewer: Erin McNamara Horvat, Temple University; and David Everington Baugh, Bensalem Township School District, Pennsylvania
A review of the reports finds that the ConnCAN report is more opinion than fact, while the AEI report provides useful information to help parents make informed decisions. However, the reviewers question whether simply having more information to make better decisions is sufficient to improve our educational system. Reviewers Horvat and Baugh

 

Report Reviewed: The Economic Benefits of Closing Education Achievement Gaps
Publisher/Think Tank: Center for American Progress (CAP)
Author: Robert Lynch and Patrick Oakford
This report from the Center for American Progress (CAP) claimed that closing racial and ethnic achievement gaps would have substantial economic benefits, including raising incomes and the size of the economy. The gains would result from higher levels of student achievement and specifically from higher achievement by minority students. The report estimates that if Black and Hispanic high school math scores converged to equal those of white high school students, the size of the U.S. economy would increase by $20 trillion. Additionally, federal and state/local tax revenues would increase as a result.
 
Think Twice Review Date: December 4, 2014
Reviewer: Clive Belfield, Queens College, City University of New York (CUNY)
In his review, Professor Clive Belfield says, “the report does not include much detail concerning specific calculations, and does not check the accuracy of its estimates.”  According to Belfield, “the report does not provide enough detail for readers to see how big the efficiency gains are, and readers are asked to accept that closing achievement gaps – rather than raising graduation rates or enhancing socioemotional skills – will lead to the biggest economic pay-off.” In his conclusion, Belfield affirms that despite being helpful to policymakers, a closer look at the economic models would be useful for deciding how much investment is needed for long-term improvements.


Report Reviewed: Federal School Finance Reform: Moving Toward Title I Funding Following the Child
Publisher/Think Tank: Reason Foundation
Authors: Katie Furtick & Lisa Snell
The report provided an overview of federal Title I funding, and attempted to examine problems with current funding. The report highlights two bills introduced during the 113th U.S. Congress, which promote portability – allowing funding to follow the child.
 
Think Twice Review Date: November 6, 2014
Reviewer: Gail L. Sunderman, University of Maryland
In her review, Gail Sunderman of the University of Maryland says “the report is little more than a polemic, using an eclectic assortment of disconnected facts and figures about Title I funding to promote choice and voucher policies.” Additionally, she finds that the report provides no evidence that its recommendations will improve student outcomes, and fails to consider the adverse impact on improving educational opportunities, a key tenet of Title I funding. Regarding the report’s use of research literature, she finds that it selectively cites research that supports its arguments while also ignoring contradictory evidence.

 

 

Report Reviewed:
Virtual Schooling and Student Learning: Evidence from the Florida Virtual School
Publisher/Think Tank: Harvard Kennedy School
Author: Matthew M. Chingos and Guido Schwerdt
A recently released paper from the Harvard Kennedy School attempted to compare the performance of students at the Florida Virtual School (FLVS) to students in traditional brick-and-mortar schools. The paper concluded that FLVS students performed about the same or somewhat better on state tests and at a lower cost.
 
Think Twice Review Date: October 28, 2014
Reviewer: Michael K. Barbour, Sacred Heart University
The paper claimed to be the first empirical study of K-12 student performance in virtual education. However, in his review, Michael Barbour specifically calls attention to the fact that this paper is not the first, and the report merely “confirms the findings and repeats the methodological flaws and limitations of previous research.” Specifically, Barbour exposes several methodological errors that need to be addressed: (1) a potential bias of student selectivity in the FLVS sample; (2) the potential impact of regression effects, differential mortality in the two groups, and (3) the fact that the virtual environment is simply a delivery medium.


Report Reviewed: A Meta-Analysis of the Literature on the Effect of Charter Schools on Student Achievement
Publisher/Think Tank: Center on Reinventing Public Education (CRPE)
Authors:

Julian R. Betts & Y. Emily Tang

A report from the Center on Reinventing Public Education (CRPE) attempted to examine whether charter schools have a positive effect on student achievement. The report reviewed 52 studies selected by the authors and found charters to be serving students well, particularly in math. The CRPE report is an update of prior reports that examine the achievement of charter schools as compared to traditional public schools.
 
Think Twice Review Date: September 30, 2014
Reviewer: Francesca López, University of Arizona
In her review, Francesca López says, “The report does a solid job describing the methodological limitations of the studies reviewed, then seemingly forgets those limits in the analysis.” Specifically, López finds fault with the authors’ use of lottery-based studies, cautioning that inclusion limits the study’s usefulness in broad comparisons of all charters vs. public schools. Her review finds several problems that prevent it from being of use to policymakers: (1) claims of positive effects when they are not statistically significant; (2) exaggeration of the magnitude effects; (3) reliance on simple vote-counts from a selected sample of studies; and (4) unwarranted extrapolation of the available evidence.

Report Reviewed: The Efficiency Index
Publisher/Think Tank: GEMS Education Solutions
Author: Peter Dolton, Oliver Marcenaro-Gutierrez, and Adam Still
This report, supported by the Organisation for Economic Co-Operation and Development (OECD), measured efficiency based on test scores, and resource use analyzed in terms of teachers wages and pupil-teacher ratios. It looked at costs for teachers and educational outcomes, and did not consider socioeconomic differences.
 
Think Twice Review Date: September 23, 2014
Reviewer: Clive Belfield, Queens College, City University of New York
In his review, Clive Belfield finds the assumptions that efficiency is best analyzed through international comparisons of test score data to be problematic. He states, “This type of analysis has contributed very little to the improvement of educational policy in the US.” Regarding the report’s usefulness, he finds “the policy implications that flow from the report's own analysis are unrealistic and by the authors' own admission ‘practically impossible.’”


Report Reviewed: Seeds of Achievement: AppleTree’s Early Childhood D.C. Charter Schools
Publisher/Think Tank: Pioneer Institute
Authors:

C. Stillings Candal

This report, funded by the Pioneer Institute and written by Cara Stillings Candal, presents a case, based on anecdotes and basic data, that the AppleTree model, a well-regarded early childhood program in the Washington D.C. area, is “unusually effective” and should be replicated across the nation.
 
Think Twice Review Date: September 16, 2014
Reviewer: W. Steven Barnett and Cynthia E. Lamy, National Institute for Early Education Research
In the review, Barnett and Lamy found that the program described in this case closely resembles the research-based approaches found in other highly effective publicly funded preschool models, but there are limitations to making the kind of claims that the report does to support expansion of preschool charters.

 

Report Reviewed: The Productivity of Public Charter Schools
Publisher/Think Tank: Department of Education Reform (DER) at the University of Arkansas
Author: Meagan Batdorff, Larry Maloney, Jay F. May, Sheree T. Speakman, Patrick J. Wolf, Albert Cheng
The authors of the report claim that this is the first national study of productivity of charter schools relative to traditional public schools. The report utilizes finding from the National Assessment of Educational Progress (NAEP) and “revenues received” to support that charter schools spend less per pupil than traditional public schools and produce achievement as good or superior to that of traditional public schools.
 
Think Twice Review Date: August 19, 2014
Reviewer: Gene V. Glass, University of Colorado Boulder and Arizona State University
In his review, Gene V. Glass found the report inaccurately utilizes NAEP test results and that the assessment of expenditures is based on questionable data, leaving readers with little evidence on which to base any valid conclusions. The report and its conclusions rest on shaking ground and suffer from multiple sources of invalidity, rendering it useless for educators or policymakers.

 


Report Reviewed: Preparing Principals to Raise Student Achievement: Implementation and Effects of the New Leaders Program in Ten Districts
Publisher/Think Tank: RAND Corporation
Authors:

Susan M. Gates, Laura S. Hamilton, Paco Martorell, Susan Burkhauser, Paul Heaton, Ashley Pierson, Matthew Baird, Mirka Vuollo, Jennifer J. Li, Diana Catherine Lavery, Melody Harvey, Kun Gu

A recently released RAND report, an evaluation of the New Leaders principal preparation program, attempted to determine the program’s impact on student test scores. It concluded that New Leaders principals were slightly more effective overall than non-New Leaders principals. The report was based on a statistical analysis of student achievement data on standardized tests.
 
Think Twice Review Date: June 26, 2014
Reviewer: Edward J. Fuller, Penn State University
Ed Fuller (Penn State University) finds no known statistical approach that can identify the independent effect of principals. In addition, Fuller finds the overall conclusion to be problematic. Specifically, he cites problems with effect sizes, the use of multiple VAMs, peer effects, and an unbalanced presentation of positive findings among his cautions for the report’s usefulness. Regarding the results, Fuller finds inconsistencies across years, grade levels, and sites and that the positive results were very small. Encouragingly, Fuller finds the study provides a thoughtful approach to studying principal and principal preparation program (PPP) effectiveness. “Those seeking to evaluate principals and especially PPPs will find the study quite informative.”

 

Report Reviewed: Expanding the Education Universe: A Fifty-State Strategy for Course Choice
Publisher/Think Tank: Thomas B. Fordham Institute
Author: Michael Brickman
Based on the presumed success of school choice programs,a new Fordham Institute report called Expanding the Education Universe: A Fifty-State Strategy for Course Choice seeks to take choice a step further. Each student would design his or her own program of online and off-line courses, choosing their courses from a marketplace of curriculum developed by for-profit and not-for-profit vendors, as well as school districts or other public entities.
 
Think Twice Review Date: June 19, 2014
Reviewer: Patricia Burch, University of Southern California
In the review, the authors found that the report simply assumes that school choice is effective, and that it represents sound public and educational policy. Course choice is presented as a logical extension of that model. In reviewing the literature on the topic of course choice, the authors turned up a dearth of existing research to either support or refute the claims made in the report. The reviewers note that the report is more of "an advocacy piece" which seeks to advance "course choice." In conclusion, the authors find "The piece lacks the empirical evidence and sophisticated discussion necessary for a serious policy proposal."

Report Reviewed: The State Education Agency: At the Helm, Not the Oar
Publisher/Think Tank: Bellwether Education Partners & The Thomas B. Fordham Institute
Authors:

Andrew Smarick & Juliet Squire

A recent report from the Fordham Institute and Bellwether Education Partners argues that states should get out of the reform business directly and hand it over to an "ecosystem of nonprofit organizations." It proposes a governance overhaul to replace the role of State Education Agencies in school improvement. The report outlines the responsibilities of SEAs and suggests that they struggle to deliver.  It cautions against strengthening SEAs to meet new accountability obligations.
 
Think Twice Review Date: June 17, 2014
Reviewer: Peter W. Cookson, Jr., Georgetown University
An academic review finds the report is well written, but reveals that the findings and recommendations are not based on original research. In his review, Peter W. Cookson, Jr. found the report fails to support its findings or recommendations with the in-depth research needed to support the sweeping claims in the report.  "There is no real substantiation of their claims about the failings of SEAs or the efficacy of their solutions; it relies on the preponderance of secondary evidence as the authors' have collected it." Cookson further states: "The methods used in this report are more akin to advocacy journalism than scholarship."


Report Reviewed: New York City's Children First: Lessons in School Reform
Publisher/Think Tank: Center for American Progress
Author: Maureen Kelleher
A recent report from the Center for American Progress summarizes and evaluates the public education reforms during the Bloomberg mayoral era in NYC. The report suggests that positive change resulted from the Children First reforms and makes several policy recommendations. The report describes four aspects of reform during the Bloomberg era, and makes recommendations for NYC and other urban districts. It focuses on top-down school reforms from the mayor's office, rather than on instructional changes inside classrooms.
 
Think Twice Review Date: June 3, 2014
Reviewer: Randal Reback, Barnard College
An academic review finds that the report does an impressive job summarizing the reforms into a single, short narrative. However, several of the claims overhype the success of small high schools and charter schools. In his review, Randall Reback found several misleading claims and endorsements for the success of New York's small school movement. Regarding these claims, he says: "It is thus difficult to separate whether successful small schools were successful due to their ‘smallness' or due to other, correlated factors." He also finds that the report exaggerates student performance outcomes for both charter schools and traditional public schools. For example, he discusses how the report fails to mention concerns about inflated high school graduation rates.

 

Report Reviewed: Time to Improve: How Federal Policy Can Promote Better Prepared Teachers and School Leaders
Publisher/Think Tank: New America Foundation
Authors:

Melissa Tooley and Laura Bornfreund

The report recommends federal regulation of teacher-education programs on the basis of how well their graduates' pupils score on standardized tests.
 
Report Reviewed: Fast Start: Training Better Teachers Faster, with Focus, Practice, and Feedback
Publisher/Think Tank: TNTP
Authors:

Ana Menezes and Adam Maier

The report recommends giving new teachers the tools they need to be successful and proposes a five-week pre-service preparation model. The recommendations include several strategic changes: (1) a narrower curriculum focused only on the most essential teaching skills; (2) an emphasis on practicing skills instead of just learning about them; and (3) intensive coaching that provides regular, specific feedback on changes to instructional techniques.
 
Think Twice Review Date: May 29, 2014
Reviewer: William J. Mathis, University of Colorado Boulder
In his review, Mathis finds that neither report is grounded in research. He also singles out problems in each report. Time to Improve ignores a large body of recent experience and research that cautions against its proposal to the extent that the use of standardized tests scores to assess the quality of teacher preparation programs. Mathis calls the proposed changes in Fast Start "basic survival training" and not teacher preparation. Fast Start fails to show how proposed changes are superior to other approaches.


Report Reviewed: Charter School Funding: Inequity Expands Department of Education Reform (DER) at the University of Arkansas
Publisher/Think Tank: Department of Education Reform (DER) at the University of Arkansas
Author: Meagan Batdorff, Larry Maloney, Jay F. May, Sheree T. Speakman, Patrick J. Wolf, Albert Cheng
The authors of the report claim large and growing inequities between district funding provided through state, local, federal and other sources and charter school revenues from those same sources, even after accounting for differences in student needs. 
 
Think Twice Review Date: May 20, 2014
Reviewer: Bruce Baker, Rutgers University
In his review, Bruce Baker finds that the report has one overarching flaw that invalidates all of its findings and conclusions, "the report displays complete lack of understanding of intergovernmental fiscal relationships, which results in the blatantly erroneous assignment of ‘revenues' between charters and district schools." Baker further states that the report ignores district funding that passes through district schools to charter schools in most states. The serious flaws in the report invalidate its conclusions and any subsequent return-on-investment comparisons claiming they're a better deal because they receive less funding and yet perform as well if not better than traditional public schools.

Report Reviewed: The Effect of Co-locations on Student Achievement in NYC Public Schools
Publisher/Think Tank: The Manhattan Institute
Authors:

Marcus A. Winters

The report from the Manhattan Institute investigated the effect of colocations of charter schools and traditional public schools on a narrow range of standardized test scores. It focused on students' test scores and looked at fourth through eighth grade ELA and math standardized assessment gains. Colocation was not found to have a significant impact on students' test scores.
 
Think Twice Review Date: April 15, 2014
Reviewer: Tina Trujillo and Marialena Rivera, UC-Berkeley
An academic review of the report finds that the report omits important details about its analysis. Additionally, the report does not build on existing research or background knowledge on colocations or related topics. Furthermore, the report expressly neglects to consider other important outcomes related to students' socio-emotional development, safety, health, and broader academic experiences. The report does little to help policymakers and practitioners evaluate the effects of colocation on students' educational experiences and outcomes.

Report Reviewed: Measuring the Impacts of Teachers I (NBER No. 19423) & II (NBER No. 19424)
Publisher/Think Tank: National Bureau of Economic Research (NBER)
Author: Raj Chetty, John N. Friedman, and Jonah E. Rockoff (both papers)
This NBER working paper - split in two parts - studied the impact of teachers on long-term outcomes using value-added scores.  The reports addressed a key question: Are teachers' impacts on students' test scores ("value added") a good measure of their quality? Economists Raj Chetty, John Friedman, & Jonah Rockoff claim in a two-part paper that higher value-added scores for teachers lead to greater economic success for their students later in life. The paper drew widespread media and politicians' attention, including the president's.
 
Think Twice Review Date: April 10, 2014
Reviewer: Moshe Adler, Columbia University & Empire State College, SUNY
Moshe Adler of the department of Urban Planning at Columbia University and the Harry Van Arsdale, Jr. Center for Labor Studies at Empire State College, SUNY reviewed the reports for the Think Twice think tank review project. In his review Adler found five problems that invalidate the paper's main claim: (1) the paper fails to mention the existence of a crucial conflicting result that was reported in an earlier version of the same paper; (2) the paper claims that there was insufficient data to investigate whether teacher value-added has an effect on income at any age after the age of 28.  This claim is untrue; (3) the method used to calculate the effect of teacher value-added on income at age 28 is biased and misleading; (4) the paper makes assumptions that inflate its main result, but the evidence contradicts these assumptions; and (5) the studies that the paper cites in support of its methodology don't actually provide that support.

Report Reviewed: The Economic Benefits of New York City's Public School Reforms, 2002-2013
Publisher/Think Tank: Sonecon, Inc.
Authors:

Robert J. Shapiro, Sonecon, Inc. &
Kevin A. Hassett, American Enterprise Institute (AEI)

An estimated $74 billion impact was attributed to the reforms (based on earnings of students who graduated under the reforms – who might not have otherwise – and on property values). The report assumes that higher graduation rates and charter school availability increased residential property values in the city.
 
Think Twice Review Date: March 31, 2014
Reviewer: Sean P. Corcoran, New York University
Professor Sean Corcoran writes, "While such estimates are always an exercise in some level of speculation, this report relies on highly inappropriate assumptions to reach its conclusions." Breaking down the math of the authors, Corcoran calculates that the impact on property values attributed to the Bloomberg-era educational reforms is comparable to "two-thirds of the entire increase in residential property values between 2007 and 2013." Corcoran found that many NYC public school students did experience " real educational and economic gains" during Bloomberg's time in office, but the estimates that the Sonecon report makes, he concludes, "are pure fantasy."

 

Report Reviewed: Updating Career and Technical Education for the 21st Century
Publisher/Think Tank: The Lexington Institute
Author: Kristen Nye Larson
A recent publication from the Lexington Institute makes the claim that schools are failing to unleash the potential of Career and Technical Education (CTE). The report claims that too many CTE programs offered by today's high schools are outdated and fail to produce 21st century workplace skills. The report showcased effective CTE models underway in the U.S. and suggests elements for replication in other schools. It identified three strategies for replication: (1) partnerships with businesses; (2) improved accountability measures; and (3) innovative curriculum options.
 
Think Twice Review Date: March 18, 2014
Reviewer: Marisa Saunders and Jaime L. Del Razo, Annenberg Institute for School Reform at Brown University
A review of the report found that it does not identify or produce evidence on the proposed practices. Furthermore, it fails to provide rationale for why the recommended practices are key to improving CTE experiences or opportunities for students. More important, the review finds that the report focuses on workforce readiness and does not identify postsecondary readiness as a critical component. It reinforces the belief that CTE and a college prep curriculum are mutually exclusive. The reviewers note: "The report is limited in its usefulness, both in practice and policy. It both over-reaches and under-reaches." The report makes broad claims that are "superficial, inconsistent, and lack a coherency."


Report Reviewed: A New Frontier: Utilizing Charter Schooling to Strengthen Rural Education
Publisher/Think Tank: Bellwether Education Partners; J. A. and Kathryn Albertson Foundation
Author: Andrew Smarick
A recent document from Bellwether Education Partners argues for expanding charter schools into rurally populated states. It recommends opening the rural charter market and removing barriers to expansion. It highlights several state-level policy recommendations and offers five directives to policymakers: (1) replace barriers to growth with smart, flexible policies; (2) provide flexibility from teacher certification rules; (3) provide fair funding; (4) make facilities accessible; and (5) leverage technology.
 
Think Twice Review Date: March 13, 2014
Reviewer: Craig Howley, Ohio University
In a review, Ohio University's Craig Howley finds that the document provides no articulated rationale for its recommendations. Citing missing research and slanted representations, Howley declares it "useless as a source of objective information." The document employed very little peer-reviewed research and no reference was found to any peer-reviewed empirical research in rural education. Howley notes that the methods employed by New Frontier are entirely rhetorical, based on unstated assumptions that charter schools provide positive outcomes in all regards. Also, Howley finds the document "obscures rather than illuminates consideration of rural education, making it more harmful than helpful." In summarizing the usefulness of New Frontier, Howley says, "The document is useful only to those who seek to expand charter schools into rural areas."

 

Report Reviewed: Pluck & Tenacity: How Five Private Schools in Ohio Have Adapted to Vouchers
Publisher/Think Tank: The Thomas B. Fordham Institute
Author: Ellen Belcher
This report highlights perceived challenges facing private schools brought on by the introduction of vouchers and calls for policymakers to strengthen Ohio's voucher programs. It praises the private schools profiled and cautions against state accountability programs that may "trample" on private schools' ability to provide a unique learning experience.
 
Think Twice Review Date: March 6, 2014
Reviewer: Christopher Lubienski and T. Jameson Brewer, University of Illinois
Overall, the reviewers found the report to be a case-study in how to engage in a slanted selection and interpretation of research evidence. The reviewers note that the report is supported by a total of three endnote references "cherry-picked" to support a contested policy agenda. "Thus, the report is grounded in a twice-skewed and intellectually dishonest view of the research on vouchers and their academic outcomes."

Report Reviewed: A Legal Lever for Enhancing Productivity
Publisher/Think Tank: George W. Bush Institute
Author: Elizabeth Ettema, Krishanu Sengupta, and Sandy Kress
This report examines the extent to which the Texas education system is efficient. Using a five-factor definition of efficiency, the authors argue that in key areas – teacher training, teacher evaluation, teacher pay-setting, and use of instructional materials – the Texas education system is unlikely to be efficient or cannot demonstrate efficiency.
 
Think Twice Review Date: February 27, 2014
Reviewer: Clive Belfield, Queens College, City University of New York
Dr. Clive Belfield found that the authors failed to prove that the Texas education system is inefficient.  The review noted positively on the theoretical framework (x-efficiency) used to expand the concept of efficiency beyond the common economic definition.  However, the report suffers from a lack of evidence and sufficient analysis to draw the conclusions made.  The report is of limited use for policymakers and education professionals.

Report Reviewed: Should Charter Schools Pay Rent? Implications for Staffing and Growth
Publisher/Think Tank: Manhattan Institute
Authors: Stephen Eide
This report from the Manhattan Institute examined the potential impact of requiring co-located charter schools to pay rent in NYC. The report reflects concerns of charter advocates and operators regarding potential policy changes under Bill de Blasio, New York's new mayor. The report claims that charging rent to co-located charters in NYC would have triggered an average budget deficit of 10.7 percent at those schools. The report also proposes that paying rent could cripple the co-located charters' growth in NYC.
 
Think Twice Review Date: February 20, 2014
Reviewer: Bruce Baker, Rutgers University
Bruce Baker finds that the report only presents a handful of poorly documented tables and graphs listing potential budget deficits, speculative layoffs, and average proficiency rates of co-located and non-co-located charter schools. Baker says that the report's greatest weakness is in its assumption that there is no possible downside when resources are transferred from city schools to charter schools. The report assumes that subsidies benefit charter schools and halting these subsidies harms charters and benefits no one. More importantly, Baker finds that the report ignores the broader and more complex policy questions of what it takes to manage a balanced and equitable system of schooling options.

 

Report Reviewed: Transfer Incentives for High-Performing Teachers: Final Results from a Multisite Randomized Experiment
Publisher/Think Tank: Mathematica Policy Research (MPR)
Authors: Steven Glazerman, Ali Protik, Bing-ru Teh, Julie Bruch, and Jeffery Max
The Teacher Transfer Initiative (TTI) actively encouraged 1,500 "high-performing" elementary teachers in 10 urban school districts to transfer to low-performing schools in return for a total stipend of $20,000 distributed over two years. The teachers were all identified as having students who were in the top 20 percent in their districts, based on a value-added measure of standardized test scores. The assumption was that better teachers would get better results on standardized tests wherever they teach.
 
Think Twice Review Date: January 28, 2014
Reviewer: Donald Gratz, Curry College
The results of this extensive study are not encouraging, writes reviewer Donald Gratz, who found several flaws that limit the report's usefulness for policy consideration. In his review, Gratz identifies several flaws. Among these: (1) the study does not consider the potential impact of other relevant school factors that affect teacher success, such as school leadership; and (2) while it considers the overall cost of the program, it only compares that cost to class size reduction, a generally expensive program.  The most significant problem, Gratz says, is that this two-year study was too short to draw conclusions either about teacher retention or student achievement, limiting its impact and usefulness

 

Report Reviewed: Fixing Classroom Observations: How Common Core Will Change the Way We Look at Teaching
Publisher/Think Tank: TNTP, Student Achievement Partners
This report suggests that many states and school districts are rolling out new teacher evaluation systems that will not succeed in improving instruction because they have not been updated to reflect the Common Core. The report asserts that new evaluation systems have not remedied practical challenges of classroom observations. It calls for increased emphasis on lesson content and a paring down of observation rubrics – to make them more focused and clear.
 
Think Twice Review Date: January 21, 2014
Reviewer: Jennie Whitcomb, University of Colorado Boulder
Dr. Jennie Whitcomb, associate dean for teacher education at the University of Colorado Boulder, concludes that simply streamlining the instruments and paying greater attention to what content is taught are unlikely to address the core problems surrounding teacher evaluations. Rather than solving implementation problems facing schools, Whitcomb says, "The report appears to be a sales-pitch for TNTP's soon-to-be-launched observation tools focused on Common Core lesson content."

 

These articles and/or reports are copyrighted material, the use of which has not been specifically authorized by the copyright owner. We are making such material available in our efforts to advance understanding of educational issues, etc. We believe this constitutes a 'fair use' of any such copyrighted material as provided for in section 107 of the U.S. Copyright Law. In accordance with Title 17 U.S.C. Section 107, the material on this site is distributed without profit to those who have expressed a prior interest in receiving the included information for research and educational purposes. For more information go to: http://www.law.cornell.edu/uscode/17/107.shtml. If you wish to use copyrighted material from this site for purposes of your own that go beyond fair use, you must obtain permission from the copyright owner.

Worth A Read

A weekly selection of thought-provoking research and commentary focused on
education reform.

Email:

Delivered by FeedBurner

rss
Read
rss

Stay Informed

Enter your email address to receive the latest news and information from The Great Lakes Center. We value your privacy and do not share email addresses with any other organizations.


A value is required.
Invalid format.