ABOUT   DONATE   SUBSCRIBE

ABOUT   DONATE   SUBSCRIBE

Facebook Twitter LinkedIn rss
Great Lakes Center Logo

2017 Think Twice Reviews

Think Twice is one of the nation's first efforts to serve as a watchdog to review think tank research on public education issues and policies, ensuring that published work meets the quality and standards of university scholarship. As think tank research becomes increasingly important reference sources in public policy debates, media and other critics have called for increased scrutiny to ensure validity and objectivity (click here to see related stories).

The goal of the Think Twice project is to provide the public, policy makers and the press with timely academically sound reviews of selected think tank publications.

Click Here To Read Our Legal Statement

Think Tank Research Quality

Think Tank Research Quality:
Lessons for Policy Makers, the Media, and the Public

Think Tank Research Quality, edited by Kevin Welner of the University of Colorado at Boulder, Patricia H. Hinchey of Pennsylvania State University, Alex Molnar of Arizona State University, and independent researcher Don Weitzman, offers clearly written, jargon-free expert reviews of studies on topics such as vouchers, charter schools, and alternative teacher certification. Friends of The Great Lakes Center receive 20% off phone, fax or mail-in orders...not valid online.

Reports & Reviews for 2017

Report Reviewed: Back to the Staffing Surge: The Great Teacher Salary Stagnation and the Decades-Long Employment Growth in American Schools
Publisher/Think Tank: EdChoice
Author: Ben Scafidi
A recent report from EdChoice, formerly the Friedman Foundation for Educational Choice, concluded that non-teaching staffing growth in schools after the Great Recession has outpaced enrollment growth and student performance. The report recommended increasing teacher pay, at the expense of non-teaching staff, and more school choice.
 
Think Twice Review Date: July 5, 2017
Reviewer: Joydeep Roy, Columbia University; and William J. Mathis, University of Colorado Boulder
An academic review finds that the report presents no logical relationship between the staffing trends presented and the solutions proposed in the report. The reviewers identify that the primary shortcoming of the report is its failure to examine why there has been a staffing surge and whether it reflects a valid use of personnel. In their conclusion, they also find that there is no linkage between the report’s call for more school choice and vouchers to remedy “over-staffing” or better education. In response, the reviewers provide an appendix list of recent evidence on voucher and charter school performance.


Report Reviewed: Renewing our Cities: A Case Study on School Choice’s Role in Urban Renewal
Publisher/Think Tank: EdChoice
Author: Bartley Danielsen, David Harrison, and Jing Zhao
Report Reviewed: CPR Scholarships: Using Private School Choice to Attack Concentrated Poverty, Crime, and Unemployment
Publisher/Think Tank: American Enterprise Institute
Author: Bartley Danielsen
EdChoice and the American Enterprise Institute (AEI) released reports suggesting that expanded school choice would promote economic development in economically distressed urban areas. The EdChoice report was a case study on the relocation decisions of families in an urban charter school, while the AEI report calls for a voucher-like program to spur economic development by luring higher income families into neighborhoods. While the EdChoice report is an empirical paper, the AEI report only consists of a hypothetical proposal for a voucher program aimed at promoting economic development in high-poverty neighborhoods.
 
Think Twice Review Date: June 20, 2017
Reviewers: Jennifer Jellison Holme and Emily Germain, University of Texas Austin
An academic review of the reports finds that both reports make unsupported claims that rely on flawed logic and data. The reviewers also note that both reports use selective research evidence and overlook other studies that contradict their claims. In their conclusion, the reviewers say: “the reports offer little guidance for policymakers seeking to reform urban schools, to support low-income students, or to uplift urban neighborhoods.”


Report Reviewed: Hidden Money: The Outsized Role of Parent Contributions in School Finance
Publisher/Think Tank: Center for American Progress
Author: Catherine Brown, Scott Sargrad, and Meg Benner
A report from the Center for American Progress (CAP) investigated private fundraising by parent groups, a source of inequality in education that receives scant attention. The report called on district leaders to take actions to address funding disparities between wealthy and low-income schools. The report found that while affluent students receive additional funding for additional resources from parent groups, low-income students (disproportionately children of color) must spend public dollars to obtain the same resources.
 
Think Twice Review Date: June 15, 2017
Reviewer: Maia Cucchiara, Temple University
An academic review of the report finds the report’s findings about the scope and beneficiaries of private fundraising are credible and important. However, the reviewer, Maia Cucchiera, Temple University, finds fault with the report’s focus on a small number of schools and districts. She adds that the report fails to provide school and community context, and the case study design limited the report’s overall relevance. More importantly, Cucchiera reminds readers that sources of educational inequality are based on structural factors, rather than on the actions of individual parents.


Report Reviewed: Leveraging ESSA to Support Quality-School Growth
Publisher/Think Tank: Thomas B. Fordham Institute and Education Cities
Author: Nelson Smith and Brandon Wright
A recent report published by the Thomas B. Fordham Institute and Education Cities sought to provide a “how-to-guide” for education reform advocates seeking to influence states’ use of Title I school improvement funds under the Every Student Succeeds Act (ESSA). The report identified three prominent school reform models for school improvement, which remove communities’ democratic control over their schools. The three models recommended were: (1) expanding charter schools; (2) creating state turnaround districts; and (3) the use of state-led, district based solutions, which remove the powers of superintendents and school boards and vests that authority in a single individual.
 
Think Twice Review Date: June 13, 2017
Reviewers: Gail L. Sunderman, University of Maryland
An academic review of the report finds that the report omits research that sheds light on the three models, and fails to take into account the opportunity costs of pursuing one set of policies over another. The review finds the report’s use of research literature to be selective, with no peer-reviewed journal articles included. Many of the cited references were from other advocacy pieces. Further, the reviewer finds the support of the effectiveness behind the three approaches is simply too limited to present them as promising school improvement strategies. In her conclusion, she states: “Policymakers, educators and state education administrators should be wary of relying on this report — or Education Cities members who use it — as a source of information that can guide them as they develop their state improvement plans.”


Report Reviewed: Understanding a Vicious Cycle: Do Out-of-School Suspensions Impact Student Test Scores?
Publisher/Think Tank: UArk Department of Ed. Reform
Author: Kaitlin P. Anderson, Gary W. Ritter, & Gema Zamarro
The Department of Education Reform at the University of Arkansas released a working paper on the association between out-of-school suspensions and student test scores. The paper concluded that punishment works, and that out-of-school suspension had a positive relationship to math and language arts test scores. The working paper used multilevel regression modeling of student discipline records across six years for all K-12 public schools in Arkansas. In contrast to prior work, the report found a modest positive relationship between days of suspensions and student test scores.
 
Think Twice Review Date: June 1, 2017
Reviewer: Brea L. Perry, Indiana University; and Daniel Losen, director of the Center for Civil Rights Remedies initiative at the Civil Rights Project at UCLA
An academic review of the paper found multiple methodological and validity concerns, which limit the claims made. The reviewers found problems with the report, including: (1) the effects of out-of-school suspensions were not measured in the same year in which the suspensions occurred; and (2) other factors were not considered (viz., lost instructional time; the deterioration of student-teacher relationships; psychological distress; and other more immediate consequences of suspensions). The reviewers also noted that past research has predominantly shown exclusionary discipline to adversely effect test scores, GPA, grade retention, and increases the chances of a student dropping out. Because of this, the findings, according to the review, have weak face validity. In their conclusion, the reviewers state: “This paper offers no value for guiding disciplinary policy and practice.”


Report Reviewed: New York Charter Schools Outperform Traditional Selective Public Schools: More Evidence that Cream-skimming is Not Driving Charters’ Success
Publisher/Think Tank: The Manhattan Institute
Author: Macus A. Winters
The Manhattan Institute’s Marcus Winters released a report that explored the extent to which charter schools’ success could be attributed to “cream-skimming” from struggling public schools in New York. The report claimed that NYC’s charter schools performed better than selective non-charter public schools. The report found that the charter schools performed no differently in ELA and significantly better in math. The report concluded that the success of NYC’s charter schools could not be explained by cream-skimming.
 
Think Twice Review Date: May 9, 2017
Reviewers: Sarah A. Cordes, Temple University
In her review, Sarah A. Cordes, Temple University, says that the conclusions of the report seem logical, but the report suffers from two primary flaws: (1) it assumes that selective school applicants are higher performing and more motivated than charter school applicants; and (2) the report relies on a single year of data to make comparisons. In her conclusion, she says that this report misses the mark in response to evaluating cream-skimming in NYC charter schools. She writes, “Addressing the question of cream-skimming in NYC charter schools will require the use of longitudinal student-level data and much more rigorous methods.”


Report Reviewed: Gender Identity Policies in Schools: What Congress, the Courts, and the Trump Administration Should Do
Publisher/Think Tank: The Heritage Foundation
Author: Ryan Anderson and Melody Wood
A report from the Heritage Foundation called on the U.S. government to respect federalism, local decision-making, and parental authority regarding decisions concerning gender identity. The report argued that schools should not be required to protect youth from discrimination based on gender identity under Title IX of the Education Amendments of 1972. The report called on the Trump administration and the courts to reverse Obama-era protections of transgender students under Title IX. It claimed that recent gender identity policies were unlawful.
 
Think Twice Review Date: May 2, 2017
Reviewer: Robert Kim, Rutgers University
Robert Kim, a William T. Grant Distinguished Fellow at Rutgers University, reviewed the report. Kim in his review identifies several weaknesses of the report; the report fails to acknowledge or address: (a) legal developments that support the argument that gender identity discrimination is a form of sex discrimination; (b) the near-consensus within the medical, scientific, and educational communities concerning how transgender students should be treated; and (c) recent research literature on the appropriate care for and education of transgender youth. In his conclusion, he says: “What is entirely missing from this report — and what state and local policymakers and educators urgently need — is information and guidance in an area that may be new or unfamiliar to them.”


Report Reviewed: Apples to Apples: The Definitive Look at Test Scores in Milwaukee and Wisconsin
Publisher/Think Tank: Wisconsin Institute for Law and Liberty (WILL)
Author: Will Flanders
The report asserted that it provided an “apples to apples” comparison of schools in Milwaukee and Wisconsin. The report’s primary findings, based on standardized test scores from a single year, were that charter schools and private schools receiving voucher funds were more effective than traditional public schools. Further, it claimed to provide “the clearest possible comparison of student outcomes in each sector in Milwaukee – as well as in Wisconsin.”
 
Think Twice Review Date: April 25, 2017
Reviewers: Benjamin Shear, University of Colorado Boulder
In this academic review, Benjamin Shear, University of Colorado Boulder, highlights the problems of using a single year’s test scores to compare schools that serve different populations. Additionally, he calls attention to the limited nature of the available data, which undermine the causal conclusions of the report. Despite providing some important descriptive statistics about test score performance in Milwaukee and Wisconsin, Shear finds some troubling inconsistencies regarding the methods and results. In conclusion, Shear says, “If policymakers or the public are interested in determining which schools or school choice policies in Wisconsin are most effective, this report cannot provide answers to such questions.”


Report Reviewed: Differences by Design? Student Composition in Charter Schools with Different Academic Models
Publisher/Think Tank: American Enterprise Institute
Author: Nat Malkus And Jenn Hatfield
A report from the American Enterprise Institute (AEI) compared differences in approaches and demographics between and among charter school models and traditional public schools. The report used three national data sets to capture the national universe of charter schools. Using enrollment demographics at different charter school models (e.g., art focused, no-excuses, single-sex, etc.), the report demonstrated that different demographic groups attend different types of charter schools. With regard to different categories of race and ethnicity, family income, and special education status, the report documents demographic sorting as an outcome of school choice.
 
Think Twice Review Date: April 13, 2017
Reviewer: T. Jameson Brewer, University of North Georgia; and Christopher Lubienski, Indiana University
An academic review of the report finds that it fails to consider: (1) a large body of research on parent-decision-making; (2) research suggesting that charter schools are not as innovative as they claim; and (3) the purpose and aims of an equitable public education system. The reviewers find that the report presents charter school de facto segregation as a benign byproduct of parental choice. In fact, the review finds that the original report actually acknowledged that this type of stratification was part and partial of a “properly” functioning charter sector - one in which parents get to choose the type of school their children attend. Ultimately, the reviewers conclude: “While the authors and AEI may have conceived this report as a rationale for advancing charter schools, their data demonstrates that charter schools may be destructive of the common good.”


Report Reviewed: A New Agenda: Research to Build a Better Teacher Preparation Program
Publisher/Think Tank: Bellwether Education Partners (BEP)
Author: Ashley LiBetti Mitchel & Melissa Steel King
Bellwether Education Partners (BEP) released a report calling into question the effectiveness of teacher preparation programs in the U.S. The report proposed that improving teacher quality could be accomplished through teacher preparation program redesign. Further, it asserted that teacher preparation programs are “blindly swinging from one popular reform to the next” and decades of input- and outcome-based research have failed to improve teacher education.
 
Think Twice Review Date: March 28, 2017
Reviewers: Marilyn Cochran-Smith, Stephani Burton, Molly Cummings Carney, Juan Gabriel Sánchez, and Andrew F. Miller; Boston College
Despite a reasonable assumption that teacher quality can be improved through redesigned teacher preparation programs, the reviewers find that the report includes mischaracterizations of teacher education research and omits relevant literature. In conclusion, the reviewers find that the report “represents a missed opportunity to offer guidance to [either] policymakers or institutions.”


Report Reviewed: Within Our Grasp: Achieving Higher Admission Standards in Teacher Prep
Publisher/Think Tank: National Council on Teacher Quality (NCTQ)
Author: Kate Walsh, Nithya Joseph and Autumn Lewis
A recently issued report from the National Council on Teacher Quality (NCTQ) advocated raising the bar for entry into teacher preparation programs as a means to improve teacher quality across the country. The report reviewed the academic requirements at 221 institutions in 25 states, and argued for boosting entry requirements.
 
Think Twice Review Date: March 23, 2017
Reviewer: Marilyn Cochran-Smith, Boston College; Megina Baker, Project Zero, Harvard University; Wen-Chia Chang, Boston College; M. Beatriz Fernández, Alberto Hurtado University, Chile; and Elizabeth Stringer Keefe, Lesley University.
The review team cautions that the report is based on ‘cherry-picked’ citations, which “ignore a substantial body of research that either provides conflicting evidence or shows that the issues are much more complex and nuanced than the report suggests.” They add: “Ultimately the report offers little guidance for policymakers or institutions.”


Report Reviewed: Reimagining Learning: A Big Bet on the Future of American Education
Publisher/Think Tank: NewSchools Venture Fund
Author: Stacey Childress and Meghan Amrofell
A recent report from the NewSchools Venture Fund offered a thought experiment on how philanthropists can help accelerate efforts to reimagine school. It suggested that philanthropic investment should make a “big bet” on innovative models. It called for a total investment of $4 billion over 10 years through three key areas: (1) creating new schools and redesigning existing ones; (2) supporting targeted technology innovation; and (3) launching a campaign to foster understanding and demand for innovative education models.
 
Think Twice Review Date: March 7, 2017
Reviewer: Jeffrey W. Snyder, Cleveland State University
An academic review finds the report’s usefulness to policy and practice is limited. Although the report presents valuable insight into the promises of philanthropy, The review finds that the report overpromises. The reviewer says, “Unfortunately, the report fails to provide a meaningful examination of research or a thorough basis for its recommendations.”


Report Reviewed: Measures of Last Resort: Assessing Strategies for State-Initiated Turnarounds
Publisher/Think Tank: Center on Reinventing Public Education (CRPE)
Author: Ashley Jochim
A recent report from the Center on Reinventing Public Education (CRPE) attempted to provide evidence for states seeking to better support state-initiated turnaround strategies. The report sought to identify effective mechanisms states can use to improve school performance. It provides a broad-brush profile of the various forms of state initiated turnarounds.
 
Think Twice Review Date: February 14, 2017
Reviewer: Betty Malen and Jennifer King Rice, University of Maryland
Although the report draws on multiple sources of information, the review finds the following limitations: (1) the report fails to elevate the research base or the policy discourse; (2) the methods used are neither well-described nor justified; (3) the evidence provided does not support the claims made; and (3) the report neglects to consider relevant research on the specific turnaround mechanisms available to states. The reviewers conclude, “As a result of these problems, the report does not enhance the evidence base or provide the substantive guidance to state policymakers require to make informed decisions about the use of the various school turnaround strategies.”


Report Reviewed: High Stakes for High Achievers
Publisher/Think Tank: Thomas B. Fordham Institute
Author: Michael J. Petrilli, David Griffith, Brandon L. Wright, and Audrey Kim
Report Reviewed: High Stakes for High Schoolers
Publisher/Think Tank: Thomas B. Fordham Institute
Author: Michael J. Petrilli, David Griffith, and Brandon L. Wright
Two recent think tank reports from the Thomas B. Fordham Institute called into question the impact of accountability systems on “high achievers.” The reports claimed that states have failed to address the needs of higher-scoring students, and that state accountability systems should be redesigned under the Every Student Succeeds Act (ESSA) with those students in mind.
 
Think Twice Review Date: January 26, 2017
Reviewer: Beth C. Rubin, Rutgers University
An academic review of the reports finds that the recommendations could result in a furthering of the inequitable educational opportunities that ESSA was designed to reduce. Despite arguing that “high achievers” are being neglected educationally, the review finds that both reports fail to address the concerns productively. Beth C. Rubin concludes that these reports contribute to the collective discussion regarding the federal role in education, but they must be approached with caution.


Report Reviewed: Better Evidence, Better Choices, Better Schools: State Supports for Evidence-Based School Improvement
Publisher/Think Tank: Center for American Progress and Knowledge Alliance
Author: Steve Fleischman, Caitlin Scott, and Scott Sargrad
The Center for American Progress (CAP) and Knowledge Alliance released a report on the evidence-based provisions of the Every Student Succeeds Act (ESSA). The report called for a shift away from federal mandates toward greater state and local autonomy. It sought to provide guidance for school districts seeking to make decisions based on evidence to improve their lowest performing schools. And lastly, the report offered eight key recommendations for implementing evidence-based strategies.
 
Think Twice Review Date: January 12, 2017
Reviewer: Terri S. Wilson, University of Colorado Boulder
An academic review of the report found that the report raises a number of important issues regarding the implementation of ESSA, yet the guidance offered to practitioners is “vague and unsubstantiated.” The advice provided to practitioners lacked evidence and specifics.


Report Reviewed: Segregation, Race, and Charter Schools: What Do We Know?
Publisher/Think Tank: Center on Children and Families at Brookings
Author: Grover J. "Russ" Whitehurst, Richard V. Reeves, Edward Rodriguez
A report from the Center on Children and Families at Brookings attempted to compare various research findings on school segregation with a focus on expanding urban charter schools. The report argued that school quality is the primary determinant of student achievement, and that instead of addressing school segregation, policy should focus on improving the schools that students of color and low-income students attend.
 
Think Twice Review Date: December 19, 2016
Reviewer: Erica Frankenberg, Penn State University College of Education
An academic review of the report finds the report has limited use for policymakers. Despite its claimed attempt to investigate research findings on school segregation, the report suggests that policy attend to school quality via school choice. The reviewer finds that the report presents a false choice between school integration and creating high-quality urban charter schools. She indicates that the report is limited because of its selective review and interpretation of the literature on race and poverty, and its questionable conclusions that are not reflective of the research consensus.


Report Reviewed:

Squeezing the Public School Districts: The Fiscal Effects of Eliminating the Louisiana Scholarship Program

and

The Fiscal Effect of Eliminating the Louisiana Scholarship Program on State Education

Publisher/Think Tank: Department of Education Reform (DER) at the University of Arkansas
Author: Corey A. DeAngelis & Julie R. Trivitt
Two recent papers from the Department of Education Reform (DER) at the University of Arkansas predicted the budgetary consequences of ending the Louisiana Scholarship Program (LSP). The two closely allied papers concluded that terminating the LSP would not relieve pressure on public spending in Louisiana. The first paper uses an economic model to conclude that terminating the LSP would increase costs. While, the second paper confirms the findings from the first paper and advances that the findings would apply to almost all school districts in Louisiana.
 
Think Twice Review Date: September 29, 2016
Reviewer: Clive Belfield, Queens College, City University of New York
A new review of the papers says that the findings are reasonable, but they fail to make a convincing case. In his review Belfield says, “These papers are useful in that they apply an economic model to understanding the consequences of voucher systems.” However, he says that the papers do not make a fully convincing case. Additionally, Belfield questions the omission of contradictory evidence provided by the state’s Legislative Fiscal Office. Belfield concludes, “There may be savings or additional expenditures, depending on several key parameters which have not been precisely estimated.” Regardless of the economic consequences, Belfield adds, “the poor performance of LSP might in itself be justification for termination.”


Report Reviewed: A 21st Century School System in the Mile-High City
Publisher/Think Tank: Progressive Policy Institute
Author: David Osborne
A recent report from the Progressive Policy Institute advocated for expanded use of the “portfolio model” of school governance. The report drew on contemporary reforms adopted by the Denver Public Schools. It claimed that reforms enacted by the Denver Public Schools (viz., portfolio model, school closings, and charter expansion) positively impacted student test scores. The goal was to highlight Denver as a success story that should be copied elsewhere.
 
Think Twice Review Date: July 26, 2016
Reviewer: Terrenda White, University of Colorado Boulder
An academic review by Terrenda White, University of Colorado Boulder, finds that the report oversteps its claims in an attempt to positively portray Denver’s reforms. The review raises serious concerns about the claims made. Most notably, she finds that the only data presented were simple charts that lacked any conventional statistical analysis. Regarding the report’s use of research literature, White finds that the report drew primarily from advocacy publications from partisan foundations. In her conclusion, White says: “Ironically, the report celebrates Denver’s portfolio strategy and its expansion of charter schools as a model of 21st century reform at a time when other cities have discovered that charters are not panaceas.”


Report Reviewed: Measuring School Turnaround Success
Publisher/Think Tank: Public Impact and The Center on School Turnaround (WestEd)
Author: Cassie Lutterloh, Jeanette P. Cornier, & Bryan C. Hassel
A report produced by Public Impact for WestEd’s Center on School Turnaround asserted that because there is a lack of a shared definition of school turnaround success for states and districts, it is difficult for school districts to learn from one another. The report claimed to provide a model for defining school turnaround success. The report bases its model framework for determining turnaround success on three school-level measures: (1) proficiency in reading and math on state assessments; (2) growth in reading and math on state assessments; and (3) graduate rate (for high schools).
 
Think Twice Review Date: July 20, 2016
Reviewer: Tina Trujillo, University of California Berkeley; and
Marialena Rivera, Texas State University
A review of the report found following shortcomings: (1) Sufficient research evidence for the claims made is not presented in the report; (2) It lacks sound methodological techniques; and (3) It omits several rigorous, peer-reviewed studies that contradict a majority of its proposals. The review concludes that the proposed model continues the trend of relying on flawed, test-centered strategies.


Report Reviewed: Bang For The Buck: Which Public Schools In Milwaukee Produce The Best Outcomes Per Dollar Spent?
Publisher/Think Tank: Wisconsin Institute for Law and Liberty (WILL)
Author: Will Flanders and C.J. Szafir
This report from the Wisconsin Institute for Law and Liberty (WILL) attempted to estimate efficiency scores for schools in Milwaukee and make the case for greater autonomy for charter schools. The scores were used in an attempt to draw conclusions about the relative “efficiency” of Milwaukee’s charter and traditional public schools. The report used a simple analysis of math and science scores divided by per pupil funding. A second analysis from the report generated “efficiency scores” by using a regression model, which accounted for selected demographic characteristics. The authors of the report claimed that schools with greater autonomy were more efficient than schools with less autonomy.
 
Think Twice Review Date: July 12, 2016
Reviewer: Casey Cobb, University of Connecticut
A review of the report found the strong claims about the relative efficiencies to be too weak to serve any useful function for policymakers. Five major problems arise from the report’s approach, Casey Cobb finds: (1) Test scores do not comprehensively represent the purposes of schools; (2) The report does not address threats to the validity of its assumption that there is uniform financial accounting across schools and types; (3) The analytic description of the study is incomplete, making interpretation difficult; (4) “Autonomy” is never really defined—it is just used as a loose term implying independence—so autonomous behavior is assumed by virtue of their charter status. The report then makes strong but unmeasured claims about the superior “efficiency” of charter schools based on their having this greater autonomy; and (5) While the report’s analysis controls for some school demographic characteristics, it does not appear to adjust for selection effects; effects that could prove fatal to their conclusions.


First Document Reviewed: A Win-Win Solution: The Empirical Evidence on School Choice
Publisher/Think Tank: Friedman Foundation for Educational Choice
Author: Greg Forster
Second Document Reviewed: The Participant Effects of Private School Vouchers across the Globe: A Meta-Analytic and Systematic Review
Publisher/Think Tank: Department of Education Reform (DER) at the University of Arkansas
Author(s): M. Danish Shakeel, Kaitlin P. Anderson, & Patrick J. Wolf
Two reports claimed to offer empirical support of the effectiveness of school vouchers. The reviewed reports include: A Win-Win Solution: The Empirical Evidence on School Choice from the Friedman Foundation for Educational Choice; and The Participant Effects of Private School Vouchers across the Globe: A Meta-Analytic and Systematic Review from the Department of Education Reform (DER) at the University of Arkansas. The reports focused on randomized studies of the effects of school vouchers on educational outcomes, concluding that school vouchers have positive effects.
 
Think Twice Review Date: June 30, 2016
Reviewer: Christopher Lubienski, University of Illinois Urbana-Champaign
An academic review released today finds the reports have serious problems and errors. Christopher Lubienski, University of Illinois Urbana-Champaign, reviewed both reports for the Think Twice think tank review project. Lubienski concludes: “Both reports are marred by a number of serious problems and errors, including misrepresentations of the research literature, a failure to acknowledge the limitations of their approaches, not addressing the shortcomings of the theoretical underpinnings of vouchers, and the use of methods that bias the selections of the studies they utilize.” He adds, “Together, their manifold serious flaws undercut the trustworthiness and usefulness of these reports.”


Report Reviewed: School Spending and Student Achievement in Michigan: What’s the Relationship?
Publisher/Think Tank: Mackinac Center for Public Policy
Author: Ben DeGrow and Ed Hoang
A report released in April by the Mackinac Center for Public Policy made the claim that spending more on Michigan schools doesn’t increase achievement. The report asserted that there is little or no relationship between student achievement and marginal increases in school spending in the state. The report characterizes spending levels in Michigan as “high,” but never substantiates the claim. In summary, the report argues that spending increases in Michigan would provide little or no gain in student achievement, and that spending on increased salaries or reduced class sizes would be inefficient or ineffective.
 
Think Twice Review Date: May 13, 2016
Reviewer: Bruce D. Baker, Rutgers University
An academic review of the Mackinac Center report finds that the report clashes with existing research about the positive impact of funding nationally and in Michigan. In his review, professor Bruce D. Baker says that the report lacks evidence to substantiate the claims. According to Baker, the report "wrongly assumes that all Michigan districts are now high spending and that none could benefit from any marginal increase to funding; it fails to evaluate thoroughly the overall level of spending in context, nor does it adequately consider whether and to what extent spending varies across children and contexts within Michigan."


Report Reviewed: 2016 Brown Center Report on American Education — Part II: Tracking and Advanced Placement
Publisher/Think Tank: Brown Center on Education Policy at Brookings
Author: Tom Loveless
Based on the logic that states with more tracking tend to have more students passing AP exams in high school, a recent report from the Brown Center on Education Policy at the Brookings Institution claimed that tracking in eighth grade would promote greater equity. The Brown Center report, authored by Tom Loveless, used state-level data from the NAEP to describe a positive association between tracking in eighth grade and larger percentages of high-scoring AP test takers. Based on the findings, the author claims that tracking is beneficial for high-achieving students.
 
Think Twice Review Date: April 26, 2016
Reviewer: Marshall Jean, University of Chicago
An academic review of the report questions the impact of tracking policies on low-achieving students, and calls attention to the potential for tracking practices to exacerbate existing educational inequalities. Ultimately, the reviewer, Marshall Jean (University of Chicago) finds the claim that tracking has the potential for promoting equity to be “dubious.” He concludes, “unless tracking systems are implemented carefully and coincide with substantial supports for struggling students, students assigned to low-ranking tracks are likely to be harmed.”


Report Reviewed: The Policy Framework for Online Charter Schools
Publisher/Think Tank: Center for Reinventing Public Education (CRPE)
Author: Rosa Pazhouh, Robin Lake, Larry Miller
A study from the Center for Reinventing Public Education (CRPE), an organization that often advocates for charter schools, attempted to provide an in-depth analysis of policy features across the states that allow for online charter schools. The report concluded that online charter schools should use a separate regulatory framework than brick-and-mortar charter schools.  The report is part of a series of three, which were sponsored by the Walton Foundation.
 
Think Twice Review Date: April 18, 2016
Reviewer: Gary Miron, Western Michigan University
An academic review of the report finds that it potentially offers a solid contribution for policymaking. In his review of the report, professor Gary Miron, Western Michigan University, describes it as a well-organized description of policy features, and that the policy recommendations generally, but not always, follow well from the study’s evidence.


Report Reviewed: The School Choice Voucher: A “Get Out of Jail” Card?
Publisher/Think Tank: Department of Education Reform, University of Arkansas
Author: Corey DeAngelis and Patrick J. Wolf
The authors of the working paper used data from the Milwaukee Parental Choice Program (MPCP) to compare crimes processed through the Wisconsin courts for program participants and a matched sample of students. Although most of these comparisons showed no association, the report finds that some subgroups of MPCP students were less likely to commit crimes as adults.
 
Think Twice Review Date: April 6, 2016
Reviewer: Clive Belfield, Queens College, City University of New York (CUNY)
An academic review of the paper finds that it has several weaknesses and does not support the implication that voucher programs caused a reduction in crime. Clive Belfield, Queens College (CUNY) is very clear in his review, the findings from the paper should not warrant any strong claims of voucher effects on crime. Belfield raises the following concerns: (1) the study’s results are highly variable, with most comparisons showing statistically insignificant results; (2) the methods used do not justify a causal inference; and (3) the evidence, despite the paper’s title, is associational, not causal. In short, Belfield cautions that a better interpretation of the report would be that vouchers and crime are, in fact, not correlated.


Report Reviewed: Learning about Learning: What Every New Teacher Needs to Know
Publisher/Think Tank: National Council on Teacher Quality (NCTQ)
Author: Laura Pomerance, Julie Greenberg and Kate Walsh
As part of a series of reports chronicling teacher education, the National Council on Teacher Quality (NCTQ) attempted to investigate the textbooks used in teacher prep coursework. The report contended that textbooks assigned in methods coursework fail to provide teacher candidates with proper preparation. The NCTQ report is based on six strategies adapted from a practice guide provided by the What Works Clearinghouse, which is funded by the U.S. Department of Education’s Institute of Education Sciences (IES). The report offers recommendations for textbook publishers, teacher education programs, and state departments of education.
 
Think Twice Review Date: March 24, 2016
Reviewer: P.L. Thomas, Furman University; and Christian Z. Goering, University of Arkansas
However, the reviewers note that the report is not grounded in a comprehensive examination of the literature on teaching methods. Moreover, the report relies on a single source, which itself was based on a narrow set of research studies. The reviewers find that, despite posing an interesting question, NCTQ’s report falls short because it uses a narrow set of criteria and applies it in a misleading and superficial way. They conclude: “this report in no way justifies taking the conclusions or recommendations seriously when determining policy or practice.”


Report Reviewed: SchoolGrades.org
Publisher/Think Tank: Manhattan Institute for Policy Research
Author: Jacob L. Vigdor and Josh B. McGee
The Manhattan Institute for Policy Research launched a website, SchoolGrades.org, which aimed to provide a means to compare how well America’s schools prepare students in core subjects. The website attempts to evaluate and assign letter grades to schools using reading and math test scores. The website claims that the school grades found on the site allow parents to compare local schools against schools in other countries using a four-step process.
 
Think Twice Review Date: March 10, 2016
Reviewer: Jaime L. Del Razo, Annenberg Institute for School Reform & Brown University
The review finds that the unsubstantiated norming involved in the process of creating school grades is too tenuous and the results are overly extrapolated, which diminishes their value. A technical analysis also finds that the website fails to explain how: (1) international scores are equated to a national standard created by the website; (2) letter grades are determined; and (3) free and reduced lunch counts were used to make socioeconomic adjustments. In his conclusion, reviewer Jaime L. Del Razo indicates that the site fails on two grounds; technical and philosophical. He says, “the Manhattan Institute’s website fails to advance policy not only on the technical shortcomings of its efforts but most importantly, for not appreciating the [broader] purposes of education.”


Report Reviewed: Lessons From State Performance on NAEP: Why Some High-Poverty Students Score Better Than Others
Publisher/Think Tank: Center for American Progress
Author: Ulrich Boser and Catherine Brown
A report from the Center for American Progress (CAP) attempted to examine the impact of standards-based policies between 2003 and 2013. The report explored whether states’ adoption of standards-based policies predicts low-income students’ NAEP achievement trends in fourth and eighth grade math and reading. The report attempted to highlight that the difference between high and low scoring states was the adoption of high standards, specifically, the Common Core State Standards.
 
Think Twice Review Date: February 25, 2016
Reviewer: Sharon L. Nichols, University of Texas San Antonio
A review finds that it employs inappropriate research methods, fails to adequately define its approach, and reports only incomplete findings from its analyses. Additionally, according to the review, the report does not adequately describe variables or analytic methods, and the data and methods used do not allow for any causal findings. The review also finds that while the report claims to analyze changes across five separate two-year intervals, it only reports findings for 2009-2011. And the positive results are statistically significant only at the generally unacceptable 0.10 level of significance.


Report Reviewed: Special Education and English Language Learner Students in Boston Charter Schools
Publisher/Think Tank: School Effectiveness & Inequality Initiative (SEII); Department of Economics at MIT
Author: Elizabeth Setren
The School Effectiveness & Inequality Initiative (SEII), housed in the Department of Economics at MIT, released a working paper in December 2015. The report sought to investigate the enrollment and achievement of students with special needs and English language learners (ELLs) in oversubscribed charter schools in Boston. The SEII working paper claimed to “debunk” the common perception that students with special needs and ELLs are underserved in charters. The report focused on Boston charter schools and Boston Public Schools (BPS) that enrolled similar special populations.
 
Think Twice Review Date: February 17, 2016
Reviewer: Julie Mead, University of Wisconsin- Madison; and Mark Weber, Rutgers University
The reviewers found some limitations to the paper’s methods; however, they find the primary claim regarding test score effects to be on solid ground. Additionally, the models used to estimate the effects were also found to be appropriate. The reviewers do point out the following limitations in the working paper: (1) the data and analyses are more limited than readers of the report might be lead to believe; (2) the effects can only be generalized to those students who enter the lottery and comply with their assignment (charter or traditional public school); and (3) there is no context provided to compare the size of reported test score gains, or how those test score gains were realized (e.g., unaccounted peer effects and spending differences). Reviewers Mead and Weber ultimately conclude that, while this report takes an important step in studying how oversubscribed charters may affect the academic achievement of special needs students, a closer examination is needed in order to accurately inform those making education policy.


Report Reviewed: Continued Progress: Promising Evidence on Personalized Learning
Publisher/Think Tank: RAND Corporation
Author: John F. Pane, Elizabeth D. Steiner, Matthew D. Baird, & Laura S. Hamilton
A recent report published by the RAND Corporation focused on three school-wide initiatives funded by the Bill and Melinda Gates Foundation. The report concentrated on approaches touted as “personalized learning.” The report sought to add to the evidence base by examining the effects of school-wide efforts to promote personalized learning and the links between implementation of particular strategies and outcomes.
 
Think Twice Review Date: January 26, 2016
Reviewer: William R. Penuel and Raymond Johnson, University of Colorado Boulder
Encouragingly, the reviewers state that the report did include high-quality elements. Yet, the conclusions about the efficacy of technology-based personalized learning are not warranted by the research presented. Readers should be skeptical of what promise the report’s evidence actually provides for any given model of personalized learning. The reviewers conclude, “the study lacks utility for judging the value of the more disruptive and digital-technology-based personalized learning.” Limitations identified by the review include: (1) sample of treatment schools unrepresentative of the general population of schools; (2) the lack of a threshold in the study for what qualified as implementing “personalized learning” in treatment schools; and (3) that several strategies highlighted were rarely implemented in the studied schools.


Report Reviewed: Smart, Skilled, and Striving: Transforming and Elevating the Teaching Profession
Publisher/Think Tank: Center for American Progress (CAP)
Author: Carmel Martin, Lisette Partelow, and Catherine Brown
This Center for American Progress (CAP) report outlined a vision for elevating and modernizing the teaching profession. The report provided ten recommendations for improving the public perceptions and experiences of teachers. The report argued that if we do not change the perception of the teaching profession, schools would not be able to recruit “high achieving young people” into teaching.
 
Think Twice Review Date: January 7, 2016
Reviewer: Elizabeth J. Meyer, University of Colorado Boulder
In an academic review, Elizabeth J. Meyer, University of Colorado Boulder, notes that elements of the report’s recommendations are likely to be beneficial. However, she also finds that the some of the report’s recommendations could likely have the opposite effect by: (1) increasing the surveillance of teachers; (2) reducing the job security of teachers; (3) evaluating teachers based on students’ test scores; and (4) using merit pay systems. The review also finds that the report relies too heavily on popular rhetoric, sound bites, opinion articles, and advocacy publications. In conclusion, Meyer says, “Other than a review of contemporary issues, the report offers little of substance to advance the teaching profession.”


Report Reviewed: The Integration Anomaly: Comparing the Effects of K-12 Education Delivery Models on Segregation in Schools
Publisher/Think Tank: Friedman Foundation for Educational Choice
Author: Benjamin Scafidi
A report from the Friedman Foundation for Educational Choice claimed that universal school choice offers a solution to increasing school segregation. The report argued that competition unleashed by unrestricted school choice would promote integration. The report suggested that housing integration has not been an effective way to pursue school integration, and it concludes with recommendations for how to structure school choice to achieve integration.
 
Think Twice Review Date: January 5, 2016
Reviewer: Genevieve Siegel-Hawley, Virginia Commonwealth University; & Erica Frankenberg, Penn State University
An academic review of the report from Genevieve Siegel-Hawley, an assistant professor at Virginia Commonwealth University, and Erica Frankenberg, an associate professor at Penn State University finds that the arguments are not based on evidence. The reviewers further find that the analysis of the empirical relationship between school and residential segregation relies on flawed methodological decisions with regard to how to define segregation and divergent trends over time. Those problematic definitions, in turn, yield biased results and prompt the reader to incorrectly assume that housing integration policies will have little bearing on school segregation. Siegel-Hawley and Frankenberg also point out that the report’s use of research literature on school choice is haphazard and incomplete, drawing conclusions either beyond what the research supports or contrary to what research has found.


Report Reviewed: The Effects of Test-based Retention on Student Outcomes Over Time: Regression Discontinuity Evidence from Florida
Publisher/Think Tank: National Bureau of Economic Research (NBER)
Author: Guido Schwerdt and Martin R. West
As part of its working paper series, the National Bureau of Economic Research (NBER) recently released a report examining the outcomes of Florida’s third-grade retention policy. The report concluded, contrary to the conventional wisdom on grade retention, that third-grade retention had positive effects on the following year’s test results, but the effects fade, with no effect on graduation. The report attempted to investigate the impact of a Florida policy, which flags students for retention, to repeat third grade, based on a state-specified cut-score on the Florida Comprehensive Achievement Test. The findings indicated that students just below the threshold (one-third of whom were retained) performed better than those just above the threshold (5% of whom were retained) on next year’s tests.
 
Think Twice Review Date: December 3, 2015
Reviewer: Joseph P. Robinson-Cimpian, University of Illinois at Urbana-Champaign
Overall, the review points out, the methods used have extremely limited generalizability, which is restricted to students at or very near the threshold and directly affected by the policy. Even setting aside the problems generated by confounding retention effects with the effects of other interventions and supports, the findings are not easily generalizable to lower- or higher-achieving students, to other grades, or to other states with similar test-based retention policies.


Report Reviewed: The Hidden Value of Curriculum Reform: Do States and Districts Receive the Most Bang for their Curriculum Buck?
Publisher/Think Tank: Center for American Progress (CAP)
Author: Ulrich Boser, Matthew Chingos, and Chelsea Straus
The Center for American Progress (CAP) issued a recent report that drew on bold conclusions about the high payoff of better textbooks. The report claimed that, compared to other reforms, a switch to better textbooks was a cost-efficient way to improve student achievement. The report pointed to very real problems with textbook adoption, including poor alignment with standards. It also made strong claims about the payoff of schools’ investment in high-quality curriculum materials.
 
Think Twice Review Date: November 19, 2015
Reviewer: Sarah Lubienski, University of Illinois at Urbana-Champaign
The review points to clear merits of the report. However, the review also points out that the report overreaches in several areas. Specifically, the report is based on a single prior study and ignores key findings within the original study. Additionally, the report attempts to compare estimated return on investment (ROI) for textbooks against another study’s calculations, ignoring findings from that report as well. The report’s conclusions are marred by highly optimistic claims about curricular ROI.


Report Reviewed: Do We Already Have Universal Preschool?
Publisher/Think Tank: Brookings Institution
Author: Grover J. (Russ) Whitehurst and Ellie Klein
A Brookings Institution project called “Evidence Speaks” claims in a recent report that advocates exaggerate unmet need as well as the cost of universal pre-kindergarten. Unfortunately, an academic review of the report released today finds the report vastly underestimates unmet need and costs. The report estimates that 69 percent of all four-year-olds already attend preschool and that universal access tops out at 80 percent enrollment. The report estimated that only $2 to $4 billion per year would be needed.
 
Think Twice Review Date: November 12, 2015
Reviewer: W. Steven Barnett, National Institute for Early Education Research (NIEER) at Rutgers University
Unfortunately, an academic review of the report released today finds the report vastly underestimates unmet need and costs. According to the review, both estimates are based on serious factual errors and unfounded assumptions. Regarding the overall usefulness of the report, Professor W. Steven Barnett determines, “The report’s conclusions regarding needs and the costs to meet them are invalid and misleading, and it should not be used as a basis for policymaking.”


Report Reviewed: Ten Years in New Orleans: Public School Resurgence and the Path Ahead
Publisher/Think Tank: Public Impact and New Schools for New Orleans
Author: Christen Holly, Tim Field, Juli Kim, Bryan C. Hassel, Maggie Runyan-Shefa, Michael Stone, and Davis Zaunbrecher
A report published by Public Impact and New Schools for New Orleans attempted to review 10 years of education reforms in post-Katrina New Orleans, and the creation of a “portfolio model.” The report argued that the reform experiment has been an unquestioned success, and other cities should duplicate efforts. Overall, the report lays out a grand vision for the future of public education in New Orleans, which was described by the authors as “America’s first great urban public school system.” It is organized into six themes: governance, schools, talent, equity, community, and funders.
 
Think Twice Review Date: October 21, 2015
Reviewer: Adrienne D. Dixson, University of Illinois at Urbana-Champaign
An academic review of the report from Adrienne D. Dixson finds that the report does little to accurately inform policymakers or practitioners about the current state of public education in New Orleans or the viability of “portfolio” districts. The review found multiple weaknesses that limit the usefulness of the report. In her review, Dixson highlights that the report exaggerates improvements, while downplaying the ways in which the enacted reforms exacerbated inequities in New Orleans. She also notes that the report erroneously presents the reforms as a result of a logical and apolitical process.


Report Reviewed: The Mirage: Confronting the Hard Truth About our Quest for Teacher Development
Publisher/Think Tank: TNTP (previously The New Teacher Project)
A recent report from TNTP, previously The New Teacher Project, argued for changes in the way public school districts think about teacher development. The report identified a divide between teacher professional development and improved teacher evaluation scores. The report offers lessons about professional development in three large school districts and one mid-size charter network. Specifically, the report provides details about teachers’ time investment, how that time is spent, and district costs related to professional development.
 
Think Twice Review Date: September 30, 2015
Reviewer: Heather C. Hill, Harvard Graduate School of Education
Professor Heather C. Hill, Harvard Graduate School of Education, identified several strengths of the report and noted that readers could benefit from the report’s empirical evidence. However, she cautions that several of the reports conclusions are problematic. Weaknesses of the report identified by the reviewer, include: (1) problems with the analysis comparing teachers’ professional development and growth in teacher evaluation scores; (2) differences between academic conventions for calculating the cost of professional development and the methods used in the report; and (3) “hyperbolic” statements presented without sufficient support.


Report Reviewed: Measuring Diversity in Charter School Offerings
Publisher/Think Tank: American Enterprise Institute (AEI)
Author: Michael McShane and Jenn Hatfield
A recent report from the American Enterprise Institute (AEI) rates the diversity of charter school programs in 17 major cities.  The authors of the report advocate for the expansion and deregulation of charter schools since they provide greater program variety and, thus, respond to parental desires. The authors find small to moderate correlations between city demographics and certain types of charter schools. They also find that specialized charters tend to morph into homogenized general schools over time.
 
Think Twice Review Date: September 1, 2015
Reviewer: Arnold Danzig, San Jose State University; and
William J. Mathis, University of Colorado Boulder
The reviewers found several weaknesses: (1) the report claims charter schools provide greater program diversity, but fails to empirically compare charter offerings to traditional public school districts, which can also have diverse offerings; (2) the report claims but does not address how charter schools are hamstrung by red tape; and (3) the report’s findings were based on website descriptions, which can be error-prone. More specifically, the reviewers find that the correlations made in the report are weak, and relied on only 17 cases. Ultimately, the report fails to support its major claims and is of little use to practitioners or policymakers. The reviewers conclude, “Assuming we accept diversity of offerings as a primary policy goal, the report presents no evidence that charter schools do any better or worse than the current mix of public school alternatives.”

 

Report Reviewed: School Closures and Student Achievement: An Analysis of Ohio’s Urban District and Charter Schools
Publisher/Think Tank: Thomas B. Fordham Institute
Author: Deven Carleson and Stéphane Lavertu
This report from the Thomas B. Fordham Institute investigated school closures in Ohio for urban district and charter schools. The report found that test scores of displaced district students showed greater gains in math and reading relative to students from non-closed schools and that displaced charter students showed gains in math but not reading. Among the findings of the report, closed schools were attended by greater percentages of African American students from low-income families.  Additionally, the report noted that displaced students performed better if they transferred to schools with higher levels of performance.
 
Think Twice Review Date: June 16, 2015
Reviewer: Ben Kirshner, University of Colorado Boulder; and Matthew Gaertner, Pearson
An academic review of the report finds that, despite the encouraging results, they leave un-addressed core questions about closure policy. National and international media extoled the report’s findings and suggested that closing schools could improve educational outcomes. However, the reviewers caution that readers should not draw strong or straightforward conclusions from the study. The reviewers cited three primary concerns: (1) availability of higher-quality receiving schools that are easily accessible to students ought to be a precondition for a decision to close a school; (2) what counts as a better schooling option needs to be better defined, with the promise of safe, reliable transportation options; and (3) school closure does not provide a lasting solution to challenges associated with economic and racial segregation found in neighborhoods of concentrated poverty. The reviewers primary concern for the report focuses on the ability of parents and communities to make decisions about schools.

 

Report Reviewed: ESEA Reauthorization: How Can We Build upon No Child Left Behind’s Progress for Students with Disabilities in a Reauthorized ESEA
Publisher/Think Tank: Center for American Progress (CAP)
Author: Chelsea Straus
A report from the Center for American Progress (CAP) links benchmarks for inclusion in state testing and stringent accountability with improved educational outcomes for students with disabilities. While noting that causal claims could not be made, the report then proceeded to try to convince readers of exactly such claims. The report compared 2000 to 2013 NAEP test scores along with NCES national-level data and found increased test scores, decreased dropout rates, and increased graduation rates for students with disabilities, as well as improved outcomes for Black and Hispanic students with disabilities.
 
Think Twice Review Date: June 8, 2015
Reviewer: Edward Fierros & Katherine Cosner, Villanova University
An academic review by Edward Fierros & Katherine Cosner, Villanova University, explains that the causal arguments made in the report are nowhere near possible given the relatively weak data and analyses used in the report. Despite the report’s suggestions that tougher accountability and higher expectations produced these better educational outcomes, the reviewers found little to justify the claims. “While these student outcomes did indeed improve for students with disabilities during this time period,” the reviewers note “the report is wrong to assert that improvements were caused by NCLB or NCLB-type reforms.”


Report Reviewed: Dramatic Action, Dramatic Improvement: the Research on School Turnaround
Publisher/Think Tank: Center for American Progress (CAP)
Author: Tiffany D. Miller and Catherine Brown
The Center for American Progress (CAP) recently released a report advocating for implementation of evidence-based best practices for turning around low-performing schools through the federal School Improvement Grant (SIG) program. The report asserts that it offers five lessons about the most effective, research-based methods for turning around low-performing schools through the federal SIG program.
 
Think Twice Review Date: May 28, 2015
Reviewer: Tina Trujillo, University of California Berkeley
Tina Trujillo, in her review, finds that the report essentially ignores large bodies of research on high-stakes accountability, school improvement, and emerging evidence on school closures and federally funded turnarounds. Specifically, Trujillo cautions that a more inclusive review of extant research “reveals that the federal SIG program’s turnaround policies are based on unwarranted claims and are contradicted by the empirical evidence.” Trujillo’s review identifies that the report failed to meet standards for evidence and analytic transparency, and is of little use to policymakers or practitioners.


Report Reviewed: Increasing Education: What it Will and Will Not Do for Earnings and Earnings Inequality
Publisher/Think Tank: The Hamilton Project
Author: Brad Hershbein, Melissa S. Kearney, and Lawrence H. Summers
A recent report from the Hamilton Project sought to study how public investment in education will further long-term prosperity, economic growth, and individual economic security. The inquiry focused on whether or not a bachelor’s degree or higher education would increase economic prosperity and reduce economic inequality.
 
Think Twice Review Date: May 18, 2015
Reviewer: Marvin Lazerson, Central European University and University of Pennsylvania; and Ryan Pfleger, University of Colorado Boulder
In the review, the authors find that the report and its assertions are straightforward and use an empirically-based simulation for future projections. However, the report has several limitations. Specifically, Marvin Lazerson and Ryan Pfleger find the following insufficiencies: (1) there is little direct evidence in the report to show that increasing educational attainment is the most “efficient and effective,” as the report describes, way to improve prosperity; (2) the data are drawn are only from males, with no attention paid to gender, race, field of study, labor-market conditions, or institutional reputation; and (3) no data were analyzed to evaluate other ways to address economic problems. The reviewers conclude, “Claiming that the primary solution to a wide array of economic problems is to improve ‘human capital,’ the report perpetuates a problematic myth that undervalues alternative ways to address poverty and economic insecurity.”


Report Reviewed: Dramatic Action, Dramatic Improvement: the Research on School Turnaround
Publisher/Think Tank: Center for American Progress (CAP)
Author: Tiffany D. Miller and Catherine Brown
The Center for American Progress (CAP) recently released a report advocating for implementation of evidence-based best practices for turning around low-performing schools through the federal School Improvement Grant (SIG) program. The report asserts that it offers five lessons about the most effective, research-based methods for turning around low-performing schools through the federal SIG program.
 
Think Twice Review Date: May 11, 2015
Reviewer: Tina Trujillo, University of California Berkeley
Tina Trujillo, in her review, finds that the report essentially ignores large bodies of research on high-stakes accountability, school improvement, and emerging evidence on school closures and federally funded turnarounds. Specifically, Trujillo cautions that a more inclusive review of extant research “reveals that the federal SIG program’s turnaround policies are based on unwarranted claims and are contradicted by the empirical evidence.” Trujillo’s review identifies that the report failed to meet standards for evidence and analytic transparency, and is of little use to policymakers or practitioners.


Report Reviewed: Pushed Out? Low-Performing Students and New York City Charter Schools
Publisher/Think Tank: Manhattan Institute
Author: Marcus A. Winters
A Manhattan Institute report by Marcus A. Winters recently investigated claims that New York City charter schools are pushing out low-performing students as a means of inflating academic achievement scores. The report suggested that charter school exit rates were similar to traditional public schools (TPS). In short, the report determines that there is no charter push-out effect for low-performing students in charter schools.
 
Think Twice Review Date: May 4, 2015
Reviewer: Erica Frankenberg, Penn State University
An academic review of the report finds, despite a rich dataset available for the analysis, that the report has little detail and fails to provide guidance to policymakers. Frankenberg, in her review, discovers that: (1) the research design does not address its primary push-out question; (2) the report has little detail; (3) the report does not examine other relevant factors; and (4) there is a substantial gap between the findings and the conclusions of the report. Specifically, Frankenberg notes that the report’s findings show attrition rates for low-achieving students are higher than attrition of higher-performing students. The report’s conclusions that NYC charter schools are not pushing out students are unsupported.


Report Reviewed: Urban Charter School Study
Publisher/Think Tank: Center for Research on Education Outcomes (CREDO) at Stanford University
A recent report from the Center for Research on Education Outcomes (CREDO) at Stanford University attempted to investigate whether charter schools generate better outcomes than traditional public schools (TPS) in urban environments. The report, part of a series of reports on the performance of charter schools relative to TPS, asserts charter schools in urban environments provide a slightly greater test score advantage than those in non-urban environments. The report utilizes a methodological approach similar to previous reports from CREDO, and finds that students in urban charter schools were estimated to score approximately 0.055 standard deviations higher on math tests and 0.039 standard deviations higher on reading tests than their peers in urban TPSs.
 
Think Twice Review Date: April 27, 2015
Reviewer: Andrew Maul, University of California Santa Barbara
An academic review issues concerns with the methodology and reporting of the CREDO study. In a review, Andrew Maul cites the following concerns: (1) the study’s “virtual twin” technique is insufficiently documented; (2) the report’s estimation of growth using “days of learning” requires accepting untested assumptions; and (3) the study includes a number of arbitrary and unexplained analytic choices. All the same, Maul states, “Even setting aside such concerns over analytic methods, the actual effect sizes reported are very small, explaining well under a tenth of one percent the variance in test scores.” In his conclusion, Maul says “The findings of this report cannot be regarded as compelling evidence of the greater effectiveness of charter school compared to traditional public schools, either overall or specifically within urban districts.”


Report Reviewed: Measuring and Understanding Education Advocacy
Publisher/Think Tank: Brown Center on Education Policy at Brookings
Author: Grover J. “Russ” Whitehurst, David Stuit, Claire Graves, and Lauren Shaw
The report looked at the causal influence of organizations for and against education reform in Louisiana, Tennessee, and North Carolina. According to the report, findings indicated that coordination of advocacy groups strengthens their impact on the introduction of policy into the legislative arena, content of legislation, and the votes of members of the legislature. The report also found that the groups’ perceived influence closely tracked outcomes. The report utilized two methodological innovations to measure the impact of advocacy groups on education reform policy: (1) Surveys with Placebo (SwP); and (2) Critical Path Analysis (CPA).
 
Think Twice Review Date: April 16, 2015
Reviewer: Robin Rogers, Queens College, CUNY
Sara Goldrick-Rab, University of Wisconsin-Madison
An academic review of the report finds there is a gap between the evidence and the conclusions presented, however, the methods used in the report may be useful in education policy research. Regarding the overall merits of the report, Rogers and Goldrick-Rab state, “this report is useful primarily for understanding the perceptions of education advocacy groups’ influence and tactics in the three cases studied.” In their review, Rogers and Goldrick-Rab, find that SwP and CPA may be useful in education policy research, but the methods have limitations that are not acknowledged in the report. Additionally, the reviewers suggest the report is further limited because the research is a small case study, included a low response rate, and were based on advocacy groups’ self-reported tactics.


Report Reviewed: Paying the Best Teachers More to Teach More Students
Publisher/Think Tank: Edunomics Lab at Georgetown University
Authors: Marguerite Roza and Amanda Warco
A recent report from the Edunomics Lab at Georgetown University proposed that school districts pay top performing teachers a bonus for accepting additional students into their existing classes. The report claimed that larger classes and reductions to the teaching force would create significant savings. According to the report, districts should pay teachers in the top quartile a bonus for increasing their class size by up to three students. The report considers that students working with more effective teachers would offset any potential sacrifice in student learning.
 
Think Twice Review Date: April 13, 2015
Reviewer: Patricia H. Hinchey, Penn State University
An academic review of the report finds that the report is largely unsubstantiated, ignores what is known about teacher pay, and fails to offer guidance for policymaking. In her review, Penn State’s Patricia Hinchey finds that the report: (1) ignores the technical problem of how the best teachers might be reliably identified; (2) neglects a strong research base that has established a link between class size and student learning; and (3) misrepresents what is known about teacher pay, teacher attitudes, and teacher job satisfaction.

 

Report Reviewed: Separating Fact and Fiction: What You Need to Know about Charter Schools
Publisher/Think Tank: National Alliance for Public Charter Schools (NAPCS)
The report addressed 21 "myths" regarding charter schools, which were rejected. Succinctly, the original report addressed various claims about charter schools in such areas as financial equality of charter schools, lower teacher qualifications, student selection demographics, academic outcomes, segregation, and innovation.
 
Think Twice Review Date: February 23, 2015
Reviewer: Gary Miron, Western Michigan University
Kevin G. Welner, University of Colorado Boulder
and William J. Mathis, University of Colorado Boulder
The reviewers found that the report’s main purpose appears to be the "repetition or 'spinning' of claims voiced by advocacy groups and think tanks that promote privatization and school choice." Furthermore, the reviewers found that it relied almost exclusively on advocacy documents rather than more careful and balanced empirical research, and provides only a superficial examination of any "criticisms" regarding charter schools. The review is organized in a format that lists each of the criticisms identified, and then provides a short commentary based on the extant research literature. Where the original document overlooked research evidence, the reviewers provide readers with a valuable tool to examine charter school criticisms.


Report Reviewed: The Texas Economy and School Choice
Publisher/Think Tank: Texas Public Policy Foundation & Texas Association of Business
Authors: Arthur Laffer, Laffer Associates Investment Research
The Texas Association of Business and the Texas Public Policy Foundation commissioned Laffer Associates to perform an analysis of the Taxpayer Savings Grant Program (TGSP), which is a universal voucher program for Texas. The report theorized that by raising graduation rates, improving education achievement, and thus increasing human capital, the TSGP would create economic growth in Texas.
 
Think Twice Review Date: February 17, 2015
Reviewer: Chris Lubienski & Ee-Seul Yoon, University of Illinois
Chris Lubienski and Ee-Seul Yoon, in their review, highlight two key problems with the report: (1) Arthur Laffer’s assertions about the educational benefits of choice represent a severe overreach with and misapplication of the available research; and (2) the economic estimations are overgeneralized and heavily biased towards those families who already have the wealth to choose and relocate. The reviewers also note that the TSGP could result in further inequities for Texas schoolchildren, as higher income families would be able to supplement their children’s education even further, while devoting fewer resources to low-income families. In their conclusion, Lubienski and Yoon state: “While this report is clearly written to recommend the TSGP to the State of Texas, the lack of comprehensiveness and transparency, as well as the problems in its methodology, literature review and analysis, make it unsuitable as a basis for public policy decisions.”

 

Report Reviewed: Turning Lightning into Electricity: Organizing Parents for Education Reform
Publisher/Think Tank: American Enterprise Institute (AEI)
Author: Andrew P. Kelly
Turning Lightning into Electricity, written by Andrew P. Kelly, presents an inside look at several Education Reform Advocacy Organizations (ERAO) to examine the issue of parent participation and community organizing. The report offers strategy for advocacy groups who support the ERAO agenda to compete at the grassroots level.
 
Think Twice Review Date: February 2, 2015
Reviewer: Mark R. Warren, University of Massachusetts Boston
A review of the report released today finds limitations to the research methods and that the report cannot be considered a reliable study of parent organizing in ERAOs. Mark Warren, in his review, finds that the research methods employed are not adequately explained. Moreover, the research presented does not appear to be systematic or representative of ERAOs. He says: “the sample of ERAO groups is biased toward groups that more highly value long-lasting forms of parent engagement.” According to Warren, the report reduces organizing to a set of tactics to engage parents around an issue or agenda, rather than a democratic practice. “This approach undermines an understanding of community organizing as a profoundly democratic practice rooted in deeply held values of equity and social justice.”


Report Reviewed: No Excuses Charter Schools: A Meta-Analysis of the Experimental Evidence
Publisher/Think Tank: Department of Education Reform (DER) at the University of Arkansas
Authors: Albert Cheng, Collin Hitt, Brian Kasida, & Jonathan N. Mill
The authors of this working paper declared that students attending No Excuses charter schools had significantly improved math and reading scores. No Excuses schools feature high academic standards, strict disciplinary codes, extended instructional time, and targeted supports for low-performing students. The hope is that by having high standards for academics and discipline, No Excuses schools will help close the achievement gap between minority students and their White peers. The authors used findings from 10 studies that utilized experimental methods to estimate achievement outcomes.
 
Think Twice Review Date: January 20, 2015
Reviewer: Jeanne M. Powers, Arizona State University
Jeanne Powers, Arizona State University, found that the working paper is of limited value to policymakers or anyone seeking to understand the advantages or disadvantages of the No Excuses model. Powers, in her review, finds several flaws in the paper: (1) students who apply to charter school lotteries are not representative of all charter school students; (2) the authors did not address student attrition from charter schools; and (3) the results are based on a small sample of schools concentrated in East Coast cities. She says, “As a result, the author’s claim that No Excuses schools can close the achievement gap substantially overstates their findings.” Furthermore, Powers cautions “the current research base is too limited to make conclusions about the effectiveness of No Excuses charter schools.”

 

Report Reviewed: Proposed 2015 Federal Teacher Preparation Regulations
The U.S. Department of Education has released proposed Teacher Preparation Regulations under Title II of the Higher Education Act with a call for public comments through the Federal Register. The comment period closes February 2. The Department of Education claims that the proposed regulations will help ensure teacher-training programs are better preparing educators to succeed in the classroom by requiring states to measure outcomes of how graduates are doing in the classroom. The proposal would require states to assess and rate every teacher preparation program every year with four Performance Assessment Levels (exceptional, effective, at-risk, and low-performing), and states would be required to provide technical assistance to “low-performing” programs. Additionally, programs that do not show improvement could lose state approval, state funding, and federal student financial aid.
 
Think Twice Review Date: January 12, 2015
Reviewer: Kevin K. Kumashiro, University of San Francisco
An academic review of the proposed regulations considers the evidentiary support and identifies concerns. In his review Kevin Kumashiro has identified seven concerns with the proposed regulations. The proposed regulations: (1) will likely burden institutions with costs that are higher than estimated; (2) inaccurately conceptualize the impact and preparedness of teachers apart from systems; (3) mandate an improperly narrow definition of teacher classroom readiness; (4) require a reliance on scientifically discredited test-based accountability and value-added measures for data analysis; (5) disincentivize teachers to work in high-needs schools; (6) could restrict federal funding for students in financial need, and restrict access to the teaching profession for underrepresented groups; and (7) create a narrow view of the purposes of education.


Report Reviewed: Better Data, Better Decisions: Informing School Choosers to Improve Education Markets
Publisher/Think Tank: American Enterprise Institute (AEI)
Authors: John Valant
This report, published by AEI, asserts that choice is seen as a mechanism, and at times a panacea, for better educational quality. The report provides an overview of what families want from schools, where they get information on schools, and how they use information to make decisions. The report concludes that informed choice and the power of the market will produce better educational outcomes.
 
Report Reviewed: A Crisis We Can Solve: Connecticut’s Failing Schools and Their Impact
Publisher/Think Tank: Connecticut Coalition for Achievement Now (ConnCAN)
Authors: No author named
The ConnCAN report makes a case that there is a desperate need to improve school quality in Connecticut’s neediest neighborhoods and promotes charter schools as the best and perhaps only method for increasing the number of high-quality educational seats in Connecticut.
 
Think Twice Review Date: January 5, 2015
Reviewer: Erin McNamara Horvat, Temple University; and David Everington Baugh, Bensalem Township School District, Pennsylvania
A review of the reports finds that the ConnCAN report is more opinion than fact, while the AEI report provides useful information to help parents make informed decisions. However, the reviewers question whether simply having more information to make better decisions is sufficient to improve our educational system. Reviewers Horvat and Baugh

Previous Reports & Reviews:

These articles and/or reports are copyrighted material, the use of which has not been specifically authorized by the copyright owner. We are making such material available in our efforts to advance understanding of educational issues, etc. We believe this constitutes a 'fair use' of any such copyrighted material as provided for in section 107 of the U.S. Copyright Law. In accordance with Title 17 U.S.C. Section 107, the material on this site is distributed without profit to those who have expressed a prior interest in receiving the included information for research and educational purposes. For more information go to: http://www.law.cornell.edu/uscode/17/107.shtml. If you wish to use copyrighted material from this site for purposes of your own that go beyond fair use, you must obtain permission from the copyright owner.