Mathematica Report on Charter School Managers Offers Insights on What Works -- but Downplays Disappointing Results
EAST LANSING, Mich. (Jan. 19, 2012) – A recent study by Mathematica Policy Research Inc. and the Center on Reinventing Public Education on the benefits stemming from private, nonprofit charter school management operators highlighted interesting trends in the U.S. charter movement, but overreached when interpreting key findings, according to a new review.
The report, Charter-School Management Organizations: Diverse Strategies and Diverse Student Impacts, was reviewed for the Think Twice review project by Professor Bruce Fuller of the University of California-Berkley. The review was produced by the National Education Policy Center with funding from the Great Lakes Center for Education Research and Practice.
Charter management organizations (CMOs) are nonprofit firms that operate multiple charter schools either directly or through management contracts. CMO-operated schools account for about one-fifth of the nation's more than 5,000 public charter schools.
Fuller's review praised the report for offering an objective assessment of the comparative benefits for middle-school teachers and students. But the review also noted that the report's analytic approach relied on a highly select slice of all CMOs operating nationwide, moving from about 130 CMOs to just 22 that qualified for the study sample. The review questioned the generalizability of the findings arising from these select CMOs, and it criticized the report's "tortured" attempts to find achievement advantages in the CMO-run schools..
Most CMOs, the report's authors found, serve urban students from low-income families. Survey data and site visits show that CMO-operated schools are typically smaller; offer longer school days, school years, or both; and recruit teachers who are loyal to their respective school's mission. The principals of CMO-run schools reported more frequent coaching and mentoring of teachers and more intensive use of student test results and other achievement data to evaluate teachers than did nearby traditional public schools. Fuller's review found the report to be solid and useful regarding these findings.
At the heart of the report is an impact analysis focusing on middle school grades. The analysis began with the more than 130 CMOs operating nationwide and then engaged in the successive narrowing of the CMOs to be included in the study down to just 22 that met several inclusion criteria.
From those 22, the report found that a small number of CMOs boosted middle-school student achievement growth at discernible levels. "Once thus narrowed, the study presents impressive results for between 4 (reading outcome) and 7 (math outcome) of the 22 CMOs," Fuller observes. "So, to whom or what can we generalize these results?" Moreover, the overall effects of CMOs on student achievement were largely flat when compared with both traditional public schools and with a comparison sample of independent charter schools not affiliated with a CMO.
"What's disappointing for charter-school adherents is the bottom line that the average effects of attending a CMO-run charter school are not significantly different from those of attending a regular public school," Fuller notes. He criticizes the report for what he calls its "unrelenting search for achievement effects in a small, selective subset of sampled CMOs," which he states "erodes its credibility."
Fuller concludes that the report does offer illuminating insights into features of successful CMOs that appear to have a modest association with student achievement growth in middle school. "This report gets us a bit closer toward pinpointing highly effective CMOs and the practices or resources that may help to explain their efficacy," he writes. "What is not known is whether these practices or resources can be exported with integrity to other charter companies or to the wider public school system. These questions should be squarely addressed in future research."
Find Bruce Fuller's review on the Great Lakes Center website at:
The review is also available on the NEPC website at:
The mission of the Great Lakes Center is to improve public education for all students in the Great Lakes region through the support and dissemination of high quality, academically sound research on education policy and practices.