Milwaukee Voucher Report Sound But Of Limited Value
Future reports examining voucher program should be improved and clarified
EAST LANSING, MI (May 28, 2009)—A recent report on the second year of a five-year evaluation of the Milwaukee school voucher program finds no achievement growth differences between voucher schools and the Milwaukee Public Schools over a one-year period. A new review of the report finds its design and methods sound, but its usefulness is limited by several factors, including a lack of clarity about the representativeness of the voucher schools included in the study.
The MPCP Longitudinal Educational Growth Study Second Year Report was written for the University of Arkansas School Choice Demonstration Project as part of a five-year evaluation of the Milwaukee Parental Choice Program (MPCP).
The report was reviewed for the Think Twice think tank review project by Casey Cobb of the University of Connecticut, an expert in policy analysis and social science research methods.
The Second Year Report’s primary finding was that there were no overall statistically significant differences in achievement growth between a group of students in the voucher schools included in the study and a matched group of students in Milwaukee Public Schools (MPS).
The most substantial concern Cobb raises in his review involves the sample of schools used for this evaluation – and likely for future evaluations. An act of the Wisconsin legislature requires schools accepting vouchered students to administer a standardized test in reading, math and science in grades 4, 8, and 10. It is these test scores that are analyzed in the new report. Because the law doesn’t require any particular exam, different schools administer different tests. The sample of voucher schools included in the report reflects only those using one particular test, the Wisconsin Knowledge and Concepts Examination (WKCE). As a result, Cobb points out in his review, the results “are only generalizable to MPCP schools that administer the WKCE.” The key question, and one that a reader of the report cannot answer, is whether this sample differs in any meaningful way from those schools that use a different exam. For instance, might a given subgroup of schools, such as those run by the Milwaukee Catholic Archdiocese, be excluded from the study because they use a different test?
Find Casey Cobb’s review on the web at:
This is the third Think Twice review in recent weeks to examine the School Choice Demonstration Project’s Milwaukee reports. Previously, a review by Dr. Clive Belfield found that a report about the Milwaukee voucher program’s fiscal impact offers valuable insights but also questioned the report’s findings regarding fiscal benefits and noted that the report omits some important factors essential to a more complete analysis. And a review by Dr. Gregory Camilli of a report about the Milwaukee voucher program’s competitive effects raised a number of questions about the report’s statistical methods, ultimately concluding that any positive effect of competition is very small, if it exists at all.
About The Think Twice Project
The mission of the Great Lakes Center is to improve public education for all students in the Great Lakes region through the support and dissemination of high quality, academically sound research on education policy and practices.
Visit the Great Lakes Center Web Site at: http://www.greatlakescenter.org