Milwaukee Voucher Study Design too Weak to Link Vouchers to Higher Graduation Rates
New review recognizes report's usefulness, but warns about causal suggestion
EAST LANSING, Mi., (March 31, 2010) – A recent report from School Choice Wisconsin finds that students who attended private high schools under the Milwaukee voucher program had higher graduation rates than their Milwaukee Public Schools counterparts. A new Think Twice review praises the report as technically accurate and descriptively useful, but notes that any real claims about whether the voucher program is actually causing higher graduation rates would depend on a much stronger research design.
The report, Graduation Rates for Choice and Public School Students in Milwaukee, 2003-2008, was reviewed by Casey Cobb, Director of the Center for Education Policy Analysis at the University of Connecticut's Neag School of Education.
The Milwaukee report compares the graduation rates of a sample of students attending traditional Milwaukee Public Schools (MPS) high schools with a sample of high school students enrolled in the Milwaukee Parental Choice Program, which pays for low-income families in MPS to send their children to private schools. Examining six years of data, it finds that voucher students' graduation rates exceeded those of MPS students in five of the six years.
"The methods used in the report are statistically precise and explained in very clear detail," Cobb writes in his review. For the most part, the data on graduation rates "represent reasonable estimates," he notes.
But one key assumption--that net migration in or out of the public and private school programs was zero--was not supported by any direct evidence, Cobb observes. This could present a source of bias, because net in-migration tends to bias a graduation rate figure upward, while net out-migration biases the figure downward.
More importantly, any claims that the voucher program causes higher graduation rates "must be based on stronger research designs." The report does acknowledge that limitation, and it directs readers to a forthcoming longitudinal study of the program that is not yet complete, Cobb notes. Yet then it undermines that caution by stating that, if the MPS graduation rate had equaled that of the choice program, 3,352 more MPS students would thus graduated. "Although mathematically accurate, such extrapolation invites causal inferences," which would mislead readers, Cobb concluded.
Find Casey Cobb's review and a link to the full School Choice Wisconsin report on the web at: www.greatlakescenter.org.
About The Think Twice Project
The mission of the Great Lakes Center is to improve public education for all students in the Great Lakes region through the support and dissemination of high quality, academically sound research on education policy and practices.
Visit the Great Lakes Center Web Site at: http://www.greatlakescenter.org