Over at the Flypaper Mike Petrilli claimed that Charter Start-Ups are 4-Times as Likely to Succeed than District Turnarounds* (Note Big Asterisk). He bases this claim on an analysis Dr. David Stuit conducted for the Fordham Institute.
Although Petrilli was transparent about the fact that the analysis has significant shortcomings by placing “big asterisk” in the title, it didn’t dissuade him from recommending taking away hundreds of millions of dollars from low-performing and many cases severely disadvantaged traditional public schools and giving them to charter schools.
“It is screwy for federal tax payers to spend 12 times as much on school turnarounds ($3 billion) as charter start-ups ($250 million) when the latter appear to be four times more likely to succeed than the former. Team Obama want to fix that?”
A quick look at the analysis gives pause to whether indeed charter start-ups should be expanded as an alternative to turnaround traditional public schools.
Shortcomings of the Analysis
The analysis is based on a small sample of schools
- Only 81 pairs of low-performing traditional public schools and charter school startups were found across the ten large states.
- Of the 81 pairs, only 19 schools (15 charter and 4 traditional public schools were found to be “successful.”
- These pairs were not necessarily representative of schools nationwide.
Differences in “success” rates were not statistically significant
- That is, the difference in success rates may have happened by chance, rather than differences in actual effectiveness.
- Because of the small sample size it was not possible to determine with any confidence whether there was any true difference in success rates.
- There is even less statistical confidence that charter school start-ups are four times more likely to succeed than low-performing traditional public schools.
The analysis only examined the success of low-performing traditional public schools where charter schools were present.
- The report ignores all other traditional public schools that may have turned around – specifically, many of those schools that received federal ‘turnaround’ funds Petrilli is recommending sending to charters instead.
- It could be that low-performing traditional public schools have a greater success rate when a charter school is not located in their neighborhood.
The analysis did not examine whether the charter schools enrolled students of similar achievement.
- In particular, the analysis did not explore whether charter school start-ups impeded low-performing traditional public schools from becoming ‘successes’ by enrolling their higher-performing students from the low-performing traditional public school.
The analysis neglects the fact that fewer than 1 in 5 charter schools are more effective than their neighboring traditional public school.
- So expanding charters, even in areas where there are low-performing schools, is not necessarily the answer.
Finding that more charter start-ups is the answer neglects the fact that students will remain in traditional public schools.
Overall, the analysis tries to answer a very important question: What is the best way to turn around schools? But its methodology has significant limitations. As such, Petrilli’s conclusion and recommendations are overblown. However, the analysis does provide the basis for future research on this topic that can more accurately answer the question, “What is a better use of resources, putting money in charter schools or turning around existing schools?” However, more sophisticated research techniques are needed to answer this very important question. In the meantime, research indicates charter start-ups may be more successful, but not with enough confidence to siphon hundreds of millions of dollars from disadvantaged traditional public schools.
For more information about charter schools check out the Center for Public Education’s Charter Schools: Finding out the Facts.