On occasion, ASBJ has advised school board members to learn more about how research can be used to better understand education issuesand also warned them how research can muddy the waters if used incorrectly.
The editors of ASBJ have had cause to heed their own words lately.
Working on an article on school choice for the November issue, senior editor Naomi Dillon came across a U.S. Department of Education study that concluded, in her words, that “students enrolled in online courses perform at consistently higher levels when compared to traditional classroom settings.”
She also noted a study indicating students at the Florida Virtual School “earned better grades, scored higher on state tests, and outperformed their traditional classroom counterparts on AP courses, too.”
All true enough. But what was the other side of the story? What of studies that found no significant difference in student achievement between online and traditional classroom learning.
Other questions arose. Did the Education Department study determine whether working online was the cause of improved student achievementor did higher scores simply reflect that more high-achieving students have online opportunities than less-successful students?
And if Florida Virtual School students earn better grades, is that because students who sign up for online courses are more naturally self-motivated? What about students who regularly skip school and never bother with homework? Are they likely to enroll in online classes? Does their absence skew the comparison between online and traditional schools?
Rather than distract readers with a litany of research findings, we simply noted the cited research wasn’t the final word and went on with the main point of the story.
But questions also arose about an accompanying article I wrote. Not wanting to bog down my story with a dull list of research findings on charter schools, I settled on simply noting the conflicting research: “Some studies have found charter schools outscoring nearby public schools; other studies have concluded just the opposite.”
That moved the story along, but it wasn’t particularly helpful for any reader hoping ASBJ would clarify the issue for them. Nor did it offer any explanation for these contradictory findings.
At the same time, my story notes the existence of studies indicating charter schools and open enrollment policies can worsen racial isolation and the economic segregation of students. That hardly does justice to the complex and nuanced forces at work in creating this segregation. But, I settled for the brief reference simply to indicate that negative results can occur if school choice policies aren’t properly designed and monitored.
My point in sharing the inner workings of ASBJ is to highlight the editors’ appreciation for the challenges superintendents and school board members confront in sorting through today’s education research.
Also, this gives me an opportunity to remind readers to revisit ASBJ‘s November 2007 issue and its cover story: “Politics and Research.” There are some useful insights about using research in your policymaking. Among those:
- Look at where research originates. These days, almost anyone can publish a study or report on the Internet, and that’s led to a rise in material that’s ideologically driven or shoddy in its methodology. You need to do a little research of your own before deciding what research merits your attention.
- Accept that no one study has all the answers. Study findings can be contradictory because researchers aren’t studying exactly the same thing. Students can be different. Schools can be different. How results are measured can vary. But answers are thereif you look at the “body of knowledge” built up by study after study after study. Look for the commonalities in the findings.
- Question any claim of “cause and effect.” I recently read a study that concluded that students do better in schools with well-equipped libraries. But dig deeper: Do good libraries boost student test scores because lots of books area available? Or is it that schools that can afford a good library serve affluent communities and thus enroll students who statistically are more likely to do well academically?
These are just some of the kind of questions the ASBJ editorial staff asks itself when examining today’s education research. Yet, we ask these questions to ensure our magazine is as informative and accurate as possible.
You, our readers, have to ask these questions so you can better educate America’s children.
Del Stover, Senior Editor
