It was a long time ago, but I still remember the multiple choice question I got wrong on one elementary school test, and the thought of it still bothers me.
The test was asking (for what reason, I can’t remember, or, more likely, never knew) the best way to dry off after going swimming. There were two obviously bogus choices, then the one I was going to choose dry off with a towel and then another: Let the sun dry you.
Hmmm: Let the sun dry you? I thought. Why not? It’s easier, feels nice, and is more basic. (I don’t think we used the word natural back then.) OK, I’ll pick that.
Why did it bug me so much? I think it’s because the test was telling me that I made a stupid choice, and therefore, must be stupid. Sure, the answer “dry off with a towel” was the most obvious — even I knew that; but it was also totally arbitrary, and I knew that too. I just didn’t have the words to express it back then.
I’m happy to say I made it through elementary school, nonetheless. But consider for a moment: What if that question, or others like it, had been used for a more consequential purpose say, to evaluate my teacher on how good a job she was doing? The stakes would have been a lot higher than one 3rd grader’s bruised ego.
James Popham, an emeritus professor at the UCLA Graduate School of Education and a former president of the American Educational Research Association, has a name for multiple choice questions like that: They’re “instructionally insensitive,” that is, they do a poor job of telling the evaluator how well a given teacher is doing.
Unfortunately, Popham says in (Mis)Measuring Teachers, his must-read article in September’s ASBJ, the people developing tests to evaluate teachers have little idea how valid their tests will be. Yet everyone from Secretary of Education Arne Duncan, to state and federal leaders, to the heads of major think tanks are calling for instituting tests now that purport to measure how well teachers are doing based on the test performance of their students.