Multiple-choice is not easy!
Sometimes I give multiple-choice questions.
People make the error of thinking these are easy.
In fact, I work hard at designing a confusing choice of answers so that it is hard to get the answer by chance.
Marks in my multiple-choice tend to be similar to marks in my long questions.
People do fail this section.
Here is an example.
- In 2016, in CA249, one person got 2 out of 15 on my multiple-choice (13 percent).
-
If you answered randomly, you would score on average 2.56 out of 15 (17 percent).
So this person scored worse than chance.
-
Curious about this, I decided to get my dog to do the test.
This is Hector.
- For each question with n answers, my daughter laid out n treats on the ground. I would read the question out to Hector and release him. Whichever treat he picked up first would be his answer.
- He scored 3 out of 15. He was not bottom of the class.
-
The questions Hector got right were questions 1, 4 and 12.
Given that Hector has never used Linux or Unix (or indeed any computer) I think he did very well.
- The conclusion is not that multiple-choice is easy. (After all, Hector failed.) The conclusion is that it is hard. (One student could not even beat my dog.)