“Not all questions are answered, commander, but fortunately some answers are questioned.”
– Terry Pratchett, Snuff

Although I think MQCs can be useful, especially for formative assessments, there are a number of alternatives that can improve upon this style and still often have the same advantages of fast marking and feedback. This makes MCQs more useful and more suitable for summative assessments.

Multiple Option Questions

The first of these is making multiple correct answers for each question, for example (stared options are the correct answers):

If the wet-bulb temperature is calculated from
$\displaystyle T_w=T-\frac{\Delta\,H_w^{\text{vap}}}{C_p}\left(\frac{1}{\text{Le}}\right)^{2/3}\left(H_w-H\right)$
what assumptions about the drying process have been made?

• The dryer operates isobarically*
• The energy for moisture evaporation is only supplied from hot gas by convective heat transfer*
• The mole fraction of moisture in the bulk gas is small*
• The dryer operates isothermally
• The drying rate is very slow
• The drying must be a batch operation

Guessing (or simply selecting all answers) can then be prevented by having negative marking on the incorrect answers. You can have the amount of negative score whatever you want for the incorrect options. I have seen lots of options including each incorrect answer being worth the same as all the correct answers in total (thus selecting any incorrect option gives 0 for the questions); however, typically if all the answers are selected then I recommend that the total mark should be zero (and never more than 0), such that,

Where $M_i$ is the mark per incorrect option, $M_c$ is the mark per correct option, $n_i$ is the number of incorrect options, and $n_c$ is the number of correct options. It is also important that the question is set so that the lowest score is 0, so even if someone selects only the incorrect options then their score is capped at 0 rather than producing a negative total for the question.

In this kind of question, it is still useful to have 2-3 distractors, and to try to make the number of correct and incorrect options the similar. However, it is important not to have the same number of each for different questions as this will cause students to guess more, e.g. if they have spotted that there are always 3 correct and 3 incorrect options.

Assertion-Reason Questions

The second type of advanced MCQ that I tend to find effective are the assertion-reason type (ARQs). This type of question consists of two statements, the first an assertion and the second a potential reason. The student must first determine whether each statement is true and then if both are true, the student must determine whether the reason correctly explains the assertion. We can see an example of this type of question,

The question below consists of an assertion and a reason. Indicate your answer from the alternatives below.

Chloroform has a dipole moment
               BECAUSE
the chloroform molecule is tetrahedral.

	Assertion	Reason
A	True	True	Reason is correct explanation
B*	True	True	Reason is NOT a correct explanation
C	True	False
D	False	True
E	False	False

These types of questions hopefully make students think more and as shown by Williams (2006) the learning outcomes are likely to be superior to those produced by traditional MCQs which tend to focus on recall rather than reasoning. However, it is very important that given the additional complexity of ARQs, the meaning of each question is clear and the wording free of ambiguity.

Scenario-Based Questions

The third type of advanced MCQs is I suppose is technically not a different type but a method of combining them to provide higher level testing. This is the Scenario-Based Questions (SBQs) where the questions based around one task or scenario and allows the student to think through a concept. This should be clearer if we look at the example from Haladyna (1994).

Two researchers were studying the relationship between amount of sleep each night and calories burned on an exercise bike for 42 men and women. They were interested if people who slept more had more energy to use during their exercise session. They obtained a correlation of 0.28, which has a two-tailed probability of 0.08. $\alpha$ was 0.10.

1. Which is an example of a properly written research question?
   • Is there a relationship between amount of sleep and energy expanded?*
   • Does amount of sleep correlate with energy used?
   • What is the cause of energy expanded?
   • What is the value of $\rho$?
2. What is the correct term for the variable amount of sleep?
   • Dependent*
   • Independent
   • Predictor
   • $y$
3. What is the correct statistical null hypothesis?
   • There is no correlation between sleep and energy expanded
   • $\rho$ equals zero*
   • $r$ equals zero
   • $\rho$ equals $r$
4. What conclusions should you draw regarding the null hypothesis?
   • Reject*
   • Accept
   • Cannot determine without more information
5. What conclusions should you draw regarding this study?
   • The correlation was significant
   • The correlation was not significant
   • A small relationship exists*
   • No relationship exists

I often use these scenario based questions with students as you can build on the information. Also multiple choice, multiple option, and even numerical calculations can be combined together to give good coverage of a topic.

Summary

These are three useful variants on the standard MCQs, there are others available, but I don’t think they add much over the standard MCQs. In addition to MCQs many online testing systems now come with other useful question types that can be automatically marked. As an engineer I tend to find the most useful types are numerical questions, Blackboard calls these Calculated Formula Question were a formula is input and the variables can be set to change for each student within a suitable range.

References

J. B. Williams (2006) “Assertion-reason multiple-choice testing as a tool for deep learning: a qualitative analysis”, Assessment & Evaluation in Higher Education, 31: 287-301.
T. M. Haladyna (1994) Developing and validating-multiple choice test items, 1st ed. Hillsdale, NJ: Lawrence Erlbaum Associates.