One frequently stated disadvantage of multiple-choice tests is that they test primarily or exclusively knowledge. The higher levels of mastery (e.g. comprehension, application or problem-solving) are usually not addressed. Studies have shown that students generally study more superficially for multiple-choice tests than they do for other types of tests (e.g. with open-ended questions). The construction of multiple-choice questions that assess higher levels of mastery is nevertheless not impossible. Although it is not simple, a few tips could help in this regard:
- Start with a knowledge question
If you are not able to arrive at a question for a higher level of mastery immediately, it is often worthwhile to cast the content to be assessed as a knowledge question, which can subsequently be transformed to the intended higher level of mastery. For example:
According to the objectives, medical students should be able to identify a disease profile based on symptoms (application level). In this case, you could proceed from the knowledge question: Which symptoms are characteristic of Disease x? (followed by several combinations of the symptoms). This question could then be easily transformed into an application question: "Ms Y is complaining of the following symptoms. What is your diagnosis of her disease profile?"
- Use a brief case, scenario or newspaper article
This makes it possible to assess students’ problem-solving skills (analysis, synthesis, evaluation). The inclusion of such questions in a test, however, often increases the testing time. This could be kept under control through such measures as asking several sub-questions on several topics to be tested based on only one case.
- Use visual material
Instead of asking for definitions of specific concepts, students could be asked to interpret concepts based on figures, tables or graphs.
- Proceed from active verbs
One convenient tool consists of using verbs that are representative of a certain level of mastery. For example: You would like to ask a question on a particular topic at the level of comprehension. The following verbs are typical of the ‘comprehension’ level: ‘in your own words, state’, ‘using your own example, illustrate’, ‘list the similarities and differences’.
- Avoid copying the course material verbatim
The use of text fragments that students could recognise from the lectures or course measures only quality of a student’s memory. With regard to testing the higher levels of mastery, therefore, it is important to use different wordings and new examples.
Want to know more?
Brame, C., (2013). Writing good multiple choice test questions. Retrieved from Vanderbilt University website (accessed August 23rd, 2019).
Burton, S., Sudweeks, R., Merrill, P. & Wood, B. (1991). How to Prepare Better Multiple-Choice Test Items: Guidelines for University Faculty. Retrieved from Brigham Young University website.
Clay, B. (2001). Is This a Is This a Trick Question? Question? Question? A Short Guide to Writing Effective Test Questions. Retrieved from Kansas State University website.
Scully, Darina (2017). Constructing Multiple-Choice Items to Measure Higher-Order Thinking. Practical Assessment, Research & Evaluation, 22(4).