Do sequentially-presented answer options prevent the use of testwiseness cues on continuing medical education tests?

ABSTRACT

Testwiseness--that is, the ability to find subtle cues towards the solution by the simultaneous comparison of the available answer options--threatens the validity of multiple-choice (MC) tests. Discrete-option multiple-choice (DOMC) has recently been proposed as a computerized alternative testing format for MC tests, and presumably allows for a better control of testwiseness. It is based on a sequential rather than simultaneous presentation of answer options. The test taker has to decide on the correctness of one option after another, until the item has been answered either correctly or incorrectly. Test items that have been criticized for being susceptible to testwiseness strategies are used in continuing medical education (CME) programs aimed at developing and maintaining the knowledge of medical professionals. In Experiment 1 with 48 adults without a special medical education, presenting answer options sequentially reduced the use of testwiseness cues on a CME test compared to their simultaneous presentation as shown by a significant interaction of answer format and the availability of cues (p = .01, η(2) = 0.13). This result was not dependent on a hint towards potential cues to the solution, as it also held when another 86 adults were not informed of the presence of testwiseness cues in Experiment 2 (p < .001, η(2) = 0.14). The result could be replicated with 106 medical students and medical doctors in Experiment 3. Items were answered correctly more often in the MC condition (71%) than in the DOMC condition (47%), but only when items contained cues to their solution. A significant interaction between answer format and the availability of cues (p = .02, η(2) = 0.05) showed that the sequential DOMC answer format allows for a better control of testwiseness than traditional MC testing.