Measuring cognitive load: performance, mental effort and simulation task complexity.

ABSTRACT

CONTEXT Interest in applying cognitive load theory in health care simulation is growing. This line of inquiry requires measures that are sensitive to changes in cognitive load arising from different instructional designs. Recently, mental effort ratings and secondary task performance have shown promise as measures of cognitive load in health care simulation. OBJECTIVES: We investigate the sensitivity of these measures to predicted differences in intrinsic load arising from variations in task complexity and learner expertise during simulation-based surgical skills training. METHODS: We randomly assigned 28 novice medical students to simulation training on a simple or complex surgical knot-tying task. Participants completed 13 practice trials, interspersed with computer-based video instruction. On trials 1, 5, 9 and 13, knot-tying performance was assessed using time and movement efficiency measures, and cognitive load was assessed using subjective rating of mental effort (SRME) and simple reaction time (SRT) on a vibrotactile stimulus-monitoring secondary task. RESULTS: Significant improvements in knot-tying performance (F(1.04,24.95)  = 41.1, p < 0.001 for movements; F(1.04,25.90)  = 49.9, p < 0.001 for time) and reduced cognitive load (F(2.3,58.5)  = 57.7, p < 0.001 for SRME; F(1.8,47.3)  = 10.5, p < 0.001 for SRT) were observed in both groups during training. The simple-task group demonstrated superior knot tying (F(1,24)  = 5.2, p = 0.031 for movements; F(1,24)  = 6.5, p = 0.017 for time) and a faster decline in SRME over the first five trials (F(1,26)  = 6.45, p = 0.017) compared with their peers. Although SRT followed a similar pattern, group differences were not statistically significant. CONCLUSIONS: Both secondary task performance and mental effort ratings are sensitive to changes in intrinsic load among novices engaged in simulation-based learning. These measures can be used to track cognitive load during skills training. Mental effort ratings are also sensitive to small differences in intrinsic load arising from variations in the physical complexity of a simulation task. The complementary nature of these subjective and objective measures suggests their combined use is advantageous in simulation instructional design research.