Teamwork Training With a Multiplayer Game in Health Care: Content Analysis of the Teamwork Principles Applied.

BACKGROUND: In health care, teamwork skills are critical for patient safety; therefore, great emphasis is placed on training these skills. Given that training is increasingly designed in a blended way, serious games may offer an efficient method of preparing face-to-face simulation training of these procedural skills. OBJECTIVE: This study aimed to investigate the teamwork principles that were used during gameplay by medical students and teamwork experts. Findings can improve our understanding of the potential of serious games for training these complex skills. METHODS: We investigated a web-based multiplayer game designed for training students' interprofessional teamwork skills. During gameplay, 4 players in different roles (physician, nurse, medical student, and student nurse) had to share information, prioritize tasks, and decide on next steps to take in web-based patient scenarios, using one-to-one and team chats. We performed a qualitative study (content analysis) on these chats with 144 fifth-year medical students and 24 health care teamwork experts (as a benchmark study) playing the game in groups of 4. Game chat data from 2 scenarios were analyzed. For the analysis, a deductive approach was used, starting with a conceptual framework based on Crew Resource Management principles, including shared situational awareness, decision-making, communication, team management, and debriefing. RESULTS: Results showed that most teamwork principles were used during gameplay: shared situational awareness, decision-making (eg, re-evaluation), communication (eg, closed loop), and team management (eg, distributing the workload). Among students, these principles were often used on a basic level. Among experts, teamwork principles were used with more open forms of speak up and more justification of decisions. Some specific Crew Resource Management principles were less observed among both groups, for example, prevention of fixation errors and use of cognitive aids. Both groups showed relatively superficial debriefing reflections. CONCLUSIONS: Playing a multiplayer game for interprofessional teamwork appears to facilitate the application of teamwork principles by students in all important teamwork domains on a basic level. Expert players applied similar teamwork principles on a moderately high complexity level. Some teamwork principles were less observed among both students and expert groups, probably owing to the artifacts of the game environment (eg, chatting instead of talking). A multiplayer game for teamwork training can elicit the application of important, basic teamwork principles, both among novices and experts, and provides them with a flexible, accessible, and engaging learning environment. This may create time for exercising more complex skills during face-to-face training.

Global versus task-specific postoperative feedback in surgical procedure learning.

BACKGROUND: Task-specific checklists and global rating scales are both recommended assessment tools to provide constructive feedback on surgical performance. This study evaluated the most effective feedback tool by comparing the effects of the Observational Clinical Human Reliability Analysis (OCHRA) and the Objective Structured Assessment of Technical Skills (OSATS) on surgical performance in relation to the visual-spatial ability of the learners. METHODS: In a randomized controlled trial, medical students were allocated to either the OCHRA (n = 25) or OSATS (n = 25) feedback group. Visual-spatial ability was measured by a Mental Rotation Test. Participants performed an open inguinal hernia repair procedure on a simulation model twice. Feedback was provided after the first procedure. Improvement in performance was evaluated blindly using a global rating scale (performance score) and hand-motion analysis (time and path length). RESULTS: Mean improvement in performance score was not significantly different between the OCHRA and OSATS feedback groups (P = .100). However, mean improvement in time (371.0 ± 223.4 vs 274.6 ± 341.6; P = .027) and path length (53.5 ± 42.4 vs 34.7 ± 39.0; P = .046) was significantly greater in the OCHRA feedback group. When stratified by mental rotation test scores, the greater improvement in time (P = .032) and path length (P = .053) was observed only among individuals with low visual-spatial abilities. CONCLUSION: A task-specific (OCHRA) feedback is more effective in improving surgical skills in terms of time and path length in novices compared to a global rating scale (OSATS). The effects of a task-specific feedback are present mostly in individuals with lower visual-spatial abilities.

One Step at a Time: Step by Step Versus Continuous Video-Based Learning to Prepare Medical Students for Performing Surgical Procedures.

OBJECTIVE: The objective of this study was to compare the effects of cognitive load and surgical performance in medical students that performed the open inguinal hernia repair after preparation with step-by-step video-demonstration versus continuous video-demonstration. Hypothetically, the step-by-step group will perceive lower extraneous load during the preparation of the surgical procedure compared to the continuous group. Subsequently, fewer errors will be made in the surgical performance assessment by the step-by-step group, resulting in better surgical performance. DESIGN: In this prospective study, participants were randomly assigned to the step-by-step or continuous video-demonstration. They completed questionnaires regarding perceived cognitive load during preparation (10-point Likert scale). Their surgical performance was assessed on a simulation hernia model using the Observational Clinical Human Reliability Assessment. SETTING: Erasmus University Medical Center, Rotterdam, the Netherlands. PARTICIPANTS: Participants included medical students who were enrolled in extracurricular anatomy courses. RESULTS: Forty-three students participated; 23 students in the step-by-step group and 20 in the continuous group. As expected, the step-by-step group perceived a lower extraneous cognitive load (2.92 ± 1.21) compared to the continuous group (3.91 ± 1.67, p = 0.030). The surgical performance was not statistically significantly different between both groups; however, in subanalyses on a selection of students that prepared for 1 to 2 hours, the step-by-step group made less procedural errors, 1.67 ± 1.11, compared to the continuous group, 3.06 ± 1.91, p = 0.018. CONCLUSIONS: Our results suggest that preparation using step-by-step video-based learning results in lower extraneous cognitive load and subsequently fewer procedural errors during the surgical performance. For learning purposes, demonstration videos of surgical procedures should be presented in a segmented format.

The Effect of Stereoscopic Augmented Reality Visualization on Learning Anatomy and the Modifying Effect of Visual-Spatial Abilities: A Double-Center Randomized Controlled Trial.

Monoscopically projected three-dimensional (3D) visualization technology may have significant disadvantages for students with lower visual-spatial abilities despite its overall effectiveness in teaching anatomy. Previous research suggests that stereopsis may facilitate a better comprehension of anatomical knowledge. This study evaluated the educational effectiveness of stereoscopic augmented reality (AR) visualization and the modifying effect of visual-spatial abilities on learning. In a double-center randomized controlled trial, first- and second-year (bio)medical undergraduates studied lower limb anatomy with stereoscopic 3D AR model (n = 20), monoscopic 3D desktop model (n = 20), or two-dimensional (2D) anatomical atlas (n = 18). Visual-spatial abilities were tested with Mental Rotation Test (MRT), Paper Folding Test (PFT), and Mechanical Reasoning (MR) Test. Anatomical knowledge was assessed by the validated 30-item paper posttest. The overall posttest scores in the stereoscopic 3D AR group (47.8%) were similar to those in the monoscopic 3D desktop group (38.5%; P = 0.240) and the 2D anatomical atlas group (50.9%; P = 1.00). When stratified by visual-spatial abilities test scores, students with lower MRT scores achieved higher posttest scores in the stereoscopic 3D AR group (49.2%) as compared to the monoscopic 3D desktop group (33.4%; P = 0.015) and similar to the scores in the 2D group (46.4%; P = 0.99). Participants with higher MRT scores performed equally well in all conditions. It is instrumental to consider an aptitude-treatment interaction caused by visual-spatial abilities when designing research into 3D learning. Further research is needed to identify contributing features and the most effective way of introducing this technology into current educational programs.

Comparative effectiveness of a serious game and an e-module to support patient safety knowledge and awareness.

BACKGROUND: Serious games have the potential to teach complex cognitive skills in an engaging way, at relatively low costs. Their flexibility in use and scalability makes them an attractive learning tool, but more research is needed on the effectiveness of serious games compared to more traditional formats such e-modules. We investigated whether undergraduate medical students developed better knowledge and awareness and were more motivated after learning about patient-safety through a serious game than peers who studied the same topics using an e-module. METHODS: Fourth-year medical students were randomly assigned to either a serious game that included video-lectures, biofeedback exercises and patient missions (n = 32) or an e-module, that included text-based lectures on the same topics (n = 34). A third group acted as a historical control-group without extra education (n = 37). After the intervention, which took place during the clinical introduction course, before the start of the first rotation, all students completed a knowledge test, a self-efficacy test and a motivation questionnaire. During the following 10-week clinical rotation they filled out weekly questionnaires on patient-safety awareness and stress. RESULTS: The results showed patient safety knowledge had equally improved in the game group and e-module group compared to controls, who received no extra education. Average learning-time was 3 h for the game and 1 h for the e-module-group. The serious game was evaluated as more engaging; the e-module as more easy to use. During rotations, students in the three groups reported low and similar levels of patient-safety awareness and stress. Students who had treated patients successfully during game missions experienced higher self-efficacy and less stress during their rotation than students who treated patients unsuccessfully. CONCLUSIONS: Video-lectures (in a game) and text-based lectures (in an e-module) can be equally effective in developing knowledge on specific topics. Although serious games are strongly engaging for students and stimulate them to study longer, they do not necessarily result in better performance in patient safety issues.

Preparing Residents Effectively in Emergency Skills Training With a Serious Game.

INTRODUCTION: Training emergency care skills is critical for patient safety but cost intensive. Serious games have been proposed as an engaging self-directed learning tool for complex skills. The objective of this study was to compare the cognitive skills and motivation of medical residents who only used a course manual as preparation for classroom training on emergency care with residents who used an additional serious game. METHODS: This was a quasi-experimental study with residents preparing for a rotation in the emergency department. The "reading" group received a course manual before classroom training; the "reading and game" group received this manual plus the game as preparation for the same training. Emergency skills were assessed before training (with residents who agreed to participate in an extra pretraining assessment), using validated competency scales and a global performance scale. We also measured motivation. RESULTS: All groups had comparable important characteristics (eg, experience with acute care). Before training, the reading and game group felt motivated to play the game and spent more self-study time (+2.5 hours) than the reading group. Game-playing residents showed higher scores on objectively measured and self-assessed clinical competencies but equal scores on the global performance scale and were equally motivated for training, compared with the reading group. After the 2-week training, no differences between groups existed. CONCLUSIONS: After preparing training with an additional serious game, residents showed improved clinical competencies, compared with residents who only studied course material. After a 2-week training, this advantage disappeared. Future research should study the retention of game effects in blended designs.

An experimental study on the effects of a simulation game on students' clinical cognitive skills and motivation.

Simulation games are becoming increasingly popular in education, but more insight in their critical design features is needed. This study investigated the effects of fidelity of open patient cases in adjunct to an instructional e-module on students' cognitive skills and motivation. We set up a three-group randomized post-test-only design: a control group working on an e-module; a cases group, combining the e-module with low-fidelity text-based patient cases, and a game group, combining the e-module with a high-fidelity simulation game with the same cases. Participants completed questionnaires on cognitive load and motivation. After a 4-week study period, blinded assessors rated students' cognitive emergency care skills in two mannequin-based scenarios. In total 61 students participated and were assessed; 16 control group students, 20 cases students and 25 game students. Learning time was 2 h longer for the cases and game groups than for the control group. Acquired cognitive skills did not differ between groups. The game group experienced higher intrinsic and germane cognitive load than the cases group (p = 0.03 and 0.01) and felt more engaged (p < 0.001). Students did not profit from working on open cases (in adjunct to an e-module), which nonetheless challenged them to study longer. The e-module appeared to be very effective, while the high-fidelity game, although engaging, probably distracted students and impeded learning. Medical educators designing motivating and effective skills training for novices should align case complexity and fidelity with students' proficiency level. The relation between case-fidelity, motivation and skills development is an important field for further study.

Assessing the assessment in emergency care training.

OBJECTIVE: Each year over 1.5 million health care professionals attend emergency care courses. Despite high stakes for patients and extensive resources involved, little evidence exists on the quality of assessment. The aim of this study was to evaluate the validity and reliability of commonly used formats in assessing emergency care skills. METHODS: Residents were assessed at the end of a 2-week emergency course; a subgroup was videotaped. Psychometric analyses were conducted to assess the validity and inter-rater reliability of the assessment instrument, which included a checklist, a 9-item competency scale and a global performance scale. RESULTS: A group of 144 residents and 12 raters participated in the study; 22 residents were videotaped and re-assessed by 8 raters. The checklists showed limited validity and poor inter-rater reliability for the dimensions "correct" and "timely" (ICC = .30 and.39 resp.). The competency scale had good construct validity, consisting of a clinical and a communication subscale. The internal consistency of the (sub)scales was high (α = .93/.91/.86). The inter-rater reliability was moderate for the clinical competency subscale (.49) and the global performance scale (.50), but poor for the communication subscale (.27). A generalizability study showed that for a reliable assessment 5-13 raters are needed when using checklists, and four when using the clinical competency scale or the global performance scale. CONCLUSIONS: This study shows poor validity and reliability for assessing emergency skills with checklists but good validity and moderate reliability with clinical competency or global performance scales. Involving more raters can improve the reliability substantially. Recommendations are made to improve this high stakes skill assessment.

A blended design in acute care training: similar learning results, less training costs compared with a traditional format.

Introduction There is a demand for more attractive and efficient training programmes in postgraduate health care training. This retrospective study aims to show the effectiveness of a blended versus traditional face-to-face training design. For nurses in postgraduate Acute and Intensive Care training, the effectiveness of a blended course design was compared with a traditional design. Methods In a first pilot study 57 students took a traditional course (2-h lecture and 2-h workshop) and 46 students took a blended course (2-h lecture and 2-h online self-study material). Test results were compared for both groups. After positive results in the pilot study, the design was replicated for the complete programme in Acute and Intensive Care. Now 16 students followed the traditional programme (11 days face-to-face education) and 31 students did the blended programme (7 days face-to-face and 40 h online self-study). An evaluation was done after the pilot and course costs were calculated. Results Results show that the traditional and blended groups were similar regarding the main characteristics and did not differ in learning results for both the pilot and the complete programme. Student evaluations of both designs were positive; however, the blended group were more confident that they had achieved the learning objectives. Training costs were reduced substantially. Conclusion The blended training design offers an effective and attractive training solution, leading to a significant reduction in costs.