Simulation-based education improves quality of care during cardiac arrest team responses at an academic teaching hospital: a case-control study.

BACKGROUND: Simulation technology is widely used in medical education. Linking educational outcomes achieved in a controlled environment to patient care improvement is a constant challenge. METHODS: This was a retrospective case-control study of cardiac arrest team responses from January to June 2004 at a university-affiliated internal medicine residency program. Medical records of advanced cardiac life support (ACLS) events were reviewed to assess adherence to ACLS response quality indicators based on American Heart Association (AHA) guidelines. All residents received traditional ACLS education. Second-year residents (simulator-trained group) also attended an educational program featuring the deliberate practice of ACLS scenarios using a human patient simulator. Third-year residents (traditionally trained group) were not trained on the simulator. During the study period, both simulator-trained and traditionally trained residents responded to ACLS events. We evaluated the effects of simulation training on the quality of the ACLS care provided. RESULTS: Simulator-trained residents showed significantly higher adherence to AHA standards (mean correct responses, 68%; SD, 20%) vs traditionally trained residents (mean correct responses, 44%; SD, 20%; p = 0.001). The odds ratio for an adherent ACLS response was 7.1 (95% confidence interval, 1.8 to 28.6) for simulator-trained residents compared to traditionally trained residents after controlling for patient age, ventilator, and telemetry status. CONCLUSIONS: A simulation-based educational program significantly improved the quality of care provided by residents during actual ACLS events. There is a growing body of evidence indicating that simulation can be a useful adjunct to traditional methods of procedural training.

Mastery learning of advanced cardiac life support skills by internal medicine residents using simulation technology and deliberate practice.

BACKGROUND: Internal medicine residents must be competent in advanced cardiac life support (ACLS) for board certification. OBJECTIVE: To use a medical simulator to assess postgraduate year 2 (PGY-2) residents' baseline proficiency in ACLS scenarios and evaluate the impact of an educational intervention grounded in deliberate practice on skill development to mastery standards. DESIGN: Pretest-posttest design without control group. After baseline evaluation, residents received 4, 2-hour ACLS education sessions using a medical simulator. Residents were then retested. Residents who did not achieve a research-derived minimum passing score (MPS) on each ACLS problem had more deliberate practice and were retested until the MPS was reached. PARTICIPANTS: Forty-one PGY-2 internal medicine residents in a university-affiliated program. MEASUREMENTS: Observational checklists based on American Heart Association (AHA) guidelines with interrater and internal consistency reliability estimates; deliberate practice time needed for residents to achieve minimum competency standards; demographics; United States Medical Licensing Examination Step 1 and Step 2 scores; and resident ratings of program quality and utility. RESULTS: Performance improved significantly after simulator training. All residents met or exceeded the mastery competency standard. The amount of practice time needed to reach the MPS was a powerful (negative) predictor of posttest performance. The education program was rated highly. CONCLUSIONS: A curriculum featuring deliberate practice dramatically increased the skills of residents in ACLS scenarios. Residents needed different amounts of training time to achieve minimum competency standards. Residents enjoy training, evaluation, and feedback in a simulated clinical environment. This mastery learning program and other competency-based efforts illustrate outcome-based medical education that is now prominent in accreditation reform of residency education.

A longitudinal study of internal medicine residents' retention of advanced cardiac life support skills.

BACKGROUND: Internal medicine residents must be competent in Advanced Cardiac Life Support (ACLS) for board certification. Traditional ACLS courses have limited ability to enable residents to achieve and maintain skills. Educational programs featuring reliable measurements and improved retention of skills would be useful for residency education. METHOD: We developed a training program using a medical simulator, small-group teaching and deliberate practice. Residents received traditional ACLS education and subsequently participated in four two-hour educational sessions using the simulator. Resident performance in six simulated ACLS scenarios was assessed using a standardized checklist. RESULTS: After the program, resident ACLS skill improved significantly. The cohort was followed prospectively for 14 months and the skills did not decay. CONCLUSIONS: Use of a simulation-based educational program enabled us to achieve and maintain high levels of resident performance in simulated ACLS events. Given the limitations of traditional methods to train, assess and maintain competence, simulation technology can be a useful adjunct in high-quality ACLS education.

Graduating internal medicine residents' self-assessment and performance of advanced cardiac life support skills.

Internal medicine residents in the US must be competent to perform procedures including Advanced Cardiac Life Support (ACLS) to become board-eligible. Our aim was to determine if residents near graduation could assess their skills in ACLS procedures accurately. Participants were 40 residents in a university-based training program. Self-assessments of confidence in managing six ACLS scenarios were measured on a 0 (very low) to 100 (very high) scale. These were compared to reliable observational ratings of residents' performance on a high-fidelity simulator using published treatment protocols. Residents expressed strong self-confidence about managing the scenarios. Residents' simulator performance varied widely (range from 45% to 94%). Self-confidence assessments correlated poorly with performance (median r = 0.075). Self-assessment of performance by graduating internal medicine residents was not accurate in this study. The use of self-assessment to document resident competence in procedures such as ACLS is not a proxy for objective evaluation.

Comparison of two standard-setting methods for advanced cardiac life support training.

BACKGROUND: This study used the Angoff and Hofstee standard-setting methods to derive minimum passing scores for six advanced cardiac life support (ACLS) procedures. METHOD: An expert panel provided item-based (Angoff) and group-based (Hofstee) judgments about six ACLS performance checklists on two occasions separated by ten weeks. Interrater reliabilities and test-retest reliability (stability) of the judgments were calculated. Derived ACLS passing standards are compared to historical ACLS performance data from two groups of ACLS-trained internal medicine residents. RESULTS: Both the Angoff and Hofstee standard-setting methods produced reliable and stable data. Hofstee minimum passing scores (MPSs) were uniformly more stringent than Angoff MPSs. Interpretation of historical ACLS performance data from medical residents shows the MPSs derived in this study would yield higher-than-expected failure rates. CONCLUSION: Systematic standard setting for ACLS procedures is a necessary step toward the creation of mastery learning educational programs.

Mastery learning of thoracentesis skills by internal medicine residents using simulation technology and deliberate practice.

BACKGROUND: Thoracentesis has been identified as a core competency for hospitalists. Residency training may not provide adequate preparation to perform this procedure. OBJECTIVE: Our aim was to use a thoracentesis simulation to assess graduating residents' proficiency in thoracentesis procedural skills and to evaluate the impact of an educational intervention on skill development to mastery standards. DESIGN: This was a pretest-posttest mastery learning design without a control group. SETTING: Participants were 40 third-year internal medicine residents in a university-affiliated program. INTERVENTION: Using an observational checklist, each resident underwent baseline assessment of thoracentesis skill using a standardized clinical history, radiograph, and thoracentesis simulation. After baseline testing, residents received two 2-hour education sessions featuring a videotaped presentation and deliberate practice with the thoracentesis simulator. MEASUREMENTS: Residents were retested after the intervention. Skill mastery was defined as meeting or exceeding the minimum passing score (MPS) set by an expert panel at thoracentesis posttest. Those who did not achieve the MPS had additional deliberate practice and were retested until the MPS was reached. RESULTS: Performance improved 71% from pretest to posttest on the clinical skills examination. All residents met or exceeded the mastery standard. The amount of practice time needed to reach the MPS was the only predictor (negative) of posttest performance. The education program was rated highly. CONCLUSIONS: A curriculum featuring deliberate practice dramatically increased the skills of residents in thoracentesis. Residents enjoy training and receiving evaluation and feedback in a simulated clinical environment. This mastery program illustrates a feasible and reliable mechanism to achieve procedural competency.

Simulation-based training of internal medicine residents in advanced cardiac life support protocols: a randomized trial.

BACKGROUND: Internal medicine residents must be competent in Advanced Cardiac Life Support (ACLS) for board certification. PURPOSE: The purpose was to use a medical simulator to assess baseline proficiency in ACLS and determine the impact of an intervention on skill development. METHOD: This was a randomized trial with wait-list controls. After baseline evaluation in all residents, the intervention group received 4 education sessions using a medical simulator. All residents were then retested. After crossover, the wait-list group received the intervention, and residents were tested again. Performance was assessed by comparison to American Heart Association guidelines for treatment of ACLS conditions with interrater and internal consistency reliability estimates. RESULTS: Performance improved significantly after simulator training. No improvement was detected as a function of clinical experience alone. The educational program was rated highly.