Development and Pilot Testing of a Programmatic System for Competency Assessment in US Anesthesiology Residency Training.

BACKGROUND: In 2018, a set of entrustable professional activities (EPAs) and procedural skills assessments were developed for anesthesiology training, but they did not assess all the Accreditation Council for Graduate Medical Education (ACGME) milestones. The aims of this study were to (1) remap the 2018 EPA and procedural skills assessments to the revised ACGME Anesthesiology Milestones 2.0, (2) develop new assessments that combined with the original assessments to create a system of assessment that addresses all level 1 to 4 milestones, and (3) provide evidence for the validity of the assessments. METHODS: Using a modified Delphi process, a panel of anesthesiology education experts remapped the original assessments developed in 2018 to the Anesthesiology Milestones 2.0 and developed new assessments to create a system that assessed all level 1 through 4 milestones. Following a 24-month pilot at 7 institutions, the number of EPA and procedural skill assessments and mean scores were computed at the end of the academic year. Milestone achievement and subcompetency data for assessments from a single institution were compared to scores assigned by the institution's clinical competency committee (CCC). RESULTS: New assessment development, 2 months of testing and feedback, and revisions resulted in 5 new EPAs, 11 nontechnical skills assessments (NTSAs), and 6 objective structured clinical examinations (OSCEs). Combined with the original 20 EPAs and procedural skills assessments, the new system of assessment addresses 99% of level 1 to 4 Anesthesiology Milestones 2.0. During the 24-month pilot, aggregate mean EPA and procedural skill scores significantly increased with year in training. System subcompetency scores correlated significantly with 15 of 23 (65.2%) corresponding CCC scores at a single institution, but 8 correlations (36.4%) were <30.0, illustrating poor correlation. CONCLUSIONS: A panel of experts developed a set of EPAs, procedural skill assessment, NTSAs, and OSCEs to form a programmatic system of assessment for anesthesiology residency training in the United States. The method used to develop and pilot test the assessments, the progression of assessment scores with time in training, and the correlation of assessment scores with CCC scoring of milestone achievement provide evidence for the validity of the assessments.

Development and Pilot Testing of Entrustable Professional Activities for US Anesthesiology Residency Training.

BACKGROUND: Modern medical education requires frequent competency assessment. The Accreditation Council for Graduate Medical Education (ACGME) provides a descriptive framework of competencies and milestones but does not provide standardized instruments to assess and track trainee competency over time. Entrustable professional activities (EPAs) represent a workplace-based method to assess the achievement of competency milestones at the point-of-care that can be applied to anesthesiology training in the United States. METHODS: Experts in education and competency assessment were recruited to participate in a 6-step process using a modified Delphi method with iterative rounds to reach consensus on an entrustment scale, a list of EPAs and procedural skills, detailed definitions for each EPA, a mapping of the EPAs to the ACGME milestones, and a target level of entrustment for graduating US anesthesiology residents for each EPA and procedural skill. The defined EPAs and procedural skills were implemented using a website and mobile app. The assessment system was piloted at 7 anesthesiology residency programs. After 2 months, faculty were surveyed on their attitudes on usability and utility of the assessment system. The number of evaluations submitted per month was collected for 1 year. RESULTS: Participants in EPA development included 18 education experts from 11 different programs. The Delphi rounds produced a final list of 20 EPAs, each differentiated as simple or complex, a defined entrustment scale, mapping of the EPAs to milestones, and graduation entrustment targets. A list of 159 procedural skills was similarly developed. Results of the faculty survey demonstrated favorable ratings on all questions regarding app usability as well as the utility of the app and EPA assessments. Over the 2-month pilot period, 1636 EPA and 1427 procedure assessments were submitted. All programs continued to use the app for the remainder of the academic year resulting in 12,641 submitted assessments. CONCLUSIONS: A list of 20 anesthesiology EPAs and 159 procedural skills assessments were developed using a rigorous methodology to reach consensus among education experts. The assessments were pilot tested at 7 US anesthesiology residency programs demonstrating the feasibility of implementation using a mobile app and the ability to collect assessment data. Adoption at the pilot sites was variable; however, the use of the system was not mandatory for faculty or trainees at any site.

Optimal Management of Endometriosis and Pain.

The pathophysiology of endometriosis-associated pain involves inflammatory and hormonal alterations and changes in brain signaling pathways. Although medical treatment can provide temporary relief, most patients can achieve long-term sustained pain relief when it is combined with surgical intervention. Owing to its complexity, there is an ongoing debate about how to optimally manage endometriosis-associated pain. We believe optimal management for this condition requires: 1) possible egg preservation in affected young patients with and without endometriomas; 2) preoperative medical suppression to inhibit ovulation and to avoid removal of functional cysts that might look like endometriomas; and 3) postoperative hormonal suppression to decrease recurrence, but this treatment should be modified according to disease severity, symptoms, and fertility goals.

Simulation-based mastery learning with deliberate practice improves clinical performance in spinal anesthesia.

Introduction. Properly performing a subarachnoid block (SAB) is a competency expected of anesthesiology residents. We aimed to determine if adding simulation-based deliberate practice to a base curriculum improved performance of a SAB. Methods. 21 anesthesia residents were enrolled. After baseline assessment of SAB on a task-trainer, all residents participated in a base curriculum. Residents were then randomized so that half received additional deliberate practice including repetition and expert-guided, real-time feedback. All residents were then retested for technique. SABs on all residents' next three patients were evaluated in the operating room (OR). Results. Before completing the base curriculum, the control group completed 81% of a 16-item performance checklist on the task-trainer and this increased to 91% after finishing the base curriculum (P < 0.02). The intervention group also increased the percentage of checklist tasks properly completed from 73% to 98%, which was a greater increase than observed in the control group (P < 0.03). The OR time required to perform SAB was not different between groups. Conclusions. The base curriculum significantly improved resident SAB performance. Deliberate practice training added a significant, independent, incremental benefit. The clinical impact of the deliberate practice intervention in the OR on patient care is unclear.

Detection of respiratory compromise by acoustic monitoring, capnography, and brain function monitoring during monitored anesthesia care.

Episodes of apnea in sedated patients represent a risk of respiratory compromise. We hypothesized that acoustic monitoring would be equivalent to capnography for detection of respiratory pauses, with fewer false alarms. In addition, we hypothesized that the patient state index (PSI) would be correlated with the frequency of respiratory pauses and therefore could provide information about the risk of apnea during sedation. Patients undergoing sedation for surgical procedures were monitored for respiration rate using acoustic monitoring and capnography and for depth of sedation using the PSI. A clinician blinded to the acoustic and sedation monitor observed the capnograph and patient to assess sedation and episodes of apnea. Another clinician retrospectively reviewed the capnography and acoustic waveform and sound files to identify true positive and false positive respiratory pauses by each method (reference method). Sensitivity, specificity, and likelihood ratio for detection of respiratory pause was calculated for acoustic monitoring and capnography. The correlation of PSI with respiratory pause events was determined. For the 51 respiratory pauses validated by retrospective analysis, the sensitivity, specificity, and likelihood ratio positive for detection were 16, 96 %, and 3.5 for clinician observation; 88, 7 %, and 1.0 for capnography; and 55, 87 %, and 4.1 for acoustic monitoring. There was no correlation between PSI and respiratory pause events. Acoustic monitoring had the highest likelihood ratio positive for detection of respiratory pause events compared with capnography and clinician observation and, therefore, may provide the best method for respiration rate monitoring during these procedures.

The stanford anesthesia faculty teaching scholars program: summary of faculty development, projects, and outcomes.

BACKGROUND: The Stanford Anesthesia Teaching Scholars Program was launched in 2007 to further pedagogic training of faculty and improve residency education. OBJECTIVES: The goals of this article are to describe the program intervention and improvements made based on participant feedback, summarize the characteristics of the faculty enrolled and projects undertaken, and report on program outcomes tracked to date. INTERVENTION: THE TEACHING SCHOLARS PROGRAM HOUSED WITHIN THE DEPARTMENT OF ANESTHESIA SUPPORTS FACULTY IN THESE AREAS: (1) attending education-related meetings; (2) engaging in a monthly seminar on core topics paired with independent study reading; and (3) undertaking a project to improve resident education. Structured interviews with all graduates (n  =  19; 47% women) were conducted using a pilot-tested questionnaire. RESULTS: A total of 15 of 19 Scholars (79%) were instructors/assistant professors. Sixteen Scholars (84%) attended an off-site education meeting. The Scholars pursued a variety of projects, including curriculum (53%), teaching (26%), administration (11%), assessment (5%), and advising/mentoring (5%). Projects were fully completed by 13 of 19 participants (68%), and 12 of 19 projects (63%) are currently integrated into the residency. Completed projects were published/presented at conferences by 4 of 13 participants (31%), and education grants were received by 3 of 19 participants (16%). CONCLUSIONS: This is the first description of a faculty development (education) program in an anesthesiology department. The program has been well accepted by participants and resulted in increased educational products, some of which have become a permanent part of the residency curriculum. This educational innovation can be replicated in other departments of anesthesiology provided that funding is available for faculty time and meeting expenses.