Motion-Tracking Machines and Sensors: Advancing Education Technology.

Graduate medical education is predominantly based on a time-based apprenticeship model, with implied acquisition of proficiency after a pre-set amount of clinical exposure. While motion metrics have been used previously to measure skill performance indicators, these assessments have largely been performed on a summative scale to describe the performance of complete tasks or procedures. By segmenting performances of interest and assessing the essential elements individually, a more comprehensive understanding of the aspects in need of improvement for a learner can be obtained. The purpose of this review is to discuss technologies applicable to motion tracking, their benefits and limitations, approaches to data processing, and potential applications based on recent improvements in this technology. Objective analysis of motion metrics may improve educational standards of learning and efficiency by both standardizing the feedback process for trainees and reducing the volume of instructors required to facilitate practice sessions. With rigorous validation and standardization, motion metric assessment may also prove useful to demonstrate competency in technical procedures as part of a comprehensive certification process.

Assessing Skill Acquisition in Anesthesiology Interns Practicing Central Venous Catheter Placement Through Advancements in Motion Analysis.

OBJECTIVES: The study authors hypothesized that a combination of previously used (path length, translational motions, and time) and novel (rotational sum) motion metrics could be used to analyze learning curves of anesthesiology interns (postgraduate year 1) practicing central venous catheter placement in the simulation setting. They also explored the feasibility of using segmented motion recordings to inform deliberate practice. DESIGN: A prospective cohort study. SETTING: A single academic medical center. PARTICIPANTS: Anesthesiology interns (postgraduate year 1). INTERVENTIONS: Anesthesiology interns underwent a 2-day training course in which they performed 9 central venous catheter placements, while attached to motion sensors on the dorsum of their dominant hand and ultrasound probe. MEASUREMENTS AND MAIN RESULTS: Motion metrics were analyzed using generalized estimating equations for both the overall procedure and predefined segments. Five attending anesthesiologists performed 3 trials each for comparison. Overall, there was a negative trend in path length, translational motions, rotational sum, and time (p < 0.001), with the exception of translational motions of the ultrasound probe. Interns reached within 1 standard deviation of the attending anesthesiologists by trials 7-to-8 for most metrics. Segmentation identified specific components of the procedure that were either significantly improved upon or required deliberate practice. The novel metric of rotational sum exhibited a moderate-to-strong positive correlation with other metrics (p < 0.001). CONCLUSIONS: A comprehensive series of motion metrics was able to describe the learning curves of novices training to perform central venous catheter placement in the simulation setting. Furthermore, it was determined that segmentation may provide additional insight into skill acquisition and inform deliberate practice.

National Delphi Survey on Anesthesiology Resident Training in Perioperative Ultrasound.

OBJECTIVE: To establish agreement among nationwide experts through a Delphi process on the key components of perioperative ultrasound and the recommended minimum number of examinations that should be performed by a resident upon graduation. DESIGN: A prospective cross-sectional study. SETTING: A survey on multiinstitutional academic medical centers. PARTICIPANTS: Anesthesiology residency program directors and/or experts in perioperative ultrasound. INTERVENTIONS: A list of components and examinations recommended for anesthesiology resident training in perioperative ultrasound was developed based on guidelines and 2 survey rounds among a steering committee of 10 experts. A questionnaire asking for a rating of each component on a 5-point Likert scale subsequently was sent to an expert panel of 120 anesthesiology residency program directors across the United States. An agreement of at least 70% of participants, rating a component as 4 or 5, was compulsory to list a component as essential for anesthesiology resident training in perioperative ultrasound. MEASUREMENTS AND MAIN RESULTS: The nationwide survey's response rate was 62.5%, and agreement was reached after 2 Delphi rounds. The final list included 44 essential components for basic ultrasound physics and knobology, cardiac ultrasound, lung ultrasound, and ultrasound-guided vascular access. Agreement was not reached for abdominal ultrasound, gastric ultrasound, and ultrasound-guided airway assessment. Agreement for the recommended minimum number of examinations that should be performed by a resident upon graduation included 50 each for transthoracic and transesophageal echocardiography, and 20 each for lung ultrasound, ultrasound-guided central line, and ultrasound-guided arterial line placements. CONCLUSIONS: The recommendations outlined in this survey can be used to establish standardized training for perioperative ultrasound by anesthesiology residency programs.

Validation of a Virtual Simulator With Haptic Feedback for Endotracheal Intubation Training.

INTRODUCTION: Endotracheal intubation (ETI) is a procedure that varies in difficulty because of patient characteristics and clinical conditions. Existing physical simulators do not encompass these variations. The Virtual Airway Skills Trainer for Endotracheal Intubation (VAST-ETI) was developed to provide different patient characteristics and high-fidelity haptic feedback to improve training. METHODS: We demonstrate the effectiveness of VAST-ETI as a training and evaluation tool for ETI. Construct validation was evaluated by scoring the performance of experts ( N = 15) and novices ( N = 15) on the simulator to ensure its ability to distinguish technical proficiency. Convergent and predictive validity were evaluated by performing a learning curve study, in which a group of novices ( N = 7) were trained for 2 weeks using VAST-ETI and then compared with a control group ( N = 9). RESULTS: The VAST-ETI was able to distinguish between expert and novice based on mean simulator scores ( t [88] = -6.61, P < 0.0005). When used during repeated practice, individuals demonstrated a significant increase in their score on VAST-ETI over the learning period ( F [11,220] = 7206, P < 0.001); however when compared with a control group, there was not a significant interaction effect on the simulator score. There was a significant difference between the simulator-trained and control groups ( t [12.85] = -2.258, P = 0.042) when tested in the operating room. CONCLUSIONS: Our results demonstrate the effectiveness of virtual simulation with haptic feedback for assessing performance and training of ETI. The simulator was not able to differentiate performance between more experienced trainees and experts because of limits in simulator difficulty.

Resident Engagement With a Web- and App-based Journal Club Curriculum Utilizing Email and Text Notifications.

Background: High learner engagement is important for the success of asynchronous and online learning for graduate medical education. Medical trainees have recently reported using medical mobile apps. App-based interactions may provide more participation than email-based interactions. We sought to investigate (1) if there were higher levels of engagement with an online curriculum using notifications sent via email as compared with via text, and (2) if there were higher levels of engagement with the mobile app or website format. Methods: We implemented an online Journal Club curriculum with weekly topics for anesthesiology residents (postgraduate years 2-4) from July 2020 to June 2021. Weekly notifications were sent to residents via email for weeks 1-10, text for weeks 11-20, then email for weeks 21-49. Based on activity logs, we compared (1) the weekly numbers of interactions when email notifications were sent with the weekly numbers of interactions when text notifications were sent, and (2) the weekly numbers of interactions via the app with the weekly numbers of interactions via the website. Results: Thirty-eight of the 54 anesthesiology residents in our department at the time of the study (70.4%) interacted with the online Journal Club at least once throughout the study. The weekly numbers of interactions with email notifications (median [interquartile range (IQR)]: 13 [7-28]) were significantly higher than with text notifications (median [IQR]: 6 [4-8]) (P = .023). The weekly numbers of interactions via the website (median [IQR]: 9 [4-24]) were significantly higher than via the app (median [IQR]: 0 [0-1]) (P < .001). Conclusions: Although mobile technology may increase engagement and participation for some educational resources, learners may prefer accessing others through more conventional methods.

Motion Analysis: An Objective Assessment of Special Operations Forces and Tactical Medics Performing Point-of-Care Ultrasound.

BACKGROUND: Point-of-care ultrasound (POCUS) is commonly employed to image the heart, lungs, and abdomen. Rapid ultrasound for shock and hypotension (RUSH) exams are a critical component of POCUS employed in austere environments by Special Operations Forces (SOF) and tactical medics for triage and diagnosis. Despite its utility, training for POCUS remains largely unstandardized with respect to feedback and markers of proficiency. We hypothesized that motion analysis could objectively identify improvement in medics' performance of RUSH exams. Furthermore, we predicted that motion metrics would correlate with qualitative ratings administered by attending anesthesiologists. METHODS: A team of civilian and military attending anesthesiologists trained 24 medics in POCUS during a 5-day course. Each medic performed eight RUSH exams using an ultrasound probe equipped with an electromagnetic motion sensor to track total distance travelled (path length), movements performed (translational motions), degrees rotated (rotational sum), and time. Instructors (experts) observed and rated the exams on the following items: image finding, image fine-tuning, speed, final image accuracy, and global assessment. Motion metrics were used to provide feedback to medics throughout the course. Generalized estimating equations were used to analyze the trends of motion metrics across all trials. Correlations amongst motion metrics and expert ratings were assessed with Pearson correlation coefficients. RESULTS: Participants exhibited a negative trend in all motion metrics (p < 0.001). Pearson correlation coefficients revealed moderate inverse correlations amongst motion metrics and expert ratings. CONCLUSION: Motion analysis was able to quantify and describe the performance of medics training in POCUS and correlated with expert ratings.

Identifying the Gap Between Novices and Experts in Fiberoptic Scope Control.

BACKGROUND: Fiberoptic intubation (FOI) is key in managing difficult airways. Good scope control increases efficiency and patient safety. Understanding the gap between novices and experts in scope control would help medical educators develop a feedback-based teaching approach for novices. We designed and used a checklist for evaluating the gap in fiberoptic scope control between novices and experts. METHODS: Twelve first-year anesthesiology residents (novice group) attended a lecture, followed by hands-on practice with a fiberoptic scope on a manikin. Five staff anesthesiologists (expert group) only did the hands-on practice. After practice, each participant was video-recorded while conducting an FOI on the manikin. Two senior anesthesiologists developed and used a 7-item checklist to assess the FOIs. Checklist scores and total times for FOIs were compared between groups using the Mann-Whitney U test. Internal consistency of the checklist items, interrater reliability, and the relationship between checklist score and total time for FOI were assessed with Cronbach alpha, Cohen kappa, and the Pearson correlation coefficient, respectively. RESULTS: Experts had higher checklist scores than novices (P = .0016). The item with the lowest success rate for novices (50%) was keeping the scope straight. Novices spent more time on the FOI than experts (P = .0005). Cronbach alpha, Cohen kappa, and the Pearson correlation coefficient were 0.8699, 0.75, and -0.9454, respectively. CONCLUSIONS: Our checklist was used to detect differences in fiberoptic scope control skills between novices and experts. With a video-based assessment method, it can be used to develop a feedback-based teaching method for fiberoptic scope control.

Comprehensive Ultrasound Course for Special Operations Combat and Tactical Medics.

BACKGROUND: Advances in ultrasound technology with enhanced portability and high-quality imaging has led to a surge in its use on the battlefield by nonphysician providers. However, there is a consistent need for comprehensive and standardized ultrasound training to improve ultrasound knowledge, manual skills, and workflow understanding of nonphysician providers. MATERIALS AND METHODS: Our team designed a multimodal ultrasound course to improve ultrasound knowledge, manual skills, and workflow understanding of nine Special Operations combat medics and Special Operations tactical medics. The course was based on a flipped classroom model with a total time of 43 hours, consisting of an online component followed by live lectures and hands-on workshops. The effectiveness of the course was determined using a knowledge exam, expert ratings of manual skills using a global rating scale, and an objective structured clinical skills examination (OSCE). RESULTS: The average knowledge exam score of the medics increased from pre-course (56% ± 6.8%) to post-course (80% ± 5.0%, p < .001). Based on expert ratings, their manual skills improved from baseline to day 4 of the course for image finding (p = .007), image optimization (p = .008), image acquisition speed (p = .008), final image quality (p = .008), and global assessment (p = .008). Their average score at every OSCE station was > 91%. CONCLUSION: A comprehensive multimodal training program can be used to improve military medics' ultrasound knowledge, manual skills, and workflow understanding for various applications of ultrasound. Further research is required to develop a reliable, sustainable course.

Formation of an Intraoperative Educational Curriculum for Anesthesiology Residents Using a Systematic Approach.

Anesthesiology residents spend most of their training in operating rooms, but intraoperative teaching is often unstructured. Needs assessment indicated a need to incorporate a more evidence-based approach to education and improvement of our methods of introducing residents to primary anesthesiology literature. Kern's 6-step approach to curriculum development was used to create a robust and innovative curriculum to increase both the evidence-based component of our curriculum and the amount of educational intraoperative discussion among trainees and faculty. Our curriculum uses a structured topic outline, an e-journal club, and other relevant resources to facilitate discussion of the topics.

Development of an Endotracheal Intubation Formative Assessment Tool.

BACKGROUND: Valid methods for providing detailed formative feedback on direct laryngoscopy and endotracheal intubation (ETI) performance do not exist. We are developing an observation-based assessment tool for measuring performance and providing feedback during ETI. METHODS: Based on the literature and interviews of experts, we proposed an initial ETI metric with 22 items. Six anesthesiology experts used it to assess the quality of ETI performance in videotaped intubations. Following metric revisions, 2 expert groups assessed 2 collections of videos (27 total) using the revised metric. Two reference standards for comparison with metric scores were created with a third and fourth group of experts; (1) an average global rating (1-100) of each ETI performance and (2) average rank-ordered performance from best to worst. Rater agreement and correlations between the 2 methods were calculated. Regression analysis determined items that optimally discriminated quality. When calculating a score based on all clinically important terms, multiple weightings were evaluated. RESULTS: Metric items had high average rater agreement (80%) with intraclass correlation coefficients averaging 0.83. Correlations of the reference rank and score were high for both video collections (-0.96, P < .05, and -0.95, P < .05). Regression coefficients for different item weighting methods indicated strong relationships with global ratings (averaging r = 0.89, P < .05) and rankings averaging -0.85, P < .05). Prediction of global ratings using regression achieved high accuracy (R 2 = 0.8218). CONCLUSIONS: High observer agreement and strong correlations between metric and rank data support the validity of using this metric to assess ETI performance. Different weighting models yielded scores that correlated strongly with the ratings and ranks from global assessment. When using the metric to predict competency, a 3-item regression model is most accurate in predicting a global score.

Tool to improve qualitative assessment of left ventricular systolic function.

Interactive online learning tools have revolutionized graduate medical education and can impart echocardiographic image interpretive skills. We created self-paced, interactive online training modules using a repository of echocardiography videos of normal and various degrees of abnormal left ventricles. In this study, we tested the feasibility of this learning tool. Thirteen anesthesia interns took a pre-test and then had 3 weeks to complete the training modules on their own time before taking a post-test. The average score on the post-test (74.6% ± 11.08%) was higher than the average score on the pre-test (57.7% ± 9.27%) (P < 0.001). Scores did not differ between extreme function (severe dysfunction or hyperdynamic function) and non-extreme function (normal function or mild or moderate dysfunction) questions on both the pre-test (P = 0.278) and post-test (P = 0.093). The interns scored higher on the post-test than the pre-test on both extreme (P = 0.0062) and non-extreme (P = 0.0083) questions. After using an online educational tool that allowed learning on their own time and pace, trainees improved their ability to correctly categorize left ventricular systolic function. Left ventricular systolic function is often a key echocardiographic question that can be difficult to master. The promising performance of this educational resource may lead to more time- and cost-effective methods for improving diagnostic accuracy among learners.