Development and Pilot Testing of a Programmatic System for Competency Assessment in US Anesthesiology Residency Training.

BACKGROUND: In 2018, a set of entrustable professional activities (EPAs) and procedural skills assessments were developed for anesthesiology training, but they did not assess all the Accreditation Council for Graduate Medical Education (ACGME) milestones. The aims of this study were to (1) remap the 2018 EPA and procedural skills assessments to the revised ACGME Anesthesiology Milestones 2.0, (2) develop new assessments that combined with the original assessments to create a system of assessment that addresses all level 1 to 4 milestones, and (3) provide evidence for the validity of the assessments. METHODS: Using a modified Delphi process, a panel of anesthesiology education experts remapped the original assessments developed in 2018 to the Anesthesiology Milestones 2.0 and developed new assessments to create a system that assessed all level 1 through 4 milestones. Following a 24-month pilot at 7 institutions, the number of EPA and procedural skill assessments and mean scores were computed at the end of the academic year. Milestone achievement and subcompetency data for assessments from a single institution were compared to scores assigned by the institution's clinical competency committee (CCC). RESULTS: New assessment development, 2 months of testing and feedback, and revisions resulted in 5 new EPAs, 11 nontechnical skills assessments (NTSAs), and 6 objective structured clinical examinations (OSCEs). Combined with the original 20 EPAs and procedural skills assessments, the new system of assessment addresses 99% of level 1 to 4 Anesthesiology Milestones 2.0. During the 24-month pilot, aggregate mean EPA and procedural skill scores significantly increased with year in training. System subcompetency scores correlated significantly with 15 of 23 (65.2%) corresponding CCC scores at a single institution, but 8 correlations (36.4%) were <30.0, illustrating poor correlation. CONCLUSIONS: A panel of experts developed a set of EPAs, procedural skill assessment, NTSAs, and OSCEs to form a programmatic system of assessment for anesthesiology residency training in the United States. The method used to develop and pilot test the assessments, the progression of assessment scores with time in training, and the correlation of assessment scores with CCC scoring of milestone achievement provide evidence for the validity of the assessments.

Oral opioid prescribing to active duty US military personnel: a cross-sectional population.

INTRODUCTION: While civilian opioid prescriptions have seen a dramatic decline in recent years, there are few studies investigating trends in opioid prescription in the active duty military population. We evaluated oral opioid prescribing patterns to active duty military personnel in the Military Health System (MHS) from 2017 to 2020 to determine the incidence of opioid prescriptions as well as demographic and military-specific risk factors for receiving an oral opioid prescription. METHODS: The MHS Data Repository was queried from 2017 to 2020 to identify all outpatient oral opioid prescriptions to active duty military personnel in August of each year as well as demographic information on the study population. Data were evaluated in a logistic regression model, and ORs of receiving an oral opioid prescription were calculated for each factor. RESULTS: The proportion of active duty military personnel receiving an oral opioid prescription declined from 2.71% to 1.26% (53% relative reduction) over the study period. Within the logistic regression model, female military personnel were significantly more likely to receive opioid prescriptions compared with men, and there was a stepwise increase in likelihood of an opioid prescription with increasing age. Army and Marine personnel, personnel without a history of military deployment and those stationed within the continental USA were significantly more likely to receive an opioid prescription. DISCUSSION: The substantial decrease in oral opioid prescriptions to active duty military personnel mirrors data published in the civilian community. The identified risk factors for receiving an opioid prescription may be potential targets for future interventions to further decrease prescribing.

The Association Between Borderline Dysnatremia and Perioperative Morbidity and Mortality: Retrospective Cohort Study of the American College of Surgeons National Surgical Quality Improvement Program Database.

BACKGROUND: Hyponatremia and hypernatremia, as conventionally defined (<135 mEq/L and >145 mEq/L, respectively), are associated with increased perioperative morbidity and mortality. However, the effects of subtle deviations in serum sodium concentration within the normal range are not well-characterized. OBJECTIVE: The purpose of this analysis is to determine the association between borderline hyponatremia (135-137 mEq/L) and hypernatremia (143-145 mEq/L) on perioperative morbidity and mortality. METHODS: A retrospective cohort study was performed using data from the American College of Surgeons National Surgical Quality Improvement Program database. This database is a repository of surgical outcome data collected from over 600 hospitals across the United States. The National Surgical Quality Improvement Program database was queried to extract all patients undergoing elective, noncardiac surgery from 2015 to 2019. The primary predictor variable was preoperative serum sodium concentration, measured less than 5 days before the index surgery. The 2 primary outcomes were the odds of morbidity and mortality occurring within 30 days of surgery. The risk of both outcomes in relation to preoperative serum sodium concentration was modeled using weighted generalized additive models to minimize the effect of selection bias while controlling for covariates. RESULTS: In the overall cohort, 1,003,956 of 4,551,726 available patients had a serum sodium concentration drawn within 5 days of their index surgery. The odds of morbidity and mortality across sodium levels of 130-150 mEq/L relative to a sodium level of 140 mEq/L followed a nonnormally distributed U-shaped curve. The mean serum sodium concentration in the study population was 139 mEq/L. All continuous covariates were significantly associated with both morbidity and mortality (P<.001). Preoperative serum sodium concentrations of less than 139 mEq/L and those greater than 144 mEq/L were independently associated with increased morbidity probabilities. Serum sodium concentrations of less than 138 mEq/L and those greater than 142 mEq/L were associated with increased mortality probabilities. Hypernatremia was associated with higher odds of both morbidity and mortality than corresponding degrees of hyponatremia. CONCLUSIONS: Among patients undergoing elective, noncardiac surgery, this retrospective analysis found that preoperative serum sodium levels less than 138 mEq/L and those greater than 142 mEq/L are associated with increased morbidity and mortality, even within currently accepted "normal" ranges. The retrospective nature of this investigation limits the ability to make causal determinations for these findings. Given the U-shaped distribution of risk, past investigations that assume a linear relationship between serum sodium concentration and surgical outcomes may need to be revisited. Likewise, these results question the current definition of perioperative eunatremia, which may require future prospective investigations.

Competency-Based Assessments: Leveraging Artificial Intelligence to Predict Subcompetency Content.

PURPOSE: Faculty feedback on trainees is critical to guiding trainee progress in a competency-based medical education framework. The authors aimed to develop and evaluate a Natural Language Processing (NLP) algorithm that automatically categorizes narrative feedback into corresponding Accreditation Council for Graduate Medical Education Milestone 2.0 subcompetencies. METHOD: Ten academic anesthesiologists analyzed 5,935 narrative evaluations on anesthesiology trainees at 4 graduate medical education (GME) programs between July 1, 2019, and June 30, 2021. Each sentence (n = 25,714) was labeled with the Milestone 2.0 subcompetency that best captured its content or was labeled as demographic or not useful. Inter-rater agreement was assessed by Fleiss' Kappa. The authors trained an NLP model to predict feedback subcompetencies using data from 3 sites and evaluated its performance at a fourth site. Performance metrics included area under the receiver operating characteristic curve (AUC), positive predictive value, sensitivity, F1, and calibration curves. The model was implemented at 1 site in a self-assessment exercise. RESULTS: Fleiss' Kappa for subcompetency agreement was moderate (0.44). Model performance was good for professionalism, interpersonal and communication skills, and practice-based learning and improvement (AUC 0.79, 0.79, and 0.75, respectively). Subcompetencies within medical knowledge and patient care ranged from fair to excellent (AUC 0.66-0.84 and 0.63-0.88, respectively). Performance for systems-based practice was poor (AUC 0.59). Performances for demographic and not useful categories were excellent (AUC 0.87 for both). In approximately 1 minute, the model interpreted several hundred evaluations and produced individual trainee reports with organized feedback to guide a self-assessment exercise. The model was built into a web-based application. CONCLUSIONS: The authors developed an NLP model that recognized the feedback language of anesthesiologists across multiple GME programs. The model was operationalized in a self-assessment exercise. It is a powerful tool which rapidly organizes large amounts of narrative feedback.

Machine Learning to Predict Discharge Destination After Total Knee Arthroplasty and Total Hip Arthroplasty.

Discharge destination impacts costs and perioperative planning for primary total knee (TKA) or hip arthroplasty (THA). The purpose of this study was to create a tool to predict discharge destination in contemporary patients. Models were developed using more than 400,000 patients from the National Surgical Quality Improvement Program database. Models were compared with a previously published model using area under the receiver operating characteristic curve (AUC) and decision curve analysis (DCA). AUC on patients with TKA was 0.729 (95% confidence interval [CI]: 0.719 to 0.738) and 0.688 (95% CI: 0.678 to 0.697) using the new and previous models, respectively. AUC on patients with THA was 0.768 (95% CI: 0.758 to 0.778) and 0.726 (95% CI: 0.714 to 0.737) using the new and previous models, respectively. DCA showed substantially improved net clinical benefit. The new models were integrated into a web-based application. This tool enhances clinical decision making for predicting discharge destination following primary TKA and THA. (Journal of Surgical Orthopaedic Advances 32(4):252-258, 2023).

A Machine-Learning Algorithm to Predict the Likelihood of Prolonged Opioid Use Following Arthroscopic Hip Surgery.

PURPOSE: To develop a machine-learning algorithm and clinician-friendly tool predicting the likelihood of prolonged opioid use (>90 days) following hip arthroscopy. METHODS: The Military Data Repository was queried for all adult patients undergoing arthroscopic hip surgery between 2012 and 2017. Demographic, health history, and prescription records were extracted for all included patients. Opioid use was divided into preoperative use (30-365 days before surgery), perioperative use (30 days before surgery through 14 days after surgery), postoperative use (14-90 days after surgery), and prolonged postoperative use (90-365 days after surgery). Six machine-learning algorithms (Naïve Bayes, Gradient Boosting Machine, Extreme Gradient Boosting, Random Forest, Elastic Net Regularization, and artificial neural network) were developed. Area under the receiver operating curve and Brier scores were calculated for each model. Decision curve analysis was applied to assess clinical utility. Local-Interpretable Model-Agnostic Explanations were used to demonstrate factor weights within the selected model. RESULTS: A total of 6,760 patients were included, of whom 2,762 (40.9%) filled at least 1 opioid prescription >90 days after surgery. The artificial neural network model showed superior discrimination and calibration with area under the receiver operating curve = 0.71 (95% confidence interval 0.68-0.74) and Brier score = 0.21 (95% confidence interval 0.20-0.22). Postsurgical opioid use, age, and preoperative opioid use had the most influence on model outcome. Lesser factors included the presence of a psychological comorbidity and strong history of a substance use disorder. CONCLUSIONS: The artificial neural network model shows sufficient validity and discrimination for use in clinical practice. The 5 identified factors (age, preoperative opioid use, postoperative opioid use, presence of a mental health comorbidity, and presence of a preoperative substance use disorder) accurately predict the likelihood of prolonged opioid use following hip arthroscopy. LEVEL OF EVIDENCE: III, retrospective comparative prognostic trial.

Pulse Arrival Time Is Associated With Hemorrhagic Volume in a Porcine Model: A Pilot Study.

BACKGROUND: Hemorrhage is a major cause of preventable death worldwide, and early identification can be lifesaving. Pulse wave contour analysis has previously been used to infer hemodynamic variables in a variety of settings. We hypothesized that pulse arrival time (PAT), a form of pulse wave contour analysis which is assessed via electrocardiography (ECG) and photoplethysmography (PPG), is associated with hemorrhage volume. METHODS: Yorkshire-Cross swine were randomized to hemorrhage (30 mL/kg over 20 minutes) vs. control. Continuous ECG and PPG waveforms were recorded with a novel monitoring device, and algorithms were developed to calculate PAT and PAT variability throughout the respiratory cycle, termed "PAT index" or "PAT_I." Mixed effects models were used to determine associations between blood loss and PAT and between blood loss and PAT_I to account for clustering within subjects and investigate inter-subject variability in these relationships. RESULTS: PAT and PAT_I data were determined for ∼150 distinct intervals from five subjects. PAT and PAT_I were strongly associated with blood loss. Mixed effects modeling with PAT alone was substantially better than PAT_I alone (R2 0.93 vs. 0.57 and Akaike information criterion (AIC) 421.1 vs. 475.5, respectively). Modeling blood loss with PAT and PAT_I together resulted in slightly improved fit compared to PAT alone (R2 0.96, AIC 419.1). Mixed effects models demonstrated significant inter-subject variability in the relationships between blood loss and PAT. CONCLUSIONS: Findings from this pilot study suggest that PAT and PAT_I may be used to detect blood loss. Because of the simple design of a single-lead ECG and PPG, the technology could be packaged into a very small form factor device for use in austere or resource-constrained environments. Significant inter-subject variability in the relationship between blood loss and PAT highlights the importance of individualized hemodynamic monitoring.

Hemodynamic Effects of Cardiovascular Medications in a Normovolemic and Hemorrhaged Yorkshire-cross Swine Model.

The Yorkshire-cross swine model is a valuable translational model commonly used to study cardiovascular physiology and response to insult. Although the effects of vasoactive medications have been well described in healthy swine, the effects of these medications during hemorrhagic shock are less studied. In this study, we sought to expand the utility of the swine model by characterizing the hemodynamic changes that occurred after the administration of commonly available vasoactive medications during euvolemic and hypovolemic states. To this end, we anesthetized and established femoral arterial, central venous, and pulmonary arterial access in 15 juvenile Yorkshire-cross pigs. The pigs then received a series of rapidly metabolized but highly vasoactive medications in a standard dosing sequence. After completion of this sequence, each pig underwent a 30-mL/kg hemorrhage over 10 min, and the standard dosing sequence was repeated. We then used standard sta- tistical techniques to compare the effects of these vasoactive medications on a variety of hemodynamic parameters between the euvolemic and hemorrhagic states. All subjects completed the study protocol. The responses in the hemorrhagic state were often attenuated or even opposite of those in the euvolemic state. For example, phenylephrine decreased the mean arterial blood pressure during the euvolemic state but increased it in the hemorrhagic state. These results clarify previously poorly defined responses to commonly used vasoactive agents during the hemorrhagic state in swine. Our findings also demonstrate the need to consider the complex and dynamic physiologic state of hemorrhage when anticipating the effects of vasoactive drugs and planning study protocols.

Natural Language Processing and Its Use in Orthopaedic Research.

PURPOSE OF REVIEW: This review aims to demonstrate how natural language processing is used in orthopaedic research. RECENT FINDINGS: Natural language processing is a form of artificial intelligence that involves encoding human-generated text or speech into a form which can be interpreted by computers to perform a variety of tasks. Natural language processing gathers, processes, and organizes large amounts of free-text data more efficiently than humans. In orthopaedics, it has been utilized for retrospective chart review, automated reporting of electronic health record data, analyzing operative notes and radiology reports, and patient reviews of physicians and practices. Although still in its infancy, natural language processing promises to be a valuable tool in the future of orthopaedic research. It will not eliminate the need for the essential human component of questioning involved in research, but natural language processing can improve the quality, efficiency, and thoroughness of research, thus improving patient care.

Machine Learning to Predict Fascial Dehiscence after Exploratory Laparotomy Surgery.

BACKGROUND: Fascial dehiscence following exploratory laparotomy is associated with significant morbidity and increased mortality. Previously published risk prediction models for fascial dehiscence are dated and limit a surgeon's ability to perform reliable risk assessment intraoperatively. We sought to determine if machine learning can predict fascial dehiscence after exploratory laparotomy. MATERIALS AND METHODS: A retrospective cohort study was conducted of 93,024 patients undergoing exploratory laparotomy from the 2011-2018 ACS NSQIP data files. Data were divided into training (2011-2016, n = 69,969) and temporal validation (2017-2018, n = 23,055) cohorts. A clinical decision support tool was developed using the model generated via machine learning techniques. RESULTS: 1,332 (1.9%) patients in the training cohort and 390 (1.7%) patients in the temporal validation cohort developed fascial dehiscence. The area under the receiver operating characteristic curve was 0.69 (95% CI 0.66 to 0.72) in the validation cohort. Model predictions demonstrated excellent probability calibration. Decision curve analysis calculates net clinical benefit within a threshold range of 0.8%-4.5%. Operative time, surgical site and deep space infections, and body mass index were among the most important features for model predictions. Finally, operative time, sodium level, and hematocrit demonstrated non-linear relationships with predicted risk. CONCLUSION: A clinical decision support tool for predicting fascial dehiscence after exploratory laparotomy was created and validated on a contemporary, national patient cohort using machine learning. The tool calculates net clinical benefit and can be used at the point of care. Some identified risk factor relationships were found to be complex and non-linear, highlighting the ability of some machine learning applications to capture nuanced, patient-specific risk profiles.

A Novel Low-Cost Ventilator for Use in a Worldwide Pandemic: The Portsmouth Ventilator.

The ongoing severe acute respiratory syndrome coronavirus 2 or coronavirus disease 2019 pandemic has demonstrated the potential need for a low-cost, rapidly deployable ventilator. Based on this premise, we sought to design a ventilator with the following criteria: 1) standard components that are accessible to the public, 2) "open-source" compatibility to allow anyone to easily recreate the system, 3) ability to ventilate in acute respiratory distress syndrome, and 4) lowest possible cost to provide adequate oxygenation and ventilation. DESIGN: We pursued development of a pneumatic-type ventilator. The basic design involves three electrically controlled solenoid valves, a pressure chamber, the patient breathing circuit, a positive end-expiratory pressure valve, and an electronics control system. Multiple safety elements were built into the design. The user-friendly interface allows simple control of ventilator settings. The ventilator delivers a hybrid form of pneumatic, assist-control ventilation, with predicted tidal volumes of 300-800 mL, positive end-expiratory pressure 0-20 cm H2O, and Fio2 21-100%. MAIN RESULTS: The ventilator was extensively tested with two separate high-fidelity lung simulators and a porcine in vivo model. Both lung simulators were able to simulate a variety of pathologic states, including obstructive lung disease and acute respiratory distress syndrome. The ventilator performed well across all simulated scenarios. Similarly, a porcine in vivo model was used to assess performance in live tissue, with a specific emphasis on gas exchange. The ventilator performed well in vivo and demonstrated noninferior ventilation and oxygenation when compared with the standard ventilator. CONCLUSIONS: The Portsmouth Ventilator was able to perform well across all simulated pathologies and in vivo. All components may be acquired by the public for a cost of approximately $250 U.S.D. Although this ventilator has limited functionality compared with modern ventilators, the simple design appears to be safe and would allow for rapid mass production if ventilator surge demand exceeded supply.