Accuracy of Risk Estimation for Surgeons Versus Risk Calculators in Emergency General Surgery.

INTRODUCTION: Surgical risk calculators have expanded in both number and sophistication of their predictive approach. These calculators are gaining popularity as validated tools to help surgeons estimate mortality and complications following emergency general surgery (EGS). However, the accuracy of risk estimates generated by these calculators compared to risk estimation by practicing surgeons has not been explored. METHODS: Acute care surgeons at a quaternary care center prospectively estimated 30-d mortality and complications for adult EGS patients (2019-2021). Surgeon predictions were compared to Predictive OpTimal Trees in Emergency Surgery Risk (POTTER) and NSQIP estimates. Observed-to-expected (O:E) ratios of median aggregate estimates were calculated. C-statistics for surgeon and calculator estimations were utilized to quantify predictive accuracy. RESULTS: Among 150 patients (median 61 y, 45% male), 30-d mortality was 15% (n = 23). Observed rates of prolonged mechanical ventilation and acute renal failures were 30% and 10%, respectively. Overall, surgeon predictions were similar to risk calculator estimates for mortality (c-statistics 0.843 [surgeon] versus 0.848 [POTTER] and 0.815 [NSQIP]) and need for prolonged ventilation (c-statistics 0.801 versus 0.722 and 0.689, respectively). Surgeons tended to overestimate complication risks. Surgeon experience was not significantly associated with mortality prediction in an adjusted model. CONCLUSIONS: Acute care surgeons at a quaternary care center predicted postoperative mortality and complications with similar discrimination when compared to surgical risk calculators. Surgeon expertise should be utilized in conjunction with risk calculators when counseling EGS patients regarding anticipated postoperative outcomes. Surgeons should be cognizant of patterns in overestimation or underestimation of complications.

Consensus Current Procedural Terminology Code Definition of Source Control for Sepsis.

INTRODUCTION: Unlike antibiotic and perfusion support, guidelines for sepsis source control lack high-quality evidence and are ungraded. Internally valid administrative data methods are needed to identify cases representing source control procedures to evaluate outcomes. METHODS: Over five modified Delphi rounds, two independent reviewers identified Current Procedural Terminology (CPT) codes pertinent to source control. In each round, codes with perfect agreement were retained or excluded, whereas disagreements were reviewed by the panelists. Manual review of 400 patient records meeting Sepsis-3 criteria (2010-2017) clinically adjudicated which encounters included source control procedures (gold standard). The performance of consensus codes was compared with the gold standard to assess sensitivity, specificity, predictive values, and likelihood ratios. RESULTS: Of 5752 CPT codes, 609 consensus codes represented source control procedures. Of 400 hospitalizations for sepsis, 39 (9.8%; 95% confidence interval [CI] 7.0%-13.1%) underwent gold standard source control procedures and 29 (7.3%; 95% CI 4.9-10.3%) consensus code-defined source control procedures. Thirty consensus codes were identified (20.0% gastrointestinal/intraabdominal, 10.0% genitourinary, 13.3% hepatopancreatobiliary, 23.3% orthopedic/cranial, 23.3% soft tissue, and 10.0% intrathoracic), which had 61.5% (95% CI 44.6%-76.6%) sensitivity, 98.6% (95% CI 96.8%-99.6%) specificity, 83.2% (95% CI 66.6%-92.4%) positive, and 95.9% (95% CI 93.9%-97.2%) negative predictive values. With pretest probability at sample prevalence, an identified consensus code had a posttest probability of 83.0% (95% CI 66.0%-92.0%), whereas consensus code absence had a probability of 4.0% (95% CI 3.0-6.0) for undergoing a source control procedure. CONCLUSIONS: Using modified Delphi methodology, we created and validated CPT codes identifying source control procedures, providing a framework for evaluation of the surgical care of patients with sepsis.

Leveraging Decision Curve Analysis to Improve Clinical Application of Surgical Risk Calculators.

BACKGROUND: Surgical risk calculators (SRCs) have been developed for estimation of postoperative complications but do not directly inform decision-making. Decision curve analysis (DCA) is a method for evaluating prediction models, measuring their utility in guiding decisions. We aimed to analyze the utility of SRCs to guide both preoperative and postoperative management of patients undergoing hepatopancreaticobiliary surgery by using DCA. METHODS: A single-institution, retrospective review of patients undergoing hepatopancreaticobiliary operations between 2015 and 2017 was performed. Estimation of postoperative complications was conducted using the American College of Surgeons SRC [ACS-SRC] and the Predictive OpTimal Trees in Emergency Surgery Risk (POTTER) calculator; risks were compared with observed outcomes. DCA was used to model optimal patient selection for risk prevention strategies and to compare the relative performance of the ACS-SRC and POTTER calculators. RESULTS: A total of 994 patients were included in the analysis. C-statistics for the ACS-SRC prediction of 12 postoperative complications ranged from 0.546 to 0.782. DCA revealed that an ACS-SRC-guided readmission prevention intervention, when compared with an all-or-none approach, yielded a superior net benefit for patients with estimated risk between 5% and 20%. Comparison of SRCs for venous thromboembolism intervention demonstrated superiority of the ACS-SRC for thresholds for intervention between 2% and 4% with the POTTER calculator performing superiorly between 4% and 8% estimated risk. CONCLUSIONS: SRCs can be used not only to predict complication risk but also to guide risk prevention strategies. This methodology should be incorporated into external validations of future risk calculators and can be applied for institution-specific quality improvement initiatives to improve patient outcomes.

Predictors of acute incisional hernia incarceration at initial hernia diagnosis on computed tomography.

BACKGROUND: Acute incisional hernia incarceration is associated with high morbidity and mortality yet there is little evidence to guide which patients will benefit most from prophylactic repair. We explored baseline computed tomography (CT) characteristics associated with incarceration. METHODS: A case-control study design was utilized to explore adults (≥18 years) diagnosed with an incisional hernia between 2010 and 2017 at a single institution with a 1-year minimum follow-up. Computed tomography imaging at the time of initial hernia diagnosis was examined. Following propensity score matching for baseline characteristics, multivariable logistic regression was performed to identify independent predictors associated with acute incarceration. RESULTS: A total of 532 patients (27.26% male, mean 61.55 years) were examined, of whom 238 experienced an acute incarceration. Between two well-matched cohorts with and without incarceration, the presence of small bowel in the hernia sac (odds ratio [OR], 7.50; 95% confidence interval [CI], 3.35-16.38), increasing sac height (OR, 1.34; 95% CI, 1.10-1.64), more acute hernia angle (OR, 0.98 per degree; 95% CI, 0.97-0.99), decreased fascial defect width (OR, 0.68; 95% CI, 0.58-0.81), and greater outer abdominal fat (OR, 1.28; 95% CI, 1.02-1.60) were associated with acute incarceration. Using threshold analysis, a hernia angle of <91 degrees and a sac height of >3.25 cm were associated with increased incarceration risk. CONCLUSION: Computed tomography features present at the time of hernia diagnosis provide insight into later acute incarceration risk. Improved understanding of acute incisional hernia incarceration can guide selection for prophylactic repair and thereby may mitigate the excess morbidity associated with incarceration. LEVEL OF EVIDENCE: Prognostic and Epidemiological; Level III.

Sex, race, and socioeconomic distinctions in incisional hernia management.

BACKGROUND: We sought to explore the impact of sex, race, and insurance status on operative management of incisional hernias. METHODS: A retrospective cohort study was conducted to explore adult patients diagnosed with an incisional hernia. Adjusted odds for non-operative versus operative management and time to repair were queried. RESULTS: Of the 29,475 patients with an incisional hernia, 20,767 (70.5%) underwent non-operative management. In relation to private insurance, Medicaid (aOR 1.40, 95% CI 1.27-1.54), Medicare (aOR 1.53, 95% CI 1.42-1.65), and uninsured status (aOR 1.99, 95% CI 1.71-2.36) were independently associated with non-operative management. African American race (aOR 1.30, 95% CI 1.17-1.47) was associated with non-operative management while female sex (aOR 0.81, 95% CI 0.77-0.86) was predictive of elective repair. For patients who underwent elective repair, both Medicare (aOR 1.40, 95% CI 1.18-1.66) and Medicaid (aOR 1.49, 95% CI 1.29-1.71) insurance, but not race, were predictive of delayed repair (>90 days after diagnosis). CONCLUSIONS: Sex, race, and insurance status influence incisional hernia management. Development of evidence-based management guidelines may help to ensure equitable care.

Postprocedural Cognitive Load Measurement With Immediate Feedback to Guide Curriculum Development.

BACKGROUND: Methods to assess competency in cardiothoracic training are essential. Here, we report a system that allows us to better assess competency from the perspective of both the trainee and educator. We hypothesized that postprocedural cognitive burden measurement (by the trainee) with immediate feedback (from the educator) could aid in identifying barriers to the acquisition of skills and knowledge so that training curricula can be individualized. METHODS: The National Aeronautics and Space Administration Task Load Index (NASA-TLX), a validated instrument to measure cognitive load, was administered with an online platform after bronchoscopy, esophagogastroduodenoscopy, and video-assisted thoracoscopic surgery for 11 residents. Immediate postprocedure feedback and standardized debriefing occurred for each procedure. RESULTS: Mean NASA-TLX scores were highest (indicating greater cognitive load) for esophagogastroduodenoscopy and video-assisted thoracoscopic surgery (P < .001). When comparing subscale measures, mental demand was significantly higher for video-assisted thoracoscopic surgery (P = .026) compared with the other procedures, whereas physical demand was highest for esophagogastroduodenoscopy (P = .018). Self-reported frustration was similar for all case types (P = .247). Cognitive burden decreased with a greater number of procedures for bronchoscopy (P = .027). Significant improvement was noted by the trainee at the end of the rotation in self-assessed procedural competency and preparedness for thoracic board topics (all P < .05). Postprocedure feedback by the attending surgeon correlated with more frequent completion of self-evaluations by the residents. CONCLUSIONS: Longitudinal assessment of cognitive load in combination with postprocedural feedback identified barriers to skill acquisition for both residents and educators. This information allows for individualized rotation development as a step toward a competency-based curriculum.

Development and validation of a five-factor score for prediction of pathologic pneumatosis.

BACKGROUND: The significance of pneumatosis intestinalis (PI) remains challenging. While certain clinical scenarios are predictive of transmural ischemia, risk models to assess the presence of pathologic PI are needed. The aim of this study was to determine what patient factors at the time of radiographic diagnosis of PI predict the risk for pathologic PI. METHODS: We conducted a retrospective cohort study examining patients with PI from 2010 to 2016 at a multicenter hospital network. Multivariate logistic regression was used to develop a predictive model for pathologic PI in a derivation cohort. Using regression-coefficient-based methods, the final multivariate model was converted into a five-factor-based score. Calibration and discrimination of the score were then assessed in a validation cohort. RESULTS: Of 305 patients analyzed, 102 (33.4%) had pathologic PI. We identified five factors associated with pathologic PI at the time of radiographic diagnosis: small bowel PI, age 70 years or older, heart rate 110 bpm or greater, lactate of 2 mmol/L or greater, and neutrophil-lymphocyte ratio 10 or greater. Using this model, patients in the validation cohort were assigned risk scores ranging from 0 to 11. Low-risk patients were categorized when scores are 0 to 4; intermediate, score of 5 to 6; high, score of 7 to 8; and very high risk, 9+. In the validation cohort, very high-risk patients (n = 17; 18.1%) had predicted rates of pathologic pneumatosis of 88.9% and an observed rate of 82.4%. In contrast, patients labeled as low risk (n = 37; 39.4%) had expected rates of pathologic pneumatosis of 1.3% and an observed rate of 0%. The model showed excellent discrimination (area under the curve, 0.90) and good calibration (Hosmer-Lemeshow goodness-of-fit, p = 0.37). CONCLUSION: Our score accurately stratifies patient risk of pathologic pneumatosis. This score has the potential to target high-risk individuals for expedient operation and spare low-risk individuals invasive interventions. LEVEL OF EVIDENCE: Prognostic Study, Level III.

Prolapse Recurrence After Sacrocolpopexy Mesh Removal: A Retrospective Cohort Study.

OBJECTIVES: There is limited literature regarding outcomes after sacrocolpopexy mesh removal. We sought to compare the proportion of prolapse recurrence in women after sacrocolpopexy mesh removal with women who underwent sacrocolpopexy without subsequent mesh removal. We hypothesize that more women will experience prolapse recurrence after mesh removal. METHODS: This is a retrospective cohort study of women who underwent sacrocolpopexy mesh removal between 2010 and 2019. These patients were time matched with women who had a sacrocolpopexy but did not undergo mesh removal. Prolapse recurrence was defined as the leading edge past the hymen or retreatment. Analysis was done using χ, Wilcoxon rank-sum, or t test with a Cox proportional hazard model to assess the association between mesh removal and time to recurrence. RESULTS: We identified 26 mesh removals, which were matched with 78 patients without mesh removal. The most common indications for mesh removal were exposure (69.2%) and pain (57.7%). Women who underwent mesh removal were more likely to have Mersilene mesh (19.2% vs 1.3%, P = 0.006). Recurrence occurred in 46% of women who had mesh removal compared with 7.7% in those without (P < 0.001). When adjusted for age, parity, menopause, smoking, and diabetes status, those who had mesh removal had a 15 times higher hazard of prolapse recurrence (adjusted hazard ratio = 15.4, 95% confidence interval = 4.3-54.8, P = <.0001). CONCLUSIONS: When compared with time-matched controls, women who underwent sacrocolpopexy mesh removal had a significantly higher proportion of prolapse recurrence. Prospective studies are needed to further explore the utility of concomitant prolapse repair at the time of mesh removal.

Predisposed to failure? The challenge of rescue in the medical intensive care unit.

BACKGROUND: Medical intensive care unit (MICU) patients develop acute surgical processes that require operative intervention. There are limited data addressing outcomes of emergency general surgery (EGS) in this population. The aim of our study was to characterize the breadth of surgical consults from the MICU and assess mortality after abdominal EGS cases. METHODS: All MICU patients with an EGS consult in an academic medical center between January 2010 and 2016 were identified from an electronic medical record-based registry. Charts were reviewed to determine reason for consult, procedures performed, and to obtain additional clinical data. A multivariate logistic regression was used to determine patient factors associated with patient mortality. RESULTS: Of 911 MICU patients seen by our service, 411(45%) required operative intervention, with 186 patients undergoing an abdominal operation. The postoperative mortality rate after abdominal operations was 37% (69/186), significantly higher than the mortality of 16% (1833/11192) for all patients admitted to the MICU over the same period (p < 0.05). Damage-control procedures were performed in 64 (34%) patients, with 46% mortality in this group. The most common procedures were bowel resections, with mortality of 42% (28/66) and procedures for severe clostridium difficile, mortality of 38% (9/24). Twenty-seven patients met our definition of surgical rescue, requiring intervention for complications of prior procedures, with mortality of 48%. Need for surgical rescue was associated with increased admission mortality (odds ratio, 13.07; 95% confidence interval, 2.86-59.77). Twenty-six patients had pathology amenable to surgical intervention but did not undergo operation, with 100% mortality. In patients with abdominal pathology at the time of operation, in-hospital delay was associated with increased mortality (odds ratio, 5.13; 95% confidence interval, 1.11-23.77). CONCLUSION: Twenty percent of EGS consults from the MICU had an abdominal process requiring an operative intervention. While the MICU population as a whole has a high baseline mortality, patients requiring abdominal surgical intervention are an even higher risk. LEVEL OF EVIDENCE: Prognostic and epidemiological, level III.

Identifying incidental findings from radiology reports of trauma patients: An evaluation of automated feature representation methods.

BACKGROUND: Radiologic imaging of trauma patients often uncovers findings that are unrelated to the trauma. These are termed as incidental findings and identifying them in radiology examination reports is necessary for appropriate follow-up. We developed and evaluated an automated pipeline to identify incidental findings at sentence and section levels in radiology reports of trauma patients. METHODS: We created an annotated dataset of 4,181 reports and investigated automated feature representations including traditional word and clinical concept (such as SNOMED CT) representations, as well as word and concept embeddings. We evaluated these representations by using them with traditional classifiers such as logistic regression and with deep learning methods such as convolutional neural networks (CNNs). RESULTS: The best performance was observed using word embeddings with CNNs with F1 scores of 0.66 and 0.52 at section and sentence levels respectively. The F1 score was statistically significantly higher for sections compared to sentences (Wilcoxon; Z < 0.001, p < 0.05). Compared to using words alone, the addition of SNOMED CT concepts did not improve performance. At the sentence level, the F1 score improved significantly from 0.46 to 0.52 when using pre-trained embeddings (Wilcoxon; Z < 0.001, p < 0.05). CONCLUSION: The results show that the best performance was achieved by using embeddings with CNNs at both sentence and section levels. This provides evidence that such a pipeline is capable of accurately identifying incidental findings in radiology reports in an automated manner.

Interactive NLP in Clinical Care: Identifying Incidental Findings in Radiology Reports.

BACKGROUND: Despite advances in natural language processing (NLP), extracting information from clinical text is expensive. Interactive tools that are capable of easing the construction, review, and revision of NLP models can reduce this cost and improve the utility of clinical reports for clinical and secondary use. OBJECTIVES: We present the design and implementation of an interactive NLP tool for identifying incidental findings in radiology reports, along with a user study evaluating the performance and usability of the tool. METHODS: Expert reviewers provided gold standard annotations for 130 patient encounters (694 reports) at sentence, section, and report levels. We performed a user study with 15 physicians to evaluate the accuracy and usability of our tool. Participants reviewed encounters split into intervention (with predictions) and control conditions (no predictions). We measured changes in model performance, the time spent, and the number of user actions needed. The System Usability Scale (SUS) and an open-ended questionnaire were used to assess usability. RESULTS: Starting from bootstrapped models trained on 6 patient encounters, we observed an average increase in F1 score from 0.31 to 0.75 for reports, from 0.32 to 0.68 for sections, and from 0.22 to 0.60 for sentences on a held-out test data set, over an hour-long study session. We found that tool helped significantly reduce the time spent in reviewing encounters (134.30 vs. 148.44 seconds in intervention and control, respectively), while maintaining overall quality of labels as measured against the gold standard. The tool was well received by the study participants with a very good overall SUS score of 78.67. CONCLUSION: The user study demonstrated successful use of the tool by physicians for identifying incidental findings. These results support the viability of adopting interactive NLP tools in clinical care settings for a wider range of clinical applications.