Implementation of a software-based decision support tool for guideline-appropriate preoperative evaluation: a prospective agreement study.

BACKGROUND: Guideline adherence in the medical field leaves room for improvement. Digitalised decision support helps improve compliance. However, the complex nature of the guidelines makes implementation in clinical practice difficult. METHODS: This single-centre prospective study included 204 adult ASA physical status 3-4 patients undergoing elective noncardiac surgery at a German university hospital. Agreement of clearance for surgery between a guideline expert and a digital guideline support tool was investigated. The decision made by the on-duty anaesthetists (standard approach) was assessed for agreement with the expert in a cross-over design. The main outcome was the level of agreement between digital guideline support and the expert. RESULTS: The digital guideline support approach cleared 18.1% of the patients for surgery, the standard approach cleared 74.0%, and the expert approach cleared 47.5%. Agreement of the expert decision with digital guideline support (66.7%) and the standard approach (67.6%) was fair (Cohen's kappa 0.37 [interquartile range 0.26-0.48] vs 0.31 [0.21-0.42], P=0.6). Taking the expert decision as a benchmark, correct clearance using digital guideline support was 50.5%, and correct clearance using the standard approach was 44.6%. Digital guideline support incorrectly asked for additional examinations in 31.4% of the patients, whereas the standard approach did not consider conditions that would have justified additional examinations before surgery in 29.4%. CONCLUSIONS: Strict guideline adherence for clearance for surgery through digitalised decision support inadequately considered patients, clinical context. Vague formulations, weak recommendations, and low-quality evidence complicate guideline translation into explicit rules. CLINICAL TRIAL REGISTRATION: NCT04058769.

Anaesthesiologists' guideline adherence in pre-operative evaluation: a retrospective observational study.

BACKGROUND: Surveys suggest a low level of implementation of clinical guidelines, although they are intended to improve the quality of treatment and patient safety. Which guideline recommendations are not followed and why has yet to be analysed. In this study, we investigate the proportion of European and national guidelines followed in the area of pre-operative anaesthetic evaluation prior to non-cardiac surgery. METHODS: We conducted this monocentric retrospective observational study at a German university hospital with the help of software that logically links guidelines in such a way that individualised recommendations can be derived from a patient's data. We included routine logs of 2003 patients who visited our pre-anaesthesia outpatient clinic between June 2018 and June 2020 and compared the actual conducted pre-operative examinations with the recommendations issued by the software. We descriptively analysed the data for examinations not performed that would have been recommended by the guidelines and examinations that were performed even though they were not covered by a guideline recommendation. The guidelines examined in this study are the 2018 ESAIC guidelines for pre-operative evaluation of adults undergoing elective non-cardiac surgery, the 2014 ESC/ESA guidelines on non-cardiac surgery and the German recommendations on pre-operative evaluation on non-cardiothoracic surgery from the year 2017. RESULTS: Performed ECG (78.1%) and cardiac stress imaging tests (86.1%) indicated the highest guideline adherence. Greater adherence rates were associated with a higher ASA score (ASA I: 23.7%, ASA II: 41.1%, ASA III: 51.8%, ASA IV: 65.8%, P < 0.001), lower BMI and age > 65 years. Adherence rates in high-risk surgery (60.5%) were greater than in intermediate (46.5%) or low-risk (44.6%) surgery (P < 0.001). 67.2% of technical and laboratory tests performed preoperatively were not covered by a guideline recommendation. CONCLUSIONS: Guideline adherence in pre-operative evaluation leaves room for improvement. Many performed pre-operative examinations, especially laboratory tests, are not recommended by the guidelines and may cause unnecessary costs. The reasons for guidelines not being followed may be the complexity of guidelines and organisational issues. A software-based decision support tool may be helpful. TRIAL REGISTRATION: ClinicalTrials.gov ID NCT04843202.

Susceptibility of AutoML mortality prediction algorithms to model drift caused by the COVID pandemic.

BACKGROUND: Concept drift and covariate shift lead to a degradation of machine learning (ML) models. The objective of our study was to characterize sudden data drift as caused by the COVID pandemic. Furthermore, we investigated the suitability of certain methods in model training to prevent model degradation caused by data drift. METHODS: We trained different ML models with the H2O AutoML method on a dataset comprising 102,666 cases of surgical patients collected in the years 2014-2019 to predict postoperative mortality using preoperatively available data. Models applied were Generalized Linear Model with regularization, Default Random Forest, Gradient Boosting Machine, eXtreme Gradient Boosting, Deep Learning and Stacked Ensembles comprising all base models. Further, we modified the original models by applying three different methods when training on the original pre-pandemic dataset: (Rahmani K, et al, Int J Med Inform 173:104930, 2023) we weighted older data weaker, (Morger A, et al, Sci Rep 12:7244, 2022) used only the most recent data for model training and (Dilmegani C, 2023) performed a z-transformation of the numerical input parameters. Afterwards, we tested model performance on a pre-pandemic and an in-pandemic data set not used in the training process, and analysed common features. RESULTS: The models produced showed excellent areas under receiver-operating characteristic and acceptable precision-recall curves when tested on a dataset from January-March 2020, but significant degradation when tested on a dataset collected in the first wave of the COVID pandemic from April-May 2020. When comparing the probability distributions of the input parameters, significant differences between pre-pandemic and in-pandemic data were found. The endpoint of our models, in-hospital mortality after surgery, did not differ significantly between pre- and in-pandemic data and was about 1% in each case. However, the models varied considerably in the composition of their input parameters. None of our applied modifications prevented a loss of performance, although very different models emerged from it, using a large variety of parameters. CONCLUSIONS: Our results show that none of our tested easy-to-implement measures in model training can prevent deterioration in the case of sudden external events. Therefore, we conclude that, in the presence of concept drift and covariate shift, close monitoring and critical review of model predictions are necessary.

Enabling personalized perioperative risk prediction by using a machine-learning model based on preoperative data.

Preoperative risk assessment is essential for shared decision-making and adequate perioperative care. Common scores provide limited predictive quality and lack personalized information. The aim of this study was to create an interpretable machine-learning-based model to assess the patient's individual risk of postoperative mortality based on preoperative data to allow analysis of personal risk factors. After ethical approval, a model for prediction of postoperative in-hospital mortality based on preoperative data of 66,846 patients undergoing elective non-cardiac surgery between June 2014 and March 2020 was created with extreme gradient boosting. Model performance and the most relevant parameters were shown using receiver operating characteristic (ROC-) and precision-recall (PR-) curves and importance plots. Individual risks of index patients were presented in waterfall diagrams. The model included 201 features and showed good predictive abilities with an area under receiver operating characteristic (AUROC) curve of 0.95 and an area under precision-recall curve (AUPRC) of 0.109. The feature with the highest information gain was the preoperative order for red packed cell concentrates followed by age and c-reactive protein. Individual risk factors could be identified on patient level. We created a highly accurate and interpretable machine learning model to preoperatively predict the risk of postoperative in-hospital mortality. The algorithm can be used to identify factors susceptible to preoperative optimization measures and to identify risk factors influencing individual patient risk.

Weak correlations between serum and cerebrospinal fluid levels of estradiol, progesterone and testosterone in males.

BACKGROUND: Neuroactive steroids seem to be implicated in a variety of neurophysiological and behavioral processes, such as sleep, learning, memory, stress, feeding and aging. Numerous studies have also addressed this implication in various cerebral disorders and diseases. Yet, the correlation and association between steroids in the periphery, e.g. blood, and the central compartments, e.g. cerebrospinal fluid (CSF), have not yet been comprehensively assessed. As the brain is not directly accessible, and the collection of human CSF usually requires invasive procedures, easier accessible compartments, such as blood, have always attracted attention. However, studies in humans are scarce. In the present study we determined estradiol, progesterone and testosterone levels in CSF and serum of 22 males without cerebral disorders or diseases. RESULTS: Samples were taken under conditions corresponding closest to basal conditions with patients expecting only spinal anesthesia and minor surgery. All samples per patient were collected concomitantly. Total estradiol, progesterone and testosterone concentrations were measured by electro-chemiluminescence immunoassay. The strength of correlation was assessed by Spearman's rank correlation coefficient. Correlation analysis revealed merely weak to very weak correlations for estradiol, progesterone and testosterone respectively between the CSF and serum compartments. CONCLUSIONS: Total steroid levels of estradiol, progesterone and testosterone in CSF and serum of males without neurological disorders were determined. Weak to very weak correlations between CSF and serum were found thus suggesting that concentrations in the periphery do not parallel concentrations in the central compartments. Further research is needed to clarify to what extent and under which conditions serum levels of estradiol, progesterone and testosterone may possibly serve as a biomarker reflecting the respective concentrations in the CSF or in the brain.