eConsent administered by Community Health Workers in a study using the Trials within Cohorts (TwiCs) design-Experiences from the Community-Based chronic Care Lesotho (ComBaCaL) project.

Improving access to essential health services requires the development of innovative health service delivery models and their scientific assessment in often large-scale pragmatic trials. In many low- and middle-income countries, lay Community Health Workers (CHWs) play an important role in delivering essential health services. As trusted members of their communities with basic medical training, they may also contribute to health data collection. Digital clinical decision support applications may facilitate the involvement of CHWs in service delivery and data collection. Electronic consent (eConsent) can streamline the consent process that is required if the collected data is used for the scientific purposes. Here, we describe the experiences of using eConsent in the Community-Based chronic Care Lesotho (ComBaCaL) cohort study and multiple nested pragmatic cluster-randomized trials assessing CHW-led care delivery models for type 2 diabetes and arterial hypertension using the Trials within Cohorts (TwiCs) design. More than a hundred CHWs, acting both as service providers and data collectors in remote villages of Lesotho utilize an eConsent application that is linked to a tailored clinical decision support and data collection application. The eConsent application presents simplified consent information and generates personalized consent forms that are signed electronically on a tablet and then uploaded to the database of the clinical decision support application. This significantly streamlines the consent process and allows for quality consent documentation through timely central monitoring, facilitating the CHW-led management of a large-scale population-based cohort in a remote low-resource area with continuous enrollment-currently at more than 16,000 participants.

The effect of clinical decision support systems on clinical outcomes in acute kidney injury: a systematic review and meta-analysis of randomized controlled trials.

OBJECTIVES: To determine whether clinical decision support systems (CDSS) for acute kidney injury (AKI) would enhance patient outcomes in terms of mortality, dialysis, and acute kidney damage progression. METHODS: The systematic review and meta-analysis included the relevant randomized controlled trials (RCTs) retrieved from PubMed, EMBASE, Web of Science, Cochrane, and SCOPUS databases until 21st January 2024. The meta-analysis was done using (RevMan 5.4.1). PROSPERO ID: CRD42024517399. RESULTS: Our meta-analysis included ten RCTs with 18,355 patients. There was no significant difference between CDSS and usual care in all-cause mortality (RR: 1.00 with 95% CI [0.93, 1.07], p = 0.91) and renal replacement therapy (RR: 1.11 with 95% CI [0.99, 1.24], p = 0.07). However, CDSS was significantly associated with a decreased incidence of hyperkalemia (RR: 0.27 with 95% CI [0.10, 0.73], p = 0.01) and increased eGFR change (MD: 1.97 with 95% CI [0.47, 3.48], p = 0.01). CONCLUSIONS: CDSS were not associated with clinical benefit in patients with AKI, with no effect on all-cause mortality or the need for renal replacement therapy. However, CDSS reduced the incidence of hyperkalemia and improved eGFR change in AKI patients.

A clinical decision support system for diagnosis and severity quantification of lumbosacral radiculopathy using intramuscular electromyography signals.

Interpreting intramuscular electromyography (iEMG) signals for diagnosing and quantifying the severity of lumbosacral radiculopathy is challenging due to the subjective evaluation of signals. To address this limitation, a clinical decision support system (CDSS) was developed for the diagnosis and quantification of the severity of lumbosacral radiculopathy based on intramuscular electromyography (iEMG) signals. The CDSS uses the EMG interference pattern method (QEMG IP) to directly extract features from the iEMG signal and provide a quantitative expression of injury severity for each muscle and overall radiculopathy severity. From 126 time and frequency domain features, a set of five features, including the crest factor, mean absolute value, peak frequency, zero crossing count, and intensity, were selected. These features were derived from raw iEMG signals, empirical mode decomposition, and discrete wavelet transform, and the wrapper method was utilized to determine the most significant features. The CDSS was trained and tested on a dataset of 75 patients, achieving an accuracy of 93.3%, sensitivity of 93.3%, and specificity of 96.6%. The system shows promise in assisting physicians in diagnosing lumbosacral radiculopathy with high accuracy and consistency using iEMG data. The CDSS's objective and standardized diagnostic process, along with its potential to reduce the time and effort required by physicians to interpret EMG signals, makes it a potentially valuable tool for clinicians in the diagnosis and management of lumbosacral radiculopathy. Future work should focus on validating the system's performance in diverse clinical settings and patient populations.

An Institutional Febrile Neutropenia Protocol Improved the Antibacterial Treatment and Encouraged the Development of a Computerized Clinical Decision Support System.

We investigated the influence of a local guideline on the quality of febrile neutropenia (FN) management and the applicability of a computerized decision support system (CDSS) using real-life data. The study included 227 FN patients between April 2016 and January 2019. The primary outcome measure was the achievement of a 20% increase in the rate of appropriate empirical treatment of FN in bacteremic patients. The compatibility of the CDSS (the development of which was completed in November 2021) with local protocols was tested using standard patient scenarios and empirical antibiotic recommendations for bacteremic FN patients. In total, 91 patients were evaluated before (P1: between April 2016 and May 2017) and 136 after (P2: between May 2017 and January 2019) the guideline's release (May 2017). The demographic characteristics were similar. Appropriate empirical antibacterial treatment was achieved in 58.3% of P1 and 88.1% of P2 patients (p = 0.006). The need for escalation of antibacterial treatment was significantly lower in P2 (49.5% vs. 35.3%; p = 0.03). In P2, the performance of the CDSS and consulting physicians was similar (CDSS 88.8% vs. physician 88.83%; p = 1) regarding appropriate empirical antibacterial treatment. The introduction of the local guideline improved the appropriateness of initial empirical treatment and reduced escalation rates in FN patients. The high rate of compliance of the CDSS with the local guideline-based decisions in P2 highlights the usefulness of the CDSS for these patients.

Limitations of a platelet count-based clinical decision support system to facilitate diagnosis of heparin-induced thrombocytopenia.

Heparin-induced thrombocytopenia (HIT) is a rare complication of heparin exposure with potential for significant morbidity and mortality. Early identification and treatment can prevent catastrophic thrombosis. Herein, we report the performance of a platelet count-based clinical decision support system (CDSS) where providers received a notification when a patient had a platelet count decline of ≥50 %. In the 90-day study period, the CDSS sent 302 notifications on 270 patients. Notifications were frequently inappropriate; 25 % had an expected platelet count decline (organ donation, stem cell transplant), an inaccurate count, or no heparin exposure. Patient testing for HIT prompted by the CDSS was not in accordance with best practice guidelines in most circumstances. For example, 36 % had a low probability 4Ts score, while 42 % with an intermediate or high probability 4Ts score were not tested. Due to concern for lack of efficacy, the CDSS was discontinued. Analysis of an 8-month period before and after discontinuation showed a significant decrease in the number of enzyme immunoassays ordered (547 vs. 386) without a change in the number of patients with HIT identified (13 vs. 13) or the rate of thrombosis in those with confirmed HIT (62 % vs. 62 %). In conclusion, a CDSS based on platelet count decline contributed to "alert fatigue" via inappropriate notification and did not improve evidence-based HIT testing. In addition, its removal did not decrease or delay HIT identification. Additional efforts are needed to better define how CDSS can support the rapid diagnosis and appropriate treatment of patients with HIT.

A large language model-based clinical decision support system for syncope recognition in the emergency department: A framework for clinical workflow integration.

Differentiation of syncope from transient loss of consciousness can be challenging in the emergency department (ED). Natural Language Processing (NLP) enables the analysis of free text in the electronic medical records (EMR). The present paper aimed to develop a large language models (LLM) for syncope recognition in the ED and proposed a framework for model integration within the clinical workflow. Two models, based on both the Italian and Multilingual Bidirectional Encoder Representations from Transformers (BERT) language model, were developed using consecutive EMRs. The "triage" model was only based on notes contained in the "triage" section of the EMR. The "anamnesis" model added data contained in the "medical history" section. Interpretation and calibration plots were generated. The Italian and Multi BERT models were developed and tested on both 15,098 and 15,222 EMRs, respectively. The triage model had an AUC of 0·95 for the Italian BERT and 0·94 for the Multi BERT. The anamnesis model had an AUC of 0·98 for the Italian BERT and 0·97 for Multi BERT. The LLM identified syncope when not explicitly mentioned in the EMR and also recognized common prodromal symptoms preceding syncope. Both models identified syncope patients in the ED with a high discriminative capability from nurses and doctors' notes, thus potentially acting as a tool helping physicians to differentiate syncope from others transient loss of consciousness.

RCC-Supporter: supporting renal cell carcinoma treatment decision-making using machine learning.

BACKGROUND: The population diagnosed with renal cell carcinoma, especially in Asia, represents 36.6% of global cases, with the incidence rate of renal cell carcinoma in Korea steadily increasing annually. However, treatment options for renal cell carcinoma are diverse, depending on clinical stage and histologic characteristics. Hence, this study aims to develop a machine learning based clinical decision-support system that recommends personalized treatment tailored to the individual health condition of each patient. RESULTS: We reviewed the real-world medical data of 1,867 participants diagnosed with renal cell carcinoma between November 2008 and June 2021 at the Pusan National University Yangsan Hospital in South Korea. Data were manually divided into a follow-up group where the patients did not undergo surgery or chemotherapy (Surveillance), a group where the patients underwent surgery (Surgery), and a group where the patients received chemotherapy before or after surgery (Chemotherapy). Feature selection was conducted to identify the significant clinical factors influencing renal cell carcinoma treatment decisions from 2,058 features. These features included subsets of 20, 50, 75, 100, and 150, as well as the complete set and an additional 50 expert-selected features. We applied representative machine learning algorithms, namely Decision Tree, Random Forest, and Gradient Boosting Machine (GBM). We analyzed the performance of three applied machine learning algorithms, among which the GBM algorithm achieved an accuracy score of 95% (95% CI, 92-98%) for the 100 and 150 feature sets. The GBM algorithm using 100 and 150 features achieved better performance than the algorithm using features selected by clinical experts (93%, 95% CI 89-97%). CONCLUSIONS: We developed a preliminary personalized treatment decision-support system (TDSS) called "RCC-Supporter" by applying machine learning (ML) algorithms to determine personalized treatment for the various clinical situations of RCC patients. Our results demonstrate the feasibility of using machine learning-based clinical decision support systems for treatment decisions in real clinical settings.

Validating the Accuracy of a Patient-Facing Clinical Decision Support System in Predicting Lumbar Disc Herniation: Diagnostic Accuracy Study.

BACKGROUND: Low back pain (LBP) is a major cause of disability globally, and the diagnosis of LBP is challenging for clinicians. OBJECTIVE: Using new software called Therapha, this study aimed to assess the accuracy level of artificial intelligence as a Clinical Decision Support System (CDSS) compared to MRI in predicting lumbar disc herniated patients. METHODS: One hundred low back pain patients aged ≥18 years old were included in the study. The study was conducted in three stages. Firstly, a case series was conducted by matching MRI and Therapha diagnosis for 10 patients. Subsequently, Delphi methodology was employed to establish a clinical consensus. Finally, to determine the accuracy of the newly developed software, a cross-sectional study was undertaken involving 100 patients. RESULTS: The software showed a significant diagnostic accuracy with the area under the curve in the ROC analysis determined as 0.84 with a sensitivity of 88% and a specificity of 80%. CONCLUSIONS: The study's findings revealed that CDSS using Therapha has a reasonable level of efficacy, and this can be utilized clinically to acquire a faster and more accurate screening of patients with lumbar disc herniation.

Early Prediction of Cardiac Arrest in the Intensive Care Unit Using Explainable Machine Learning: Retrospective Study.

BACKGROUND: Cardiac arrest (CA) is one of the leading causes of death among patients in the intensive care unit (ICU). Although many CA prediction models with high sensitivity have been developed to anticipate CA, their practical application has been challenging due to a lack of generalization and validation. Additionally, the heterogeneity among patients in different ICU subtypes has not been adequately addressed. OBJECTIVE: This study aims to propose a clinically interpretable ensemble approach for the timely and accurate prediction of CA within 24 hours, regardless of patient heterogeneity, including variations across different populations and ICU subtypes. Additionally, we conducted patient-independent evaluations to emphasize the model's generalization performance and analyzed interpretable results that can be readily adopted by clinicians in real-time. METHODS: Patients were retrospectively analyzed using data from the Medical Information Mart for Intensive Care-IV (MIMIC-IV) and the eICU-Collaborative Research Database (eICU-CRD). To address the problem of underperformance, we constructed our framework using feature sets based on vital signs, multiresolution statistical analysis, and the Gini index, with a 12-hour window to capture the unique characteristics of CA. We extracted 3 types of features from each database to compare the performance of CA prediction between high-risk patient groups from MIMIC-IV and patients without CA from eICU-CRD. After feature extraction, we developed a tabular network (TabNet) model using feature screening with cost-sensitive learning. To assess real-time CA prediction performance, we used 10-fold leave-one-patient-out cross-validation and a cross-data set method. We evaluated MIMIC-IV and eICU-CRD across different cohort populations and subtypes of ICU within each database. Finally, external validation using the eICU-CRD and MIMIC-IV databases was conducted to assess the model's generalization ability. The decision mask of the proposed method was used to capture the interpretability of the model. RESULTS: The proposed method outperformed conventional approaches across different cohort populations in both MIMIC-IV and eICU-CRD. Additionally, it achieved higher accuracy than baseline models for various ICU subtypes within both databases. The interpretable prediction results can enhance clinicians' understanding of CA prediction by serving as a statistical comparison between non-CA and CA groups. Next, we tested the eICU-CRD and MIMIC-IV data sets using models trained on MIMIC-IV and eICU-CRD, respectively, to evaluate generalization ability. The results demonstrated superior performance compared with baseline models. CONCLUSIONS: Our novel framework for learning unique features provides stable predictive power across different ICU environments. Most of the interpretable global information reveals statistical differences between CA and non-CA groups, demonstrating its utility as an indicator for clinical decisions. Consequently, the proposed CA prediction system is a clinically validated algorithm that enables clinicians to intervene early based on CA prediction information and can be applied to clinical trials in digital health.

Artificial intelligence to assist decision-making on pharmacotherapy: A feasibility study.

Background: Artificial intelligence (AI) has the capability to analyze vast amounts of data and has been applied in various healthcare sectors. However, its effectiveness in aiding pharmacotherapy decision-making remains uncertain due to the intricate, patient-specific, and dynamic nature of this field. Objective: This study sought to investigate the potential of AI in guiding pharmacotherapy decisions using clinical data such as diagnoses, laboratory results, and vital signs obtained from routine patient care. Methods: Data of a previous study on medication therapy optimization was updated and adapted for the purpose of this study. Analysis was conducted using R software along with the tidymodels extension packages. The dataset was split into 74% for training and 26% for testing. Decision trees were selected as the primary model due to their simplicity, transparency, and interpretability. To prevent overfitting, bootstrapping techniques were employed, and hyperparameters were fine-tuned. Performance metrics such as areas under the curve and accuracies were computed. Results: The study cohort comprised 101 elderly patients with multiple diagnoses and complex medication regimens. The AI model demonstrated prediction accuracies ranging from 38% to 100% for various cardiovascular drug classes. Laboratory data and vital signs could not be interpreted, as the effect and dependence were unclear for the model. The study revealed that the issue of AI lag time in responding to sudden changes could be addressed by manually adjusting decision trees, a task not feasible with neural networks. Conclusion: In conclusion, the AI model exhibited promise in recommending appropriate medications for individual patients. While the study identified several obstacles during model development, most were successfully resolved. Future AI studies need to include the drug effect, not only the drug, if laboratory data is part of the decision. This could assist with interpreting their potential relationship. Human oversight and intervention remain essential for an AI-driven pharmacotherapy decision support system to ensure safe and effective patient care.

What Determines the Use of Clinical Decision Support Systems in Nursing? Results of a Multiple Regression Analysis.

INTRODUCTION: In nursing, professionals are expected to base their practice on evidence-based knowledge, however the successful implementation of this knowledge into nursing practice is not always assured. Clinical Decision Support Systems (CDSS) are considered to bridge this evidence-practice gap. METHODS: This study examines the extent to which evidence-based nursing (EBN) practices influence the use of CDSS and identifies what additional factors from acceptance theories such as UTAUT play a role. RESULTS AND DISCUSSION: Our findings from three regression models revealed that nursing professionals and nursing students who employ evidence-based practices are not more likely to use an evidence-based CDSS. The relationship between an EBN composite score (model 1) or is individual dimensions (model 2) and CDSS use was not significant. However, a more comprehensive model (model 3), incorporating items from the UTAUT such as Social Influences, Facilitating Conditions, Performance Expectancy, and Effort Expectancy, supplemented by Satisfaction demonstrated a significant variance explained (R2 = 0.279). Performance Expectancy and Satisfaction were found to be significantly associated with CDSS utilization. CONCLUSION: This underscores the importance of user-friendliness and practical utility of a CDSS. Despite potential limitations in generalizability and a limited sample size, the results provide insights into that CDSS first and foremost underly the same mechanisms of use as other health IT systems.

Challenges and Facilitation Approaches for the Participatory Design of AI-Based Clinical Decision Support Systems: Protocol for a Scoping Review.

BACKGROUND: In the last few years, there has been an increasing interest in the development of artificial intelligence (AI)-based clinical decision support systems (CDSS). However, there are barriers to the successful implementation of such systems in practice, including the lack of acceptance of these systems. Participatory approaches aim to involve future users in designing applications such as CDSS to be more acceptable, feasible, and fundamentally more relevant for practice. The development of technologies based on AI, however, challenges the process of user involvement and related methods. OBJECTIVE: The aim of this review is to summarize and present the main approaches, methods, practices, and specific challenges for participatory research and development of AI-based decision support systems involving clinicians. METHODS: This scoping review will follow the Joanna Briggs Institute approach to scoping reviews. The search for eligible studies was conducted in the databases MEDLINE via PubMed; ACM Digital Library; Cumulative Index to Nursing and Allied Health; and PsycInfo. The following search filters, adapted to each database, were used: Period January 01, 2012, to October 31, 2023, English and German studies only, abstract available. The scoping review will include studies that involve the development, piloting, implementation, and evaluation of AI-based CDSS (hybrid and data-driven AI approaches). Clinical staff must be involved in a participatory manner. Data retrieval will be accompanied by a manual gray literature search. Potential publications will then be exported into reference management software, and duplicates will be removed. Afterward, the obtained set of papers will be transferred into a systematic review management tool. All publications will be screened, extracted, and analyzed: title and abstract screening will be carried out by 2 independent reviewers. Disagreements will be resolved by involving a third reviewer. Data will be extracted using a data extraction tool prepared for the study. RESULTS: This scoping review protocol was registered on March 11, 2023, at the Open Science Framework. The full-text screening had already started at that time. Of the 3,118 studies screened by title and abstract, 31 were included in the full-text screening. Data collection and analysis as well as manuscript preparation are planned for the second and third quarter of 2024. The manuscript should be submitted towards the end of 2024. CONCLUSIONS: This review will describe the current state of knowledge on participatory development of AI-based decision support systems. The aim is to identify knowledge gaps and provide research impetus. It also aims to provide relevant information for policy makers and practitioners. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/58185.

An Advanced Machine Learning Model for a Web-Based Artificial Intelligence-Based Clinical Decision Support System Application: Model Development and Validation Study.

BACKGROUND: Breast cancer is a leading global health concern, necessitating advancements in recurrence prediction and management. The development of an artificial intelligence (AI)-based clinical decision support system (AI-CDSS) using ChatGPT addresses this need with the aim of enhancing both prediction accuracy and user accessibility. OBJECTIVE: This study aims to develop and validate an advanced machine learning model for a web-based AI-CDSS application, leveraging the question-and-answer guidance capabilities of ChatGPT to enhance data preprocessing and model development, thereby improving the prediction of breast cancer recurrence. METHODS: This study focused on developing an advanced machine learning model by leveraging data from the Tri-Service General Hospital breast cancer registry of 3577 patients (2004-2016). As a tertiary medical center, it accepts referrals from four branches-3 branches in the northern region and 1 branch on an offshore island in our country-that manage chronic diseases but refer complex surgical cases, including breast cancer, to the main center, enriching our study population's diversity. Model training used patient data from 2004 to 2012, with subsequent validation using data from 2013 to 2016, ensuring comprehensive assessment and robustness of our predictive models. ChatGPT is integral to preprocessing and model development, aiding in hormone receptor categorization, age binning, and one-hot encoding. Techniques such as the synthetic minority oversampling technique address the imbalance of data sets. Various algorithms, including light gradient-boosting machine, gradient boosting, and extreme gradient boosting, were used, and their performance was evaluated using metrics such as the area under the curve, accuracy, sensitivity, and F1-score. RESULTS: The light gradient-boosting machine model demonstrated superior performance, with an area under the curve of 0.80, followed closely by the gradient boosting and extreme gradient boosting models. The web interface of the AI-CDSS tool was effectively tested in clinical decision-making scenarios, proving its use in personalized treatment planning and patient involvement. CONCLUSIONS: The AI-CDSS tool, enhanced by ChatGPT, marks a significant advancement in breast cancer recurrence prediction, offering a more individualized and accessible approach for clinicians and patients. Although promising, further validation in diverse clinical settings is recommended to confirm its efficacy and expand its use.

Clinical Decision Support and Natural Language Processing in Medicine: Systematic Literature Review.

BACKGROUND: Ensuring access to accurate and verified information is essential for effective patient treatment and diagnosis. Although health workers rely on the internet for clinical data, there is a need for a more streamlined approach. OBJECTIVE: This systematic review aims to assess the current state of artificial intelligence (AI) and natural language processing (NLP) techniques in health care to identify their potential use in electronic health records and automated information searches. METHODS: A search was conducted in the PubMed, Embase, ScienceDirect, Scopus, and Web of Science online databases for articles published between January 2000 and April 2023. The only inclusion criteria were (1) original research articles and studies on the application of AI-based medical clinical decision support using NLP techniques and (2) publications in English. A Critical Appraisal Skills Programme tool was used to assess the quality of the studies. RESULTS: The search yielded 707 articles, from which 26 studies were included (24 original articles and 2 systematic reviews). Of the evaluated articles, 21 (81%) explained the use of NLP as a source of data collection, 18 (69%) used electronic health records as a data source, and a further 8 (31%) were based on clinical data. Only 5 (19%) of the articles showed the use of combined strategies for NLP to obtain clinical data. In total, 16 (62%) articles presented stand-alone data review algorithms. Other studies (n=9, 35%) showed that the clinical decision support system alternative was also a way of displaying the information obtained for immediate clinical use. CONCLUSIONS: The use of NLP engines can effectively improve clinical decision systems' accuracy, while biphasic tools combining AI algorithms and human criteria may optimize clinical diagnosis and treatment flows. TRIAL REGISTRATION: PROSPERO CRD42022373386; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=373386.

Proof of concept study on early forecasting of antimicrobial resistance in hospitalized patients using machine learning and simple bacterial ecology data.

Antibiotic resistance in bacterial pathogens is a major threat to global health, exacerbated by the misuse of antibiotics. In hospital practice, results of bacterial cultures and antibiograms can take several days. Meanwhile, prescribing an empirical antimicrobial treatment is challenging, as clinicians must balance the antibiotic spectrum against the expected probability of susceptibility. We present here a proof of concept study of a machine learning-based system that predicts the probability of antimicrobial susceptibility and explains the contribution of the different cofactors in hospitalized patients, at four different stages prior to the antibiogram (sampling, direct examination, positive culture, and species identification), using only historical bacterial ecology data that can be easily collected from any laboratory information system (LIS) without GDPR restrictions once the data have been anonymised. A comparative analysis of different state-of-the-art machine learning and probabilistic methods was performed using 44,026 instances over 7 years from the Hôpital Européen Marseille, France. Our results show that multilayer dense neural networks and Bayesian models are suitable for early prediction of antibiotic susceptibility, with AUROCs reaching 0.88 at the positive culture stage and 0.92 at the species identification stage, and even 0.82 and 0.92, respectively, for the least frequent situations. Perspectives and potential clinical applications of the system are discussed.

Vignette-based comparative analysis of ChatGPT and specialist treatment decisions for rheumatic patients: results of the Rheum2Guide study.

BACKGROUND: The complex nature of rheumatic diseases poses considerable challenges for clinicians when developing individualized treatment plans. Large language models (LLMs) such as ChatGPT could enable treatment decision support. OBJECTIVE: To compare treatment plans generated by ChatGPT-3.5 and GPT-4 to those of a clinical rheumatology board (RB). DESIGN/METHODS: Fictional patient vignettes were created and GPT-3.5, GPT-4, and the RB were queried to provide respective first- and second-line treatment plans with underlying justifications. Four rheumatologists from different centers, blinded to the origin of treatment plans, selected the overall preferred treatment concept and assessed treatment plans' safety, EULAR guideline adherence, medical adequacy, overall quality, justification of the treatment plans and their completeness as well as patient vignette difficulty using a 5-point Likert scale. RESULTS: 20 fictional vignettes covering various rheumatic diseases and varying difficulty levels were assembled and a total of 160 ratings were assessed. In 68.8% (110/160) of cases, raters preferred the RB's treatment plans over those generated by GPT-4 (16.3%; 26/160) and GPT-3.5 (15.0%; 24/160). GPT-4's plans were chosen more frequently for first-line treatments compared to GPT-3.5. No significant safety differences were observed between RB and GPT-4's first-line treatment plans. Rheumatologists' plans received significantly higher ratings in guideline adherence, medical appropriateness, completeness and overall quality. Ratings did not correlate with the vignette difficulty. LLM-generated plans were notably longer and more detailed. CONCLUSION: GPT-4 and GPT-3.5 generated safe, high-quality treatment plans for rheumatic diseases, demonstrating promise in clinical decision support. Future research should investigate detailed standardized prompts and the impact of LLM usage on clinical decisions.

Active Learning Pipeline to Identify Candidate Terms for a CDSS Ontology.

Ontology is essential for achieving health information and information technology application interoperability in the biomedical fields and beyond. Traditionally, ontology construction is carried out manually by human domain experts (HDE). Here, we explore an active learning approach to automatically identify candidate terms from publications, with manual verification later as a part of a deep learning model training and learning process. We introduce the overall architecture of the active learning pipeline and present some preliminary results. This work is a critical and complementary component in addition to manually building the ontology, especially during the long-term maintenance stage.

Evaluating Inflammatory Bowel Disease-Related Quality of Life Using an Interpretable Machine Learning Approach: A Multicenter Study in China.

Purpose: Impaired quality of life (QOL) is common in patients with inflammatory bowel disease (IBD). A tool to more quickly identify IBD patients at high risk of impaired QOL improves opportunities for earlier intervention and improves long-term prognosis. The purpose of this study was to use a machine learning (ML) approach to develop risk stratification models for evaluating IBD-related QOL impairments. Patients and Methods: An online questionnaire was used to collect clinical data on 2478 IBD patients from 42 hospitals distributed across 22 provinces in China from September 2021 to May 2022. Eight ML models used to predict the risk of IBD-related QOL impairments were developed and validated. Model performance was evaluated using a set of indexes and the best ML model was explained using a Local Interpretable Model-Agnostic Explanations (LIME) algorithm. Results: The support vector machine (SVM) classifier algorithm-based model outperformed other ML models with an area under the receiver operating characteristic curve (AUC) and an accuracy of 0.80 and 0.71, respectively. The feature importance calculated by the SVM classifier algorithm revealed that glucocorticoid use, anxiety, abdominal pain, sleep disorders, and more severe disease contributed to a higher risk of impaired QOL, while longer disease course and the use of biological agents and immunosuppressants were associated with a lower risk. Conclusion: An ML approach for assessing IBD-related QOL impairments is feasible and effective. This mechanism is a promising tool for gastroenterologists to identify IBD patients at high risk of impaired QOL.

How Trueness of Clinical Decision Support Systems Based on Machine Learning Is Assessed?

The application of machine learning algorithms in clinical decision support systems (CDSS) holds great promise for advancing patient care, yet practical implementation faces significant evaluation challenges. Through a scoping review, we investigate the common definitions of ground truth to collect clinically relevant reference values, as well as the typical metrics and combinations employed for assessing trueness. Our analysis reveals that ground truth definition is mostly not in accordance with the standard ISO expectation and that used combination of metrics does not usually cover all aspects of CDSS trueness, particularly neglecting the negative class perspective.