Hospital antimicrobial stewardship team perceptions and usability of a computerized clinical decision support system.

BACKGROUND: Antimicrobial stewardship (AMS) programs aim to optimize antibiotic use through a panel of interventions. The implementation of computerized clinical decision support systems (CDSSs) offers new opportunities for semiautomated antimicrobial review by AMS teams. This study aimed to evaluate the perceived facilitators, barriers and benefits of end-users related to a commercial CDSS recently implemented in a hospital and to assess its usability. METHODS: A mixed-method approach was used among AMS team members nine months after the implementation of the CDSS in a university hospital in northeastern France. A qualitative analysis based on individual semistructured interviews was conducted to collect end-users' perceptions. A quantitative analysis was performed using the System Usability Scale (SUS). RESULTS: Eleven AMS team members agreed to participate. The qualitative analysis revealed technical, organizational and human barriers and facilitators of CDSS implementation. Effective collaboration with information technology teams was crucial for ensuring the installation and configuration of the software. CDSS adoption by the AMS team required time, human resources, training, adaptation and a clinical leader. Moreover, the CDSS had to be well designed, user-friendly and provide benefits to AMS activities. The quantitative analysis indicated that the CDSS was a "good" system in terms of perceived ease of use (median SUS score: 77.5/100). CONCLUSIONS: This study shows the value of the studied CDSS to support AMS activities. It reveals barriers, facilitators and benefits to the implementation and adoption of the CDSS. These barriers and facilitators could be considered to facilitate the implementation of the software in other hospitals.

Computerized Decision Support Systems Informing Community-Acquired Pneumonia Surveillance, France, 2017-2023.

We show the value of real-time data generated by a computerized decision support system in primary care in strengthening pneumonia surveillance. The system showed a 66% (95% CI 64%-67%) increase in community-acquired pneumonia from 2018 to 2023 for the population of France, 1 month before a national alert was issued.

Appropriateness of intended antibiotic prescribing using clinical case vignettes in primary care, and related factors.

BACKGROUND: Factors associated with the appropriateness of antibiotic prescribing in primary care have been poorly explored. In particular, the impact of computerised decision-support systems (CDSS) remains unknown. OBJECTIVES: We aim at investigating the uptake of CDSS and its association with physician characteristics and professional activity. METHODS: Since May 2022, users of a CDSS for antibiotic prescribing in primary care in France have been invited, when registering, to complete three case vignettes assessing clinical situations frequently encountered in general practice and identified as at risk of antibiotic misuse. Appropriateness of antibiotic prescribing was defined as the rate of answers in line with the current guidelines, computed by individuals and by specific questions. Physician's characteristics associated with individual appropriate antibiotic prescribing (< 50%, 50-75% and > 75% appropriateness) were identified by multivariate ordinal logistic regression. RESULTS: In June 2023, 60,067 physicians had registered on the CDSS. Among the 13,851 physicians who answered all case vignettes, the median individual appropriateness level of antibiotic prescribing was 77.8% [Interquartile range, 66.7%-88.9%], and was < 50% for 1,353 physicians (10%). In the multivariate analysis, physicians' characteristics associated with appropriateness were prior use of the CDSS (OR = 1.71, 95% CI 1.56-1.87), being a general practitioner vs. other specialist (OR = 1.34, 95% CI 1.20-1.49), working in primary care (OR = 1.14, 95% CI 1.02-1.27), mentoring students (OR = 1.12, 95% CI 1.04-1.21) age (OR = 0.69 per 10 years increase, 95% CI 0.67-0.71). CONCLUSION: Individual appropriateness for antibiotic prescribing was high among CDSS users, with a higher rate in young general practitioners, previously using the system. CDSS could improve antibiotic prescribing in primary care.

Individual appropriateness for antibiotic prescribing is high among CDSS users.CDSS use could passively improve antibiotic prescribing in primary care.Factors associated with appropriateness for antibiotic prescribing for primary care diseases are: prior use of CDSS, general practice speciality vs. other specialities, younger age and mentoring of students.

Artificial intelligence predictive analytics in heart failure: results of the pilot phase of a pragmatic randomized clinical trial.

OBJECTIVES: We conducted an implementation planning process during the pilot phase of a pragmatic trial, which tests an intervention guided by artificial intelligence (AI) analytics sourced from noninvasive monitoring data in heart failure patients (LINK-HF2). MATERIALS AND METHODS: A mixed-method analysis was conducted at 2 pilot sites. Interviews were conducted with 12 of 27 enrolled patients and with 13 participating clinicians. iPARIHS constructs were used for interview construction to identify workflow, communication patterns, and clinician's beliefs. Interviews were transcribed and analyzed using inductive coding protocols to identify key themes. Behavioral response data from the AI-generated notifications were collected. RESULTS: Clinicians responded to notifications within 24 hours in 95% of instances, with 26.7% resulting in clinical action. Four implementation themes emerged: (1) High anticipatory expectations for reliable patient communications, reduced patient burden, and less proactive provider monitoring. (2) The AI notifications required a differential and tailored balance of trust and action advice related to role. (3) Clinic experience with other home-based programs influenced utilization. (4) Responding to notifications involved significant effort, including electronic health record (EHR) review, patient contact, and consultation with other clinicians. DISCUSSION: Clinician's use of AI data is a function of beliefs regarding the trustworthiness and usefulness of the data, the degree of autonomy in professional roles, and the cognitive effort involved. CONCLUSION: The implementation planning analysis guided development of strategies that addressed communication technology, patient education, and EHR integration to reduce clinician and patient burden in the subsequent main randomized phase of the trial. Our results provide important insights into the unique implications of implementing AI analytics into clinical workflow.

Development, Design and Utilization of a CDSS for Refeeding Syndrome in Real Life Inpatient Care-A Feasibility Study.

BACKGROUND: The refeeding syndrome (RFS) is an oftentimes-unrecognized complication of reintroducing nutrition in malnourished patients that can lead to fatal cardiovascular failure. We hypothesized that a clinical decision support system (CDSS) can improve RFS recognition and management. METHODS: We developed an algorithm from current diagnostic criteria for RFS detection, tested the algorithm on a retrospective dataset and combined the final algorithm with therapy and referral recommendations in a knowledge-based CDSS. The CDSS integration into clinical practice was prospectively investigated for six months. RESULTS: The utilization of the RFS-CDSS lead to RFS diagnosis in 13 out of 21 detected cases (62%). It improved patient-related care and documentation, e.g., RFS-specific coding (E87.7), increased from once coded in 30 month in the retrospective cohort to four times in six months in the prospective cohort and doubled the rate of nutrition referrals in true positive patients (retrospective referrals in true positive patients 33% vs. prospective referrals in true positive patients 71%). CONCLUSION: CDSS-facilitated RFS diagnosis is possible and improves RFS recognition. This effect and its impact on patient-related outcomes needs to be further investigated in a large randomized-controlled trial.

Electronic alerts for ambulatory patients with atrial fibrillation not prescribed anticoagulation: A randomized, controlled trial (AF-ALERT2).

BACKGROUND: Despite widely available risk stratification tools, safe and effective anticoagulants, and guideline recommendations, anticoagulation for stroke prevention in atrial fibrillation (AF) is under-prescribed in ambulatory patients. To assess the impact of alert-based computerized decision support (CDS) on anticoagulation prescription in ambulatory patients with AF and high-risk for stroke, we conducted this randomized controlled trial. METHODS: Patients with AF and CHA2DS2-VASc score ≥ 2 who were not prescribed anticoagulation and had a clinic visit at Brigham and Women's Hospital were enrolled. Patients were randomly allocated, according to Attending Physician of record, to intervention (alert-based CDS) versus control (no notification). The primary efficacy outcome was the frequency of anticoagulant prescription. RESULTS: The CDS tool assigned 395 and 403 patients to the alert and control groups, respectively. Alert patients were more likely to be prescribed anticoagulation within 48 h of the clinic visit (15.4 % vs. 7.7 %, p < 0.001) and at 90 days (17.2 % vs. 9.9 %, p < 0.01). Direct oral anticoagulants were the predominantly prescribed form of anticoagulation. No significant differences were observed in stroke, TIA, or systemic embolic events (0 % vs. 0.8 %, p = 0.09), symptomatic VTE (0.5 % vs. 1 %, p = 0.43), all-cause mortality (2 % vs. 0.7 %, p = 0.12), or major adverse cardiovascular events (2.8 % vs. 2.5 %, p = 0.79) at 90 days. CONCLUSIONS: An alert-based CDS strategy increased a primary efficacy outcome of anticoagulation in clinic patients with AF and high-risk for stroke who were not receiving anticoagulation at the time of the office visit. The study was likely underpowered to assess an impact on clinical outcomes. TRIAL REGISTRATION: ClinicalTrials.gov Identifier- NCT02958943.

Stakeholder Perspectives of Clinical Artificial Intelligence Implementation: Systematic Review of Qualitative Evidence.

BACKGROUND: The rhetoric surrounding clinical artificial intelligence (AI) often exaggerates its effect on real-world care. Limited understanding of the factors that influence its implementation can perpetuate this. OBJECTIVE: In this qualitative systematic review, we aimed to identify key stakeholders, consolidate their perspectives on clinical AI implementation, and characterize the evidence gaps that future qualitative research should target. METHODS: Ovid-MEDLINE, EBSCO-CINAHL, ACM Digital Library, Science Citation Index-Web of Science, and Scopus were searched for primary qualitative studies on individuals' perspectives on any application of clinical AI worldwide (January 2014-April 2021). The definition of clinical AI includes both rule-based and machine learning-enabled or non-rule-based decision support tools. The language of the reports was not an exclusion criterion. Two independent reviewers performed title, abstract, and full-text screening with a third arbiter of disagreement. Two reviewers assigned the Joanna Briggs Institute 10-point checklist for qualitative research scores for each study. A single reviewer extracted free-text data relevant to clinical AI implementation, noting the stakeholders contributing to each excerpt. The best-fit framework synthesis used the Nonadoption, Abandonment, Scale-up, Spread, and Sustainability (NASSS) framework. To validate the data and improve accessibility, coauthors representing each emergent stakeholder group codeveloped summaries of the factors most relevant to their respective groups. RESULTS: The initial search yielded 4437 deduplicated articles, with 111 (2.5%) eligible for inclusion (median Joanna Briggs Institute 10-point checklist for qualitative research score, 8/10). Five distinct stakeholder groups emerged from the data: health care professionals (HCPs), patients, carers and other members of the public, developers, health care managers and leaders, and regulators or policy makers, contributing 1204 (70%), 196 (11.4%), 133 (7.7%), 129 (7.5%), and 59 (3.4%) of 1721 eligible excerpts, respectively. All stakeholder groups independently identified a breadth of implementation factors, with each producing data that were mapped between 17 and 24 of the 27 adapted Nonadoption, Abandonment, Scale-up, Spread, and Sustainability subdomains. Most of the factors that stakeholders found influential in the implementation of rule-based clinical AI also applied to non-rule-based clinical AI, with the exception of intellectual property, regulation, and sociocultural attitudes. CONCLUSIONS: Clinical AI implementation is influenced by many interdependent factors, which are in turn influenced by at least 5 distinct stakeholder groups. This implies that effective research and practice of clinical AI implementation should consider multiple stakeholder perspectives. The current underrepresentation of perspectives from stakeholders other than HCPs in the literature may limit the anticipation and management of the factors that influence successful clinical AI implementation. Future research should not only widen the representation of tools and contexts in qualitative research but also specifically investigate the perspectives of all stakeholder HCPs and emerging aspects of non-rule-based clinical AI implementation. TRIAL REGISTRATION: PROSPERO (International Prospective Register of Systematic Reviews) CRD42021256005; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=256005. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): RR2-10.2196/33145.

Acceptance, Barriers, and Facilitators to Implementing Artificial Intelligence-Based Decision Support Systems in Emergency Departments: Quantitative and Qualitative Evaluation.

BACKGROUND: Despite the increasing availability of clinical decision support systems (CDSSs) and rising expectation for CDSSs based on artificial intelligence (AI), little is known about the acceptance of AI-based CDSS by physicians and its barriers and facilitators in emergency care settings. OBJECTIVE: We aimed to evaluate the acceptance, barriers, and facilitators to implementing AI-based CDSSs in the emergency care setting through the opinions of physicians on our newly developed, real-time AI-based CDSS, which alerts ED physicians by predicting aortic dissection based on numeric and text information from medical charts, by using the Unified Theory of Acceptance and Use of Technology (UTAUT; for quantitative evaluation) and the Consolidated Framework for Implementation Research (CFIR; for qualitative evaluation) frameworks. METHODS: This mixed methods study was performed from March to April 2021. Transitional year residents (n=6), emergency medicine residents (n=5), and emergency physicians (n=3) from two community, tertiary care hospitals in Japan were included. We first developed a real-time CDSS for predicting aortic dissection based on numeric and text information from medical charts (eg, chief complaints, medical history, vital signs) with natural language processing. This system was deployed on the internet, and the participants used the system with clinical vignettes of model cases. Participants were then involved in a mixed methods evaluation consisting of a UTAUT-based questionnaire with a 5-point Likert scale (quantitative) and a CFIR-based semistructured interview (qualitative). Cronbach α was calculated as a reliability estimate for UTAUT subconstructs. Interviews were sampled, transcribed, and analyzed using the MaxQDA software. The framework analysis approach was used during the study to determine the relevance of the CFIR constructs. RESULTS: All 14 participants completed the questionnaires and interviews. Quantitative analysis revealed generally positive responses for user acceptance with all scores above the neutral score of 3.0. In addition, the mixed methods analysis identified two significant barriers (System Performance, Compatibility) and two major facilitators (Evidence Strength, Design Quality) for implementation of AI-based CDSSs in emergency care settings. CONCLUSIONS: Our mixed methods evaluation based on theoretically grounded frameworks revealed the acceptance, barriers, and facilitators of implementation of AI-based CDSS. Although the concern of system failure and overtrusting of the system could be barriers to implementation, the locality of the system and designing an intuitive user interface could likely facilitate the use of optimal AI-based CDSS. Alleviating and resolving these factors should be key to achieving good user acceptance of AI-based CDSS.

Surgeons' risk perception in ASD surgery: The value of objective risk assessment on decision making and patient counselling.

BACKGROUND: Surgeons often rely on their intuition, experience and published data for surgical decision making and informed consent. Literature provides average values that do not allow for individualized assessments. Accurate validated machine learning (ML) risk calculators for adult spinal deformity (ASD) patients, based on 10 year multicentric prospective data, are currently available. The objective of this study is to assess surgeon ASD risk perception and compare it to validated risk calculator estimates. METHODS: Nine ASD complete (demographics, HRQL, radiology, surgical plan) preoperative cases were distributed online to 100 surgeons from 22 countries. Surgeons were asked to determine the risk of major complications and reoperations at 72 h, 90 d and 2 years postop, using a 0-100% risk scale. The same preoperative parameters circulated to surgeons were used to obtain ML risk calculator estimates. Concordance between surgeons' responses was analyzed using intraclass correlation coefficients (ICC) (poor < 0.5/excellent > 0.85). Distance between surgeons' and risk calculator predictions was assessed using the mean index of agreement (MIA) (poor < 0.5/excellent > 0.85). RESULTS: Thirty-nine surgeons (74.4% with > 10 years' experience), from 12 countries answered the survey. Surgeons' risk perception concordance was very low and heterogeneous. ICC ranged from 0.104 (reintervention risk at 72 h) to 0.316 (reintervention risk at 2 years). Distance between calculator and surgeon prediction was very large. MIA ranged from 0.122 to 0.416. Surgeons tended to overestimate the risk of major complications and reintervention in the first 72 h and underestimated the same risks at 2 years postop. CONCLUSIONS: This study shows that expert surgeon ASD risk perception is heterogeneous and highly discordant. Available validated ML ASD risk calculators can enable surgeons to provide more accurate and objective prognosis to adjust patient expectations, in real time, at the point of care.

Effect of Having, but Not Consulting, a Computerized Diagnostic Aid.

Previous research has described physicians' reluctance to use computerized diagnostic aids (CDAs) but has never experimentally examined the effects of not consulting an aid that was readily available. Experiment 1. Participants read about a diagnosis made either by a physician or an auto mechanic (to control for perceived expertise). Half read that a CDA was available but never actually consulted; no mention of a CDA was made for the remaining half. For the physician, failure to consult the CDA had no significant effect on competence ratings for either the positive or negative outcome. For the auto mechanic, failure to consult the CDA actually increased competence ratings following a negative but not a positive outcome. Negligence judgments were greater for the mechanic than for the physician overall. Experiment 2. Using only a negative outcome, we included 2 different reasons for not consulting the aid and provided accuracy information highlighting the superiority of the CDA over the physician. In neither condition was the physician rated lower than when no aid was mentioned. Ratings were lower when the physician did not trust the CDA and, surprisingly, higher when the physician believed he or she already knew what the CDA would say. Finally, consistent with our previous research, ratings were also high when the physician consulted and then followed the advice of a CDA and low when the CDA was consulted but ignored. Individual differences in numeracy did not qualify these results. Implications for the literature on algorithm aversion and clinical practice are discussed.

A framework for validating AI in precision medicine: considerations from the European ITFoC consortium.

BACKGROUND: Artificial intelligence (AI) has the potential to transform our healthcare systems significantly. New AI technologies based on machine learning approaches should play a key role in clinical decision-making in the future. However, their implementation in health care settings remains limited, mostly due to a lack of robust validation procedures. There is a need to develop reliable assessment frameworks for the clinical validation of AI. We present here an approach for assessing AI for predicting treatment response in triple-negative breast cancer (TNBC), using real-world data and molecular -omics data from clinical data warehouses and biobanks. METHODS: The European "ITFoC (Information Technology for the Future Of Cancer)" consortium designed a framework for the clinical validation of AI technologies for predicting treatment response in oncology. RESULTS: This framework is based on seven key steps specifying: (1) the intended use of AI, (2) the target population, (3) the timing of AI evaluation, (4) the datasets used for evaluation, (5) the procedures used for ensuring data safety (including data quality, privacy and security), (6) the metrics used for measuring performance, and (7) the procedures used to ensure that the AI is explainable. This framework forms the basis of a validation platform that we are building for the "ITFoC Challenge". This community-wide competition will make it possible to assess and compare AI algorithms for predicting the response to TNBC treatments with external real-world datasets. CONCLUSIONS: The predictive performance and safety of AI technologies must be assessed in a robust, unbiased and transparent manner before their implementation in healthcare settings. We believe that the consideration of the ITFoC consortium will contribute to the safe transfer and implementation of AI in clinical settings, in the context of precision oncology and personalized care.

Decision support for computed tomography in the emergency department: a multicenter cluster-randomized controlled trial.

OBJECTIVES: Clinical decision support may facilitate evidence-based imaging, but most studies to date examining the impact of decision support have used non-randomized designs which limit the conclusions that can be drawn from them. This randomized trial examines if decision support can reduce computed tomography (CT) utilization for patients with mild traumatic brain injuries and suspected pulmonary embolism in the emergency department. This study was funded by a competitive public research grant and registered on ClinicalTrials.gov (NCT02410941). METHODS: Emergency physicians at five urban sites were assigned to voluntary decision support for CT imaging of patients with either head injuries or suspected pulmonary embolism using a cluster-randomized design over a 1-year intervention period. The co-primary outcomes were CT head and CT pulmonary angiography utilization. CT pulmonary angiography diagnostic yield (proportion of studies diagnostic for acute pulmonary embolism) was a secondary outcome. RESULTS: A total of 225 physicians were randomized and studied over a 2-year baseline and 1-year intervention period. Physicians interacted with the decision support in 38.0% and 45.0% of eligible head injury and suspected pulmonary embolism cases, respectively. A mixed effects logistic regression model demonstrated no significant impact of decision support on head CT utilization (OR 0.93, 95% CI 0.79-1.10, p = 0.31), CT pulmonary angiography utilization (OR 0.98, 95% CI 0.88-1.11, p = 0.74) or diagnostic yield (OR 1.23, 95% CI 0.96-1.65, p = 0.10). However, overall CT pulmonary diagnostic yield (17.7%) was almost three times higher than that reported by a recent large US study, suggesting that selective imaging was already being employed. CONCLUSION: Voluntary decision support addressing many commonly cited barriers to evidence-based imaging did not significantly reduce CT utilization or improve diagnostic yield but was limited by low rates of participation and high baseline rates of selective imaging. Demonstrating value to clinicians through interventions that improve workflow is likely necessary to meaningfully change imaging practices.

RéSUMé: OBJECTIFS: Le soutien à la décision clinique peut faciliter l'imagerie fondée sur des données probantes, mais la plupart des études à ce jour examinant l'impact du soutien à la décision ont utilisé des modèles non randomisés qui limitent les conclusions qui peuvent en être tirées. Cet essai randomisé examine si l'aide à la décision peut réduire l'utilisation de la tomodensitométrie chez les patients présentant des lésions cérébrales traumatiques légères et une embolie pulmonaire présumée au service des urgences. Cette étude a été financée par une subvention de recherche publique compétitive et enregistrée sur ClinicalTrials.gov (NCT02410941). MéTHODES: Les médecins urgentistes de cinq sites urbains ont été assignés à une aide à la décision volontaire pour l'imagerie par tomodensitométrie des patients présentant soit un traumatisme crânien, soit une suspicion d'embolie pulmonaire, selon une conception randomisée en grappes sur une période d'intervention d'un an. Les résultats co-primaires étaient l'utilisation de la tomodensitométrie de la tête et de la tomodensitométrie par angiographie pulmonaire. Le rendement diagnostique de l'angiographie pulmonaire par TDM (proportion d'études diagnostiquant une embolie pulmonaire aiguë) était un résultat secondaire. RéSULTATS: Au total, 225 médecins ont été randomisés et étudiés au cours d'une période de référence de deux ans et d'une période d'intervention d'un an. Les médecins ont interagi avec l'aide à la décision dans 38,0 % et 45,0 % des cas admissibles de blessure à la tête et d'embolie pulmonaire soupçonnée, respectivement. Un modèle de régression logistique à effets mixtes n'a démontré aucun impact significatif de l'aide à la décision sur l'utilisation de la tomodensitométrie de la tête (OR 0,93, IC 95 % 0,79-1,10, p = 0,31), l'utilisation de l'angiographie pulmonaire par tomodensitométrie (OR 0,98, IC 95 % 0,88-1,11, p = 0,74) ou le rendement diagnostique (OR 1,23, IC 95 % 0,96-1,65, p = 0,10). Toutefois, le rendement global du diagnostic pulmonaire par TDM (17,7 %) était près de trois fois supérieur à celui rapporté par une étude récente aux États-Unis, ce qui laisse supposer que l'imagerie sélective était déjà utilisée. CONCLUSIONS: L'aide à la décision volontaire visant à éliminer de nombreux obstacles fréquemment cités à l'imagerie fondée sur des données probantes n'a pas réduit de façon significative l'utilisation de la tomodensitométrie ni amélioré le rendement diagnostique, mais a été limitée par de faibles taux de participation et des taux de base élevés d'imagerie sélective. La démonstration de la valeur pour les cliniciens par des interventions qui améliorent le flux de travail est probablement nécessaire pour changer de manière significative les pratiques d'imagerie.

Optimizing pharmacists' detection of prescribing errors: Comparison of on-ward and central pharmacy services.

WHAT IS KNOWN AND OBJECTIVE: Prescribing errors are the leading cause of adverse drug events in hospitalized patients. Pharmaceutical validation, defined as the review of drug orders by a pharmacist, associated with clinical decision support (CDS) systems, significantly reduces these errors and adverse drug events. In Belgium, because clinical pharmacy services have limited public financial support, most pharmaceutical validations are performed at the central pharmacy instead of on-ward, by hospital pharmacists doing dispensing activities. In that context, we aimed at evaluating whether the strategy of CDS-guided central validation was the most appropriate method to improve the quality and safety of medicines' use compared to an on-ward pharmaceutical validation. METHODS: Our retrospective observational study was conducted in a Belgian tertiary care hospital, in 2018-2019. Data were extracted from our validation software and pharmacists' charts. The outcomes of the study were the number of pharmaceutical interventions due to the detection of prescribing errors, reasons for interventions, their acceptance rate and their potential clinical impact (according to two blinded experts) in the central pharmacy and on-ward validation groups. RESULTS AND DISCUSSION: Despite the use of the same CDS, a pharmaceutical intervention following the detection of a prescribing error was made for 2.9% (20/698) of central group patients and 13.3% (93/701) of on-ward patients (χ2  = 49.97, p < 0.001). Interventions made at the central pharmacy (n = 20) mostly relied on CDS-alerts (i.e. drug-drug interaction [25%] or overdosing [20%]) while interventions made on-ward (n = 93) were also for pharmacotherapy optimization (i.e. no valid indication [25%] or inappropriate drug's choice [11%]). The on-ward validation group showed a higher acceptance rate compared to the central group (84% and 65%, respectively [Fisher's test, p = 0.053]). Proportions of interventions with significant or very significant clinical impact were similar between the two groups but as fewer interventions were made centrally, a significant proportion of errors were probably not detected by the central validation. WHAT IS NEW AND CONCLUSION: On-ward pharmaceutical validation leads to a higher rate of prescribing error detection. Pharmaceutical interventions made by on-ward pharmacists are also better accepted and more relevant, going further than CDS-alerts.

Consistency of decision support software-integrated telephone triage and associated factors: a systematic review.

BACKGROUND: In the recent decades, the use of computerized decision support software (CDSS)-integrated telephone triage (TT) has become an important tool for managing rising healthcare demands and overcrowding in the emergency department. Though these services have generally been shown to be effective, large gaps in the literature exist with regards to the overall quality of these systems. In the current systematic review, we aim to document the consistency of decisions that are generated in CDSS-integrated TT. Furthermore, we also seek to map those factors in the literature that have been identified to have an impact on the consistency of generated triage decisions. METHODS: As part of the TRANS-SENIOR international training and research network, a systematic review of the literature was conducted in November 2019. PubMed, Web of Science, CENTRAL, and the CINAHL database were searched. Quantitative articles including a CDSS component and addressing consistency of triage decisions and/or factors associated with triage decisions were eligible for inclusion in the current review. Studies exploring the use of other types of digital support systems for triage (i.e. web chat, video conferencing) were excluded. Quality appraisal of included studies were performed independently by two authors using the Methodological Index for Non-Randomized Studies. RESULTS: From a total of 1551 records that were identified, 39 full-texts were assessed for eligibility and seven studies were included in the review. All of the studies (n = 7) identified as part of our search were observational and were based on nurse-led telephone triage. Scientific efforts investigating our first aim was very limited. In total, two articles were found to investigate the consistency of decisions that are generated in CDSS-integrated TT. Research efforts were targeted largely towards the second aim of our study-all of the included articles reported factors related to the operator- (n = 6), patient- (n = 1), and/or CDSS-integrated (n = 2) characteristics to have an influence on the consistency of CDSS-integrated TT decisions. CONCLUSION: To date, some efforts have been made to better understand how the use of CDSS-integrated TT systems may vary across settings. In general, however, the evidence-base surrounding this field of literature is largely inconclusive. Further evaluations must be prompted to better understand this area of research. PROTOCOL REGISTRATION: The protocol for this study is registered in the PROSPERO database (registration number: CRD42020146323).

Computerized Decision Support Systems for Nursing Homes: A Scoping Review.

OBJECTIVES: To summarize the research literature describing the outcomes of computerized decision support systems (CDSSs) implemented in nursing homes (NHs). DESIGN: Scoping review. METHODS: Search of relevant articles published in the English language between January 1, 2000, and February 29, 2020, in the Medline database. The quality of the selected studies was assessed according to PRISMA guidelines and the Mixed Method Appraisal Tool. RESULTS: From 1828 articles retrieved, 24 studies were selected for review, among which only 6 were randomized controlled trials. Although clinical outcomes are seldom studied, some studies show that CDSSs have the potential to decrease pressure ulcer incidence and malnutrition prevalence. Improvement of process outcomes such as increased compliance with practice guidelines, better documentation of nursing assessment, improved teamwork and communication, and cost saving, also are reported. CONCLUSIONS AND IMPLICATIONS: Overall, the use of CDSSs in NHs may be effective to improve patient clinical outcomes and health care delivery; however, most of the retrieved studies were observational studies, which significantly weakens the evidence. High-quality studies are needed to investigate CDSS effects and limitations in NHs.

A computerized decision support system did not improve personalization of exercise-based cardiac rehabilitation according to latest recommendations.

AIMS: Recent studies showed that exercise-based cardiac rehabilitation (ECR) programmes are often not personalized to individual patient characteristics according to latest recommendations. This study investigates whether a computerized decision support (CDS) system based on latest recommendations and guidelines can improve personalization of ECR prescriptions. Pseudo-randomized intervention study. METHODS AND RESULTS: Among participating Dutch cardiac rehabilitation centres, ECR programme characteristics of consecutive patients were recorded during 1 year. CDS was used during a randomly assigned 4-month period within this year. Primary outcome was concordance to latest recommendations in three phases (before, during, and after CDS) for 12 ECR programme characteristics. Secondary outcome was variation in training characteristics. We recruited ten Dutch CR centres and enrolled 2258 patients to the study. Overall concordance of ECR prescriptions was 59.9% in Phase 1, 62.1% in Phase 2 (P = 0.82), and 59.9% in Phase 3 (P = 0.56). Concordance varied from 0.0% to 99.9% for the 12 ECR characteristics. There was significant between-centre variation for most training characteristics in Phases 1 and 2. In Phase 3, there was only a significant variation for aerobic and resistance training intensity (P = 0.01), aerobic training volume (P < 0.01), and the number of strengthening exercises but no longer for the other characteristics. Aerobic training volume was often below recommended (28.2%) and declined during the study. CONCLUSION: CDS did not substantially improve concordance with ECR prescriptions. As aerobic training volume was often lower than recommended and reduced during the study, a lack of institutional resources might be an important barrier in personalizing ECR prescriptions.

Real-time imputation of missing predictor values improved the application of prediction models in daily practice.

OBJECTIVES: In clinical practice, many prediction models cannot be used when predictor values are missing. We, therefore, propose and evaluate methods for real-time imputation. STUDY DESIGN AND SETTING: We describe (i) mean imputation (where missing values are replaced by the sample mean), (ii) joint modeling imputation (JMI, where we use a multivariate normal approximation to generate patient-specific imputations), and (iii) conditional modeling imputation (CMI, where a multivariable imputation model is derived for each predictor from a population). We compared these methods in a case study evaluating the root mean squared error (RMSE) and coverage of the 95% confidence intervals (i.e., the proportion of confidence intervals that contain the true predictor value) of imputed predictor values. RESULTS: -RMSE was lowest when adopting JMI or CMI, although imputation of individual predictors did not always lead to substantial improvements as compared to mean imputation. JMI and CMI appeared particularly useful when the values of multiple predictors of the model were missing. Coverage reached the nominal level (i.e., 95%) for both CMI and JMI. CONCLUSION: Multiple imputations using either CMI or JMI is recommended when dealing with missing predictor values in real-time settings.

Real-time imputation of missing predictor values in clinical practice.

Aims: Use of prediction models is widely recommended by clinical guidelines, but usually requires complete information on all predictors, which is not always available in daily practice. We aim to describe two methods for real-time handling of missing predictor values when using prediction models in practice. Methods and results: We compare the widely used method of mean imputation (M-imp) to a method that personalizes the imputations by taking advantage of the observed patient characteristics. These characteristics may include both prediction model variables and other characteristics (auxiliary variables). The method was implemented using imputation from a joint multivariate normal model of the patient characteristics (joint modelling imputation; JMI). Data from two different cardiovascular cohorts with cardiovascular predictors and outcome were used to evaluate the real-time imputation methods. We quantified the prediction model's overall performance [mean squared error (MSE) of linear predictor], discrimination (c-index), calibration (intercept and slope), and net benefit (decision curve analysis). When compared with mean imputation, JMI substantially improved the MSE (0.10 vs. 0.13), c-index (0.70 vs. 0.68), and calibration (calibration-in-the-large: 0.04 vs. 0.06; calibration slope: 1.01 vs. 0.92), especially when incorporating auxiliary variables. When the imputation method was based on an external cohort, calibration deteriorated, but discrimination remained similar. Conclusions: We recommend JMI with auxiliary variables for real-time imputation of missing values, and to update imputation models when implementing them in new settings or (sub)populations.

Patients with perceived high-bleeding risk and computerized decision support for stroke prevention in atrial fibrillation: an AF-ALERT substudy : Piazza: outcomes of high-bleeding risk AF patients.

A perceived increased risk of bleeding is one of the most frequent reasons for withholding anticoagulation for stroke prevention in atrial fibrillation (AF). We previously conducted a randomized controlled trial of alert-based computerized decision support to increase prescription of anticoagulation in hospitalized patients with AF. To determine the clinical characteristics and outcomes of those patients whose inpatient health care providers received a computer alert, we analyzed all 248 patients in the alert group. Patients for whom providers elected to omit anticoagulation and provided a rationale of a perceived high risk of bleeding were compared with those who were not designated as high-risk. Perceived high risk of bleeding was the most common reason (77%) for omitting anticoagulation. Median HAS-BLED scores were similar in these patients compared with those who were not deemed to have an increased bleeding risk (3 vs. 3, p = 0.44). Despite being categorized as too high-risk for bleeding to receive antithrombotic therapy at the time of the alert, nearly 12% of these patients were ultimately prescribed anticoagulation by 90 days. The frequency of major and clinically-relevant non-major bleeding was similar between the groups. The frequency of death, myocardial infarction, stroke, or systemic embolic event was similar in both groups (10.2% vs. 12.4%, p = 0.59). In conclusion, a perceived high risk of bleeding was the most common reason for omission of anticoagulation in patients with AF after a computerized alert. Perceived high risk of bleeding was not reflected in a higher HAS-BLED score.Clinical trial registration: ClinicalTrials.gov Identifier: NCT02339493 https://clinicaltrials.gov/ct2/show/NCT02339493 In a randomized controlled trial of computerized decision support to increase prescription of antithrombotic therapy in hospitalized patients with atrial fibrillation (AF), a perceived high risk of bleeding was the most common reason (77%) for omitting antithrombotic therapy after an on-screen alert. Median HAS-BLED scores were similar in these patients compared with those who were not deemed to have an increased bleeding risk (3 vs. 3, p = 0.44). Despite being categorized as too high-risk for bleeding to receive antithrombotic therapy for stroke prevention at the time of the alert, nearly 12% of these patients were ultimately prescribed anticoagulation over the ensuing 90 days.

From sensitization to adoption? A qualitative study of the implementation of a digitally supported intervention for clinical decision making in polypharmacy.

OBJECTIVE: Formative evaluation of the implementation process for a digitally supported intervention in polypharmacy in Germany. Qualitative research was conducted within a cluster randomized controlled trial (C-RCT). It focused on understanding how the intervention influences behavior-related outcomes in the prescription and medication review process. METHODS/SETTING: Twenty-seven general practitioners (GPs) were included in the study in the two groups of the C-RCT, the intervention, and the wait list control group. Behavior-related outcomes were investigated using three-step data analysis (content analytic approach, documentary method, and design of a model of implementation pathways). RESULTS: Content analysis showed that physicians were more intensely aware of polypharmacy-related risks, described positive learning effects of the digital technology on their prescribing behavior, and perceived a change in communication with patients and pharmacists. Conversely, they felt uncertain about their own responsibility when prescribing. Three main dimensions were discovered which influenced adoption behavior: (1) the physicians' interpretation of the relevance of pharmaceutical knowledge provided by the intervention in changing decision-making situations in polypharmacy; (2) their medical code of ethics for clinical decision making in the context of progressing digitalization; and (3) their concepts of evidence-based medicine on the basis of professional experiences with polypharmacy in primary care settings. In our sample, both simple and complex pathways from sensitization to adoption were observed. The resulting model on adoption behavior includes a paradigmatic description of different pathways and a visualization of different observed levels and applied methodological approaches. We assumed that the GP habitus can weaken or strengthen interventional effects towards intervention uptake. This formative evaluation strategy is beneficial for the identification of behavior-related implementation barriers and facilitators. CONCLUSION: Our analyses of the adoption behavior of a digitally supported intervention in polypharmacy revealed both simple and complex pathways from awareness to adoption, which may impact the implementation of the intervention and therefore, its effectiveness. Future consideration of adoption behavior in the planning and evaluation of digitally supported interventions may enhance uptake and support the interpretation of effects. TRIAL REGISTRATION: NCT03430336 , 12 February 2018.