An innovative artificial intelligence-based method to compress complex models into explainable, model-agnostic and reduced decision support systems with application to healthcare (NEAR).

BACKGROUND AND OBJECTIVE: In everyday clinical practice, medical decision is currently based on clinical guidelines which are often static and rigid, and do not account for population variability, while individualized, patient-oriented decision and/or treatment are the paradigm change necessary to enter into the era of precision medicine. Most of the limitations of a guideline-based system could be overcome through the adoption of Clinical Decision Support Systems (CDSSs) based on Artificial Intelligence (AI) algorithms. However, the black-box nature of AI algorithms has hampered a large adoption of AI-based CDSSs in clinical practice. In this study, an innovative AI-based method to compress AI-based prediction models into explainable, model-agnostic, and reduced decision support systems (NEAR) with application to healthcare is presented and validated. METHODS: NEAR is based on the Shapley Additive Explanations framework and can be applied to complex input models to obtain the contributions of each input feature to the output. Technically, the simplified NEAR models approximate contributions from input features using a custom library and merge them to determine the final output. Finally, NEAR estimates the confidence error associated with the single input feature contributing to the final score, making the result more interpretable. Here, NEAR is evaluated on a clinical real-world use case, the mortality prediction in patients who experienced Acute Coronary Syndrome (ACS), applying three different Machine Learning/Deep Learning models as implementation examples. RESULTS: NEAR, when applied to the ACS use case, exhibits performances like the ones of the AI-based model from which it is derived, as in the case of the Adaptive Boosting classifier, whose Area Under the Curve is not statistically different from the NEAR one, even the model's simplification. Moreover, NEAR comes with intrinsic explainability and modularity, as it can be tested on the developed web application platform (https://neardashboard.pythonanywhere.com/). CONCLUSIONS: An explainable and reliable CDSS tailored to single-patient analysis has been developed. The proposed AI-based system has the potential to be used alongside the clinical guidelines currently employed in the medical setting making them more personalized and dynamic and assisting doctors in taking their everyday clinical decisions.

Predicting the future: introducing business analytics to endoscopy units

Background and aims: currently, most endoscopy software only provides limited statistics of past procedures, while none allows patterns to be extrapolated. To overcome this need, the authors applied business analytic models to predict future demand and the need for endoscopists in a tertiary hospital Endoscopy Unit. Methods: a query to the endoscopy database was performed to retrieve demand from 2015 to 2021. The graphical inspection allowed inferring of trends and seasonality, perceiving the impact of the COVID-19 pandemic, and selecting the best forecasting models. Considering COVID-19’s impact in the second quarter of 2020, data for esophagogastroduodenoscopy (EGD) and colonoscopy was estimated using linear regression of historical data. The actual demand in the first two quarters of 2022 was used to validate the models. Results: during the study period, 53,886 procedures were requested. The best forecasting models were: a) simple seasonal exponential smoothing for EGD, colonoscopy and percutaneous endoscopic gastrostomy (PEG); b) double exponential smoothing for capsule endoscopy and deep enteroscopy; and c) simple exponential smoothing for endoscopic retrograde cholangiopancreatography (ERCP) and endoscopic ultrasound (EUS). The mean average percentage error ranged from 6.1 % (EGD) to 33.5 % (deep enteroscopy). Overall, 8,788 procedures were predicted for 2022. The actual demand in the first two quarters of 2022 was within the predicted range. Considering the usual time allocation for each technique, 3.2 full-time equivalent endoscopists (40 hours-dedication to endoscopy) will be required to perform all procedures in 2022. Conclusions: the incorporation of business analytics into the endoscopy software and clinical practice may enhance resource allocation, improving patient-focused decision-making and healthcare quality (AU)

Validation of Pediatric Emergency Care Applied Research Network (PECARN) rule in children with minor head trauma.

The Pediatric Emergency Care Applied Research Network (PECARN) rule is commonly used for predicting the need for computed tomography (CT) scans in children with mild head trauma. The objective of this study was to validate the PECARN rule in Korean children presenting to the pediatric emergency department (PED) with head trauma. This study was a multicenter, retrospective, observational cohort study in two teaching PEDs in Korea between August 2015 and August 2016. In this observational study, 448 patients who visited PEDs were included in the final analysis. Risk stratification was performed with clinical decision support software based on the PECARN rule, and decisions to perform CT scans were subsequently made. Patients were followed-up by phone call between 7 days and 90 days after discharge from the PED. The sensitivity and specificity were analyzed. The sensitivity was 100% for all age groups, and no cases of clinically important traumatic brain injury (ciTBI) were identified in the very-low-risk group. CT scans were performed for 14.7% of patients in this study and for 33.8% in the original PECARN study. The PECARN rule successfully identified low-risk patients, and no cases of ciTBI were missed despite the reduced proportion of patients undergoing CT scans.

Web Services and Cloud Computing in Pediatric Care.

Electronic health record (EHR) systems do not uniformly implement pediatric-supportive functionalities. One method of adding these capabilities across EHR platforms is to integrate Web services and Web applications that may perform decision support and store data in the cloud when the EHR platform is able to integrate Web services. Specific examples of these services are described, such as immunization clinical decision support services, consumer health resources, and bilirubin nomograms. Health care providers, EHR vendors, and developers share responsibilities in the appropriate development, integration, and use of Web services and Web applications as they relate to best practices in the areas of data security and confidentiality, technical availability, audit trails, terminology and messaging standards, compliance with the Health Insurance Portability and Accountability Act, testing, usability, and other considerations. It is desirable for health care providers to have knowledge of Web services and Web applications that can improve pediatric capabilities in their own EHRs because this will naturally inform discussions concerning EHR features and facilitate implementation and subsequent use of these capabilities by clinicians caring for children.

User testing of a diagnostic decision support system with machine-assisted chart review to facilitate clinical genomic diagnosis.

OBJECTIVES: There is a need in clinical genomics for systems that assist in clinical diagnosis, analysis of genomic information and periodic reanalysis of results, and can use information from the electronic health record to do so. Such systems should be built using the concepts of human-centred design, fit within clinical workflows and provide solutions to priority problems. METHODS: We adapted a commercially available diagnostic decision support system (DDSS) to use extracted findings from a patient record and combine them with genomic variant information in the DDSS interface. Three representative patient cases were created in a simulated clinical environment for user testing. A semistructured interview guide was created to illuminate factors relevant to human factors in CDS design and organisational implementation. RESULTS: Six individuals completed the user testing process. Tester responses were positive and noted good fit with real-world clinical genetics workflow. Technical issues related to interface, interaction and design were minor and fixable. Testers suggested solving issues related to terminology and usability through training and infobuttons. Time savings was estimated at 30%-50% and additional uses such as in-house clinical variant analysis were suggested for increase fit with workflow and to further address priority problems. CONCLUSION: This study provides preliminary evidence for usability, workflow fit, acceptability and implementation potential of a modified DDSS that includes machine-assisted chart review. Continued development and testing using principles from human-centred design and implementation science are necessary to improve technical functionality and acceptability for multiple stakeholders and organisational implementation potential to improve the genomic diagnosis process.

An electronic decision support-based complex intervention to improve management of cardiovascular risk in primary health care: a cluster randomised trial (INTEGRATE).

OBJECTIVES: To determine whether a multifaceted primary health care intervention better controlled cardiovascular disease (CVD) risk factors in patients with high risk of CVD than usual care. DESIGN, SETTING: Parallel arm, cluster randomised trial in 71 Australian general practices, 5 December 2016 - 13 September 2019. PARTICIPANTS: General practices that predominantly used an electronic medical record system compatible with the HealthTracker electronic decision support tool, and willing to implement all components of the INTEGRATE intervention. INTERVENTION: Electronic point-of-care decision support for general practices; combination cardiovascular medications (polypills); and a pharmacy-based medication adherence program. MAIN OUTCOME MEASURES: Proportion of patients with high CVD risk not on an optimal preventive medication regimen at baseline who had achieved both blood pressure and low-density lipoprotein (LDL) cholesterol goals at study end. RESULTS: After a median 15 months' follow-up, primary outcome data were available for 4477 of 7165 patients in the primary outcome cohort (62%). The proportion of patients who achieved both treatment targets was similar in the intervention (423 of 2156; 19.6%) and control groups (466 of 2321; 20.1%; relative risk, 1.06; 95% CI, 0.85-1.32). Further, no statistically significant differences were found for a number of secondary outcomes, including risk factor screening, preventive medication prescribing, and risk factor levels. Use of intervention components was low; it was highest for HealthTracker, used at least once for 347 of 3236 undertreated patients with high CVD risk (10.7%). CONCLUSIONS: Despite evidence for the efficacy of its individual components, the INTEGRATE intervention was not broadly implemented and did not improve CVD risk management in participating Australian general practices. TRIAL REGISTRATION: Australian New Zealand Clinical Trials Registry, ACTRN12616000233426 (prospective).

Optimizing pharmacists' detection of prescribing errors: Comparison of on-ward and central pharmacy services.

WHAT IS KNOWN AND OBJECTIVE: Prescribing errors are the leading cause of adverse drug events in hospitalized patients. Pharmaceutical validation, defined as the review of drug orders by a pharmacist, associated with clinical decision support (CDS) systems, significantly reduces these errors and adverse drug events. In Belgium, because clinical pharmacy services have limited public financial support, most pharmaceutical validations are performed at the central pharmacy instead of on-ward, by hospital pharmacists doing dispensing activities. In that context, we aimed at evaluating whether the strategy of CDS-guided central validation was the most appropriate method to improve the quality and safety of medicines' use compared to an on-ward pharmaceutical validation. METHODS: Our retrospective observational study was conducted in a Belgian tertiary care hospital, in 2018-2019. Data were extracted from our validation software and pharmacists' charts. The outcomes of the study were the number of pharmaceutical interventions due to the detection of prescribing errors, reasons for interventions, their acceptance rate and their potential clinical impact (according to two blinded experts) in the central pharmacy and on-ward validation groups. RESULTS AND DISCUSSION: Despite the use of the same CDS, a pharmaceutical intervention following the detection of a prescribing error was made for 2.9% (20/698) of central group patients and 13.3% (93/701) of on-ward patients (χ2  = 49.97, p < 0.001). Interventions made at the central pharmacy (n = 20) mostly relied on CDS-alerts (i.e. drug-drug interaction [25%] or overdosing [20%]) while interventions made on-ward (n = 93) were also for pharmacotherapy optimization (i.e. no valid indication [25%] or inappropriate drug's choice [11%]). The on-ward validation group showed a higher acceptance rate compared to the central group (84% and 65%, respectively [Fisher's test, p = 0.053]). Proportions of interventions with significant or very significant clinical impact were similar between the two groups but as fewer interventions were made centrally, a significant proportion of errors were probably not detected by the central validation. WHAT IS NEW AND CONCLUSION: On-ward pharmaceutical validation leads to a higher rate of prescribing error detection. Pharmaceutical interventions made by on-ward pharmacists are also better accepted and more relevant, going further than CDS-alerts.

Clinical efficacy of a Decision Support Tool (Link-me) to guide intensity of mental health care in primary practice: a pragmatic stratified randomised controlled trial.

BACKGROUND: The volume and heterogeneity of mental health problems that primary care patients present with is a substantial challenge for health systems, and both undertreatment and overtreatment are common. We developed Link-me, a patient-completed Decision Support Tool, to predict severity of depression or anxiety, identify priorities, and recommend interventions. In this study, we aimed to examine if Link-me reduces psychological distress among individuals predicted to have minimal/mild or severe symptoms of anxiety or depression. METHODS: In this pragmatic stratified randomised controlled trial, adults aged 18-75 years reporting depressive or anxiety symptoms or use of mental health medication were recruited from 23 general practices in Australia. Participants completed the Decision Support Tool and were classified into three prognostic groups (minimal/mild, moderate, severe), and those in the minimal/mild and severe groups were eligible for inclusion. Participants were individually and randomly assigned (1:1) by a computer-generated allocation sequence to receive either prognosis-matched care (intervention group) or usual care plus attention control (control group). Participants were not blinded but intervention providers were only notified of those allocated to the intervention group. Outcome assessment was blinded. The primary outcome was the difference in the change in scores between the intervention and control group, and within prognostic groups, on the 10-item Kessler Psychological Distress Scale at 6 months post randomisation. The trial was registered on the Australian and New Zealand Clinical Trials Registry, ACTRN12617001333303. OUTCOMES: Between Nov 21, 2017, and Oct 31, 2018, 24 616 patients were invited to complete the eligibility screening survey. 1671 of these patients were included and randomly assigned to either the intervention group (n=834) or the control group (n=837). Prognosis-matched care was associated with greater reductions in psychological distress than usual care plus attention control at 6 months (p=0·03), with a standardised mean difference (SMD) of -0·09 (95% CI -0·17 to -0·01). This reduction was also seen in the severe prognostic group (p=0·003), with a SMD of -0·26 (-0·43 to -0·09), but not in the minimal/mild group (p=0·73), with a SMD of 0·04 (-0·17 to 0·24). In the complier average causal effect analysis in the severe prognostic group, differences were larger among those who received some or all aspects of the intervention (SMD range -0·58 to -1·15). No serious adverse effects were recorded. INTERPRETATION: Prognosis-based matching of interventions reduces psychological distress in patients with anxiety or depressive symptoms, particularly in those with severe symptoms, and is associated with better outcomes when patients access the recommended treatment. Optimisation of the Link-me approach and implementation into routine practice could help reduce the burden of disease associated with common mental health conditions such as anxiety and depression. FUNDING: Australian Government Department of Health.

Development of a pharmacy 'patient prioritization tool' for use in a Tertiary Paediatric Hospital.

WHAT IS KNOWN AND OBJECTIVE: Pharmacists play an integral role in paediatric patient care by ensuring the safe and optimal use of medications. There are increasing demands on pharmacists' time and challenges to meet them within allocated resources, and therefore, it is important to ensure that resources are used efficiently. Patient prioritization tools for clinical pharmacists have been proposed via many studies, but are generally adult-based and/or have not been validated to confirm their effectiveness. The aim of this study was to create, pilot and validate a patient prioritization tool to be used by pharmacists providing clinical pharmacy services to paediatric patients. METHODS: A two-phase (retrospective and prospective) observational audit of pharmacists' interventions collected via notes made on their ward handover information sheets and patient case notes was conducted over a 2-year period in a tertiary paediatric hospital. A patient prioritization tool was created based on pharmacists' interventions in real time. This tool could be used at the start of the working day (without the need to review the patient or their case notes) to identify patients who would benefit most from a clinical pharmacist review. The tool was validated for effectiveness and selectivity. RESULTS AND DISCUSSION: The tool was easy to use and effective in identifying that 43% of paediatric inpatients did not require a routine clinical pharmacist review. It had 98% specificity in identifying patients who require a pharmacist intervention. It could be easily used at the start of the day to select patients for pharmacist review. WHAT IS NEW AND CONCLUSION: A new patient prioritization tool has been developed and validated for identifying paediatric inpatients requiring clinical pharmacist review.

Effects of a multifaceted intervention to promote the use of intravenous iron sucrose complex instead of ferric carboxymaltose in patients admitted for more than 24 h.

PURPOSE: Although more practical for use, the impact of ferric carboxymaltose (FCM) on the hospital budget is considerable, and intravenous iron sucrose complex (ISC) represents a cost-saving alternative for the management of iron deficiency anemia in patients during hospitalization. The Drug Committee decided to reserve FCM for day hospitalizations and contraindications to ISC, especially allergy. ISC was available for prescription for all other situations. METHODS: The impact of a multifaceted intervention promoting a switch from FCM to ISC was evaluated using an interrupted time series model with segmented regression analysis. The standardized rate of the dispensing of FCM, ISC, and oral iron by the hospital pharmacy, as well as the rate of the dispensing of packed red blood cells and the number of biological iron status measurements, was analyzed before and after the intervention. RESULTS: There was an immediate decrease in FCM consumption following the intervention, with a reduction of 88% (RR: 0.12 [CI95% 0.10 to 0.15]). Conversely, there was a large increase in ISC use (RR: 5.1 [CI95% 4.4 to 5.9]). We did not observe a prescription shift to packed red blood cells or oral iron after the intervention. The time series analysis showed the frequency of iron status testing to remain stable before and after. The direct savings for intravenous iron for 8 months were 187,417.54 €. CONCLUSION: Our intervention to lower the impact of intravenous iron therapy on the hospital budget was effective.

Transitioning Clinical Practice Guidelines into the Electronic Health Record through Clinical Pathways.

Clinical practice guidelines in oncology provide an evidence-based roadmap for most cancer care delivery but often lack directions for specific patient factors and disease conditions. Clinical pathways serve as a real-time clinical decision support system to translate guidelines to clinical practice. Pathways allow for the creation of a standardized, multidimensional roadmap for the continuum of care that can support clinical decision-making, maintain optimal outcomes, and limit unnecessary variation in cancer care. Here we describe the process to develop and implement clinical pathways in the electronic health record. This process includes building the appropriate foundation for a clinical pathways team with supports in the institutional ecosystem, creating visual representations of care paths, formalizing the pathway approval process, and translating clinical pathways into an electronic health record-integrated clinical decision support tool.

Optimizing clinical decision support alerts in electronic medical records: a systematic review of reported strategies adopted by hospitals.

OBJECTIVE: To identify and summarize the current internal governance processes adopted by hospitals, as reported in the literature, for selecting, optimizing, and evaluating clinical decision support (CDS) alerts in order to identify effective approaches. MATERIALS AND METHODS: Databases (Medline, Embase, CINAHL, Scopus, Web of Science, IEEE Xplore Digital Library, CADTH, and WorldCat) were searched to identify relevant papers published from January 2010 to April 2020. All paper types published in English that reported governance processes for selecting and/or optimizing CDS alerts in hospitals were included. RESULTS: Eight papers were included in the review. Seven papers focused specifically on medication-related CDS alerts. All papers described the use of a multidisciplinary committee to optimize alerts. Other strategies included the use of clinician feedback, alert data, literature and drug references, and a visual dashboard. Six of the 8 papers reported evaluations of their CDS alert modifications following the adoption of optimization strategies, and of these, 5 reported a reduction in alert rate. CONCLUSIONS: A multidisciplinary committee, often in combination with other approaches, was the most frequent strategy reported by hospitals to optimize their CDS alerts. Due to the limited number of published processes, variation in system changes, and evaluation results, we were unable to compare the effectiveness of different strategies, although employing multiple strategies appears to be an effective approach for reducing CDS alert numbers. We recommend hospitals report on descriptions and evaluations of governance processes to enable identification of effective strategies for optimization of CDS alerts in hospitals.

Perinatal e-screening and clinical decision support: the Maternity Case-finding Help Assessment Tool (MatCHAT).

INTRODUCTION Screening tools assist primary care clinicians to identify mental health, addiction and family violence problems. Electronic tools have many advantages, but there are none yet available in the perinatal context. AIM To assess the acceptability and feasibility of the Maternity Case-finding Help Assessment Tool (MatCHAT), a tool designed to provide e-screening and clinical decision support for depression, anxiety, cigarette smoking, use of alcohol or illicit substances, and family violence among pre- and post-partum women under the care of midwives. METHODS A co-design approach and an extensive consultation process was used to tailor a pre-existing electronic case-finding help assessment tool (eCHAT) to a maternity context. Quantitative MatCHAT data and qualitative data from interviews with midwives were analysed following implementation. RESULTS Five midwives participated in the study. They reported that MatCHAT was useful and acceptable and among the 20 mothers screened, eight reported substance use, one depression and five anxiety. Interviews highlighted extensive contextual barriers of importance to the implementation of maternity-specific screening. DISCUSSION MatCHAT has potential to optimise e-screening and decision support in maternity settings, but in this study, use was impeded by multiple contextual barriers. The information from this study is relevant to policymakers and future researchers when considering how to improve early identification of common mental health, substance use and family violence problems.

The human resource costs of implementing autopend clinical decision support to improve health maintenance.

OBJECTIVES: Sutter Health developed a novel autopend, or automated laboratory test ordering, clinical decision support (CDS) tool to coordinate the patient and physician process of completing preventive services. This study estimated the costs of developing and implementing the autopend functionality within an existing electronic health maintenance (HM) reminder system. STUDY DESIGN: Human resource time was measured by triangulating in-depth key informant interviews with Microsoft Outlook Calendar metadata (meetings attended) for managers and hourly data from a time-based project management tool (Project Web App) for Epic programmers. Employee time spent was multiplied by the Bureau of Labor Statistics California state hourly wages. Sutter Health is an integrated health care delivery network with more than 12,000 physicians across 100 communities serving 3 million patients. METHODS: Activity-based costing methodology was used to divide the implementation into activities and the human resources required to complete them. RESULTS: Developing and implementing the autopend CDS took more than 3 years, involved 6 managers and 3 Epic programmers, and cost $201,500 (2013 US$) (2670 total hours), which excluded the costs of implementing the initial HM reminder system. Managers spent 90.5% of the total costs (86.6% of total hours) integrating autopend into the health system compared with 9.5% of the total costs (13.4% of total hours) spent programming the functionality. CONCLUSIONS: The autopend CDS might be similarly costly for other organizations to implement if their managers need to complete comparable activities. However, electronic health record vendors could include autopend as a standard package to reduce development costs and improve the uptake of this promising CDS tool.

A NICE perspective on computable biomedical knowledge.

INTRODUCTION: The National Institute for Health and Care Excellence (NICE) plays a central role in the NHS. We distill knowledge of best practice from the best available sources of evidence and share this across the health and care system, typically in the form of recommendations. We want to ensure that this knowledge is shared in a form that supports improved decision making by professionals working together with patients, leading to improved outcomes. Understanding the role of computable knowledge in the context of a learning health system is therefore of deep interest to NICE. METHODS: The Agency for Healthcare Research and Quality (AHRQ) 4 levels of knowledge have been used as a framework to review current NICE products and services and envisage how they may need to evolve. DISCUSSION: NICE is mostly still at level 1 of the AHRQ knowledge hierarchy but aspires to work towards structured and computable products. The NHS Long Term Plan makes clear that the wider health and care system is seeking to drive up interoperability with standards for data exchange at the heart of this. NICE Connect is the name given to NICE's ambition to change in order to keep pace with changing technologies, advances in guideline development and analytical methods and the shifting needs of the system, and to ensure that it can sustainably and efficiently manage its portfolio of guidance. It is seen as crucial that NICE Connect and the wider Mobilising Computable Biomedical Knowledge (MCBK) agenda align for either of them to truly succeed.

Improving the Primary Care Consultation for Diabetes and Depression Through Digital Medical Interview Assistant Systems: Narrative Review.

BACKGROUND: Digital medical interview assistant (DMIA) systems, also known as computer-assisted history taking (CAHT) systems, have the potential to improve the quality of care and the medical consultation by exploring more patient-related aspects without time constraints and, therefore, acquiring more and better-quality information prior to the face-to-face consultation. The consultation in primary care is the broadest in terms of the amount of topics to be covered and, at the same time, the shortest in terms of time spent with the patient. OBJECTIVE: Our aim is to explore how DMIA systems may be used specifically in the context of primary care, to improve the consultations for diabetes and depression, as exemplars of chronic conditions. METHODS: A narrative review was conducted focusing on (1) the characteristics of the primary care consultation in general, and for diabetes and depression specifically, and (2) the impact of DMIA and CAHT systems on the medical consultation. Through thematic analysis, we identified the characteristics of the primary care consultation that a DMIA system would be able to improve. Based on the identified primary care consultation tasks and the potential benefits of DMIA systems, we developed a sample questionnaire for diabetes and depression to illustrate how such a system may work. RESULTS: A DMIA system, prior to the first consultation, could aid in the essential primary care tasks of case finding and screening, diagnosing, and, if needed, timely referral to specialists or urgent care. Similarly, for follow-up consultations, this system could aid with the control and monitoring of these conditions, help check for additional health issues, and update the primary care provider about visits to other providers or further testing. Successfully implementing a DMIA system for these tasks would improve the quality of the data obtained, which means earlier diagnosis and treatment. Such a system would improve the use of face-to-face consultation time, thereby streamlining the interaction and allowing the focus to be the patient's needs, which ultimately would lead to better health outcomes and patient satisfaction. However, for such a system to be successfully incorporated, there are important considerations to be taken into account, such as the language to be used and the challenges for implementing eHealth innovations in primary care and health care in general. CONCLUSIONS: Given the benefits explored here, we foresee that DMIA systems could have an important impact in the primary care consultation for diabetes and depression and, potentially, for other chronic conditions. Earlier case finding and a more accurate diagnosis, due to more and better-quality data, paired with improved monitoring of disease progress should improve the quality of care and keep the management of chronic conditions at the primary care level. A somewhat simple, easily scalable technology could go a long way to improve the health of the millions of people affected with chronic conditions, especially if working in conjunction with already-established health technologies such as electronic medical records and clinical decision support systems.

Designing, Conducting, and Reporting Clinical Decision Support Studies: Recommendations and Call to Action.

By enabling more efficient and effective medical decision making, computer-based clinical decision support (CDS) could unlock widespread benefits from the significant investment in electronic health record (EHR) systems in the United States. Evidence from high-quality CDS studies is needed to enable and support this vision of CDS-facilitated care optimization, but limited guidance is available in the literature for designing and reporting CDS studies. To address this research gap, this article provides recommendations for designing, conducting, and reporting CDS studies to: 1) ensure that EHR data to inform the CDS are available; 2) choose decision rules that are consistent with local care processes; 3) target the right users and workflows; 4) make the CDS easy to access and use; 5) minimize the burden placed on users; 6) incorporate CDS success factors identified in the literature, in particular the automatic provision of CDS as a part of clinician workflow; 7) ensure that the CDS rules are adequately tested; 8) select meaningful evaluation measures; 9) use as rigorous a study design as is feasible; 10) think about how to deploy the CDS beyond the original host organization; 11) report the study in context; 12) help the audience understand why the intervention succeeded or failed; and 13) consider the financial implications. If adopted, these recommendations should help advance the vision of more efficient, effective care facilitated by useful and widely available CDS.

Reporting and Implementing Interventions Involving Machine Learning and Artificial Intelligence.

Increasingly, interventions aimed at improving care are likely to use such technologies as machine learning and artificial intelligence. However, health care has been relatively late to adopt them. This article provides clinical examples in which machine learning and artificial intelligence are already in use in health care and appear to deliver benefit. Three key bottlenecks toward increasing the pace of diffusion and adoption are methodological issues in evaluation of artificial intelligence-based interventions, reporting standards to enable assessment of model performance, and issues that need to be addressed for an institution to adopt these interventions. Methodological best practices will include external validation, ideally at a different site; use of proactive learning algorithms to correct for site-specific biases and increase robustness as algorithms are deployed across multiple sites; addressing subgroup performance; and communicating to providers the uncertainty of predictions. Regarding reporting, especially important issues are the extent to which implementing standardized approaches for introducing clinical decision support has been followed, describing the data sources, reporting on data assumptions, and addressing biases. Although most health care organizations in the United States have adopted electronic health records, they may be ill prepared to adopt machine learning and artificial intelligence. Several steps can enable this: preparing data, developing tools to get suggestions to clinicians in useful ways, and getting clinicians engaged in the process. Open challenges and the role of regulation in this area are briefly discussed. Although these techniques have enormous potential to improve care and personalize recommendations for individuals, the hype regarding them is tremendous. Organizations will need to approach this domain carefully with knowledgeable partners to obtain the hoped-for benefits and avoid failures.