Overcoming Major Barriers to Build Efficient Decision Support Systems in Pharmacovigilance.

Many decision support methods and systems in pharmacovigilance are built without explicitly addressing specific challenges that jeopardize their eventual success. We describe two sets of challenges and appropriate strategies to address them. The first are data-related challenges, which include using extensive multi-source data of poor quality, incomplete information integration, and inefficient data visualization. The second are user-related challenges, which encompass users' overall expectations and their engagement in developing automated solutions. Pharmacovigilance decision support systems will need to rely on advanced methods, such as natural language processing and validated mathematical models, to resolve data-related issues and provide properly contextualized data. However, sophisticated approaches will not provide a complete solution if end-users do not actively participate in their development, which will ensure tools that efficiently complement existing processes without creating unnecessary resistance. Our group has already tackled these issues and applied the proposed strategies in multiple projects.

Protocol for an intervention development and pilot implementation evaluation study of an e-health solution to improve newborn care quality and survival in two low-resource settings, Malawi and Zimbabwe: Neotree.

INTRODUCTION: Every year 2.4 million deaths occur worldwide in babies younger than 28 days. Approximately 70% of these deaths occur in low-resource settings because of failure to implement evidence-based interventions. Digital health technologies may offer an implementation solution. Since 2014, we have worked in Bangladesh, Malawi, Zimbabwe and the UK to develop and pilot Neotree: an android app with accompanying data visualisation, linkage and export. Its low-cost hardware and state-of-the-art software are used to improve bedside postnatal care and to provide insights into population health trends, to impact wider policy and practice. METHODS AND ANALYSIS: This is a mixed methods (1) intervention codevelopment and optimisation and (2) pilot implementation evaluation (including economic evaluation) study. Neotree will be implemented in two hospitals in Zimbabwe, and one in Malawi. Over the 2-year study period clinical and demographic newborn data will be collected via Neotree, in addition to behavioural science informed qualitative and quantitative implementation evaluation and measures of cost, newborn care quality and usability. Neotree clinical decision support algorithms will be optimised according to best available evidence and clinical validation studies. ETHICS AND DISSEMINATION: This is a Wellcome Trust funded project (215742_Z_19_Z). Research ethics approvals have been obtained: Malawi College of Medicine Research and Ethics Committee (P.01/20/2909; P.02/19/2613); UCL (17123/001, 6681/001, 5019/004); Medical Research Council Zimbabwe (MRCZ/A/2570), BRTI and JREC institutional review boards (AP155/2020; JREC/327/19), Sally Mugabe Hospital Ethics Committee (071119/64; 250418/48). Results will be disseminated via academic publications and public and policy engagement activities. In this study, the care for an estimated 15 000 babies across three sites will be impacted. TRIAL REGISTRATION NUMBER: NCT0512707; Pre-results.

Validation and clinical value of the MANAGE-PD tool: A clinician-reported tool to identify Parkinson's disease patients inadequately controlled on oral medications.

INTRODUCTION: Making Informed Decisions to Aid Timely Management of Parkinson's Disease (MANAGE-PD) is a clinician-reported tool designed to facilitate timely identification and management of patients with advancing Parkinson's disease (PD) with suboptimal symptom control while on standard therapy. The objective of this study was to evaluate the validity and clinical value of the tool. METHODS: Driven by structured inputs from a steering committee and panel of PD experts, the tool was developed to classify patients into 3 categories. Validity and clinical value were elucidated using a two-pronged approach: (i) hypothetical patient vignettes (n = 10) developed based on the MANAGE-PD tool and rated by 17 PD specialists and 400 general neurologists (GN) and (ii) patients with PD (n = 2546) managed in real-world clinical settings. Vignette validity was based on concordance between PD experts' clinical judgement and MANAGE-PD vignette categorization. Patient-level data was used for known-group comparisons (validity) and discordant pair analysis (clinical value). RESULTS: The tool demonstrated strong validity and clinical value among PD specialists (intraclass coefficient [ICC] 0.843; Fleiss weighted kappa [ƙweighted] 0.79) and GN (ICC 0.690; ƙweighted 0.65) using patient vignettes. MANAGE-PD also demonstrated real-world validity and clinical value based on ability to identify patients with incrementally higher clinical, economic, and humanistic PD burden across categories of the tool (p < 0.01). CONCLUSIONS: MANAGE-PD demonstrated robust validity and clinical value in identifying patients with suboptimal PD symptom control. Clinical use of MANAGE-PD may complement treatment decision-making and facilitate timely and comprehensive management of patients with advancing PD.

Widely Used Sepsis Warning System Missed Two-Thirds of Cases.

Study raises questions about adopting commercial technology without scientific review.

Pharmacogenomic testing to support prescribing in primary care: a structured review of implementation models.

The application of pharmacogenomics could meaningfully contribute toward medicines optimization within primary care. This review identified 13 studies describing eight implementation models utilizing a multi-gene pharmacogenomic panel within a primary care or community setting. These were small feasibility studies (n <200). They demonstrated importance and feasibility of pre-test counseling, the role of the pharmacist, data integration into the electronic medical record and point-of-care clinical decision support systems (CDSS). Findings were considered alongside existing primary care prescribing practices and implementation frameworks to demonstrate how issues may be addressed by existing nationalized healthcare and primary care infrastructure. Development of point-of-care CDSS should be prioritized; establishing clinical leadership, education programs, defining practitioner roles and responsibilities and addressing commissioning issues will also be crucial.

New Standards for Clinical Decision Support: A Survey of The State of Implementation.

OBJECTIVES: To review the current state of research on designing and implementing clinical decision support (CDS) using four current interoperability standards: Fast Healthcare Interoperability Resources (FHIR); Substitutable Medical Applications and Reusable Technologies (SMART); Clinical Quality Language (CQL); and CDS Hooks. METHODS: We conducted a review of original studies describing development of specific CDS tools or infrastructures using one of the four targeted standards, regardless of implementation stage. Citations published any time before the literature search was executed on October 21, 2020 were retrieved from PubMed. Two reviewers independently screened articles and abstracted data according to a protocol designed by team consensus. RESULTS: Of 290 articles identified via PubMed search, 44 were included in this study. More than three quarters were published since 2018. Forty-three (98%) used FHIR; 22 (50%) used SMART; two (5%) used CQL; and eight (18%) used CDS Hooks. Twenty-four (55%) were in the design stage, 15 (34%) in the piloting stage, and five (11%) were deployed in a real-world setting. Only 12 (27%) of the articles reported an evaluation of the technology under development. Three of the four articles describing a deployed technology reported an evaluation. Only two evaluations with randomized study components were identified. CONCLUSION: The diversity of topics and approaches identified in the literature highlights the utility of these standards. The infrequency of reported evaluations, as well as the high number of studies in the design or piloting stage, indicate that these technologies are still early in their life cycles. Informaticists will require a stronger evidence base to understand the implications of using these standards in CDS design and implementation.

A transfer learning approach to facilitate ComBat-based harmonization of multicentre radiomic features in new datasets.

PURPOSE: To facilitate the demonstration of the prognostic value of radiomics, multicenter radiomics studies are needed. Pooling radiomic features of such data in a statistical analysis is however challenging, as they are sensitive to the variability in scanner models, acquisition protocols and reconstruction settings, which is often unavoidable in a multicentre retrospective analysis. A statistical harmonization strategy called ComBat was utilized in radiomics studies to deal with the "center-effect". The goal of the present work was to integrate a transfer learning (TL) technique within ComBat-and recently developed alternate versions of ComBat with improved flexibility (M-ComBat) and robustness (B-ComBat)-to allow the use of a previously determined harmonization transform to the radiomic feature values of new patients from an already known center. MATERIAL AND METHODS: The proposed TL approach were incorporated in the four versions of ComBat (standard, B, M, and B-M ComBat). The proposed approach was evaluated using a dataset of 189 locally advanced cervical cancer patients from 3 centers, with magnetic resonance imaging (MRI) and positron emission tomography (PET) images, with the clinical endpoint of predicting local failure. The impact performance of the TL approach was evaluated by comparing the harmonization achieved using only parts of the data to the reference (harmonization achieved using all the available data). It was performed through three different machine learning pipelines. RESULTS: The proposed TL technique was successful in harmonizing features of new patients from a known center in all versions of ComBat, leading to predictive models reaching similar performance as the ones developed using the features harmonized with all the data available. CONCLUSION: The proposed TL approach enables applying a previously determined ComBat transform to new, previously unseen data.

External Validation of a Widely Implemented Proprietary Sepsis Prediction Model in Hospitalized Patients.

Importance: The Epic Sepsis Model (ESM), a proprietary sepsis prediction model, is implemented at hundreds of US hospitals. The ESM's ability to identify patients with sepsis has not been adequately evaluated despite widespread use. Objective: To externally validate the ESM in the prediction of sepsis and evaluate its potential clinical value compared with usual care. Design, Setting, and Participants: This retrospective cohort study was conducted among 27 697 patients aged 18 years or older admitted to Michigan Medicine, the academic health system of the University of Michigan, Ann Arbor, with 38 455 hospitalizations between December 6, 2018, and October 20, 2019. Exposure: The ESM score, calculated every 15 minutes. Main Outcomes and Measures: Sepsis, as defined by a composite of (1) the Centers for Disease Control and Prevention surveillance criteria and (2) International Statistical Classification of Diseases and Related Health Problems, Tenth Revision diagnostic codes accompanied by 2 systemic inflammatory response syndrome criteria and 1 organ dysfunction criterion within 6 hours of one another. Model discrimination was assessed using the area under the receiver operating characteristic curve at the hospitalization level and with prediction horizons of 4, 8, 12, and 24 hours. Model calibration was evaluated with calibration plots. The potential clinical benefit associated with the ESM was assessed by evaluating the added benefit of the ESM score compared with contemporary clinical practice (based on timely administration of antibiotics). Alert fatigue was evaluated by comparing the clinical value of different alerting strategies. Results: We identified 27 697 patients who had 38 455 hospitalizations (21 904 women [57%]; median age, 56 years [interquartile range, 35-69 years]) meeting inclusion criteria, of whom sepsis occurred in 2552 (7%). The ESM had a hospitalization-level area under the receiver operating characteristic curve of 0.63 (95% CI, 0.62-0.64). The ESM identified 183 of 2552 patients with sepsis (7%) who did not receive timely administration of antibiotics, highlighting the low sensitivity of the ESM in comparison with contemporary clinical practice. The ESM also did not identify 1709 patients with sepsis (67%) despite generating alerts for an ESM score of 6 or higher for 6971 of all 38 455 hospitalized patients (18%), thus creating a large burden of alert fatigue. Conclusions and Relevance: This external validation cohort study suggests that the ESM has poor discrimination and calibration in predicting the onset of sepsis. The widespread adoption of the ESM despite its poor performance raises fundamental concerns about sepsis management on a national level.

Unexpected Inequality: Disparate-Impact From Artificial Intelligence in Healthcare Decisions.

Systemic discrimination in healthcare plagues marginalized groups. Physicians incorrectly view people of color as having high pain tolerance, leading to undertreatment. Women with disabilities are often undiagnosed because their symptoms are dismissed. Low-income patients have less access to appropriate treatment. These patterns, and others, reflect long-standing disparities that have become engrained in U.S. health systems. As the healthcare industry adopts artificial intelligence and algorithminformed (AI) tools, it is vital that regulators address healthcare discrimination. AI tools are increasingly used to make both clinical and administrative decisions by hospitals, physicians, and insurers--yet there is no framework that specifically places nondiscrimination obligations on AI users. The Food and Drug Administration has limited authority to regulate AI and has not sought to incorporate anti-discrimination principles in its guidance. Section 1557 of the Affordable Care Act has not been used to enforce nondiscrimination in healthcare AI and is under-utilized by the Office of Civil Rights. State level protections by medical licensing boards or malpractice liability are similarly untested and have not yet extended nondiscrimination obligations to AI. This Article discusses the role of each legal obligation on healthcare AI and the ways in which each system can improve to address discrimination. It highlights the ways in which industries can self-regulate to set nondiscrimination standards and concludes by recommending standards and creating a super-regulator to address disparate impact by AI. As the world moves towards automation, it is imperative that ongoing concerns about systemic discrimination are removed to prevent further marginalization in healthcare.

Clinical Decision Support Reduces Unnecessary Tetanus Vaccinations in the Emergency Department.

STUDY OBJECTIVE: Tetanus is the most common vaccination given in the emergency department; yet, administrations of tetanus vaccine boosters in the ED may not comply with the US Centers for Disease Control and Prevention's recommended vaccination schedule. We implemented a clinical decision support alert in the electronic health record that warned providers when ordering a tetanus vaccine if a prior one had been given within 10 years and studied its efficacy to reduce potentially unnecessary vaccines in the ED. METHODS: This was a retrospective, quasi-experimental, 1-group, pretest-posttest study in 3 hospital EDs in Boston, MA. We studied adult patients for whom tetanus vaccines were ordered despite a history of vaccination within the prior 10 years. We compared the number of potentially unnecessary tetanus vaccine administrations in a baseline phase (when the clinical decision support alert was not visible) versus an intervention phase. RESULTS: Of eligible patients, 22.1% (95% confidence interval [CI] 21.8% to 22.4%) had prior tetanus vaccines within 5 years, 12.8% (95% CI 12.5% to 13.0%) within 5 to 10 years, 3.8% (95% CI 3.6% to 3.9%) more than 10 years ago, and 61.3% (95% CI 60.9% to 61.7%) had no prior tetanus vaccination documentation. Of 60,983 encounters, 337 met the inclusion criteria. A tetanus vaccination was administered in 91% (95% CI 87% to 96%) of encounters in the baseline phase, compared to 55% (95% CI 47% to 62%) during the intervention. The absolute risk reduction was 36.7% (95% CI 28.0% to 45.4%), and the number of encounters needed to alert to avoid 1 potentially unnecessary tetanus vaccine (number needed to treat) was 2.7 (95% CI 2.2% to 3.6%). For patients with tetanus vaccines within the prior 5 years, the absolute risk reduction was 47.9% (95% CI 35.5 % to 60.3%) and the number needed to treat was 2.1 (95% CI 1.7% to 2.8%). CONCLUSION: A clinical decision support alert that warns ED clinicians that a patient may have an up-to-date tetanus vaccination status reduces potentially unnecessary vaccinations.

Accuracy assessment methods for physiological model selection toward evaluation of closed-loop controlled medical devices.

Physiological closed-loop controlled (PCLC) medical devices are complex systems integrating one or more medical devices with a patient's physiology through closed-loop control algorithms; introducing many failure modes and parameters that impact performance. These control algorithms should be tested through safety and efficacy trials to compare their performance to the standard of care and determine whether there is sufficient evidence of safety for their use in real care setting. With this aim, credible mathematical models have been constructed and used throughout the development and evaluation phases of a PCLC medical device to support the engineering design and improve safety aspects. Uncertainties about the fidelity of these models and ambiguities about the choice of measures for modeling performance need to be addressed before a reliable PCLC evaluation can be achieved. This research develops tools for evaluating the accuracy of physiological models and establishes fundamental measures for predictive capability assessment across different physiological models. As a case study, we built a refined physiological model of blood volume (BV) response by expanding an original model we developed in our prior work. Using experimental data collected from 16 sheep undergoing hemorrhage and fluid resuscitation, first, we compared the calibration performance of the two candidate physiological models, i.e., original and refined, using root-mean-squared error (RMSE), Akiake information criterion (AIC), and a new multi-dimensional approach utilizing normalized features extracted from the fitting error. Compared to the original model, the refined model demonstrated a significant improvement in calibration performance in terms of RMSE (9%, P = 0.03) and multi-dimensional measure (48%, P = 0.02), while a comparable AIC between the two models verified that the enhanced calibration performance in the refined model is not due to data over-fitting. Second, we compared the physiological predictive capability of the two models under three different scenarios: prediction of subject-specific steady-state BV response, subject-specific transient BV response to hemorrhage perturbation, and leave-one-out inter-subject BV response. Results indicated enhanced accuracy and predictive capability for the refined physiological model with significantly larger proportion of measurements that were within the prediction envelope in the transient and leave-one-out prediction scenarios (P < 0.02). All together, this study helps to identify and merge new methods for credibility assessment and physiological model selection, leading to a more efficient process for PCLC medical device evaluation.

The role of digital clinical decision support tool in improving quality of intrapartum and postpartum care: experiences from two states of India.

BACKGROUND: Computerized clinical decision support (CDSS) -digital information systems designed to improve clinical decision making by providers - is a promising tool for improving quality of care. This study aims to understand the uptake of ASMAN application (defined as completeness of electronic case sheets), the role of CDSS in improving adherence to key clinical practices and delivery outcomes. METHODS: We have conducted secondary analysis of program data (government data) collected from 81 public facilities across four districts each in two sates of Madhya Pradesh and Rajasthan. The data collected between August -October 2017 (baseline) and the data collected between December 2019 - March 2020 (latest) was analysed. The data sources included: digitized labour room registers, case sheets, referral and discharge summary forms, observation checklist and complication format. Descriptive, univariate and multivariate and interrupted time series regression analyses were conducted. RESULTS: The completeness of electronic case sheets was low at postpartum period (40.5%), and in facilities with more than 300 deliveries a month (20.9%). In multivariate logistic regression analysis, the introduction of technology yielded significant improvement in adherence to key clinical practices. We have observed reduction in fresh still births rates and asphyxia, but these results were not statistically significant in interrupted time series analysis. However, our analysis showed that identification of maternal complications has increased over the period of program implementation and at the same time referral outs decreased. CONCLUSIONS: Our study indicates CDSS has a potential to improve quality of intrapartum care and delivery outcome. Future studies with rigorous study design is required to understand the impact of technology in improving quality of maternity care.

Effects of patients' hospital discharge preferences on uptake of clinical decision support.

The Centers for Medicare and Medicaid Services identified unplanned hospital readmissions as a critical healthcare quality and cost problem. Improvements in hospital discharge decision-making and post-discharge care are needed to address the problem. Utilization of clinical decision support (CDS) can improve discharge decision-making but little is known about the empirical significance of two opposing problems that can occur: (1) negligible uptake of CDS by providers or (2) over-reliance on CDS and underuse of other information. This paper reports an experiment where, in addition to electronic medical records (EMR), clinical decision-makers are provided subjective reports by standardized patients, or CDS information, or both. Subjective information, reports of being eager or reluctant for discharge, was obtained during examinations of standardized patients, who are regularly employed in medical education, and in our experiment had been given scripts for the experimental treatments. The CDS tool presents discharge recommendations obtained from econometric analysis of data from de-identified EMR of hospital patients. 38 clinical decision-makers in the experiment, who were third and fourth year medical students, discharged eight simulated patient encounters with an average length of stay 8.1 in the CDS supported group and 8.8 days in the control group. When the recommendation was "Discharge," CDS uptake of "Discharge" recommendation was 20% higher for eager than reluctant patients. Compared to discharge decisions in the absence of patient reports: (i) odds of discharging reluctant standardized patients were 67% lower in the CDS-assisted group and 40% lower in the control (no-CDS) group; whereas (ii) odds of discharging eager standardized patients were 75% higher in the control group and similar in CDS-assisted group. These findings indicate that participants were neither ignoring nor over-relying on CDS.

Decision support systems for the prediction of lymph node involvement in early breast cancer.

The prediction of lymph node involvement represents an important task which could reduce unnecessary surgery and improve the definition of oncological therapies. An artificial intelligence model able to predict it in pre-operative phase requires the interface among multiple data structures. The trade-off among time consuming, expensive and invasive methodologies is emerging in experimental setups exploited for the analysis of sentinel lymph nodes, where machine learning algorithms represent a key ingredient in recorded data elaboration. The accuracy required for clinical applications is obtainable matching different kind of data. Statistical associations of prognostic factors with symptoms and predictive models implemented also through on-line softwares represent useful diagnostic support tools which translate into patients quality of life improvement and costs reduction.

Development of safety and usability guideline for clinical information system.

ABSTRACT: Clinical information systems (CISs) that do not consider usability and safety could lead to harmful events. Therefore, we aimed to develop a safety and usability guideline of CISs that is comprehensive for both users and developers. And the guideline was categorized to apply actual clinical workflow and work environment.The guideline components were extracted through a systematic review of the articles published between 2000 and 2015, and existing CIS safety and/or usability design guidelines. The guideline components were categorized according to clinical workflow and types of user interface (UI). The contents of the guideline were evaluated and validated by experts with 3 specialties: medical informatics, patient safety, and human engineering.Total 1276 guideline components were extracted through article and guideline review. Of these, 464 guideline components were categorized according to 5 divisions of the clinical workflow: "Data identification and selection," "Document entry," "Order entry," "Clinical decision support and alert," and "Management". While 521 guideline components were categorized according to 4 divisions of UI: UIs related to information process steps, "Perception," "Recognition," "Control," and "Feedback". We developed a guideline draft with 219 detailed guidance for clinical task and 70 for UI. Overall appropriateness and comprehensiveness were proven to achieve more than 90% in experts' survey. However, there were significant differences among the groups of specialties in the judgment of appropriateness (P < .001) and comprehensiveness (P = .038).We developed and verified a safety and usability guideline for CIS that qualifies the requirements of both clinical workflows and usability issues. The developed guideline can be a practical tool to enhance the usability and safety of CISs. Further validation is required by applying the guideline for designing the actual CIS.

[Clinical history in times of big data; a plea for a standard for the structured recording of the clinical history]. / Anamnese in tijden van big data.

Clinical decision support systems to aid the clinician in making a correct diagnosis will only succeed if data from the clinical history are taken into account. However, currently, very little is known on diagnostic test characteristics of specific symptoms, let alone of a pattern of several symptoms with all their cardinal features. We plead for the nation-wide introduction of a standard for the structured recording of the clinical history. To allow for such structured recording, user interfaces of electronic healthcare records must become far more user-friendly. Furthermore, scribes may be used, or, ideally, a digital scribe, a computer application that records the conversation between healthcare professional and patient and creates an automated summary. So far, to our knowledge, no digital scribe encompassing the entire patient history has been implemented into medical practice. We are currently trying to develop such a digital scribe.