Effectiveness of fall prevention interventions in residential aged care and community settings: an umbrella review.

INTRODUCTION: Preventing falls is a priority for aged care providers. Research to date has focused on fall prevention strategies in single settings (e.g., residential aged care (RAC) or community settings). However, some aged care providers deliver care, including fall prevention interventions, across RAC and community settings. We conducted an umbrella review to identify what type of fall prevention interventions had the greatest impact on falls outcomes in RAC and community settings. METHODS: Five databases were searched for systematic reviews of falls prevention randomised control trials in older adults living in the community or RAC. Data extracted included systematic review methods, population characteristics, intervention characteristics, setting details (RAC or community), and fall-related outcomes (falls, people who have had a fall, fall-related hospitalisations, and fall-related fractures). Review quality was appraised using the Assessment of Multiple Systematic Reviews-2 tool. RESULTS: One-hundred and six systematic reviews were included; 63 and 19 of these stratified results by community and RAC settings respectively, the remainder looked at both settings. The most common intervention types discussed in reviews included 'exercise' (61%, n = 65), 'multifactorial' (two or more intervention types delivered together) (26%, n = 28), and 'vitamin D' (18%, n = 19). In RAC and community settings, 'exercise' interventions demonstrated the most consistent reduction in falls and people who have had a fall compared to other intervention types. 'Multifactorial' interventions were also beneficial in both settings however demonstrated more consistent reduction in falls and people who fall in RAC settings compared to community settings. 'Vitamin D' interventions may be beneficial in community-dwelling populations but not in RAC settings. It was not possible to stratify fall-related hospitalisation and fall-related fracture outcomes by setting due to limited number of RAC-specific reviews (n = 3 and 0 respectively). CONCLUSION: 'Exercise' interventions may be the most appropriate falls prevention intervention for older adults in RAC and community settings as it is beneficial for multiple fall-related outcomes (falls, fall-related fractures, and people who have had a fall). Augmenting 'exercise' interventions to become 'multifactorial' interventions may also improve the incidence of falls in both settings.

Is evidence of effectiveness a driver for clinical decision support selection? A qualitative descriptive study of senior hospital staff.

Limited research has focused on understanding if and how evidence of health information technology (HIT) effectiveness drives the selection and implementation of technologies in practice. This study aimed to explore the views of senior hospital staff on the role evidence plays in the selection and implementation of HIT, with a particular focus on clinical decision support (CDS) alerts in electronic medication management systems. A qualitative descriptive design was used. Twenty senior hospital staff from six Australian hospitals in New South Wales and Queensland took part in a semistructured interview. Interviews were audio-recorded and transcribed, and a general inductive content analysis approach was used to identify themes. Participants acknowledged the importance of an evidence base, but reported that selection of CDS alerts, and HIT more broadly, was rarely underpinned by evidence that technologies improve patient care. Instead, investments in technologies were guided by the expectation that benefits will be achieved, bolstered by vendor assurances, and a perception that implementation of HIT is unavoidable. Postponing implementation of a technology until an evidence base is available was not always feasible. Although some technologies were seen as not requiring an evidence base, stakeholders viewed evidence as extremely valuable for informing decisions about selection of CDS alerts. In the absence of evidence, evaluation or monitoring of technologies postimplementation is critical, particularly to identify new errors or risks associated with HIT implementation and use. Increased transparency from vendors, with technology evaluation outcomes made directly available to healthcare organizations, may result in less reliance on logic, intuition, and vendor assertions and more evidence-based selection of HIT.

Communicating deprescribing decisions made in hospital with general practitioners in the community.

BACKGROUND: Deprescribing, the supervised withdrawal of inappropriate medications, intends to manage polypharmacy, which is prevalent in older patients. AIMS: To examine general practitioner (GP) perceptions of communication processes between clinicians in hospital and GP in the community about deprescribing decisions made in hospital. METHODS: Focus groups and interviews were held with 15 GP, exploring deprescribing in hospitals, communication of deprescribing information and the format of communications. Sessions were audiotaped, transcribed and analysed using an inductive approach. RESULTS: GP stated that they should be involved in deprescribing decisions, especially for older complex patients, because of their good knowledge of their patients. Barriers to effective communication included the acute nature of hospital stays and lack of time. Facilitators included long-term relationships of GP with their patients and engaged patients. GP preferred communication of deprescribing decisions to be over the telephone while the patient was still in hospital, and with a concise, electronic discharge summary at the time of discharge. GP indicated that rationale for medication changes and recommended follow-up actions were crucial in a discharge summary to enable care post-discharge. CONCLUSIONS: GP welcome increased communication with hospital clinicians regarding deprescribing decisions made while patients are in hospital. Communication needs to be timely, transparent, succinct and accessible. Lack of time and difficulties contacting hospital clinicians challenge this process.

Supporting deprescribing in hospitalised patients: formative usability testing of a computerised decision support tool.

BACKGROUND: Despite growing evidence that deprescribing can improve clinical outcomes, quality of life and reduce the likelihood of adverse drug events, the practice is not widespread, particularly in hospital settings. Clinical risk assessment tools, like the Drug Burden Index (DBI), can help prioritise patients for medication review and prioritise medications to deprescribe, but are not integrated within routine care. The aim of this study was to conduct formative usability testing of a computerised decision support (CDS) tool, based on DBI, to identify modifications required to the tool prior to trialling in practice. METHODS: Our CDS tool comprised a DBI MPage in the electronic medical record (clinical workspace) that facilitated review of a patient's DBI and medication list, access to deprescribing resources, and the ability to deprescribe. Two rounds of scenario-based formative usability testing with think-aloud protocol were used. Seventeen end-users participated in the testing, including junior and senior doctors, and pharmacists. RESULTS: Participants expressed positive views about the DBI CDS tool but testing revealed a number of clear areas for improvement. These primarily related to terminology used (i.e. what is a DBI and how is it calculated?), and consistency of functionality and display. A key finding was that users wanted the CDS tool to look and function in a similar way to other decision support tools in the electronic medical record. Modifications were made to the CDS tool in response to user feedback. CONCLUSION: Usability testing proved extremely useful for identifying components of our CDS tool that were confusing, difficult to locate or to understand. We recommend usability testing be adopted prior to implementation of any digital health intervention. We hope our revised CDS tool equips clinicians with the knowledge and confidence to consider discontinuation of inappropriate medications in routine care of hospitalised patients. In the next phase of our project, we plan to pilot test the tool in practice to evaluate its uptake and effectiveness in supporting deprescribing in routine hospital care.

The efficiency-thoroughness trade-off after implementation of electronic medication management: a qualitative study in paediatric oncology.

OBJECTIVE: The efficiency-thoroughness trade-off (ETTO) principle proposes that people and organizations are often required to make a trade-off between being efficient and being thorough, as it is difficult to be both efficient and thorough at the same time. This study aimed to compare pre- electronic medication management system (EMMS) expectation of how an EMMS is likely to impact on efficiency and thoroughness to post-EMM experiences of an EMMS and the ETTO. DESIGN: Qualitative interview study. SETTING: A paediatric oncology cancer centre in a large paediatric tertiary teaching hospital in Sydney, Australia. PARTICIPANTS: Forty-four semi-structured interviews with doctors, nurses and pharmacists six months prior to and two years following implementation of an EMMS. RESULTS: Prior to EMM implementation, staff identified a number of areas of work where both efficiency and thoroughness were expected to improve with EMM. These included ease of accessibility of the medication record, and organization and legibility of medication information. Following EMMS implementation, staff reported improvements in these areas. However, the EMMS was perceived to drive thoroughness (safety) benefits at the expense of efficiency (time). Measures to improve safety in the EMMS enforced processes that required time, such as medication double-checking procedures. CONCLUSIONS: Overall, staff were aware of the competitive interplay between thoroughness and efficiency and reported that introduction of an EMMS had imposed processes that favoured improvements in thoroughness at the expense of efficiency.

A Late Attempt to Involve End Users in the Design of Medication-Related Alerts: Survey Study.

BACKGROUND: When users of electronic medical records (EMRs) are presented with large numbers of irrelevant computerized alerts, they experience alert fatigue, begin to ignore alert information, and override alerts without processing or heeding alert recommendations. Anecdotally, doctors at our study site were dissatisfied with the medication-related alerts being generated, both in terms of volume being experienced and clinical relevance. OBJECTIVE: This study aimed to involve end users in the redesign of medication-related alerts in a hospital EMR, 4 years post implementation. METHODS: This work was undertaken at a private not-for-profit teaching hospital in Sydney, Australia. Since EMR implementation in 2015, the organization elected to implement all medication-related alert types available in the system for prescribers: allergy and intolerance alerts, therapeutic duplication alerts, pregnancy alerts, and drug-drug interaction alerts. The EMR included no medication administration alerts for nurses. To obtain feedback on current alerts and suggestions for redesign, a Web-based survey was distributed to all doctors and nurses at the site via hospital mailing lists. RESULTS: Despite a general dissatisfaction with alerts, very few end users completed the survey. In total, only 3.37% (36/1066) of doctors and 14.5% (60/411) of nurses took part. Approximately 90% (30/33) of doctors who responded held the view that too many alerts were triggered in the EMR. Doctors suggested that most alerts be removed and that alerts be more specific and less sensitive. In contrast, 97% (58/60) of the nurse respondents indicated that they would like to receive medication administration alerts in the EMR. Most nurses indicated that they would like to receive all the alert types available at all severity levels. CONCLUSIONS: Attempting to engage with end users several years post implementation was challenging. Involving users so late in the implementation process may lead to clinicians viewing the provision of feedback to be futile. Seeking user feedback on usefulness, volume, and design of alerts is extremely valuable; however, we suggest this is undertaken early, preferably before system implementation.

Drug-drug interactions and their harmful effects in hospitalised patients: a systematic review and meta-analysis.

PURPOSE: Drug-drug interactions (DDIs) are often avoidable and, if undetected, can lead to patient harm. This review aimed to determine the prevalence of potential DDIs (pDDIs), clinically relevant DDIs (DDIs that could lead to measurable patient harm, taking into account the patient's individual clinical profile) and DDIs that resulted in actual patient harm during hospitalisation. METHOD: Four databases were scanned for English papers published from 2000 to 2016. Papers that reported prevalence of DDIs in the outpatient setting, at admission or discharge, involving only specific drugs, or in specific disease populations or age groups were excluded. RESULTS: Twenty-seven papers met the inclusion criteria and were graded for quality using the Critical Appraisal Skills Programme (CASP) cohort study checklist. Ten papers were rated as 'poor', 14 as 'fair' and only three papers as 'good'. Overall, the meta-analysis revealed that 33% of general patients and 67% of intensive care patients experienced a pDDI during their hospital stay. It was not possible to determine the prevalence of clinically relevant DDIs or DDIs that resulted in actual patient harm as data on these categories were limited. Of the very few studies that reported on harm, only a small proportion of DDIs were found to have resulted in actual patient harm. CONCLUSIONS: Standardisation of DDI definitions and research methods are required to allow meaningful prevalence rates to be obtained and compared. Studies that go further than measuring pDDIs are critically needed to determine the impact of DDIs on patient safety.

Reliability, ease of use and usefulness of I-MeDeSA for evaluating drug-drug interaction alerts in an Australian context.

BACKGROUND: Recently, attention has shifted to improving the design of computerized alerts via the incorporation of human factors design principles. The Instrument for Evaluating Human Factors Principles in Medication-Related Decision Support Alerts (I-MeDeSA) is a tool developed in the United States to guide improvements to alert design and facilitate selection of electronic systems with superior design. In this study, we aimed to determine the reliability, ease of use and usefulness of I-MeDeSA for assessing drug-drug interaction (DDI) alerts in an Australian context. METHODS: Using the I-MeDeSA, three reviewers independently evaluated DDI alert interfaces of seven electronic systems used in Australia. Inter-rater reliability was assessed and reviewers met to discuss difficulties in using I-MeDeSA and the tool's usefulness. RESULTS: Inter-rater reliability was high (Krippendorff's alpha = 0.76), however, ambiguous wording and the inclusion of conditional items impacted ease of use. A number of items were not relevant to Australian implementations and as a result, most systems achieved an I-MeDeSA score of less than 50%. CONCLUSIONS: The I-MeDeSA proved to be reliable, but item wording and structure made application difficult. Future studies should investigate potential modifications to the I-MeDeSA to improve ease of use and increase applicability to a variety of system configurations.

An Analysis of Incident Reports Related to Electronic Medication Management: How They Change Over Time.

OBJECTIVE: Electronic medication management (EMM) systems have been shown to introduce new patient safety risks that were not possible, or unlikely to occur, with the use of paper charts. Our aim was to examine the factors that contribute to EMM-related incidents and how these incidents change over time with ongoing EMM use. METHODS: Incidents reported at 3 hospitals between January 1, 2010, and December 31, 2019, were extracted using a keyword search and then screened to identify EMM-related reports. Data contained in EMM-related incident reports were then classified as unsafe acts made by users and the latent conditions contributing to each incident. RESULTS: In our sample, 444 incident reports were determined to be EMM related. Commission errors were the most frequent unsafe act reported by users (n = 298), whereas workarounds were reported in only 13 reports. User latent conditions (n = 207) were described in the highest number of incident reports, followed by conditions related to the organization (n = 200) and EMM design (n = 184). Over time, user unfamiliarity with the system remained a key contributor to reported incidents. Although fewer articles to electronic transfer errors were reported over time, incident reports related to the transfer of information between different computerized systems increased as hospitals adopted more clinical information systems. CONCLUSIONS: Electronic medication management-related incidents continue to occur years after EMM implementation and are driven by design, user, and organizational conditions. Although factors contribute to reported incidents in varying degrees over time, some factors are persistent and highlight the importance of continuously improving the EMM system and its use.

Qualitative study of challenges with recruitment of hospitals into a cluster controlled trial of clinical decision support in Australia.

OBJECTIVE: To identify barriers to hospital participation in controlled cluster trials of clinical decision support (CDS) and potential strategies for addressing barriers. DESIGN: Qualitative descriptive design comprising semistructured interviews. SETTING: Five hospitals in New South Wales and one hospital in Queensland, Australia. PARTICIPANTS: Senior hospital staff, including department directors, chief information officers and those working in health informatics teams. RESULTS: 20 senior hospital staff took part. Barriers to hospital-level recruitment primarily related to perceptions of risk associated with not implementing CDS as a control site. Perceived risks included reductions in patient safety, reputational risk and increased likelihood that benefits would not be achieved following electronic medical record (EMR) implementation without CDS alerts in place. Senior staff recommended clear communication of trial information to all relevant stakeholders as a key strategy for boosting hospital-level participation in trials. CONCLUSION: Hospital participation in controlled cluster trials of CDS is hindered by perceptions that adopting an EMR without CDS is risky for both patients and organisations. The improvements in safety expected to follow CDS implementation makes it challenging and counterintuitive for hospitals to implement EMR without incorporating CDS alerts for the purposes of a research trial. To counteract these barriers, clear communication regarding the evidence base and rationale for a controlled trial is needed.

Drug-Drug Interactions and Actual Harm to Hospitalized Patients: A Multicentre Study Examining the Prevalence Pre- and Post-Electronic Medication System Implementation.

INTRODUCTION: Drug-drug interactions (DDIs) have potential to cause patient harm, including lowering therapeutic efficacy. This study aimed to (i) determine the prevalence of potential DDIs (pDDIs); clinically relevant DDIs (cDDIs), that is, DDIs that could lead to patient harm, taking into account a patient's individual clinical profile, drug effects and severity of potential harmful outcome; and subsequent actual harm among hospitalized patients and (ii) examine the impact of transitioning from paper-based medication charts to electronic medication management (eMM) on DDIs and patient harms. METHODS: This was a secondary analysis of the control arm of a controlled pre-post study. Patients were randomly selected from three Australian hospitals. Retrospective chart review was conducted before and after the implementation of an eMM system, without accompanying clinical decision support alerts for DDIs. Harm was assessed by an expert panel. RESULTS: Of 1186 patient admissions, 70.1% (n = 831) experienced a pDDI, 42.6% (n = 505) a cDDI and 0.9% (n = 11) an actual harm in hospital. Of 15,860 pDDIs identified, 27.0% (n = 4285) were classified as cDDIs. The median number of pDDIs and cDDIs per 10 drugs were 6 [interquartile range (IQR) 2-13] and 0 (IQR 0-2), respectively. In cases where a cDDI was identified, both drugs were 44% less likely to be co-administered following eMM (adjusted odds ratio 0.56, 95% confidence interval 0.46-0.73). CONCLUSION: Although most patients experienced a pDDI during their hospital stay, less than one-third of pDDIs were clinically relevant. The low prevalence of harm identified raises questions about the value of incorporating DDI decision support into systems given the potential negative impacts of DDI alerts.

Comparison of the validity, perceived usefulness, and usability of I-MeDeSA and TEMAS, two tools to evaluate alert system usability.

OBJECTIVE: Two tools are currently available in the literature to evaluate the usability of medication alert systems, the instrument for evaluating human factors principles in medication-related decision support alerts (I-MeDeSA) and the tool for evaluating medication alerting systems (TEMAS). This study aimed to compare their convergent validity, perceived usability, usefulness, strengths, and weaknesses, as well as users' preferences. METHOD: To evaluate convergent validity, two experts mapped TEMAS' items against I-MeDeSA's items with respect to the usability dimensions they target. To assess perceived usability, usefulness, strengths, and weaknesses of both tools, staff with expertise in their medication alerting system were asked to use French versions of the TEMAS and I-MeDeSA. After the use of each tool, participants were asked to complete the System Usability Scale (SUS) and answer questions about the understandability and usefulness of each tool. Finally, participants were asked to name their preferred tool. Numeric scores were statistically compared. Free-text responses were analyzed using an inductive approach. RESULTS: Forty-five participants from 10 hospitals took part in the study. In terms of convergent validity, I-MeDeSA focuses more on the usability of the graphical user interface while TEMAS considers a wider range of usability principles. Both tools have a fair level of perceived usability (I-MeDeSA' SUS score = 61.85 and TEMAS' SUS score = 62.87), but results highlight that revisions are necessary to both tools to improve their usability. Participants found TEMAS more useful than I-MeDeSA (t = -3.63, p =.005) and had a clear preference for TEMAS to identify problems in formative evaluation (39 of 45; 0.867, p <.001) and to compare the usability of alert systems during the procurement process (36 of 45; 0.8, p <.001). CONCLUSIONS: The TEMAS is perceived as more useful and is preferred by participants. The I-MeDeSA seems more relevant for quick evaluations that focus on the graphical user interface. The TEMAS seems to be more suitable for in-depth usability evaluations of alert systems. Even if both tools are perceived to be equally usable, they suffer from wording, instructional, and organizational problems that hinder their use. The results of this study will be used to improve the design of I-MeDeSA and TEMAS.

Machine learning identifies a COVID-19-specific phenotype in university students using a mental health app.

Background: Advances in smartphone technology have allowed people to access mental healthcare via digital apps from wherever and whenever they choose. University students experience a high burden of mental health concerns. Although these apps improve mental health symptoms, user engagement has remained low. Studies have shown that users can be subgrouped based on unique characteristics that just-in-time adaptive interventions (JITAIs) can use to improve engagement. To date, however, no studies have examined the effect of the COVID-19 pandemic on these subgroups. Objective: Here, we sought to examine user subgroup characteristics across three COVID-19-specific timepoints: during lockdown, immediately following lockdown, and three months after lockdown ended. Methods: To do this, we used a two-step machine learning approach combining unsupervised and supervised machine learning. Results: We demonstrate that there are three unique subgroups of university students who access mental health apps. Two of these, with either higher or lower mental well-being, were defined by characteristics that were stable across COVID-19 timepoints. The third, situational well-being, had characteristics that were timepoint-dependent, suggesting that they are highly influenced by traumatic stressors and stressful situations. This subgroup also showed feelings and behaviours consistent with burnout. Conclusions: Overall, our findings clearly suggest that user subgroups are unique: they have different characteristics and therefore likely have different mental healthcare goals. Our findings also highlight the importance of including questions and additional interventions targeting traumatic stress(ors), reason(s) for use, and burnout in JITAI-style mental health apps to improve engagement.

Protocol for a bandit-based response adaptive trial to evaluate the effectiveness of brief self-guided digital interventions for reducing psychological distress in university students: the Vibe Up study.

INTRODUCTION: Meta-analytical evidence confirms a range of interventions, including mindfulness, physical activity and sleep hygiene, can reduce psychological distress in university students. However, it is unclear which intervention is most effective. Artificial intelligence (AI)-driven adaptive trials may be an efficient method to determine what works best and for whom. The primary purpose of the study is to rank the effectiveness of mindfulness, physical activity, sleep hygiene and an active control on reducing distress, using a multiarm contextual bandit-based AI-adaptive trial method. Furthermore, the study will explore which interventions have the largest effect for students with different levels of baseline distress severity. METHODS AND ANALYSIS: The Vibe Up study is a pragmatically oriented, decentralised AI-adaptive group sequential randomised controlled trial comparing the effectiveness of one of three brief, 2-week digital self-guided interventions (mindfulness, physical activity or sleep hygiene) or active control (ecological momentary assessment) in reducing self-reported psychological distress in Australian university students. The adaptive trial methodology involves up to 12 sequential mini-trials that allow for the optimisation of allocation ratios. The primary outcome is change in psychological distress (Depression, Anxiety and Stress Scale, 21-item version, DASS-21 total score) from preintervention to postintervention. Secondary outcomes include change in physical activity, sleep quality and mindfulness from preintervention to postintervention. Planned contrasts will compare the four groups (ie, the three intervention and control) using self-reported psychological distress at prespecified time points for interim analyses. The study aims to determine the best performing intervention, as well as ranking of other interventions. ETHICS AND DISSEMINATION: Ethical approval was sought and obtained from the UNSW Sydney Human Research Ethics Committee (HREC A, HC200466). A trial protocol adhering to the requirements of the Guideline for Good Clinical Practice was prepared for and approved by the Sponsor, UNSW Sydney (Protocol number: HC200466_CTP). TRIAL REGISTRATION NUMBER: ACTRN12621001223820.

Prescribers' reported acceptance and use of drug-drug interaction alerts: An Australian survey.

Drug-drug interaction (DDI) alerts are frequently included in electronic medical record (eMR) systems to provide users with relevant information and guidance at the point of care. In this study, we aimed to examine views of DDI alerts among prescribers, including junior doctors, registrars and senior doctors, across Australia. A validated survey for assessing prescribers' reported acceptance and use of DDI alerts was distributed among researcher networks and in newsletters. Fifty useable responses were received, more than half (n = 28) from senior doctors. Prescribers at all levels expected DDI alerts to improve performance but junior doctors reported that this was at a high cost, with respect to time and effort. Senior doctors and registrars reported rarely reading alerts and rarely changing prescribing decisions based on alerts. Respondents identified a number of problems with current alerts including limited relevance, repetition, and poor design, highlighting some clear areas for alert improvement.

Comparison of the validity, perceived usefulness and usability of I-MeDeSA and TEMAS, two tools to evaluate alert system usability: a study protocol.

INTRODUCTION: Research has shown that improvements to the usability of medication alert systems are needed. For designers and decisions-makers to assess usability of their alert systems, two paper-based tools are currently available: the instrument for evaluating human-factors principles in medication-related decision support alerts (I-MeDeSA) and the tool for evaluating medication alerting systems (TEMAS). This study aims to compare the validity, usability and usefulness of both tools to identify their strengths and limitations and assist designers and decision-makers in making an informed decision about which tool is most suitable for assessing their current or prospective system. METHODS AND ANALYSIS: First, TEMAS and I-MeDeSA will be translated into French. This translation will be validated by three experts in human factors. Then, in 12 French hospitals with a medication alert system in place, staff with expertise in the system will evaluate their alert system using the two tools successively. After the use of each tool, participants will be asked to fill in the System Usability Scale (SUS) and complete a survey on the understandability and perceived usefulness of each tool. Following the completion of both assessments, participants will be asked to nominate their preferred tool and relay their opinions on the tools. The design philosophy of TEMAS and I-MeDeSA differs on the calculation of a score, impacting the way the comparison between the tools can be performed. Convergent validity will be evaluated by matching the items of the two tools with respect to the usability dimensions they assess. SUS scores and answers to the survey will be statistically compared for I-MeDeSA and TEMAS to identify differences. Free-text responses in surveys will be analysed using an inductive approach. ETHICS AND DISSEMINATION: Ethical approval is not required in France for a study of this nature. The results will be published in a peer-reviewed journal.

Medication errors related to computerized provider order entry systems in hospitals and how they change over time: A narrative review.

BACKGROUND: Evaluations of computerized provider order entry (CPOE) systems have revealed that reductions in certain types of medication errors occur simultaneously with the emergence of system-related errors - errors that are unlikely or not possible to occur with the use of paper-based medication charts. System-related errors appear to persist many years post-implementation of CPOE, although little is known about whether the types and rates of system-related errors that occur immediately following CPOE implementation are similar to those that endure or emerge after years of system use. OBJECTIVE: To analyze and synthesize the literature on system-related errors, specifically in relation to the length of time that CPOE systems have been in use, to determine what is currently known about how system-related errors change over time. METHODS: A literature search was undertaken using the PubMed database to identify English language articles published between January 2005 and March 2020 that provided original data on system-related errors resulting from CPOE system use. Studies were included if they provided results on system-related errors and information relating to the length of time that CPOE had been in use. RESULTS: Thirty-one studies met the inclusion criteria for this narrative review. System-related errors were identified and described during short, medium and long-term use of CPOE systems, but no single study examined how errors changed over time. In comparing findings across studies, results suggest that system-related errors persist with long-term use of CPOE systems, although likely to occur at a reduced rate. CONCLUSIONS: This review has highlighted a significant gap in knowledge on how system-related errors change over time. Determining what and when system-related errors occur and the system factors that contribute to their occurrence at different time points after CPOE implementation is necessary for the future prevention and mitigation of these errors.

The impact of introducing automated dispensing cabinets, barcode medication administration, and closed-loop electronic medication management systems on work processes and safety of controlled medications in hospitals: A systematic review.

BACKGROUND: Technology in the form of Automated Dispensing Cabinets (ADCs), Barcode Medication Administration (BCMA), and closed-loop Electronic Medication Management Systems (EMMS) are implemented in hospitals to assist with the supply, use and monitoring of medications. Although there is evidence to suggest that these technologies can reduce errors and improve monitoring of medications in general, little is known about their impact on controlled medications such as opioids. OBJECTIVES: This review aimed to fill this knowledge gap by synthesising literature to determine the impact of ADCs, BCMA and closed-loop EMMS on clinical work processes, medication safety, and drug diversion associated with controlled medications in the inpatient setting. METHODS: Eight databases (Medline, Pubmed, Embase, Scopus, Web of Science, PsycINFO, CINAHL, and ScienceDirect) were searched for relevant papers published between January 2000 and May 2019. Qualitative, quantitative, and mixed-methods empirical studies published in English that reported findings on the impact of ADCs, BCMA and/or closed-loop EMMS on controlled medications in the inpatient setting were included. RESULTS: In total, 16 papers met the inclusion criteria. Eleven studies reported on ADCs, four on BCMA, and only one on closed-loop EMMS. Only four studies focused on controlled medications, with the remainder reporting only incidental findings. Studies reported the elimination of manual end-of-shift counts of controlled medications after ADC implementation but cases of drug diversion were reported despite introducing ADCs. Three quantitative studies reported reductions in medication errors after implementing BCMA, but medications labelled with wrong barcodes and unreadable barcodes led to confusion and administration errors. CONCLUSIONS: More quality, targeted research is needed to provide evidence on the benefits and also risks of implementing technology to safeguard against inappropriate use of controlled medications in the inpatient setting. Processes need to be in place to supplement technological capabilities, and resources should be made available for post-implementation evaluations and interventions.

Optimizing clinical decision support alerts in electronic medical records: a systematic review of reported strategies adopted by hospitals.

OBJECTIVE: To identify and summarize the current internal governance processes adopted by hospitals, as reported in the literature, for selecting, optimizing, and evaluating clinical decision support (CDS) alerts in order to identify effective approaches. MATERIALS AND METHODS: Databases (Medline, Embase, CINAHL, Scopus, Web of Science, IEEE Xplore Digital Library, CADTH, and WorldCat) were searched to identify relevant papers published from January 2010 to April 2020. All paper types published in English that reported governance processes for selecting and/or optimizing CDS alerts in hospitals were included. RESULTS: Eight papers were included in the review. Seven papers focused specifically on medication-related CDS alerts. All papers described the use of a multidisciplinary committee to optimize alerts. Other strategies included the use of clinician feedback, alert data, literature and drug references, and a visual dashboard. Six of the 8 papers reported evaluations of their CDS alert modifications following the adoption of optimization strategies, and of these, 5 reported a reduction in alert rate. CONCLUSIONS: A multidisciplinary committee, often in combination with other approaches, was the most frequent strategy reported by hospitals to optimize their CDS alerts. Due to the limited number of published processes, variation in system changes, and evaluation results, we were unable to compare the effectiveness of different strategies, although employing multiple strategies appears to be an effective approach for reducing CDS alert numbers. We recommend hospitals report on descriptions and evaluations of governance processes to enable identification of effective strategies for optimization of CDS alerts in hospitals.

A Tool for Evaluating Medication Alerting Systems: Development and Initial Assessment.

BACKGROUND: It is well known that recommendations from electronic medication alerts are seldom accepted or acted on by users. Key factors affecting the effectiveness of medication alerts include system usability and alert design. Thus, human factors principles that apply knowledge of human capabilities and limitations are increasingly used in the design of health technology to improve the usability of systems. OBJECTIVE: This study aims to evaluate a newly developed evidence-based self-assessment tool that allows the valid and reliable evaluation of computerized medication alerting systems. This tool was developed to be used by hospital staff with detailed knowledge of their hospital's computerized provider order entry system and alerts to identify and address potential system deficiencies. In this initial assessment, we aim to determine whether the items in the tool can measure compliance of medication alerting systems with human factors principles of design, the tool can be consistently used by multiple users to assess the same system, and the items are easy to understand and perceived to be useful for assessing medication alerting systems. METHODS: The Tool for Evaluating Medication Alerting Systems (TEMAS) was developed based on human factors design principles and consisted of 66 items. In total, 18 staff members recruited across 6 hospitals used the TEMAS to assess their medication alerting systems. Data collected from participant assessments were used to evaluate the validity, reliability, and usability of the TEMAS. Validity was assessed by comparing the results of the TEMAS with those of prior in-house evaluations. Reliability was measured using Krippendorff α to determine agreement among assessors. A 7-item survey was used to determine usability. RESULTS: The participants reported mostly negative (n=8) and neutral (n=7) perceptions of alerts in their medication alerting system. However, the validity of the TEMAS could not be directly tested, as participants were unaware of any results from prior in-house evaluations. The reliability of the TEMAS, as measured by Krippendorff α, was low to moderate (range 0.26-0.46); however, participant feedback suggests that individuals' knowledge of the system varied according to their professional background. In terms of usability, 61% (11/18) of participants reported that the TEMAS items were generally easy to understand; however, participants suggested the revision of 22 items to improve clarity. CONCLUSIONS: This initial assessment of the TEMAS allowed the identification of its components that required modification to improve usability and usefulness. It also revealed that for the TEMAS to be effective in facilitating a comprehensive assessment of a medication alerting system, it should be completed by a multidisciplinary team of hospital staff from both clinical and technical backgrounds to maximize their knowledge of systems.