Adult patient access to electronic health records.

BACKGROUND: To support patient-centred care, healthcare organisations increasingly offer patients access to data stored in the institutional electronic health record (EHR). OBJECTIVES: Primary objective 1. To assess the effects of providing adult patients with access to electronic health records (EHRs) alone or with additional functionalities on a range of patient, patient-provider, and health resource consumption outcomes, including patient knowledge and understanding, patient empowerment, patient adherence, patient satisfaction with care, adverse events, health-related quality of life, health-related outcomes, psychosocial health outcomes, health resource consumption, and patient-provider communication. Secondary objective 1. To assess whether effects of providing adult patients with EHR access alone versus EHR access with additional functionalities differ among patient groups according to age, educational level, or different status of disease (chronic or acute). SEARCH METHODS: We searched CENTRAL, MEDLINE, Embase, PsycINFO, CINAHL, and Scopus in June 2017 and in April 2020. SELECTION CRITERIA: Randomised controlled trials and cluster-randomised trials of EHR access with or without additional functionalities for adults with any medical condition. DATA COLLECTION AND ANALYSIS: We used standard Cochrane methodological procedures. MAIN RESULTS: We included 10 studies with 78 to 4500 participants and follow-up from 3 to 24 months. Nine studies assessed the effects of EHR with additional functionalities, each addressing a subset of outcomes sought by this review. Five studies focused on patients with diabetes mellitus, four on patients with specific diseases, and one on all patients. All studies compared EHR access alone or with additional functionalities plus usual care versus usual care only. No studies assessing the effects of EHR access alone versus EHR access with additional functionalities were identified. Interventions required a variety of data within the EHR, such as patient history, problem list, medication, allergies, and lab results. In addition to EHR access, eight studies allowed patients to share self-documented data, seven offered individualised disease management functions, seven offered educational disease-related information, six supported secure communication, and one offered preventive reminders. Only two studies were at low or unclear risk of bias across domains. Meta-analysis could not be performed, as participants, interventions, and outcomes were too heterogeneous, and most studies presented results based on different adjustment methods or variables. The quality of evidence was rated as low or very low across outcomes. Overall differences between intervention and control groups, if any, were small. The relevance of any small effects remains unclear for most outcomes because in most cases, trial authors did not define a minimal clinically important difference. Overall, results suggest that the effects of EHR access alone and with additional functionalities are mostly uncertain when compared with usual care. Patient knowledge and understanding: very low-quality evidence is available from one study, so we are uncertain about effects of the intervention on patient knowledge about diabetes and blood glucose testing. Patient empowerment: low-quality evidence from three studies suggests that the intervention may have little or no effect on patient empowerment measures. Patient adherence: low-quality evidence from two studies suggests that the intervention may slightly improve adherence to the process of monitoring risk factors and preventive services. Effects on medication adherence are conflicting in two studies; this may or may not improve to a clinically relevant degree. Patient satisfaction with care: low-quality evidence from three studies suggests that the intervention may have little or no effect on patient satisfaction, with conflicting results. Adverse events: two small studies reported on mortality; one of these also reported on serious and other adverse events, but sample sizes were too small for small differences to be detected. Therefore, low-quality evidence suggests that the intervention may have little to no effect on mortality and other adverse events. Health-related quality of life: only very low-quality evidence from one study is available. We are uncertain whether the intervention improves disease-specific quality of life of patients with asthma. Health-related outcomes: low-quality evidence from eight studies suggests that the intervention may have little to no effect on asthma control, glycosylated haemoglobin (HbA1c) levels, blood pressure, low-density lipoprotein or total cholesterol levels, body mass index or weight, or 10-year Framingham risk scores. Low-quality evidence from one study suggests that the composite scores of risk factors for diabetes mellitus may improve slightly with the intervention, but there is uncertainty about effects on ophthalmic medications or intraocular pressure. Psychosocial health outcomes: no study investigated psychosocial health outcomes in a more than anecdotal way. Health resource consumption: low-quality evidence for adult patients in three studies suggests that there may be little to no effect of the intervention on different measures of healthcare use. Patient-provider communication: very low-quality evidence is available from a single small study, and we are uncertain whether the intervention improves communication measures, such as the number of messages sent. AUTHORS' CONCLUSIONS: The effects of EHR access with additional functionalities in comparison with usual care for the most part are uncertain. Only adherence to the process of monitoring risk factors and providing preventive services as well as a composite score of risk factors for diabetes mellitus may improve slightly with EHR access with additional functionalities. Due to inconsistent terminology in this area, our search may have missed relevant studies. As the overall quality of evidence is very low to low, future research is likely to change these results. Further trials should investigate the impact of EHR access in a broader range of countries and clinical settings, including more patients over a longer period of follow-up, as this may increase the likelihood of detecting effects of the intervention, should these exist. More studies should focus on assessing outcomes such as patient empowerment and behavioural outcomes, rather than concentrating on health-related outcomes alone. Future studies should distinguish between effects of EHR access only and effects of additional functionalities, and investigate the impact of mobile EHR tools. Future studies should include information on usage patterns, and consider the potential for widening health inequalities with implementation of EHR access. A taxonomy for EHR access and additional functionalities should be developed to promote consistency and comparability of outcome measures, and facilitate future reviews by better enabling cross-study comparisons.

Improvement and Evaluation of the TOPCOP Taxonomy of Patient Portals: Taxonomy-Evaluation-Delphi (TED) Approach.

BACKGROUND: Patient portals have been introduced in many countries over the last 10 years, but many health information managers still feel they have too little knowledge of patient portals. A taxonomy can help them to better compare and select portals. This has led us to develop the TOPCOP taxonomy for classifying and comparing patient portals. However, the taxonomy has not been evaluated by users. OBJECTIVE: This study aimed to evaluate the taxonomy's usefulness to support health information managers in comparing, classifying, defining a requirement profile for, and selecting patient portals and to improve the taxonomy where needed. METHODS: We used a modified Delphi approach. We sampled a heterogeneous panel of 13 health information managers from 3 countries using the criterion sampling strategy. We conducted 4 anonymous survey rounds with qualitative and quantitative questions. In round 1, the panelists assessed the appropriateness of each dimension, and we collected new ideas to improve the dimensions. In rounds 2 and 3, the panelists iteratively evaluated the taxonomy that was revised based on round 1. In round 4, the panelists assessed the need for a taxonomy and the appropriateness of patient engagement as a distinguishing concept. Then, they compared 2 real portals with the final taxonomy and evaluated its usefulness for comparing portals, creating an initial requirement profile, and selecting patient portals. To determine group consensus, we applied the RAND/UCLA Appropriateness Method. RESULTS: The final taxonomy consists of 25 dimensions with 65 characteristics. Five new dimensions were added to the original taxonomy, with 8 characteristics added to already existing dimensions. Group consensus was achieved on the need for such a taxonomy to compare portals, on patient engagement as an appropriate distinguishing concept, and on the comprehensibility of the taxonomy's form. Further, consensus was achieved on the taxonomy's usefulness for classifying and comparing portals, assisting users in better understanding portals, creating a requirement profile, and selecting portals. This allowed us to test the usefulness of the final taxonomy with the intended users. CONCLUSIONS: The TOPCOP taxonomy aims to support health information managers in comparing and selecting patient portals. By providing a standardized terminology to describe various aspects of patient portals independent of clinical setting or country, the taxonomy will also be useful for advancing research and evaluation of patient portals.

Applying and Extending the FITT Framework to Identify the Challenges and Opportunities of Successful eHealth Services for Patient Self-Management: Qualitative Interview Study.

BACKGROUND: The number of public eHealth services that support patient self-management is rapidly increasing. However, the implementation of these eHealth services for self-management has encountered challenges. OBJECTIVE: The purpose of this paper was to analyze the challenges and opportunities of implementing eHealth services for self-management by focusing on the fit between the technical solution and clinical use. METHODS: We performed in-depth interviews with 10 clinical project coordinators and managers who were responsible for developing and implementing various eHealth services for self-management interventions in five university hospitals in Finland. The results were analyzed using content analysis and open coding. The Fit between Individuals, Task, and Technology (FITT) framework was used to interpret the findings. RESULTS: The implementation of self-management services involved many challenges related to technical problems, health professional acceptance, patient motivation, and health organization and management. The implementers identified practices to manage the identified challenges, including improving the design of the technology, supporting health professionals in the adoption of the eHealth services, changing the work processes and tasks, involving patients, and collectively planning the implementation inside an organization. The findings could be mostly attributed to the dimensions of the FITT framework. CONCLUSIONS: The FITT framework helped to analyze the challenges related to the implementation, and most of them were related to poor fit. The importance of patients as stakeholders in eHealth services for patient self-management needs to be highlighted. Thus, we propose that patients should be added as a different type of individual dimension to the FITT framework. In addition, the framework could be extended to include organization and management in a new context dimension.

Evidence-based usability design principles for medication alerting systems.

BACKGROUND: Usability flaws in medication alerting systems may have a negative impact on clinical use and patient safety. In order to prevent the release of alerting systems that contain such flaws, it is necessary to provide designers and evaluators with evidence-based usability design principles. The objective of the present study was to develop a comprehensive, structured list of evidence-based usability design principles for medication alerting systems. METHODS: Nine sets of design principles for medication alerting systems were analyzed, summarized, and structured. We then matched the summarized principles with a list of usability flaws in order to determine the level of underlying evidence. RESULTS: Fifty-eight principles were summarized from the literature and two additional principles were defined, so that each flaw was matched with a principle. We organized the 60 summarized usability design principles into 6 meta-principles, 38 principles, and 16 sub-principles. Only 15 principles were not matched with a usability flaw. The 6 meta-principles respectively covered the improvement of the signal-to-noise ratio, the support for collaborative working, the fit with a clinician's workflow, the data display, the transparency of the alerting system, and the actionable tools to be provided within an alert. CONCLUSIONS: It is possible to develop an evidence-based, structured, comprehensive list of usability design principles that are specific to medication alerting systems and are illustrated by the corresponding usability flaws. This list represents an improvement over the current literature. Each principle is now associated with the best available evidence of its violation. This knowledge may help to improve the usability of medication alerting systems and, ultimately, decrease the harmful consequences of the systems' usability flaws.

Innovative Power of Health Care Organisations Affects IT Adoption: A bi-National Health IT Benchmark Comparing Austria and Germany.

Multinational health IT benchmarks foster cross-country learning and have been employed at various levels, e.g. OECD and Nordic countries. A bi-national benchmark study conducted in 2007 revealed a significantly higher adoption of health IT in Austria compared to Germany, two countries with comparable healthcare systems. We now investigated whether these differences still persisted. We further studied whether these differences were associated with hospital intrinsic factors, i.e. the innovative power of the organisation and hospital demographics. We thus performed a survey to measure the "perceived IT availability" and the "innovative power of the hospital" of 464 German and 70 Austrian hospitals. The survey was based on a questionnaire with 52 items and was given to the directors of nursing in 2013/2014. Our findings confirmed a significantly greater IT availability in Austria than in Germany. This was visible in the aggregated IT adoption composite score "IT function" as well as in the IT adoption for the individual functions "nursing documentation" (OR = 5.98), "intensive care unit (ICU) documentation" (OR = 2.49), "medication administration documentation" (OR = 2.48), "electronic archive" (OR = 2.27) and "medication" (OR = 2.16). "Innovative power" was the strongest factor to explain the variance of the composite score "IT function". It was effective in hospitals of both countries but significantly more effective in Austria than in Germany. "Hospital size" and "hospital system affiliation" were also significantly associated with the composite score "IT function", but they did not differ between the countries. These findings can be partly associated with the national characteristics. Indicators point to a more favourable financial situation in Austrian hospitals; we thus argue that Austrian hospitals may possess a larger degree of financial freedom to be innovative and to act accordingly. This study is the first to empirically demonstrate the effect of "innovative power" in hospitals on health IT adoption in a bi-national health IT benchmark. We recommend directly including the financial situation into future regression models. On a political level, measures to stimulate the "innovative power" of hospitals should be considered to increase the digitalisation of healthcare.

Usability flaws of medication-related alerting functions: A systematic qualitative review.

INTRODUCTION: Medication-related alerting functions may include usability flaws that limit their optimal use. A first step on the way to preventing usability flaws is to understand the characteristics of these usability flaws. This systematic qualitative review aims to analyze the type of usability flaws found in medication-related alerting functions. METHOD: Papers were searched via PubMed, Scopus and Ergonomics Abstracts databases, along with references lists. Paper selection, data extraction and data analysis was performed by two to three Human Factors experts. Meaningful semantic units representing instances of usability flaws were the main data extracted. They were analyzed through qualitative methods: categorization following general usability heuristics and through an inductive process for the flaws specific to medication-related alerting functions. MAIN RESULTS: From the 6380 papers initially identified, 26 met all eligibility criteria. The analysis of the papers identified a total of 168 instances of usability flaws that could be classified into 13 categories of usability flaws representing either violations of general usability principles (i.e. they could be found in any system, e.g. guidance and workload issues) or infractions specific to medication-related alerting functions. The latter refer to issues of low signal-to-noise ratio, incomplete content of alerts, transparency, presentation mode and timing, missing alert features, tasks and control distribution. MAIN CONCLUSION: The list of 168 instances of usability flaws of medication-related alerting functions provides a source of knowledge for checking the usability of medication-related alerting functions during their design and evaluation process and ultimately constructs evidence-based usability design principles for these functions.

Implementation of the Austrian Nursing Minimum Data Set (NMDS-AT): A Feasibility Study.

BACKGROUND: An Austrian Nursing Minimum Data Set (NMDS-AT) has been developed to describe the diversity of patient populations and variability of nursing care based on nursing diagnoses, nursing interventions, and nursing outcomes. The aim of this study is to test the feasibility of using this NMDS-AT by assessing the availability of data needed for the NMDS-AT in routine nursing documentation, and to assess its reliability and usefulness. METHODS: Data were collected in a general hospital from patient records of 20 patients representing 457 patient days. Availability of needed data was assessed by two raters in a chart review based on an NMDS-AT form. The interrater reliability (n=20) and intrarater reliability (n=5) was assessed using Cohen's kappa coefficient and intraclass correlation coefficient (ICC). Usefulness was assessed by verifying whether typical analysis questions can be answered by the documented NMDS-AT data. RESULTS: In the 20 patient records, thirteen nursing diagnoses, 50 nursing interventions, and five nursing outcomes occurred, representing 68 (58.6%) of the overall 116 data elements of the NMDS-AT. The data were found at different data sources (e.g., electronic nursing record or paper-based fever chart) and in various forms (e.g., standardized or free text). The interrater reliability of the thirteen nursing diagnoses showed kappa values (percentage of agreement) ranging from 0.35 (85%) to 1.00 (100%). The 50 nursing interventions showed ICCs ranging from 0.03 to 1.00. All nursing outcomes showed an ICC of 1.00. The intrarater reliability showed 100% agreement. Performing typical analysis questions showed that the extracted NMDS-AT data are able to answer questions of clinical management, of policy makers, and of nursing science. CONCLUSIONS: The NMDS-AT was found to be feasible: needed data was available in the analysed patient records, data extraction showed good reliability, and typical analysis could be performed and showed interesting results. Before the NMDS-AT can be introduced in healthcare institutions, the following challenges need to be addressed: 1. improve the quality of nursing documentation; 2. reduce fragmentation of documentation; 3. use a standardized nursing classification system; and 4. establish mappings between nursing classification systems and the NMDS-AT.

Reflections on 'health care in the information society--a prognosis for the year 2013'.

In the year 2000 a talk was given and later published on how health care could look like in the year 2013. The aim was to "identify priorities in medical informatics research and necessary activities by policymakers in order to ensure an efficient, advanced, and yet affordable future health care system" (Haux R, Ammenwerth E, Herzog W, Knaup P. Health care in the information society. A prognosis for the year 2013. Int J Med Inform. 2002; 66: 3-21). In the year 2013 a workshop took place, critically reflecting on the prognoses and theses made for the year 2013. This editorial is reporting on the 2013-workshop and is introducing three papers, in which the workshop results have been summarized.

Assessing the prognoses on Health care in the information society 2013--thirteen years after.

Health care and information technology in health care is advancing at tremendous speed. We analysed whether the prognoses by Haux et al. - first presented in 2000 and published in 2002 - have been fulfilled in 2013 and which might be the reasons for match or mismatch. Twenty international experts in biomedical and health informatics met in May 2013 in a workshop to discuss match or mismatch of each of the 71 prognoses. After this meeting a web-based survey among workshop participants took place. Thirty-three prognoses were assessed matching; they reflect e.g. that there is good progress in storing patient data electronically in health care institutions. Twenty-three prognoses were assessed mismatching; they reflect e.g. that telemedicine and home monitoring as well as electronic exchange of patient data between institutions is not established as widespread as expected. Fifteen prognoses were assessed neither matching nor mismatching. ICT tools have considerably influenced health care in the last decade, but in many cases not as far as it was expected by Haux et al. in 2002. In most cases this is not a matter of the availability of technical solutions but of organizational and ethical issues. We need innovative and modern information system architectures which support multiple use of data for patient care as well as for research and reporting and which are able to integrate data from home monitoring into a patient centered health record. Since innovative technology is available the efficient and wide-spread use in health care has to be enabled by systematic information management.

Past and next 10 years of medical informatics.

More than 10 years ago Haux et al. tried to answer the question how health care provision will look like in the year 2013. A follow-up workshop was held in Braunschweig, Germany, for 2 days in May, 2013, with 20 invited international experts in biomedical and health informatics. Among other things it had the objectives to discuss the suggested goals and measures of 2002 and how priorities on MI research in this context should be set from the viewpoint of today. The goals from 2002 are now as up-to-date as they were then. The experts stated that the three goals: "patient-centred recording and use of medical data for cooperative care"; "process-integrated decision support through current medical knowledge" and "comprehensive use of patient data for research and health care reporting" have not been reached yet and are still relevant. A new goal for ICT in health care should be the support of patient centred personalized (individual) medicine. MI as an academic discipline carries out research concerning tools that support health care professionals in their work. This research should be carried out without the pressure that it should lead to systems that are immediately and directly accepted in practice.

[Nursing minimum data sets (NMDS) - a literature review relating to objective and data elements]. / Nursing Minimum Data Sets (NMDS) - eine Literaturübersicht bezüglich Zielsetzungen und Datenelementen.

BACKGROUND: In order to be able to follow the predicted changes in healthcare systems, there has long been a need for a unified database that could transparently compare nursing care data from different service providers. In Austria, a recommendation has been lacking thus far as to which nursing care data need to be documented as "basic data", and as a result of this, comparisons of nursing care data on a national basis have been hindered. The international development of Nursing Minimum Data Sets (NMDS) has demonstrated that nursing care data can be sufficiently compared. AIM: The aim of the systematic literature review is to raise the current level of knowledge regarding NMDS and to develop a structured description of NMDS which, above all else, can document the recorded data elements and the associated objectives with the use of NMDS. RESULTS: A total of seventy publications on the subject of NMDS were included in the literature overview. The analysis of the eight NMDS that were presented yielded six central objectives and six higher-level data elements. The identified objectives include: the description of the nursing care practice, distribution of the financial means, benchmarking, human resources planning, trend analyses, and quality assurance. The six data elements that were identified comprise operating data, demographic data, and the diagnoses, interventions, results (quality indicators), and intensity of nursing care (clinical data). DISCUSSION: What emerged was that no clear association is present between the objectives and the data elements of the minimum nursing care datasets that were studied.

Clinical Utility and Usability of the Digital Box and Block Test: Mixed Methods Study.

Background: The Box and Block Test (BBT) is a clinical tool used to measure hand dexterity, which is often used for tracking disease progression or the effectiveness of therapy, particularly benefiting older adults and those with neurological conditions. Digitizing the measurement of hand function may enhance the quality of data collection. We have developed and validated a prototype that digitizes this test, known as the digital BBT (dBBT), which automatically measures time and determines and displays the test result. Objective: This study aimed to investigate the clinical utility and usability of the newly developed dBBT and to collect suggestions for future improvements. Methods: A total of 4 occupational therapists participated in our study. To evaluate the clinical utility, we compared the dBBT to the BBT across dimensions such as acceptance, portability, energy and effort, time, and costs. We observed therapists using the dBBT as a dexterity measurement tool and conducted a quantitative usability questionnaire using the System Usability Scale (SUS), along with a focus group. Evaluative, structured, and qualitative content analysis was used for the qualitative data, whereas quantitative analysis was applied to questionnaire data. The qualitative and quantitative data were merged and analyzed using a convergent mixed methods approach. Results: Overall, the results of the evaluative content analysis suggested that the dBBT had a better clinical utility than the original BBT, with ratings of all collected participant statements for the dBBT being 45% (45/99) equal to, 48% (48/99) better than, and 6% (6/99) lesser than the BBT. Particularly in the subcategories "acceptance," "time required for evaluation," and "purchase costs," the dBBT was rated as being better than the original BBT. The dBBT achieved a mean SUS score of 83 (95% CI 76-96). Additionally, several suggested changes to the system were identified. Conclusions: The study demonstrated an overall positive evaluation of the clinical utility and usability of the dBBT. Valuable insights were gathered for future system iterations. These pioneering results highlight the potential of digitizing hand dexterity assessments.

Agile eHealth Usability Evaluation: A Triangulative Approach to Promoting the Usability of eHealth Systems.

BACKGROUND: Usability evaluation is difficult to reconcile with agile software development for eHealth systems, because traditional usability evaluation is often complex and cumbersome to implement. However, obtaining prospective users' feedback during agile software development is crucial for improving the usability of eHealth systems, which is why there is an increasing need for agile eHealth usability evaluation. OBJECTIVE: This study investigates whether agile usability evaluations are suitable to evaluate patient-centered eHealth systems being agile developed in health care and are applicable for prospective users, such as older persons suffering from age-related declines. METHODS: A triangulation study was conducted combining iterative expert interviews with an exploratory case study. RESULTS: The triangulation study revealed that the implementation of an agile eHealth usability evaluation with prospective users such as older persons proved to be possible. CONCLUSION: Established eHealth usability evaluation methods must be further evolved to address age-related impairments of older persons.

Visualizing Study Profiles Using the GMDS Competency Catalog for Medical Informatics Programs.

BACKGROUND: Medical informatics programs cover a variety of topics. OBJECTIVES: To test the utility of the GMDS medical informatics competency catalog in comparing programs by developing study profiles. METHODS: Coverage of 234 competencies is recorded and visualized in a spider diagram. RESULTS: Spider diagrams allow visualizing various study profiles. CONCLUSION: The GMDS catalog seems useful for comparing medical informatics study programs, e.g., for interested students, employers, or accreditation reviewers.

Automatic Classification of Online Discussions and Other Learning Traces to Detect Cognitive Presence.

Cognitive presence is a core construct of the Community of Inquiry (CoI) framework. It is considered crucial for deep and meaningful online-based learning. CoI-based real-time dashboards visualizing students' cognitive presence may help instructors to monitor and support students' learning progress. Such real-time classifiers are often based on the linguistic analysis of the content of posts made by students. It is unclear whether these classifiers could be improved by considering other learning traces, such as files attached to students' posts. We aimed to develop a German-language cognitive presence classifier that includes linguistic analysis using the Linguistic Inquiry and Word Count (LIWC) tool and other learning traces based on 1,521 manually coded meaningful units from an online-based university course. As learning traces, we included not only the linguistic features from the LIWC tool, but also features such as attaching files to a post, tagging, or using terms from the course glossary. We used the k-nearest neighbor method, a random forest model, and a multilayer perceptron as classifiers. The results showed an accuracy of up to 82% and a Cohen's κ of 0.76 for the cognitive presence classifier for German posts. Including learning traces did not improve the predictive ability. In conclusion, we developed an automatic classifier for German-language courses based on a linguistic analysis of students' posts. This classifier is a step toward a teacher dashboard. Our work also provides the first fully CoI-coded German dataset for future research on cognitive presence.

Clinical evaluation of the ADE scorecards as a decision support tool for adverse drug event analysis and medication safety management.

AIMS: The prevention of adverse drug events (ADEs) demands co-ordination of different health care professionals. ADE scorecards are a novel approach to raise the team awareness regarding ADE risks and causes. It makes information on numbers and on possible causes of possible ADE cases available to the clinical team. The aim of the study was to investigate the usage and acceptance of ADE scorecards by healthcare professionals and their impact on rates of possible ADEs. METHODS: ADE scorecards were introduced in three departments of a French hospital. A controlled time series analysis of ADE data was conducted to assess the impact of the ADE scorecards. In addition, qualitative interviews and a standardized survey with all participating staff members were performed. RESULTS: Physicians, nurses and pharmacists found ADE scorecards effective to increase medication safety and recommended future usage. The time-series analysis did not show changes in rates of possible ADEs. CONCLUSION: ADE scorecards appear to be useful to raise awareness of ADE-related issues among professionals. Although the evaluation did not show significant reductions of ADE rates, the participating physicians, nurses and pharmacists believed that the ADE scorecards could contribute to increased patient safety and to a reduction in ADE rates. Strategies need to be designed to integrate ADE scorecards better into the clinical routine and to increase the precision of ADE detection.

On the alert: future priorities for alerts in clinical decision support for computerized physician order entry identified from a European workshop.

BACKGROUND: Clinical decision support (CDS) for electronic prescribing systems (computerized physician order entry) should help prescribers in the safe and rational use of medicines. However, the best ways to alert users to unsafe or irrational prescribing are uncertain. Specifically, CDS systems may generate too many alerts, producing unwelcome distractions for prescribers, or too few alerts running the risk of overlooking possible harms. Obtaining the right balance of alerting to adequately improve patient safety should be a priority. METHODS: A workshop funded through the European Regional Development Fund was convened by the University Hospitals Birmingham NHS Foundation Trust to assess current knowledge on alerts in CDS and to reach a consensus on a future research agenda on this topic. Leading European researchers in CDS and alerts in electronic prescribing systems were invited to the workshop. RESULTS: We identified important knowledge gaps and suggest research priorities including (1) the need to determine the optimal sensitivity and specificity of alerts; (2) whether adaptation to the environment or characteristics of the user may improve alerts; and (3) whether modifying the timing and number of alerts will lead to improvements. We have also discussed the challenges and benefits of using naturalistic or experimental studies in the evaluation of alerts and suggested appropriate outcome measures. CONCLUSIONS: We have identified critical problems in CDS, which should help to guide priorities in research to evaluate alerts. It is hoped that this will spark the next generation of novel research from which practical steps can be taken to implement changes to CDS systems that will ultimately reduce alert fatigue and improve the design of future systems.