Utilizing a user-centered approach to develop and assess pharmacogenomic clinical decision support for thiopurine methyltransferase.

BACKGROUND: A pharmacogenomic clinical decision support tool (PGx-CDS) for thiopurine medications can help physicians incorporate pharmacogenomic results into prescribing decisions by providing up-to-date, real-time decision support. However, the PGx-CDS user interface may introduce errors and promote alert fatigue. The objective of this study was to develop and evaluate a prototype of a PGx-CDS user interface for thiopurine medications with user-centered design methods. METHODS: This study had two phases: In phase I, we conducted qualitative interviews to assess providers' information needs. Interview transcripts were analyzed through a combination of inductive and deductive qualitative analysis to develop design requirements for a PGx-CDS user interface. Using these requirements, we developed a user interface prototype and evaluated its usability (phase II). RESULTS: In total, 14 providers participated: 10 were interviewed in phase I, and seven providers completed usability testing in phase II (3 providers participated in both phases). Most (90%) participants were interested in PGx-CDS systems to help improve medication efficacy and patient safety. Interviews yielded 11 themes sorted into two main categories: 1) health care providers' views on PGx-CDS and 2) important design features for PGx-CDS. We organized these findings into guidance for PGx-CDS content and display. Usability testing of the PGx-CDS prototype showed high provider satisfaction. CONCLUSION: This is one of the first studies to utilize a user-centered design approach to develop and assess a PGx-CDS interface prototype for Thiopurine Methyltransferase (TPMT). This study provides guidance for the development of a PGx-CDS, and particularly for biomarkers such as TPMT.

Alerts for community pharmacist-provided medication therapy management: recommendations from a heuristic evaluation.

BACKGROUND: Medication therapy management (MTM) is a service, most commonly provided by pharmacists, intended to identify and resolve medication therapy problems (MTPs) to enhance patient care. MTM is typically documented by the community pharmacist in an MTM vendor's web-based platform. These platforms often include integrated alerts to assist the pharmacist with assessing MTPs. In order to maximize the usability and usefulness of alerts to the end users (e.g., community pharmacists), MTM alert design should follow principles from human factors science. Therefore, the objectives of this study were to 1) evaluate the extent to which alerts for community pharmacist-delivered MTM align with established human factors principles, and 2) identify areas of opportunity and recommendations to improve MTM alert design. METHODS: Five categories of MTM alerts submitted by community pharmacists were evaluated: 1) indication, 2) effectiveness; 3) safety; 4) adherence; and 5) cost-containment. This heuristic evaluation was guided by the Instrument for Evaluating Human-Factors Principles in Medication-Related Decision Support Alerts (I-MeDeSA) which we adapted and contained 32 heuristics. For each MTM alert, four analysts' individual ratings were summed and a mean score on the modified I-MeDeSA computed. For each heuristic, we also computed the percent of analyst ratings indicating alignment with the heuristic. We did this for all alerts evaluated to produce an "overall" summary of analysts' ratings for a given heuristic, and we also computed this separately for each alert category. Our results focus on heuristics where ≤50% of analysts' ratings indicated the alerts aligned with the heuristic. RESULTS: I-MeDeSA scores across the five alert categories were similar. Heuristics pertaining to visibility and color were generally met. Opportunities for improvement across all MTM alert categories pertained to the principles of alert prioritization; text-based information; alarm philosophy; and corrective actions. CONCLUSIONS: MTM alerts have several opportunities for improvement related to human factors principles, resulting in MTM alert design recommendations. Enhancements to MTM alert design may increase the effectiveness of MTM delivery by community pharmacists and result in improved patient outcomes.

Medication decision-making for patients with renal insufficiency in inpatient and outpatient care at a US Veterans Affairs Medical Centre: a qualitative, cognitive task analysis.

BACKGROUND: Many studies identify factors that contribute to renal prescribing errors, but few examine how healthcare professionals (HCPs) detect and recover from an error or potential patient safety concern. Knowledge of this information could inform advanced error detection systems and decision support tools that help prevent prescribing errors. OBJECTIVE: To examine the cognitive strategies that HCPs used to recognise and manage medication-related problems for patients with renal insufficiency. DESIGN: HCPs submitted documentation about medication-related incidents. We then conducted cognitive task analysis interviews. Qualitative data were analysed inductively. SETTING: Inpatient and outpatient facilities at a major US Veterans Affairs Medical Centre. PARTICIPANTS: Physicians, nurses and pharmacists who took action to prevent or resolve a renal-drug problem in patients with renal insufficiency. OUTCOMES: Emergent themes from interviews, as related to recognition of renal-drug problems and decision-making processes. RESULTS: We interviewed 20 HCPs. Results yielded a descriptive model of the decision-making process, comprised of three main stages: detect, gather information and act. These stages often followed a cyclical path due largely to the gradual decline of patients' renal function. Most HCPs relied on being vigilant to detect patients' renal-drug problems rather than relying on systems to detect unanticipated cues. At each stage, HCPs relied on different cognitive cues depending on medication type: for renally eliminated medications, HCPs focused on gathering renal dosing guidelines, while for nephrotoxic medications, HCPs investigated the need for particular medication therapy, and if warranted, safer alternatives. CONCLUSIONS: Our model is useful for trainees so they can gain familiarity with managing renal-drug problems. Based on findings, improvements are warranted for three aspects of healthcare systems: (1) supporting the cyclical nature of renal-drug problem management via longitudinal tracking mechanisms, (2) providing tools to alleviate HCPs' heavy reliance on vigilance and (3) supporting HCPs' different decision-making needs for renally eliminated versus nephrotoxic medications.

Faculty perceptions of a tobacco cessation train-the-trainer workshop and experiences with implementation: A qualitative follow-up study.

BACKGROUND: Between 2003 and 2005, pharmacy faculty members (n = 191) participated in a national train-the-trainer workshop designed to equip faculty with the necessary knowledge and skills to implement a shared curriculum, Rx for Change: Clinician-Assisted Tobacco Cessation, at pharmacy schools across the United States. OBJECTIVE: To conduct a long-term, qualitative follow-up study of faculty participants to describe (a) perceptions of the train-the-trainer workshop, and (b) subsequent experiences with curricular implementation. Results of this investigation will inform a national survey of all train-the-trainer participants. METHODS: Participants were selected via random sampling from the group of 191 faculty members who participated in the workshop. Semi-structured telephone interviews with participants were audio-recorded and transcribed, and qualitative thematic analysis was conducted. RESULTS: Eighteen (62%) of 29 invited individuals participated in the interviews. All participants reported implementing components of Rx for Change at their institution. The analysis yielded eight major themes pertaining to faculty perceptions and experiences with implementation: (1) accessibility to tools for teaching, (2) increased confidence and skills, (3) flexibility delivering the curriculum, (4) factors facilitating implementation and challenges encountered by faculty, (5) enhancement in treating tobacco users in clinical practice, (6) students' confidence and cognizance of the pharmacists' role as a public health advocate, (7) networking and career development opportunities, and (8) useful background for research. CONCLUSION: Participation in the train-the-trainer workshop increased self-reported confidence for teaching tobacco cessation, and faculty valued access to useful, updated tools for teaching. Furthermore, their newly acquired counseling skills were deemed helpful for treating patients' tobacco use and dependence in clinical practice. Participants also perceived improved pharmacy students' confidence and beneficial networking opportunities. Results can help future trainers understand faculty experiences with implementing a shared, national curriculum and inform faculty participants of some of the potential long-term outcomes as a result of participation.

Adapting Cognitive Task Analysis to Investigate Clinical Decision Making and Medication Safety Incidents.

OBJECTIVES: Cognitive task analysis (CTA) can yield valuable insights into healthcare professionals' cognition and inform system design to promote safe, quality care. Our objective was to adapt CTA-the critical decision method, specifically-to investigate patient safety incidents, overcome barriers to implementing this method, and facilitate more widespread use of cognitive task analysis in healthcare. METHODS: We adapted CTA to facilitate recruitment of healthcare professionals and developed a data collection tool to capture incidents as they occurred. We also leveraged the electronic health record (EHR) to expand data capture and used EHR-stimulated recall to aid reconstruction of safety incidents. We investigated 3 categories of medication-related incidents: adverse drug reactions, drug-drug interactions, and drug-disease interactions. Healthcare professionals submitted incidents, and a subset of incidents was selected for CTA. We analyzed several outcomes to characterize incident capture and completed CTA interviews. RESULTS: We captured 101 incidents. Eighty incidents (79%) met eligibility criteria. We completed 60 CTA interviews, 20 for each incident category. Capturing incidents before interviews allowed us to shorten the interview duration and reduced reliance on healthcare professionals' recall. Incorporating the EHR into CTA enriched data collection. CONCLUSIONS: The adapted CTA technique was successful in capturing specific categories of safety incidents. Our approach may be especially useful for investigating safety incidents that healthcare professionals "fix and forget." Our innovations to CTA are expected to expand the application of this method in healthcare and inform a wide range of studies on clinical decision making and patient safety.

Comparative usability evaluation of consultation order templates in a simulated primary care environment.

Communication breakdowns in the referral process negatively impact clinical workflow and patient safety. There is a lack of evidence demonstrating the impact of published design recommendations addressing contributing issues with consultation order templates. This study translated the recommendations into a computer-based prototype and conducted a comparative usability evaluation. With a scenario-based simulation, 30 clinicians (referrers) participated in a within-group, counterbalanced experiment comparing the prototype with their present electronic order entry system. The prototype significantly increased satisfaction (Cohen's d = 1.80, 95% CI [1.19, 2.41], p < .001), and required significantly less mental effort (d = 0.67 [0.14, 1.20], p < .001). Regarding efficiency, the prototype required significantly fewer mouse clicks (mean difference = 29 clicks, p < .001). Although overall task time did not differ significantly (d = -0.05 [-0.56, 0.47]), the prototype significantly quickened identification of the appropriate specialty clinic (mean difference = 12 s, d = 0.98 [0.43, 1.52], p < .001). The experimental evidence demonstrated that clinician-centered interfaces significantly improved system usability during ordering of consultations.

A cognitive systems engineering design approach to improve the usability of electronic order forms for medical consultation.

BACKGROUND: During medical referrals, communication barriers between referring and consulting outpatient clinics delay patients' access to health care. One notable opportunity for reducing these barriers is improved usefulness and usability of electronic medical consultation order forms. The cognitive systems engineering (CSE) design approach focuses on supporting humans in managing cognitive complexity in sociotechnical systems. Cognitive complexity includes communication, decision-making, problem solving, and planning. OBJECTIVE: The objective of this research was to implement a CSE design approach to develop a template that supports the cognitive needs of referring clinicians and improves referral communication. METHODS: We conducted interviews and observations with primary care providers and specialists at two major tertiary, urban medical facilities. Using qualitative analysis, we identified cognitive requirements and design guidelines. Next, we designed user interface (UI) prototypes and compared their usability with that of a currently implemented UI at a major Midwestern medical facility. RESULTS: Physicians' cognitive challenges were summarized in four cognitive requirements and 13 design guidelines. As a result, two UI prototypes were developed to support order template search and completion. To compare UIs, 30 clinicians (referrers) participated in a consultation ordering simulation complemented with the think-aloud elicitation method. Oral comments about the UIs were coded for both content and valence (i.e., positive, neutral, or negative). Across 619 comments, the odds ratio for the UI prototype to elicit higher-valenced comments than the implemented UI was 13.5 (95% CI = [9.2, 19.8]), p < .001. CONCLUSION: This study reinforced the significance of applying a CSE design approach to inform the design of health information technology. In addition, knowledge elicitation methods enabled identification of physicians' cognitive requirements and challenges when completing electronic medical consultation orders. The resultant knowledge was used to derive design guidelines and UI prototypes that were more useful and usable for referring physicians. Our results support the implementation of a CSE design approach for electronic medical consultation orders.

Usability Assessment of Secure Messaging for Clinical Document Sharing between Health Care Providers and Patients.

BACKGROUND: Web-based patient portals feature secure messaging systems that enable health care providers and patients to communicate information. However, little is known about the usability of these systems for clinical document sharing. OBJECTIVE: This article evaluates the usability of a secure messaging system for providers and patients in terms of its ability to support sharing of electronic clinical documents. METHODS: We conducted usability testing with providers and patients in a human-computer interaction laboratory at a Midwestern U.S. hospital. Providers sent a medication list document to a fictitious patient via secure messaging. Separately, patients retrieved the clinical document from a secure message and returned it to a fictitious provider. We collected use errors, task completion, task time, and satisfaction. RESULTS: Twenty-nine individuals participated: 19 providers (6 physicians, 6 registered nurses, and 7 pharmacists) and 10 patients. Among providers, 11 (58%) attached and sent the clinical document via secure messaging without requiring assistance, in a median (range) of 4.5 (1.8-12.7) minutes. No patients completed tasks without moderator assistance. Patients accessed the secure messaging system within 3.6 (1.2-15.0) minutes; retrieved the clinical document within 0.8 (0.5-5.7) minutes; and sent the attached clinical document in 6.3 (1.5-18.1) minutes. Although median satisfaction ratings were high, with 5.8 for providers and 6.0 for patients (scale, 0-7), we identified 36 different use errors. Physicians and pharmacists requested additional features to support care coordination via health information technology, while nurses requested features to support efficiency for their tasks. CONCLUSION: This study examined the usability of clinical document sharing, a key feature of many secure messaging systems. Our results highlight similarities and differences between provider and patient end-user groups, which can inform secure messaging design to improve learnability and efficiency. The observations suggest recommendations for improving the technical aspects of secure messaging for clinical document sharing.

Usability evaluation of a medication reconciliation tool: Embedding safety probes to assess users' detection of medication discrepancies.

OBJECTIVE: An electronic medication reconciliation tool was previously developed by another research team to aid provider-patient communication for medication reconciliation. To evaluate the usability of this tool, we integrated artificial safety probes into standard usability methods. The objective of this article is to describe this method of using safety probes, which enabled us to evaluate how well the tool supports users' detection of medication discrepancies. MATERIALS AND METHODS: We completed a mixed-method usability evaluation in a simulated setting with 30 participants: 20 healthcare professionals (HCPs) and 10 patients. We used factual scenarios but embedded three artificial safety probes: (1) a missing medication (i.e., omission); (2) an extraneous medication (i.e., commission); and (3) an inaccurate dose (i.e., dose discrepancy). We measured users' detection of each probe to estimate the probability that a HCP or patient would detect these discrepancies. Additionally, we recorded participants' detection of naturally occurring discrepancies. RESULTS: Each safety probe was detected by ≤50% of HCPs. Patients' detection rates were generally higher. Estimates indicate that a HCP and patient, together, would detect 44.8% of these medication discrepancies. Additionally, HCPs and patients detected 25 and 45 naturally-occurring discrepancies, respectively. DISCUSSION: Overall, detection of medication discrepancies was low. Findings indicate that more advanced interface designs are warranted. Future research is needed on how technologies can be designed to better aid HCPs' and patients' detection of medication discrepancies. CONCLUSION: This is one of the first studies to evaluate the usability of a collaborative medication reconciliation tool and assess HCPs' and patients' detection of medication discrepancies. Results demonstrate that embedded safety probes can enhance standard usability methods by measuring additional, clinically-focused usability outcomes. The novel safety probes we used may serve as an initial, standard set for future medication reconciliation research. More prevalent use of safety probes could strengthen usability research for a variety of health information technologies.

Ten factors to consider when developing usability scenarios and tasks for health information technology.

The quality of usability testing is highly dependent upon the associated usability scenarios. To promote usability testing as part of electronic health record (EHR) certification, the Office of the National Coordinator (ONC) for Health Information Technology requires that vendors test specific capabilities of EHRs with clinical end-users and report their usability testing process - including the test scenarios used - along with the results. The ONC outlines basic expectations for usability testing, but there is little guidance in usability texts or scientific literature on how to develop usability scenarios for healthcare applications. The objective of this article is to outline key factors to consider when developing usability scenarios and tasks to evaluate computer-interface based health information technologies. To achieve this goal, we draw upon a decade of our experience conducting usability tests with a variety of healthcare applications and a wide range of end-users, to include healthcare professionals as well as patients. We discuss 10 key factors that influence scenario development: objectives of usability testing; roles of end-user(s); target performance goals; evaluation time constraints; clinical focus; fidelity; scenario-related bias and confounders; embedded probes; minimize risks to end-users; and healthcare related outcome measures. For each factor, we present an illustrative example. This article is intended to aid usability researchers and practitioners in their efforts to advance health information technologies. The article provides broad guidance on usability scenario development and can be applied to a wide range of clinical information systems and applications.

Systematic Heuristic Evaluation of Computerized Consultation Order Templates: Clinicians' and Human Factors Engineers' Perspectives.

We assessed the usability of consultation order templates and identified problems to prioritize in design efforts for improving referral communication. With a sample of 26 consultation order templates, three evaluators performed a usability heuristic evaluation. The evaluation used 14 domain-independent heuristics and the following three supplemental references: 1 new domain-specific heuristic, 6 usability goals, and coded clinicians' statements regarding ease of use for 10 sampled templates. Evaluators found 201 violations, a mean of 7.7 violations per template. Minor violations outnumbered major violations almost twofold, 115 (57%) to 62 (31%). Approximately 68% of violations were linked to 5 heuristics: aesthetic and minimalist design (17%), error prevention (16%), consistency and standards (14%), recognition rather than recall (11%), and meet referrers' information needs (10%). Severe violations were attributed mostly to meet referrers' information needs and recognition rather than recall. Recorded violations yielded potential negative consequences for efficiency, effectiveness, safety, learnability, and utility. Evaluators and clinicians demonstrated 80% agreement in usability assessment. Based on frequency and severity of usability heuristic violations, the consultation order templates reviewed may impede clinical efficiency and risk patient safety. Results support the following design considerations: communicate consultants' requirements, facilitate information seeking, and support communication. While the most frequent heuristic violations involved interaction design and presentation, the most severe violations lacked information desired by referring clinicians. Violations related to templates' inability to support referring clinicians' information needs had the greatest potential negative impact on efficiency and safety usability goals. Heuristics should be prioritized in future design efforts.

Crossing the Communication Chasm: Challenges and Opportunities in Transitions of Care from the Hospital to the Primary Care Clinic.

BACKGROUND: Transitions of care from specialty and acute settings to primary care abound. Compared to the continuity in end-of-shift handoffs, care transitions involve provider communication between practices and facilities with their own cultures and bureaucracies. Using the transition from acute care to outpatient primary care for stroke/transient ischemic attack (TIA) patients as a case study, this qualitative research explored communication practices and institutional arrangements among clinical providers responsible for longitudinal management of hypertension. In this study, researchers investigated the barriers and facilitators of effective communication between acute stroke/TIA inpatient and primary care providers at a Veterans Affairs Medical Center. METHODS: A multidisciplinary team conducted consensus-based coding and thematic analysis of semistructured interviews with 21 clinical providers (9 with primary responsibilities for inpatient care and 12 with primary responsibilities in outpatient, primary care). RESULTS: Thematic analysis of responses identified three factors that influenced communication between clinical providers: (1) consistent, concise but complete medication and treatment plans; (2) reliable, standardized discharge documentation; (3) use of multiple modes of communication. Participants identified cultural barriers, including challenges with rotating providers at a teaching hospital and local discharge practices. CONCLUSION: Ambiguity about who is being handed off to and time pressures in the acute setting may lead inpatient providers to give lower priority to discharge communication, leaving outpatient providers with low-quality information. While electronic templates have standardized key components of discharge documentation, improvement opportunities remain. Increased awareness of the challenges and opportunities on each side of the care transfer could foster communication practices that systematically account for the information needs of inpatient and outpatient providers.

Pilot evaluation of a method to assess prescribers' information processing of medication alerts.

BACKGROUND: Prescribers commonly receive alerts during medication ordering. Prescribers work in a complex, time-pressured environment; to enhance the effectiveness of safety alerts, the effort needed to cognitively process these alerts should be minimized. Methods to evaluate the extent to which computerized alerts support prescribers' information processing are lacking. OBJECTIVE: To develop a methodological protocol to assess the extent to which alerts support prescribers' information processing at-a-glance; specifically, the incorporation of information into their working memory. We hypothesized that the method would be feasible and that we would be able to detect a significant difference in prescribers' information processing with a revised alert display that incorporates warning design guidelines compared to the original alert display. METHODS: A counterbalanced, within-subject study was conducted with 20 prescribers in a human-computer interaction laboratory. We tested a single alert that was displayed in two different ways. Prescribers were informed that an alert would appear for 10s. After the alert was shown, a white screen was displayed, and prescribers were asked to verbally describe what they saw; indicate how many total warnings; and describe anything else they remembered about the alert. We measured information processing via the accuracy of prescribers' free recall and their ability to identify that three warning messages were present. Two analysts independently evaluated participants' responses against a comprehensive catalog of alert elements and then discussed discrepancies until reaching consensus. RESULTS: This feasibility study demonstrated that the method seemed to be effective for evaluating prescribers' information processing of medication alert displays. With this method, we were able to detect significant differences in prescribers' recall of alert information. The proportion of total data elements that prescribers were able to accurately recall was significantly greater for the revised versus original alert display (p=0.006). With the revised display, more prescribers accurately reported that three warnings were shown (p=0.002). CONCLUSIONS: The methodological protocol was feasible for evaluating the alert display and yielded important findings on prescribers' information processing. Study methods supplement traditional usability evaluation methods and may be useful for evaluating information processing of other healthcare technologies.

Iterative Development and Evaluation of a Pharmacogenomic-Guided Clinical Decision Support System for Warfarin Dosing.

OBJECTIVE: Pharmacogenomic-guided dosing has the potential to improve patient outcomes but its implementation has been met with clinical challenges. Our objective was to develop and evaluate a clinical decision support system (CDSS) for pharmacogenomic-guided warfarin dosing designed for physicians and pharmacists. METHODS: Twelve physicians and pharmacists completed 6 prescribing tasks using simulated patient scenarios in two iterations (development and validation phases) of a newly developed pharmacogenomic-driven CDSS prototype. For each scenario, usability was measured via efficiency, recorded as time to task completion, and participants' perceived satisfaction which were compared using Kruskal-Wallis and Mann Whitney U tests, respectively. Debrief interviews were conducted and qualitatively analyzed. Usability findings from the first (i.e. development) iteration were incorporated into the CDSS design for the second (i.e. validation) iteration. RESULTS: During the CDSS validation iteration, participants took more time to complete tasks with a median (IQR) of 183 (124-247) seconds versus 101 (73.5-197) seconds in the development iteration (p=0.01). This increase in time on task was due to the increase in time spent in the CDSS corresponding to several design changes. Efficiency differences that were observed between pharmacists and physicians in the development iteration were eliminated in the validation iteration. The increased use of the CDSS corresponded to a greater acceptance of CDSS recommended doses in the validation iteration (4% in the first iteration vs. 37.5% in the second iteration, p<0.001). Overall satisfaction did not change statistically between the iterations but the qualitative analysis revealed greater trust in the second prototype. CONCLUSIONS: A pharmacogenomic-guided CDSS has been developed using warfarin as the test drug. The final CDSS prototype was trusted by prescribers and significantly increased the time using the tool and acceptance of the recommended doses. This study is an important step toward incorporating pharmacogenomics into CDSS design for clinical testing.

A Novel Design for Drug-Drug Interaction Alerts Improves Prescribing Efficiency.

BACKGROUND: Drug-drug interactions (DDIs) are common in clinical care and pose serious risks for patients. Electronic health records display DDI alerts that can influence prescribers, but the interface design of DDI alerts has largely been unstudied. In this study, the objective was to apply human factors engineering principles to alert design. It was hypothesized that redesigned DDI alerts would significantly improve prescribers' efficiency and reduce prescribing errors. METHODS: In a counterbalanced, crossover study with prescribers, two DDI alert designs were evaluated. Department of Veterans Affairs (VA) prescribers were video recorded as they completed fictitious patient scenarios, which included DDI alerts of varying severity. Efficiency was measured from time-stamped recordings. Prescribing errors were evaluated against predefined criteria. Efficiency and prescribing errors were analyzed with the Wilcoxon signed-rank test. Other usability data were collected on the adequacy of alert content, prescribers' use of the DDI monograph, and alert navigation. RESULTS: Twenty prescribers completed patient scenarios for both designs. Prescribers resolved redesigned alerts in about half the time (redesign: 52 seconds versus original design: 97 seconds; p<.001). Prescribing errors were not significantly different between the two designs. Usability results indicate that DDI alerts might be enhanced by facilitating easier access to laboratory data and dosing information and by allowing prescribers to cancel either interacting medication directly from the alert. Results also suggest that neither design provided adequate information for decision making via the primary interface. CONCLUSION: Applying human factors principles to DDI alerts improved overall efficiency. Aspects of DDI alert design that could be further enhanced prior to implementation were also identified.

Reducing prescribing errors through creatinine clearance alert redesign.

BACKGROUND: Literature has shown that computerized creatinine clearance alerts reduce errors during prescribing, and applying human factors principles may further reduce errors. Our objective was to apply human factors principles to creatinine clearance alert design and assess whether the redesigned alerts increase usability and reduce prescribing errors compared with the original alerts. METHODS: Twenty Veterans Affairs (VA) outpatient providers (14 physicians, 2 nurse practitioners, and 4 clinical pharmacists) completed 2 usability sessions in a counterbalanced study to evaluate original and redesigned alerts. Each session consisted of fictional patient scenarios with 3 medications that warranted prescribing changes because of renal impairment, each associated with creatinine clearance alerts. Quantitative and qualitative data were collected to assess alert usability and the occurrence of prescribing errors. RESULTS: There were 43% fewer prescribing errors with the redesigned alerts compared with the original alerts (P = .001). Compared with the original alerts, redesigned alerts significantly reduced prescribing errors for allopurinol and ibuprofen (85% vs 40% and 65% vs 25%, P = .012 and P = .008, respectively), but not for spironolactone (85% vs 65%). Nine providers (45%) voiced confusion about why the alert was appearing when they encountered the original alert design. When laboratory links were presented on the redesigned alert, laboratory information was accessed 3.5 times more frequently. CONCLUSIONS: Although prescribing errors were high with both alert designs, the redesigned alerts significantly improved prescribing outcomes. This investigation provides some of the first evidence on how alerts may be designed to support safer prescribing for patients with renal impairment.

Recommendations to improve the usability of drug-drug interaction clinical decision support alerts.

OBJECTIVE: To establish preferred strategies for presenting drug-drug interaction (DDI) clinical decision support alerts. MATERIALS AND METHODS: A DDI Clinical Decision Support Conference Series included a workgroup consisting of 24 clinical, usability, and informatics experts representing academia, health information technology (IT) vendors, healthcare organizations, and the Office of the National Coordinator for Health IT. Workgroup members met via web-based meetings 12 times from January 2013 to February 2014, and two in-person meetings to reach consensus on recommendations to improve decision support for DDIs. We addressed three key questions: (1) what, how, where, and when do we display DDI decision support? (2) should presentation of DDI decision support vary by clinicians? and (3) how should effectiveness of DDI decision support be measured? RESULTS: Our recommendations include the consistent use of terminology, visual cues, minimal text, formatting, content, and reporting standards to facilitate usability. All clinicians involved in the medication use process should be able to view DDI alerts and actions by other clinicians. Override rates are common but may not be a good measure of effectiveness. DISCUSSION: Seven core elements should be included with DDI decision support. DDI information should be presented to all clinicians. Finally, in their current form, override rates have limited capability to evaluate alert effectiveness. CONCLUSION: DDI clinical decision support alerts need major improvements. We provide recommendations for healthcare organizations and IT vendors to improve the clinician interface of DDI alerts, with the aim of reducing alert fatigue and improving patient safety.

Design and Evaluation of an Electronic Override Mechanism for Medication Alerts to Facilitate Communication Between Prescribers and Pharmacists.

BACKGROUND: Computerized medication alerts can often be bypassed by entering an override rationale, but prescribers' override reasons are frequently ambiguous to pharmacists who review orders. OBJECTIVE: To develop and evaluate a new override mechanism for adverse reaction and drug-drug interaction alerts. We hypothesized that the new mechanism would improve usability for prescribers and increase the clinical appropriateness of override reasons. METHODS: A counterbalanced, crossover study was conducted with 20 prescribers in a simulated prescribing environment. We modified the override mechanism timing, navigation, and text entry. Instead of free-text entry, the new mechanism presented prescribers with a predefined set of override reasons. We assessed usability (learnability, perceived efficiency, and usability errors) and used a priori criteria to evaluate the clinical appropriateness of override reasons entered. RESULTS: Prescribers rated the new mechanism as more efficient (Wilcoxon signed-rank test, P = 0.032). When first using the new design, 5 prescribers had difficulty finding the new mechanism, and 3 interpreted the navigation to mean that the alert could not be overridden. The number of appropriate override reasons significantly increased with the new mechanism compared with the original mechanism (median change of 3.0; interquartile range = 3.0; P < 0.0001). CONCLUSIONS: When prescribers were given a menu-based choice for override reasons, clinical appropriateness of these reasons significantly improved. Further enhancements are necessary, but this study is an important first step toward a more standardized menu of override choices. Findings may be used to improve communication through e-prescribing systems between prescribers and pharmacists.