Usability of a novel clinician interface for genetic results.

The complexity and rapid growth of genetic data demand investment in information technology to support effective use of this information. Creating infrastructure to communicate genetic information to healthcare providers and enable them to manage that data can positively affect a patient's care in many ways. However, genetic data are complex and present many challenges. We report on the usability of a novel application designed to assist providers in receiving and managing a patient's genetic profile, including ongoing updated interpretations of the genetic variants in those patients. Because these interpretations are constantly evolving, managing them represents a challenge. We conducted usability tests with potential users of this application and reported findings to the application development team, many of which were addressed in subsequent versions. Clinicians were excited about the value this tool provides in pushing out variant updates to providers and overall gave the application high usability ratings, but had some difficulty interpreting elements of the interface. Many issues identified required relatively little development effort to fix suggesting that consistently incorporating this type of analysis in the development process can be highly beneficial. For genetic decision support applications, our findings suggest the importance of designing a system that can deliver the most current knowledge and highlight the significance of new genetic information for clinical care. Our results demonstrate that using a development and design process that is user focused helped optimize the value of this application for personalized medicine.

Development and evaluation of a novel user interface for reviewing clinical microbiology results.

Background: Microbiology laboratory results are complex and cumbersome to review. We sought to develop a new review tool to improve the ease and accuracy of microbiology results review. Methods: We observed and informally interviewed clinicians to determine areas in which existing microbiology review tools were lacking. We developed a new tool that reorganizes microbiology results by time and organism. We conducted a scenario-based usability evaluation to compare the new tool to existing legacy tools, using a balanced block design. Results: The average time-on-task decreased from 45.3 min for the legacy tools to 27.1 min for the new tool (P < .0001). Total errors decreased from 41 with the legacy tools to 19 with the new tool (P = .0068). The average Single Ease Question score was 5.65 (out of 7) for the new tool, compared to 3.78 for the legacy tools (P < .0001). The new tool scored 88 ("Excellent") on the System Usability Scale. Conclusions: The new tool substantially improved efficiency, accuracy, and usability. It was subsequently integrated into the electronic health record and rolled out system-wide. This project provides an example of how clinical and informatics teams can innovative alongside a commercial Electronic Health Record (EHR).

User Requirements for a Chronic Kidney Disease Clinical Decision Support Tool to Promote Timely Referral.

BACKGROUND: Timely referral of patients with CKD has been associated with cost and mortality benefits, but referrals are often done too late in the course of the disease. Clinical decision support (CDS) offers a potential solution, but interventions have failed because they were not designed to support the physician workflow. We sought to identify user requirements for a chronic kidney disease (CKD) CDS system to promote timely referral. METHODS: We interviewed primary care physicians (PCPs) to identify data needs for a CKD CDS system that would encourage timely referral and also gathered information about workflow to assess risk factors for progression of CKD. Interviewees were general internists recruited from a network of 14 primary care clinics affiliated with Brigham and Women's Hospital (BWH). We then performed a qualitative analysis to identify user requirements and system attributes for a CKD CDS system. RESULTS: Of the 12 participants, 25% were women, the mean age was 53 (range 37-82), mean years in clinical practice was 27 (range 11-58). We identified 21 user requirements. Seven of these user requirements were related to support for the referral process workflow, including access to pertinent information and support for longitudinal co-management. Six user requirements were relevant to PCP management of CKD, including management of risk factors for progression, interpretation of biomarkers of CKD severity, and diagnosis of the cause of CKD. Finally, eight user requirements addressed user-centered design of CDS, including the need for actionable information, links to guidelines and reference materials, and visualization of trends. CONCLUSION: These 21 user requirements can be used to design an intuitive and usable CDS system with the attributes necessary to promote timely referral.

Usability of the Massachusetts Prescription Drug Monitoring Program in the Emergency Department: A Mixed-methods Study.

OBJECTIVES: Prescription drug monitoring programs (PDMPs) are underutilized, despite evidence showing that they may reduce the epidemic of opioid-related addiction, diversion, and overdose. We evaluated the usability of the Massachusetts (MA) PDMP by emergency medicine providers (EPs), as a system's usability may affect how often it is used. METHODS: This was a mixed-methods study of 17 EPs. We compared the time and number of clicks required to review one patient's record in the PDMP to three other commonly performed computer-based tasks in the emergency department (ED: ordering a computed tomography [CT] scan, writing a prescription, and searching a medication history service integrated within the electronic medical record [EMR]). We performed semistructured interviews and analyzed participant comments and responses regarding their experience using the MA PDMP. RESULTS: The PDMP task took a longer time to complete (mean = 4.22 minutes) and greater number of mouse clicks to complete (mean = 50.3 clicks) than the three other tasks (CT-pulmonary embolism = 1.42 minutes, 24.8 clicks; prescription = 1.30 minutes, 19.5 clicks; SureScripts = 1.45 minutes, 9.5 clicks). Qualitative analysis yielded four main themes about PDMP usability, three negative and one positive: 1) difficulty accessing the PDMP, 2) cumbersome acquiring patient medication history information within the PDMP, 3) nonintuitive display of patient medication history information within the PDMP, and 4) overall perceived value of the PDMP despite an inefficient interface. CONCLUSIONS: The complicated processes of gaining access to, logging in, and using the MA PDMP are barriers to preventing its more frequent use. All states should evaluate the PDMP usability in multiple practice settings including the ED and work to improve provider enrollment, login procedures, patient information input, prescription data display, and ultimately, PDMP data integration into EMRs.

Evaluation: A Qualitative Pilot Study of Novel Information Technology Infrastructure to Communicate Genetic Variant Updates.

BACKGROUND: Partners HealthCare Personalized Medicine developed GeneInsight Clinic (GIC), a tool designed to communicate updated variant information from laboratory geneticists to treating clinicians through automated alerts, categorized by level of variant interpretation change. OBJECTIVES: The study aimed to evaluate feedback from the initial users of the GIC, including the advantages and challenges to receiving this variant information and using this technology at the point of care. METHODS: Healthcare professionals from two clinics that ordered genetic testing for cardiomyopathy and related disorders were invited to participate in one-hour semi-structured interviews and/ or a one-hour focus group. Using a Grounded Theory approach, transcript concepts were coded and organized into themes. RESULTS: Two genetic counselors and two physicians from two treatment clinics participated in individual interviews. Focus group participants included one genetic counselor and four physicians. Analysis resulted in 8 major themes related to structuring and communicating variant knowledge, GIC's impact on the clinic, and suggestions for improvements. The interview analysis identified longitudinal patient care, family data, and growth in genetic testing content as potential challenges to optimization of the GIC infrastructure. DISCUSSION: Participants agreed that GIC implementation increased efficiency and effectiveness of the clinic through increased access to genetic variant information at the point of care. CONCLUSION: Development of information technology (IT) infrastructure to aid in the organization and management of genetic variant knowledge will be critical as the genetic field moves towards whole exome and whole genome sequencing. Findings from this study could be applied to future development of IT support for genetic variant knowledge management that would serve to improve clinicians' ability to manage and care for patients.

Association of note quality and quality of care: a cross-sectional study.

BACKGROUND: While physician notes are known to vary in organisation, content and quality, the relationship between note quality and clinical quality is uncertain. METHODS: We performed a cross-sectional study of outpatient visit physician notes by adult patients with coronary artery disease or diabetes mellitus seen in 2010. We assessed physician note quality using the 9-item Physician Documentation Quality Instrument (PDQI-9) and compared this to disease-specific clinical quality scores constructed from data extracted from the electronic health record (EHR). We also assessed the presence of typical note subsections, and indicators of quality care in physician notes. RESULTS: We evaluated 239 notes, written by 111 physicians; 110 notes were written by primary care physicians, 52 by cardiologists and 77 by endocrinologists. Reason for visit was absent in 10% of notes, medication list was not present in the note in 19.7% and timing for follow-up was absent in 18.0% of notes. Significant copy/pasted material was present in 10.5% of notes. Laboratory quality indicators were more often found in other EHR sections than in the physician note. Clinical quality scores for diabetes and coronary artery disease (CAD) showed no significant association with subjective note quality (diabetes: r -0.119, p=0.065, CAD: r -0.124, p=0.06). CONCLUSIONS: Notes varied in documentation method and length, and important note subsections were frequently missing. Key clinical data to support quality patient care were often not present in physician notes, but were often found elsewhere in the EHR. Subjective assessment of note quality did not correlate with clinical quality scores, suggesting that writing high-quality notes and meeting quality measures are not mutually reinforcing activities.

Efficiency and usability of a near field communication-enabled tablet for medication administration.

BACKGROUND: Barcode-based technology coupled with the electronic medication administration record (e-MAR) reduces medication errors and potential adverse drug events (ADEs). However, many current barcode-enabled medication administration (BCMA) systems are difficult to maneuver and often require multiple barcode scans. We developed a prototype, next generation near field communication-enabled medication administration (NFCMA) system using a tablet. OBJECTIVE: We compared the efficiency and usability of the prototype NFCMA system with the traditional BCMA system. METHODS: We used a mixed-methods design using a randomized observational cross-over study, a survey, and one-on-one interviews to compare the prototype NFCMA system with a traditional BCMA system. The study took place at an academic medical simulation center. Twenty nurses with BCMA experience participated in two simulated patient medication administration scenarios: one using the BCMA system, and the other using the prototype NFCMA system. We collected overall scenario completion time and number of medication scanning attempts per scenario, and compared those using paired t tests. We also collected participant feedback on the prototype NFCMA system using the modified International Business Machines (IBM) Post-Study System Usability Questionnaire (PSSUQ) and a semistructured interview. We performed descriptive statistics on participant characteristics and responses to the IBM PSSUQ. Interview data was analyzed using content analysis with a qualitative description approach to review and categorize feedback from participants. RESULTS: Mean total time to complete the scenarios using the NFCMA and the BCMA systems was 202 seconds and 182 seconds, respectively (P=.09). Mean scan attempts with the NFCMA was 7.6 attempts compared with 6.5 attempts with the BCMA system (P=.12). In the usability survey, 95% (19/20) of participants agreed that the prototype NFCMA system was easy to use and easy to learn, with a pleasant interface. Participants expressed interest in using the NFCMA tablet in the hospital; suggestions focused on implementation issues, such as storage of the mobile devices and infection control methods. CONCLUSIONS: The NFCMA system had similar efficiency to the BCMA system in a simulated scenario. The prototype NFCMA system was well received by nurses and offers promise to improve nurse medication administration efficiency.

A novel clinician interface to improve clinician access to up-to-date genetic results.

OBJECTIVES: To understand the impact of GeneInsight Clinic (GIC), a web-based tool designed to manage genetic information and facilitate communication of test results and variant updates from the laboratory to the clinics, we measured the use of GIC and the time it took for new genetic knowledge to be available to clinicians. METHODS: Usage data were collected across four study sites for the GIC launch and post-GIC implementation time periods. The primary outcome measures were the time (average number of days) between variant change approval and notification of clinic staff, and the time between notification and viewing the patient record. RESULTS: Post-GIC, time between a variant change approval and provider notification was shorter than at launch (average days at launch 503.8, compared to 4.1 days post-GIC). After e-mail alerts were sent at launch, providers clicked into the patient record associated with 91% of these alerts. In the post period, clinic providers clicked into the patient record associated with 95% of the alerts, on average 12 days after the e-mail was sent. DISCUSSION: We found that GIC greatly increased the likelihood that a provider would receive updated variant information as well as reduced the time associated with distributing that variant information, thus providing a more efficient process for incorporating new genetic knowledge into clinical care. CONCLUSIONS: Our study results demonstrate that health information technology systems have the potential effectively to assist providers in utilizing genetic information in patient care.

How physicians document outpatient visit notes in an electronic health record.

BACKGROUND: Clinical documentation, an essential process within electronic health records (EHRs), takes a significant amount of clinician time. How best to optimize documentation methods to deliver effective care remains unclear. OBJECTIVE: We evaluated whether EHR visit note documentation method was influenced by physician or practice characteristics, and the association of physician satisfaction with an EHR notes module. MEASUREMENTS: We surveyed primary care physicians (PCPs) and specialists, and used EHR and provider data to perform a multinomial logistic regression of visit notes from 2008. We measured physician documentation method use and satisfaction with an EHR notes module and determined the relationship between method and physician and practice characteristics. RESULTS: Of 1088 physicians, 85% used a single method to document the majority of their visits. PCPs predominantly documented using templates (60%) compared to 34% of specialists, while 38% of specialists predominantly dictated. Physicians affiliated with academic medical centers (OR 1.96, CI (1.23, 3.12)), based at a hospital (OR 1.57, 95% CI (1.04, 2.36)) and using the EHR for longer (OR 1.13, 95% CI (1.03, 1.25)) were more likely to dictate than use templates. Most physicians of 383 survey responders were satisfied with the EHR notes module, regardless of their preferred documentation method. CONCLUSIONS: Physicians predominantly utilized a single method of visit note documentation and were satisfied with their approach, but the approaches they chose varied. Demographic characteristics were associated with preferred documentation method. Further research should focus on why variation exists, and the quality of the documentation resulting from different methods used.

Development and preliminary evidence for the validity of an instrument assessing implementation of human-factors principles in medication-related decision-support systems--I-MeDeSA.

BACKGROUND: Medication-related decision support can reduce the frequency of preventable adverse drug events. However, the design of current medication alerts often results in alert fatigue and high over-ride rates, thus reducing any potential benefits. METHODS: The authors previously reviewed human-factors principles for relevance to medication-related decision support alerts. In this study, instrument items were developed for assessing the appropriate implementation of these human-factors principles in drug-drug interaction (DDI) alerts. User feedback regarding nine electronic medical records was considered during the development process. Content validity, construct validity through correlation analysis, and inter-rater reliability were assessed. RESULTS: The final version of the instrument included 26 items associated with nine human-factors principles. Content validation on three systems resulted in the addition of one principle (Corrective Actions) to the instrument and the elimination of eight items. Additionally, the wording of eight items was altered. Correlation analysis suggests a direct relationship between system age and performance of DDI alerts (p=0.0016). Inter-rater reliability indicated substantial agreement between raters (κ=0.764). CONCLUSION: The authors developed and gathered preliminary evidence for the validity of an instrument that measures the appropriate use of human-factors principles in the design and display of DDI alerts. Designers of DDI alerts may use the instrument to improve usability and increase user acceptance of medication alerts, and organizations selecting an electronic medical record may find the instrument helpful in meeting their clinicians' usability needs.