Human Factors and Ergonomics in Healthcare: Industry Demands and a Path Forward.

This article reviews three industry demands that will impact the future of Human Factors and Ergonomics in Healthcare settings. These demands include the growing population of older adults, the increasing use of telemedicine, and a focus on patient-centered care. Following, we discuss a path forward through improved medical teams, error management, and safety testing of medical devices and tools. Future challenges are discussed.

A Sociotechnical Framework for Safety-Related Electronic Health Record Research Reporting: The SAFER Reporting Framework.

Electronic health record (EHR)-based interventions to improve patient safety are complex and sensitive to who, what, where, why, when, and how they are delivered. Success or failure depends not only on the characteristics and behaviors of individuals who are targeted by an intervention, but also on the technical characteristics of the intervention and the culture and environment of the health system that implements it. Current reporting guidelines do not capture the complexity of sociotechnical factors (technical and nontechnical factors, such as workflow and organizational issues) that confound or influence these interventions. This article proposes a methodological reporting framework for EHR interventions targeting patient safety and builds on an 8-dimension sociotechnical model previously developed by the authors for design, development, implementation, use, and evaluation of health information technology. The Safety-related EHR Research (SAFER) Reporting Framework enables reporting of patient safety-focused EHR-based interventions while accounting for the multifaceted, dynamic sociotechnical context affecting intervention implementation, effectiveness, and generalizability. As an example, an EHR-based intervention to improve communication and timely follow-up of subcritical abnormal test results to operationalize the framework is presented. For each dimension, reporting should include what sociotechnical changes were made to implement an EHR-related intervention to improve patient safety, why the intervention did or did not lead to safety improvements, and how this intervention can be applied or exported to other health care organizations. A foundational list of research and reporting recommendations to address implementation, effectiveness, and generalizability of EHR-based interventions needed to effectively reduce preventable patient harm is provided. The SAFER Reporting Framework is not meant to replace previous research reporting guidelines, but rather provides a sociotechnical adjunct that complements their use.

Leveraging Eye Tracking to Prioritize Relevant Medical Record Data: Comparative Machine Learning Study.

BACKGROUND: Electronic medical record (EMR) systems capture large amounts of data per patient and present that data to physicians with little prioritization. Without prioritization, physicians must mentally identify and collate relevant data, an activity that can lead to cognitive overload. To mitigate cognitive overload, a Learning EMR (LEMR) system prioritizes the display of relevant medical record data. Relevant data are those that are pertinent to a context-defined as the combination of the user, clinical task, and patient case. To determine which data are relevant in a specific context, a LEMR system uses supervised machine learning models of physician information-seeking behavior. Since obtaining information-seeking behavior data via manual annotation is slow and expensive, automatic methods for capturing such data are needed. OBJECTIVE: The goal of the research was to propose and evaluate eye tracking as a high-throughput method to automatically acquire physician information-seeking behavior useful for training models for a LEMR system. METHODS: Critical care medicine physicians reviewed intensive care unit patient cases in an EMR interface developed for the study. Participants manually identified patient data that were relevant in the context of a clinical task: preparing a patient summary to present at morning rounds. We used eye tracking to capture each physician's gaze dwell time on each data item (eg, blood glucose measurements). Manual annotations and gaze dwell times were used to define target variables for developing supervised machine learning models of physician information-seeking behavior. We compared the performance of manual selection and gaze-derived models on an independent set of patient cases. RESULTS: A total of 68 pairs of manual selection and gaze-derived machine learning models were developed from training data and evaluated on an independent evaluation data set. A paired Wilcoxon signed-rank test showed similar performance of manual selection and gaze-derived models on area under the receiver operating characteristic curve (P=.40). CONCLUSIONS: We used eye tracking to automatically capture physician information-seeking behavior and used it to train models for a LEMR system. The models that were trained using eye tracking performed like models that were trained using manual annotations. These results support further development of eye tracking as a high-throughput method for training clinical decision support systems that prioritize the display of relevant medical record data.

Practicing Clinicians' Recommendations to Reduce Burden from the Electronic Health Record Inbox: a Mixed-Methods Study.

BACKGROUND: Workload from electronic health record (EHR) inbox notifications leads to information overload and contributes to job dissatisfaction and physician burnout. Better understanding of physicians' inbox requirements and workflows could optimize inbox designs, enhance efficiency, and reduce safety risks from information overload. DESIGN: We conducted a mixed-methods study to identify strategies to enhance EHR inbox design and workflow. First, we performed a secondary analysis of national survey data of all Department of Veterans Affairs (VA) primary care practitioners (PCP) to identify major themes in responses to a free-text question soliciting suggestions to improve EHR inbox design and workflows. We then conducted expert interviews of clinicians at five health care systems (1 VA and 4 non-VA settings using 4 different EHRs) to understand existing optimal strategies to improve efficiency and situational awareness related to EHR inbox use. Themes from survey data were cross-validated with interview findings. RESULTS: We analyzed responses from 2104 PCPs who completed the free-text inbox question (of 5001 PCPs who responded to survey) and used an inductive approach to identify five themes: (1) Inbox notification content should be actionable for patient care and relevant to recipient clinician, (2) Inboxes should reduce risk of losing messages, (3) Inbox functionality should be optimized to improve efficiency of processing notifications, (4) Team support should be leveraged to help with EHR inbox notification burden, (5) Sufficient time should be provided to all clinicians to process EHR inbox notifications. We subsequently interviewed 15 VA and non-VA clinicians and identified 11 unique strategies, each corresponding directly with one of these five themes. CONCLUSION: Feedback from practicing end-user clinicians provides robust evidence to improve content and design of the EHR inbox and related clinical workflows and organizational policies. Several strategies we identified could improve clinicians' EHR efficiency and satisfaction as well as empower them to work with their local administrators, health IT personnel, and EHR developers to improve these systems.

Using machine learning to selectively highlight patient information.

BACKGROUND: Electronic medical record (EMR) systems need functionality that decreases cognitive overload by drawing the clinician's attention to the right data, at the right time. We developed a Learning EMR (LEMR) system that learns statistical models of clinician information-seeking behavior and applies those models to direct the display of data in future patients. We evaluated the performance of the system in identifying relevant patient data in intensive care unit (ICU) patient cases. METHODS: To capture information-seeking behavior, we enlisted critical care medicine physicians who reviewed a set of patient cases and selected data items relevant to the task of presenting at morning rounds. Using patient EMR data as predictors, we built machine learning models to predict their relevancy. We prospectively evaluated the predictions of a set of high performing models. RESULTS: On an independent evaluation data set, 25 models achieved precision of 0.52, 95% CI [0.49, 0.54] and recall of 0.77, 95% CI [0.75, 0.80] in identifying relevant patient data items. For data items missed by the system, the reviewers rated the effect of not seeing those data from no impact to minor impact on patient care in about 82% of the cases. CONCLUSION: Data-driven approaches for adaptively displaying data in EMR systems, like the LEMR system, show promise in using information-seeking behavior of clinicians to identify and highlight relevant patient data.

Development and Validation of Trigger Algorithms to Identify Delays in Diagnostic Evaluation of Gastroenterological Cancer.

BACKGROUND & AIMS: Colorectal cancer (CRC) and hepatocellular cancer (HCC) are common causes of death and morbidity, and patients benefit from early detection. However, delays in follow-up of suspicious findings are common, and methods to efficiently detect such delays are needed. We developed, refined, and tested trigger algorithms that identify patients with delayed follow-up evaluation of findings suspicious of CRC or HCC. METHODS: We developed and validated two trigger algorithms that detect delays in diagnostic evaluation of CRC and HCC using laboratory, diagnosis, procedure, and referral codes from the Department of Veteran Affairs National Corporate Data Warehouse. The algorithm initially identified patients with positive test results for iron deficiency anemia or fecal immunochemical test (for CRC) and elevated α-fetoprotein results (for HCC). Our algorithm then excluded patients for whom follow-up evaluation was unnecessary, such as patients with a terminal illness or those who had already completed a follow-up evaluation within 60 days. Clinicians reviewed samples of both delayed and nondelayed records, and review data were used to calculate trigger performance. RESULTS: We applied the algorithm for CRC to 245,158 patients seen from January 1, 2013, through December 31, 2013 and identified 1073 patients with delayed follow up. In a review of 400 randomly selected records, we found that our algorithm identified patients with delayed follow-up with a positive predictive value of 56.0% (95% CI, 51.0%-61.0%). We applied the algorithm for HCC to 333,828 patients seen from January 1, 2011 through December 31, 2014, and identified 130 patients with delayed follow-up. During manual review of all 130 records, we found that our algorithm identified patients with delayed follow-up with a positive predictive value of 82.3% (95% CI, 74.4%-88.2%). When we extrapolated the findings to all patients with abnormal results, the algorithm identified patients with delayed follow-up evaluation for CRC with 68.6% sensitivity (95% CI, 65.4%-71.6%) and 81.1% specificity (95% CI, 79.5%-82.6%); it identified patients with delayed follow-up evaluation for HCC with 89.1% sensitivity (95% CI, 81.8%-93.8%) and 96.5% specificity (95% CI, 94.8%-97.7%). Compared to nonselective methods, use of the algorithm reduced the number of records required for review to identify a delay by more than 99%. CONCLUSIONS: Using data from the Veterans Affairs electronic health record database, we developed an algorithm that greatly reduces the number of record reviews necessary to identify delays in follow-up evaluations for patients with suspected CRC or HCC. This approach offers a more efficient method to identify delayed diagnostic evaluation of gastroenterological cancers.

Defining and Measuring Diagnostic Uncertainty in Medicine: A Systematic Review.

BACKGROUND: Physicians routinely encounter diagnostic uncertainty in practice. Despite its impact on health care utilization, costs and error, measurement of diagnostic uncertainty is poorly understood. We conducted a systematic review to describe how diagnostic uncertainty is defined and measured in medical practice. METHODS: We searched OVID Medline and PsycINFO databases from inception until May 2017 using a combination of keywords and Medical Subject Headings (MeSH). Additional search strategies included manual review of references identified in the primary search, use of a topic-specific database (AHRQ-PSNet) and expert input. We specifically focused on articles that (1) defined diagnostic uncertainty; (2) conceptualized diagnostic uncertainty in terms of its sources, complexity of its attributes or strategies for managing it; or (3) attempted to measure diagnostic uncertainty. KEY RESULTS: We identified 123 articles for full review, none of which defined diagnostic uncertainty. Three attributes of diagnostic uncertainty were relevant for measurement: (1) it is a subjective perception experienced by the clinician; (2) it has the potential to impact diagnostic evaluation-for example, when inappropriately managed, it can lead to diagnostic delays; and (3) it is dynamic in nature, changing with time. Current methods for measuring diagnostic uncertainty in medical practice include: (1) asking clinicians about their perception of uncertainty (surveys and qualitative interviews), (2) evaluating the patient-clinician encounter (such as by reviews of medical records, transcripts of patient-clinician communication and observation), and (3) experimental techniques (patient vignette studies). CONCLUSIONS: The term "diagnostic uncertainty" lacks a clear definition, and there is no comprehensive framework for its measurement in medical practice. Based on review findings, we propose that diagnostic uncertainty be defined as a "subjective perception of an inability to provide an accurate explanation of the patient's health problem." Methodological advancements in measuring diagnostic uncertainty can improve our understanding of diagnostic decision-making and inform interventions to reduce diagnostic errors and overuse of health care resources.

Electronic Detection of Delayed Test Result Follow-Up in Patients with Hypothyroidism.

BACKGROUND: Delays in following up abnormal test results are a common problem in outpatient settings. Surveillance systems that use trigger tools to identify delayed follow-up can help reduce missed opportunities in care. OBJECTIVE: To develop and test an electronic health record (EHR)-based trigger algorithm to identify instances of delayed follow-up of abnormal thyroid-stimulating hormone (TSH) results in patients being treated for hypothyroidism. DESIGN: We developed an algorithm using structured EHR data to identify patients with hypothyroidism who had delayed follow-up (>60 days) after an abnormal TSH. We then retrospectively applied the algorithm to a large EHR data warehouse within the Department of Veterans Affairs (VA), on patient records from two large VA networks for the period from January 1, 2011, to December 31, 2011. Identified records were reviewed to confirm the presence of delays in follow-up. KEY RESULTS: During the study period, 645,555 patients were seen in the outpatient setting within the two networks. Of 293,554 patients with at least one TSH test result, the trigger identified 1250 patients on treatment for hypothyroidism with elevated TSH. Of these patients, 271 were flagged as potentially having delayed follow-up of their test result. Chart reviews confirmed delays in 163 of the 271 flagged patients (PPV = 60.1%). CONCLUSIONS: An automated trigger algorithm applied to records in a large EHR data warehouse identified patients with hypothyroidism with potential delays in thyroid function test results follow-up. Future prospective application of the TSH trigger algorithm can be used by clinical teams as a surveillance and quality improvement technique to monitor and improve follow-up.

Electronic health records and national patient-safety goals.

Hospitals and clinics are adapting to new technologies and implementing electronic health records, but the efforts need to be aligned explicitly with goals for patient safety. EHRs bring the risks of both technical failures and inappropriate use, but they can also help to monitor and improve patient safety.

The use of sequential pattern mining to predict next prescribed medications.

BACKGROUND: Therapy for certain medical conditions occurs in a stepwise fashion, where one medication is recommended as initial therapy and other medications follow. Sequential pattern mining is a data mining technique used to identify patterns of ordered events. OBJECTIVE: To determine whether sequential pattern mining is effective for identifying temporal relationships between medications and accurately predicting the next medication likely to be prescribed for a patient. DESIGN: We obtained claims data from Blue Cross Blue Shield of Texas for patients prescribed at least one diabetes medication between 2008 and 2011, and divided these into a training set (90% of patients) and test set (10% of patients). We applied the CSPADE algorithm to mine sequential patterns of diabetes medication prescriptions both at the drug class and generic drug level and ranked them by the support statistic. We then evaluated the accuracy of predictions made for which diabetes medication a patient was likely to be prescribed next. RESULTS: We identified 161,497 patients who had been prescribed at least one diabetes medication. We were able to mine stepwise patterns of pharmacological therapy that were consistent with guidelines. Within three attempts, we were able to predict the medication prescribed for 90.0% of patients when making predictions by drug class, and for 64.1% when making predictions at the generic drug level. These results were stable under 10-fold cross validation, ranging from 89.1%-90.5% at the drug class level and 63.5-64.9% at the generic drug level. Using 1 or 2 items in the patient's medication history led to more accurate predictions than not using any history, but using the entire history was sometimes worse. CONCLUSION: Sequential pattern mining is an effective technique to identify temporal relationships between medications and can be used to predict next steps in a patient's medication regimen. Accurate predictions can be made without using the patient's entire medication history.

Multiple perspectives on clinical decision support: a qualitative study of fifteen clinical and vendor organizations.

BACKGROUND: Computerized clinical decision support (CDS) can help hospitals to improve healthcare. However, CDS can be problematic. The purpose of this study was to discover how the views of clinical stakeholders, CDS content vendors, and EHR vendors are alike or different with respect to challenges in the development, management, and use of CDS. METHODS: We conducted ethnographic fieldwork using a Rapid Assessment Process within ten clinical and five health information technology (HIT) vendor organizations. Using an inductive analytical approach, we generated themes from the clinical, content vendor, and electronic health record vendor perspectives and compared them. RESULTS: The groups share views on the importance of appropriate manpower, careful knowledge management, CDS that fits user workflow, the need for communication among the groups, and for mutual strategizing about the future of CDS. However, views of usability, training, metrics, interoperability, product use, and legal issues differed. Recommendations for improvement include increased collaboration to address legal, manpower, and CDS sharing issues. CONCLUSIONS: The three groups share thinking about many aspects of CDS, but views differ in a number of important respects as well. Until these three groups can reach a mutual understanding of the views of the other stakeholders, and work together, CDS will not reach its potential.

Developing a model for understanding patient collection of observations of daily living: A qualitative meta-synthesis of the Project HealthDesign Program.

We conducted a meta-synthesis of five different studies that developed, tested, and implemented new technologies for the purpose of collecting Observations of Daily Living (ODL). From this synthesis, we developed a model to explain user motivation as it relates to ODL collection. We describe this model that includes six factors that motivate patients' collection of ODL data: usability, illness experience, relevance of ODLs, information technology infrastructure, degree of burden, and emotional activation. We show how these factors can act as barriers or facilitators to the collection of ODL data and how interacting with care professionals and sharing ODL data may also influence ODL collection, health-related awareness, and behavior change. The model we developed and used to explain ODL collection can be helpful to researchers and designers who study and develop new, personal health technologies to empower people to improve their health.

Development of a clinician reputation metric to identify appropriate problem-medication pairs in a crowdsourced knowledge base.

BACKGROUND: Correlation of data within electronic health records is necessary for implementation of various clinical decision support functions, including patient summarization. A key type of correlation is linking medications to clinical problems; while some databases of problem-medication links are available, they are not robust and depend on problems and medications being encoded in particular terminologies. Crowdsourcing represents one approach to generating robust knowledge bases across a variety of terminologies, but more sophisticated approaches are necessary to improve accuracy and reduce manual data review requirements. OBJECTIVE: We sought to develop and evaluate a clinician reputation metric to facilitate the identification of appropriate problem-medication pairs through crowdsourcing without requiring extensive manual review. APPROACH: We retrieved medications from our clinical data warehouse that had been prescribed and manually linked to one or more problems by clinicians during e-prescribing between June 1, 2010 and May 31, 2011. We identified measures likely to be associated with the percentage of accurate problem-medication links made by clinicians. Using logistic regression, we created a metric for identifying clinicians who had made greater than or equal to 95% appropriate links. We evaluated the accuracy of the approach by comparing links made by those physicians identified as having appropriate links to a previously manually validated subset of problem-medication pairs. RESULTS: Of 867 clinicians who asserted a total of 237,748 problem-medication links during the study period, 125 had a reputation metric that predicted the percentage of appropriate links greater than or equal to 95%. These clinicians asserted a total of 2464 linked problem-medication pairs (983 distinct pairs). Compared to a previously validated set of problem-medication pairs, the reputation metric achieved a specificity of 99.5% and marginally improved the sensitivity of previously described knowledge bases. CONCLUSION: A reputation metric may be a valuable measure for identifying high quality clinician-entered, crowdsourced data.

Development and field testing of a self-assessment guide for computer-based provider order entry.

Despite the benefits of computerized provider order entry (CPOE), numerous reports of unexpected CPOE-related safety concerns have surfaced. As part of a larger project to improve the safety of electronic health records (EHRs), we developed and field tested a CPOE "safety self-assessment" guide through literature searches, expert opinion, and site visits. We then conducted a field test of this guide with nine hospital chief medical informatics officers (CMIOs), who were identified through the Association of Medical Directors of Information Systems. The CPOE safety self-assessment guide was sent electronically to the CMIOs. Once the assessments were returned, we conducted structured telephone interviews for further comments about the guide's format and content. The CMIOs in our study found the CPOE safety guide useful and relatively easy to complete, taking no more than 30 minutes. Analysis of responses to the guide suggest that most recommended practices were implemented inconsistently across facilities. Despite consensus for certain CPOE best practices in the medical literature and among experts, there appeared to be considerable variation among CMIOs' opinions of best practices. Interview data suggested this inconsistency was mostly due to system limitations and/or differing opinions about the necessity of certain EHR-related safety measures. Despite the absence of consensus on best practices, a self-assessment safety guide provides a practical starting point for organizations to assess and improve safety and the effectiveness of their CPOE system.