Design and implementation of electronic health record-based tools to support a weight management program in primary care.

Objectives: This paper reports on a mixed methods formative evaluation to support the design and implementation of information technology (IT) tools for a primary care weight management intervention delivered through the patient portal using primary care staff as coaches. Methods: We performed a qualitative needs assessment, designed the IT tools to support the weight management program, and developed implementation tracking metrics. Implementation tracking metrics were designed to use real world electronic health record (EHR) data. Results: The needs assessment revealed IT requirements as well as barriers and facilitators to implementation of EHR-based weight management interventions in primary care. We developed implementation metrics for the IT tools. These metrics were used in weekly project team calls to make sure that project resources were allocated to areas of need. Conclusion: This study identifies the important role of IT in supporting weight management through patient identification, weight and activity tracking in the patient portal, and the use of the EHR as a population management tool. An intensive multi-level implementation approach is required for successful primary care-based weight management interventions including well-designed IT tools, comprehensive involvement of clinic leadership, and implementation tracking metrics to guide the process of workflow integration. This study helps to bridge the gap between informatics and implementation by using socio-technical formative evaluation methods early in order to support the implementation of IT tools. Trial registration: clinicaltrials.gov, NCT04420936. Registered June 9, 2020.

Pneumonia diagnosis performance in the emergency department: a mixed-methods study about clinicians' experiences and exploration of individual differences and response to diagnostic performance feedback.

OBJECTIVES: We sought to (1) characterize the process of diagnosing pneumonia in an emergency department (ED) and (2) examine clinician reactions to a clinician-facing diagnostic discordance feedback tool. MATERIALS AND METHODS: We designed a diagnostic feedback tool, using electronic health record data from ED clinicians' patients to establish concordance or discordance between ED diagnosis, radiology reports, and hospital discharge diagnosis for pneumonia. We conducted semistructured interviews with 11 ED clinicians about pneumonia diagnosis and reactions to the feedback tool. We administered surveys measuring individual differences in mindset beliefs, comfort with feedback, and feedback tool usability. We qualitatively analyzed interview transcripts and descriptively analyzed survey data. RESULTS: Thematic results revealed: (1) the diagnostic process for pneumonia in the ED is characterized by diagnostic uncertainty and may be secondary to goals to treat and dispose the patient; (2) clinician diagnostic self-evaluation is a fragmented, inconsistent process of case review and follow-up that a feedback tool could fill; (3) the feedback tool was described favorably, with task and normative feedback harnessing clinician values of high-quality patient care and personal excellence; and (4) strong reactions to diagnostic feedback varied from implicit trust to profound skepticism about the validity of the concordance metric. Survey results suggested a relationship between clinicians' individual differences in learning and failure beliefs, feedback experience, and usability ratings. DISCUSSION AND CONCLUSION: Clinicians value feedback on pneumonia diagnoses. Our results highlight the importance of feedback about diagnostic performance and suggest directions for considering individual differences in feedback tool design and implementation.

Design of an interface to communicate artificial intelligence-based prognosis for patients with advanced solid tumors: a user-centered approach.

OBJECTIVES: To design an interface to support communication of machine learning (ML)-based prognosis for patients with advanced solid tumors, incorporating oncologists' needs and feedback throughout design. MATERIALS AND METHODS: Using an interdisciplinary user-centered design approach, we performed 5 rounds of iterative design to refine an interface, involving expert review based on usability heuristics, input from a color-blind adult, and 13 individual semi-structured interviews with oncologists. Individual interviews included patient vignettes and a series of interfaces populated with representative patient data and predicted survival for each treatment decision point when a new line of therapy (LoT) was being considered. Ongoing feedback informed design decisions, and directed qualitative content analysis of interview transcripts was used to evaluate usability and identify enhancement requirements. RESULTS: Design processes resulted in an interface with 7 sections, each addressing user-focused questions, supporting oncologists to "tell a story" as they discuss prognosis during a clinical encounter. The iteratively enhanced interface both triggered and reflected design decisions relevant when attempting to communicate ML-based prognosis, and exposed misassumptions. Clinicians requested enhancements that emphasized interpretability over explainability. Qualitative findings confirmed that previously identified issues were resolved and clarified necessary enhancements (eg, use months not days) and concerns about usability and trust (eg, address LoT received elsewhere). Appropriate use should be in the context of a conversation with an oncologist. CONCLUSION: User-centered design, ongoing clinical input, and a visualization to communicate ML-related outcomes are important elements for designing any decision support tool enabled by artificial intelligence, particularly when communicating prognosis risk.

Implementation of Lung Cancer Screening in Primary Care and Pulmonary Clinics: Pragmatic Clinical Trial of Electronic Health Record-Integrated Everyday Shared Decision-Making Tool and Clinician-Facing Prompts.

BACKGROUND: Although low-dose CT (LDCT) scan imaging lung cancer screening (LCS) can reduce lung cancer mortality, it remains underused. Shared decision-making (SDM) is recommended to assess the balance of benefits and harms for each patient. RESEARCH QUESTION: Do clinician-facing electronic health record (EHR) prompts and an EHR-integrated everyday SDM tool designed to support routine incorporation of SDM into primary care improve LDCT scan imaging ordering and completion? STUDY DESIGN AND METHODS: A preintervention and postintervention analysis was conducted in 30 primary care and four pulmonary clinics for visits with patients who met United States Preventive Services Task Force criteria for LCS. Propensity scores were used to adjust for covariates. Subgroup analyses were conducted based on the expected benefit from screening (high benefit vs intermediate benefit), pulmonologist involvement (ie, whether the patient was seen in a pulmonary clinic in addition to a primary care clinic), sex, and race and ethnicity. RESULTS: In the 12-month preintervention phase among 1,090 eligible patients, 77 patients (7.1%) had LDCT scan imaging orders and 48 patients (4.4%) completed screenings. In the 9-month intervention phase among 1,026 eligible patients, 280 patients (27.3%) had LDCT scan imaging orders and 182 patients (17.7%) completed screenings. Adjusted ORs were 4.9 (95% CI, 3.4-6.9; P < .001) and 4.7 (95% CI, 3.1-7.1; P < .001) for LDCT imaging ordering and completion, respectively. Subgroup analyses showed increases in ordering and completion for all patient subgroups. In the intervention phase, the SDM tool was used by 23 of 102 ordering providers (22.5%) and for 69 of 274 patients (25.2%) for whom LDCT scan imaging was ordered and who needed SDM at the time of ordering. INTERPRETATION: Clinician-facing EHR prompts and an EHR-integrated everyday SDM tool are promising approaches to improving LCS in the primary care setting. However, room for improvement remains. As such, further research is warranted. TRIAL REGISTRY: ClinicalTrials.gov; No.: NCT04498052; URL: www. CLINICALTRIALS: gov.

"Are we there yet?" Ten persistent hazards and inefficiencies with the use of medication administration technology from the perspective of practicing nurses.

OBJECTIVES: (1) Characterize persistent hazards and inefficiencies in inpatient medication administration; (2) Explore cognitive attributes of medication administration tasks; and (3) Discuss strategies to reduce medication administration technology-related hazards. MATERIALS AND METHODS: Interviews were conducted with 32 nurses practicing at 2 urban, eastern and western US health systems. Qualitative analysis using inductive and deductive coding included consensus discussion, iterative review, and coding structure revision. We abstracted hazards and inefficiencies through the lens of risks to patient safety and the cognitive perception-action cycle (PAC). RESULTS: Persistent safety hazards and inefficiencies related to MAT organized around the PAC cycle included: (1) Compatibility constraints create information silos; (2) Missing action cues; (3) Intermittent communication flow between safety monitoring systems and nurses; (4) Occlusion of important alerts by other, less helpful alerts; (5) Dispersed information: Information required for tasks is not collocated; (6) Inconsistent data organization: Mismatch of the display and the user's mental model; (7) Hidden medication administration technologies (MAT) limitations: Inaccurate beliefs about MAT functionality contribute to overreliance on the technology; (8) Software rigidity causes workarounds; (9) Cumbersome dependencies between technology and the physical environment; and (10) Technology breakdowns require adaptive actions. DISCUSSION: Errors might persist in medication administration despite successful Bar Code Medication Administration and Electronic Medication Administration Record deployment for reducing errors. Opportunities to improve MAT require a deeper understanding of high-level reasoning in medication administration, including control over the information space, collaboration tools, and decision support. CONCLUSION: Future medication administration technology should consider a deeper understanding of nursing knowledge work for medication administration.

A Systematic Review of Quantitative Methods for Evaluating Electronic Medication Administration Record and Bar-Coded Medication Administration Usability.

BACKGROUND: Although electronic medication administration records (eMARs) and bar-coded medication administration (BCMA) have improved medication safety, poor usability of these technologies can increase patient safety risks. OBJECTIVES: The objective of our systematic review was to identify the impact of eMAR and BCMA design on usability, operationalized as efficiency, effectiveness, and satisfaction. METHODS: We retrieved peer-reviewed journal articles on BCMA and eMAR quantitative usability measures from PsycInfo and MEDLINE (1946-August 20, 2019), and EMBASE (1976-October 23, 2019). Following Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, we screened articles, extracted and categorized data into the usability categories of effectiveness, efficiency, and satisfaction, and evaluated article quality. RESULTS: We identified 1,922 articles and extracted data from 41 articles. Twenty-four articles (58.5%) investigated BCMA only, 10 (24.4%) eMAR only, and seven (17.1%) both BCMA and eMAR. Twenty-four articles (58.5%) measured effectiveness, 8 (19.5%) efficiency, and 17 (41.5%) satisfaction. Study designs included randomized controlled trial (n = 1; 2.4%), interrupted time series (n = 1; 2.4%), pretest/posttest (n = 21; 51.2%), posttest only (n = 14; 34.1%), and pretest/posttest and posttest only for different dependent variables (n = 4; 9.8%). Data collection occurred through observations (n = 19, 46.3%), surveys (n = 17, 41.5%), patient safety event reports (n = 9, 22.0%), surveillance (n = 6, 14.6%), and audits (n = 3, 7.3%). CONCLUSION: Of the 100 measures across the 41 articles, implementing BCMA and/or eMAR broadly resulted in an increase in measures of effectiveness (n = 23, 52.3%) and satisfaction (n = 28, 62.2%) compared to measures of efficiency (n = 3, 27.3%). Future research should focus on eMAR efficiency measures, utilize rigorous study designs, and generate specific design requirements.

Clinical trial recruitment in primary care: exploratory factor analysis of a questionnaire to measure barriers and facilitators to primary care providers' involvement.

BACKGROUND: Recruitment of sufficient participants for clinical trials remains challenging. Primary care is an important avenue for patient recruitment but is underutilized. We developed and pilot tested a questionnaire to measure relevant barriers and facilitators to primary care providers' involvement in recruiting patients for clinical trials. METHODS: Prior research informed the development of the questionnaire. The initial instrument was revised using feedback obtained from cognitive interviews. We invited all primary care providers practicing within the University of Utah Health system to complete the revised questionnaire. We used a mixed-mode design to collect paper responses via in-person recruitment and email contacts to collect responses online. Descriptive statistics, exploratory factor analysis, Cronbach's alpha, and multivariable regression analyses were conducted. RESULTS: Sixty-seven primary care providers participated in the survey. Exploratory factor analysis suggested retaining five factors, representing the importance of clinical trial recruitment in providers' professional identity, clinic-level interventions to facilitate referral, patient-related barriers, concerns about patient health management, and knowledge gaps. The five factors exhibited good or high internal consistency reliability. Professional identity and clinic-level intervention factors were significant predictors of providers' intention to participate in clinical trial recruitment activities. CONCLUSIONS: Results of this exploratory analysis provide preliminary evidence of the internal structure, internal consistency reliability, and predictive validity of the questionnaire to measure factors relevant to primary care providers' involvement in clinical trial recruitment.

Inaccuracies in electronic health records smoking data and a potential approach to address resulting underestimation in determining lung cancer screening eligibility.

OBJECTIVE: The US Preventive Services Task Force (USPSTF) requires the estimation of lifetime pack-years to determine lung cancer screening eligibility. Leading electronic health record (EHR) vendors calculate pack-years using only the most recently recorded smoking data. The objective was to characterize EHR smoking data issues and to propose an approach to addressing these issues using longitudinal smoking data. MATERIALS AND METHODS: In this cross-sectional study, we evaluated 16 874 current or former smokers who met USPSTF age criteria for screening (50-80 years old), had no prior lung cancer diagnosis, and were seen in 2020 at an academic health system using the Epic® EHR. We described and quantified issues in the smoking data. We then estimated how many additional potentially eligible patients could be identified using longitudinal data. The approach was verified through manual review of records from 100 subjects. RESULTS: Over 80% of evaluated records had inaccuracies, including missing packs-per-day or years-smoked (42.7%), outdated data (25.1%), missing years-quit (17.4%), and a recent change in packs-per-day resulting in inaccurate lifetime pack-years estimation (16.9%). Addressing these issues by using longitudinal data enabled the identification of 49.4% more patients potentially eligible for lung cancer screening (P < .001). DISCUSSION: Missing, outdated, and inaccurate smoking data in the EHR are important barriers to effective lung cancer screening. Data collection and analysis strategies that reflect changes in smoking habits over time could improve the identification of patients eligible for screening. CONCLUSION: The use of longitudinal EHR smoking data could improve lung cancer screening.

Evaluation in Life Cycle of Information Technology (ELICIT) framework: Supporting the innovation life cycle from business case assessment to summative evaluation.

OBJECTIVE: Our objective was to develop an evaluation framework for electronic health record (EHR)-integrated innovations to support evaluation activities at each of four information technology (IT) life cycle phases: planning, development, implementation, and operation. METHODS: The evaluation framework was developed based on a review of existing evaluation frameworks from health informatics and other domains (human factors engineering, software engineering, and social sciences); expert consensus; and real-world testing in multiple EHR-integrated innovation studies. RESULTS: The resulting Evaluation in Life Cycle of IT (ELICIT) framework covers four IT life cycle phases and three measure levels (society, user, and IT). The ELICIT framework recommends 12 evaluation steps: (1) business case assessment; (2) stakeholder requirements gathering; (3) technical requirements gathering; (4) technical acceptability assessment; (5) user acceptability assessment; (6) social acceptability assessment; (7) social implementation assessment; (8) initial user satisfaction assessment; (9) technical implementation assessment; (10) technical portability assessment; (11) long-term user satisfaction assessment; and (12) social outcomes assessment. DISCUSSION: Effective evaluation requires a shared understanding and collaboration across disciplines throughout the entire IT life cycle. In contrast with previous evaluation frameworks, the ELICIT framework focuses on all phases of the IT life cycle across the society, user, and IT levels. Institutions seeking to establish evaluation programs for EHR-integrated innovations could use our framework to create such shared understanding and justify the need to invest in evaluation. CONCLUSION: As health care undergoes a digital transformation, it will be critical for EHR-integrated innovations to be systematically evaluated. The ELICIT framework can facilitate these evaluations.

Implementing lung cancer screening in primary care: needs assessment and implementation strategy design.

Lung cancer screening with low-dose computed tomography (CT) could help avert thousands of deaths each year. Since the implementation of screening is complex and underspecified, there is a need for systematic and theory-based strategies. Explore the implementation of lung cancer screening in primary care, in the context of integrating a decision aid into the electronic health record. Design implementation strategies that target hypothesized mechanisms of change and context-specific barriers. The study had two phases. The Qualitative Analysis phase included semi-structured interviews with primary care physicians to elicit key task behaviors (e.g., ordering a low-dose CT) and understand the underlying behavioral determinants (e.g., social influence). The Implementation Strategy Design phase consisted of defining implementation strategies and hypothesizing causal pathways to improve screening with a decision aid. Three key task behaviors and four behavioral determinants emerged from 14 interviews. Implementation strategies were designed to target multiple levels of influence. Strategies included increasing provider self-efficacy toward performing shared decision making and using the decision aid, improving provider performance expectancy toward ordering a low-dose CT, increasing social influence toward performing shared decision making and using the decision aid, and addressing key facilitators to using the decision aid. This study contributes knowledge about theoretical determinants of key task behaviors associated with lung cancer screening. We designed implementation strategies according to causal pathways that can be replicated and tested at other institutions. Future research is needed to evaluate the effectiveness of these strategies and to determine the contexts in which they can be effectively applied.

Establishing a multidisciplinary initiative for interoperable electronic health record innovations at an academic medical center.

OBJECTIVE: To establish an enterprise initiative for improving health and health care through interoperable electronic health record (EHR) innovations. MATERIALS AND METHODS: We developed a unifying mission and vision, established multidisciplinary governance, and formulated a strategic plan. Key elements of our strategy include establishing a world-class team; creating shared infrastructure to support individual innovations; developing and implementing innovations with high anticipated impact and a clear path to adoption; incorporating best practices such as the use of Fast Healthcare Interoperability Resources (FHIR) and related interoperability standards; and maximizing synergies across research and operations and with partner organizations. RESULTS: University of Utah Health launched the ReImagine EHR initiative in 2016. Supportive infrastructure developed by the initiative include various FHIR-related tooling and a systematic evaluation framework. More than 10 EHR-integrated digital innovations have been implemented to support preventive care, shared decision-making, chronic disease management, and acute clinical care. Initial evaluations of these innovations have demonstrated positive impact on user satisfaction, provider efficiency, and compliance with evidence-based guidelines. Return on investment has included improvements in care; over $35 million in external grant funding; commercial opportunities; and increased ability to adapt to a changing healthcare landscape. DISCUSSION: Key lessons learned include the value of investing in digital innovation initiatives leveraging FHIR; the importance of supportive infrastructure for accelerating innovation; and the critical role of user-centered design, implementation science, and evaluation. CONCLUSION: EHR-integrated digital innovation initiatives can be key assets for enhancing the EHR user experience, improving patient care, and reducing provider burnout.

Feeling and thinking: can theories of human motivation explain how EHR design impacts clinician burnout?

The psychology of motivation can help us understand the impact of electronic health records (EHRs) on clinician burnout both directly and indirectly. Informatics approaches to EHR usability tend to focus on the extrinsic motivation associated with successful completion of clearly defined tasks in clinical workflows. Intrinsic motivation, which includes the need for autonomy, sense-making, creativity, connectedness, and mastery is not well supported by current designs and workflows. This piece examines existing research on the importance of 3 psychological drives in relation to healthcare technology: goal-based decision-making, sense-making, and agency/autonomy. Because these motives are ubiquitous, foundational to human functioning, automatic, and unconscious, they may be overlooked in technological interventions. The results are increased cognitive load, emotional distress, and unfulfilling workplace environments. Ultimately, we hope to stimulate new research on EHR design focused on expanding functionality to support intrinsic motivation, which, in turn, would decrease burnout and improve care.

Integrated displays to improve chronic disease management in ambulatory care: A SMART on FHIR application informed by mixed-methods user testing.

OBJECTIVE: The study sought to evaluate a novel electronic health record (EHR) add-on application for chronic disease management that uses an integrated display to decrease user cognitive load, improve efficiency, and support clinical decision making. MATERIALS AND METHODS: We designed a chronic disease management application using the technology framework known as SMART on FHIR (Substitutable Medical Applications and Reusable Technologies on Fast Healthcare Interoperability Resources). We used mixed methods to obtain user feedback on a prototype to support ambulatory providers managing chronic obstructive pulmonary disease. Each participant managed 2 patient scenarios using the regular EHR with and without access to our prototype in block-randomized order. The primary outcome was the percentage of expert-recommended ideal care tasks completed. Timing, keyboard and mouse use, and participant surveys were also collected. User experiences were captured using a retrospective think-aloud interview analyzed by concept coding. RESULTS: With our prototype, the 13 participants completed more recommended care (81% vs 48%; P < .001) and recommended tasks per minute (0.8 vs 0.6; P = .03) over longer sessions (7.0 minutes vs 5.4 minutes; P = .006). Keystrokes per task were lower with the prototype (6 vs 18; P < .001). Qualitative themes elicited included the desire for reliable presentation of information which matches participants' mental models of disease and for intuitive navigation in order to decrease cognitive load. DISCUSSION: Participants completed more recommended care by taking more time when using our prototype. Interviews identified a tension between using the inefficient but familiar EHR vs learning to use our novel prototype. Concept coding of user feedback generated actionable insights. CONCLUSIONS: Mixed methods can support the design and evaluation of SMART on FHIR EHR add-on applications by enhancing understanding of the user experience.

Primary care perspectives on implementation of clinical trial recruitment.

INTRODUCTION: Poor clinical trial (CT) recruitment is a significant barrier to translating basic science discoveries into medical practice. Improving support for primary care provider (PCP) referral of patients to CTs may be an important part of the solution. However, implementing CT referral support in primary care is not only technically challenging, but also presents challenges at the person and organization levels. METHODS: The objectives of this study were (1) to characterize provider and clinical supervisor attitudes and perceptions regarding CT research, recruitment, and referrals in primary care and (2) to identify perceived workflow strategies and facilitators relevant to designing a technology-supported primary care CT referral program. Focus groups were conducted with PCPs, directors, and supervisors. RESULTS: Analysis indicated widespread support for the intrinsic scientific value of CTs, while at the same time deep concerns regarding protecting patient well-being, perceived loss of control when patients participate in trials, concern about the impact of point-of-care referrals on clinic workflow, the need for standard processes, and the need for CT information that enables referring providers to quickly confirm that the burdens are justified by the benefits at both patient and provider levels. PCP suggestions pertinent to implementing a CT referral decision support system are reported. CONCLUSION: The results from this work contribute to developing an implementation approach to support increased referral of patients to CTs.

Association of an Electronic Health Record Add-on App for Neonatal Bilirubin Management With Physician Efficiency and Care Quality.

Importance: The usefulness of electronic health record (EHR) systems could be significantly enhanced by innovative, third-party EHR add-on apps. Objective: To evaluate whether an EHR add-on app for neonatal bilirubin management can save clinicians time and improve patient care. Design, Setting, and Participants: This quality improvement study was conducted at the University of Utah Health Well Baby nursery and outpatient clinics and consisted of 4 substudies: (1) time savings were estimated in an experimental task-timing study comparing the time required for physicians to manage newborns' bilirubin levels with and without the add-on app, (2) app use was estimated from app logs, (3) health care use measures and guideline compliance were compared retrospectively before and after the intervention, and (4) clinician-perceived usability was measured through System Usability Scale surveys. The study took place between April 1, 2016, and September 3, 2019. Data analyses were conducted from October 30, 2018, to September 23, 2019. Interventions: At baseline, clinicians used a manual approach to ensure compliance with an evidence-based clinical guideline for neonatal bilirubin management. To facilitate guideline compliance, an EHR add-on app that automatically retrieves, organizes, and visualizes relevant patient data was developed. The app provides patient-specific assessments and recommendations, including the risk of rebound hyperbilirubinemia following phototherapy based on a predictive model. The add-on app was integrated with the University of Utah Health EHR on April 12, 2017. Main Outcomes and Measures: Clinician time savings, app use, health care use measures, guideline-compliant phototherapy ordering, and perceived usability as measured by the System Usability Scale survey. The survey is composed of 10 statements with responses ranging from 1 (strongly disagree) to 5 (strongly agree). The survey results in a single score ranging from 0 to 100, with ratings described as worst imaginable (mean System Usability Scale score, 12.5), awful (20.3), poor (35.7), okay (50.9), good (71.4), excellent (85.5), and best imaginable (90.9). Results: In 2018, the application was used 20 516 times by clinicians for 91.84% of eligible newborns. Use of the app saved 66 seconds for bilirubin management tasks compared with a commonly used tool (95% CI, 53-79 seconds; P < .001). Following the intervention, health care use rates remained stable, while orders for clinically appropriate phototherapy during hospitalization increased for newborns with bilirubin levels above the guideline-recommended threshold (odds ratio, 1.84; 95% CI, 1.16-2.90; P = .009). Surveys indicated excellent usability (System Usability Scale score, 83.90; 95% CI, 81.49-86.31). Conclusions and Relevance: Well-designed EHR add-on apps may save clinicians time and improve patient care. If time-saving apps, such as the bilirubin app, were implemented widely across institutions and care domains, the potential association with improved patient care and clinician efficiency could be significant. The University of Utah Health bilirubin app is being prepared for release into EHR app stores as free-to-use software.

Understanding Primary Care Providers' Information Gathering Strategies in the Care of Children and Youth with Special Health Care Needs.

Background: Effective care coordination of children and youth with special health care needs (CYSHCN) is critical but challenging. Objective: To investigate clinicians' information-gathering strategies while preparing for visits with CYSHCN. Methods: Critical incident interviews with primary care physicians and care coordinators. Results: Six themes emerged indicating 1) substantial reliance on the electronic health record; 2) a central role of the problem list in organizing and summarizing information; 3) Medical Home's central role in organizing clinical documentation; 4) universal need to integrate information from external records; 5) lack of well-organized and labeled encounter documentation; and 6) lack of tools reconcile medication lists. Conclusion: Our findings have important implications to the design of informatics tools to support information-gathering in the care of CYSHCN.

A Pragmatic Guide to Establishing Clinical Decision Support Governance and Addressing Decision Support Fatigue: a Case Study.

There is limited guidance available in the literature for establishing clinical decision support (CDS) governance and improving CDS effectiveness in a pragmatic, resource-efficient manner. Here, we describe how University of Utah Health established enterprise CDS governance in 2015 leveraging existing resources. Key components of the governance include a multi-stakeholder CDS Committee that vets new requests and reviews existing content; a requirement that proposed CDS is actually desired by intended recipients; coordination with other governance bodies; basic data analytics to identify high-frequency, low-value CDS and monitor progress; active solicitation of user issues; the transition of alert and reminder content to other, more appropriate areas in the electronic health record; and the judicious use of experimental designs to guide decision-making regarding CDS effectiveness. In the three years since establishing this governance, new CDS has been continuously added while the overall burden of clinician-facing alerts and reminders has been reduced by 53.8%.

When an Alert is Not an Alert: A Pilot Study to Characterize Behavior and Cognition Associated with Medication Alerts.

Introduction. Preventable adverse drug events are a significant patient-safety concern, yet most medication alerts are disregarded. Pharmacists encounter the highest number of medication alerts and likely have developed behaviors to cope with alerting inefficiencies. The study objective was to better understand alert override behavior relating to a motivational construct framework. Methods. Mixed-methods study of 10 pharmacists (567 verifications) with eye-tracking observations and retrospective think aloud interviews. Results. Pharmacists spent on average 14 seconds longer verifying orders with alerts than orders without alerts (p<0.001). Verification occurred before alerts were triggered, and no order changes occurred after alerts. Pharmacists reported 62% of alerts as unhelpful and 21% as frustrating. Alert interactions took on average 3.9 seconds. Discussion. Pharmacists anticipate alerts by making appropriate checks and changes before alert prompts. Medication alerts seem to be useful. However, the observed pharmacists' behavior suggests changes in the alert context are needed to match cognition.

Physician Information Needs in Managing Delirium.

Delirium has the highest occurrence rate of any complication in hospitalized adults over the age of 65. The study objective was to determine physician information needs for use in the development of electronic clinical decision support for physicians managing the care of patients with delirium. Critical incident interviews were conducted with 8 experienced internal medicine physicians and 1 cardiologist. Thematic analysis revealed the following 6 themes: 1) Clinician's experience an impoverished information field for mental status, 2) Uncertainty is pervasive, 3) Extensive information foraging effort is required for cohesive story building, 4) Goal Conflict leads to missed diagnosis and early closure, 5) Diffusion of Responsibility for treating delirium is common, and 6) Use of structured delirium resources is minimal. Elicited information needs were identified and physician recommendations for improving access to information needed in managing the care of patient's with delirium are reported. Information elicited in this study is useful for designing delirium clinical decision support that supports physician cognition.

What are they trying to do?: An analysis of Action Identities in using electronic documentation in an EHR.

Documentation processes have changed substantially with EHR adoption. User satisfaction studies have focused on usability or cognitive analysis perspectives. Few studies have provided useful information to developers to improve designs. The purpose of this study is to report a 3-pronged approach to deepen understanding of the documentation process, with the intent to provide useful information for future design. This study was conducted in two phases, beginning with cognitive task interviews and observations, followed by post-observation interviews. Twenty-five constructs were identified across the phases, and we observed several patterns of note writing. Participants provided useful information to potentially inform future design. Our study illustrates how electronic documentation serves many clinical processes and is at the core of the medical record. Providers need multiple kinds of notes and ways to display notes. In order to meet provider goals, we must completely re-think the way electronic documentation is composed and displayed.