Clinician perspectives on how situational context and augmented intelligence design features impact perceived usefulness of sepsis prediction scores embedded within a simulated electronic health record.

OBJECTIVE: Obtain clinicians' perspectives on early warning scores (EWS) use within context of clinical cases. MATERIAL AND METHODS: We developed cases mimicking sepsis situations. De-identified data, synthesized physician notes, and EWS representing deterioration risk were displayed in a simulated EHR for analysis. Twelve clinicians participated in semi-structured interviews to ascertain perspectives across four domains: (1) Familiarity with and understanding of artificial intelligence (AI), prediction models and risk scores; (2) Clinical reasoning processes; (3) Impression and response to EWS; and (4) Interface design. Transcripts were coded and analyzed using content and thematic analysis. RESULTS: Analysis revealed clinicians have experience but limited AI and prediction/risk modeling understanding. Case assessments were primarily based on clinical data. EWS went unmentioned during initial case analysis; although when prompted to comment on it, they discussed it in subsequent cases. Clinicians were unsure how to interpret or apply the EWS, and desired evidence on its derivation and validation. Design recommendations centered around EWS display in multi-patient lists for triage, and EWS trends within the patient record. Themes included a "Trust but Verify" approach to AI and early warning information, dichotomy that EWS is helpful for triage yet has disproportional signal-to-high noise ratio, and action driven by clinical judgment, not the EWS. CONCLUSIONS: Clinicians were unsure of how to apply EWS, acted on clinical data, desired score composition and validation information, and felt EWS was most useful when embedded in multi-patient views. Systems providing interactive visualization may facilitate EWS transparency and increase confidence in AI-generated information.

Impact of integrated graphical display on expert and novice diagnostic performance in critical care.

OBJECTIVE: To determine the impact of a graphical information display on diagnosing circulatory shock. MATERIALS AND METHODS: This was an experimental study comparing integrated and conventional information displays. Participants were intensivists or critical care fellows (experts) and first-year medical residents (novices). RESULTS: The integrated display was associated with higher performance (87% vs 82%; P < .001), less time (2.9 vs 3.5 min; P = .008), and more accurate etiology (67% vs 54%; P = .048) compared to the conventional display. When stratified by experience, novice physicians using the integrated display had higher performance (86% vs 69%; P < .001), less time (2.9 vs 3.7 min; P = .03), and more accurate etiology (65% vs 42%; P = .02); expert physicians using the integrated display had nonsignificantly improved performance (87% vs 82%; P = .09), time (2.9 vs 3.3; P = .28), and etiology (69% vs 67%; P = .81). DISCUSSION: The integrated display appeared to support efficient information processing, which resulted in more rapid and accurate circulatory shock diagnosis. Evidence more strongly supported a difference for novices, suggesting that graphical displays may help reduce expert-novice performance gaps.

Patient information organization in the intensive care setting: expert knowledge elicitation with card sorting methods.

Introduction: Many electronic health records fail to support information uptake because they impose low-level information organization tasks on users. Clinical concept-oriented views have shown information processing improvements, but the specifics of this organization for critical care are unclear. Objective: To determine high-level cognitive processes and patient information organization schema in critical care. Methods: We conducted an open card sort of 29 patient data elements and a modified Delphi card sort of 65 patient data elements. Study participants were 39 clinicians with varied critical care training and experience. We analyzed the open sort with a hierarchical cluster analysis (HCA) and factor analysis (FA). The Delphi sort was split into three initiating groups that resulted in three unique solutions. We compared results between open sort analyses (HCA and FA), between card sorting exercises (open and Delphi), and across the Delphi solutions. Results: Between the HCA and FA, we observed common constructs including cardiovascular and hemodynamics, infectious disease, medications, neurology, patient overview, respiratory, and vital signs. The more comprehensive Delphi sort solutions also included gastrointestinal, renal, and imaging constructs. Conclusions: We identified primarily system-based groupings (e.g., cardiovascular, respiratory). Source-based (e.g., medications, laboratory) groups became apparent when participants were asked to sort a longer list of concepts. These results suggest a hybrid approach to information organization, which may combine systems, source, or problem-based groupings, best supports clinicians' mental models. These results can contribute to the design of information displays to better support clinicians' access and interpretation of information for critical care decisions.

Toward Designing Information Display to Support Critical Care. A Qualitative Contextual Evaluation and Visioning Effort.

OBJECTIVES: Electronic health information overload makes it difficult for providers to quickly find and interpret information to support care decisions. The purpose of this study was to better understand how clinicians use information in critical care to support the design of improved presentation of electronic health information. METHODS: We conducted a contextual analysis and visioning project. We used an eye-tracker to record 20 clinicians' information use activities in critical care settings. We played video recordings back to clinicians in retrospective cued interviews and queried: 1) context and goals of information use, 2) impacts of current display design on use, and 3) processes related to information use. We analyzed interview transcripts using grounded theory-based content analysis techniques and identified emerging themes. From these, we conducted a visioning activity with a team of subject matter experts and identified key areas for focus of design and research for future display designs. RESULTS: Analyses revealed four unique critical care information use activities including new patient assessment, known patient status review, specific directed information seeking, and review and prioritization of multiple patients. Emerging themes were primarily related to a need for better representation of dynamic data such as vital signs and laboratory results, usability issues associated with reducing cognitive load and supporting efficient interaction, and processes for managing information. Visions for the future included designs that: 1) provide rapid access to new information, 2) organize by systems or problems as well as by current versus historical patient data, and 3) apply intelligence toward detecting and representing change and urgency. CONCLUSIONS: The results from this study can be used to guide the design of future acute care electronic health information display. Additional research and collaboration is needed to refine and implement intelligent graphical user interfaces to improve clinical information organization and prioritization to support care decisions.

The neuroscience nursing 2005 role delineation study: implications for certification.

A task force appointed by the American Board of Neuroscience Nursing conducted a role delineation study to define current practice in neuroscience nursing. The results were used to validate the content matrix for future Certified Neuroscience Registered Nurse (CNRN) examinations. The study employed a survey design for which the Nursing Intervention Classification taxonomy was the guiding theoretical framework. The eligible sample included all current CNRNs and all members of the American Association of Neuroscience Nursing. An invitation to participate in an online survey was successfully emailed to 2,462 neuroscience nurses; the survey was completed by 477 respondents. They rated the performance and importance of 175 neuroscience nursing activities. On the basis of data analysis conducted by Schroeder Measurement Technologies, Inc., the task force recommended revisions to the CNRN examination matrix to reflect current practice in neuroscience nursing.