Clinical workflow of sonographers performing fetal anomaly ultrasound scans: deep-learning-based analysis.

OBJECTIVE: Despite decades of obstetric scanning, the field of sonographer workflow remains largely unexplored. In the second trimester, sonographers use scan guidelines to guide their acquisition of standard planes and structures; however, the scan-acquisition order is not prescribed. Using deep-learning-based video analysis, the aim of this study was to develop a deeper understanding of the clinical workflow undertaken by sonographers during second-trimester anomaly scans. METHODS: We collected prospectively full-length video recordings of routine second-trimester anomaly scans. Important scan events in the videos were identified by detecting automatically image freeze and image/clip save. The video immediately preceding and following the important event was extracted and labeled as one of 11 commonly acquired anatomical structures. We developed and used a purposely trained and tested deep-learning annotation model to label automatically the large number of scan events. Thus, anomaly scans were partitioned as a sequence of anatomical planes or fetal structures obtained over time. RESULTS: A total of 496 anomaly scans performed by 14 sonographers were available for analysis. UK guidelines specify that an image or videoclip of five different anatomical regions must be stored and these were detected in the majority of scans: head/brain was detected in 97.2% of scans, coronal face view (nose/lips) in 86.1%, abdomen in 93.1%, spine in 95.0% and femur in 92.3%. Analyzing the clinical workflow, we observed that sonographers were most likely to begin their scan by capturing the head/brain (in 24.4% of scans), spine (in 23.2%) or thorax/heart (in 22.8%). The most commonly identified two-structure transitions were: placenta/amniotic fluid to maternal anatomy, occurring in 44.5% of scans; head/brain to coronal face (nose/lips) in 42.7%; abdomen to thorax/heart in 26.1%; and three-dimensional/four-dimensional face to sagittal face (profile) in 23.7%. Transitions between three or more consecutive structures in sequence were uncommon (up to 13% of scans). None of the captured anomaly scans shared an entirely identical sequence. CONCLUSIONS: We present a novel evaluation of the anomaly scan acquisition process using a deep-learning-based analysis of ultrasound video. We note wide variation in the number and sequence of structures obtained during routine second-trimester anomaly scans. Overall, each anomaly scan was found to be unique in its scanning sequence, suggesting that sonographers take advantage of the fetal position and acquire the standard planes according to their visibility rather than following a strict acquisition order. © 2022 The Authors. Ultrasound in Obstetrics & Gynecology published by John Wiley & Sons Ltd on behalf of International Society of Ultrasound in Obstetrics and Gynecology.

Using patient flow analysis with real-time patient tracking to optimize radiation oncology consultation visits.

PURPOSE: Clinical efficiency is a key component of the value-based care model and a driver of patient satisfaction. The purpose of this study was to identify and address inefficiencies at a high-volume radiation oncology clinic. METHODS AND MATERIALS: Patient flow analysis (PFA) was used to create process maps and optimize the workflow of consultation visits in a gastrointestinal radiation oncology clinic at a large academic cancer center. Metrics such as cycle times, waiting times, and rooming times were assessed by using a real-time patient status function in the electronic medical record for 556 consults and compared between before vs after implementation of the PFA recommendations. RESULTS: The initial PFA revealed four inefficiencies: (1) protracted rooming time, (2) inefficient communications, (3) duplicated tasks, and (4) ambiguous clinical roles. We analyzed 485 consult-visits before the PFA and 71 after the PFA. The PFA recommendations led to reductions in overall median cycle time by 21% (91 min vs 72 min, p < 0.001), in cumulative waiting times by 64% (45 min vs 16 min; p < 0.001), which included waiting room time (14 min vs 5 min; p < 0.001) and wait for physician (20 min vs. 6 min; p < 0.001). Slightly less than one-quarter (22%) of consult visits before the PFA lasted > 2 h vs. 0% after implementation of the recommendations (p < 0.001). Similarly, the proportion of visits requiring < 1 h was 16% before PFA vs 34% afterward (p < 0.001). CONCLUSIONS: PFA can be used to identify clinical inefficiencies and optimize workflows in radiation oncology consultation clinics, and implementing their findings can significantly improve cycle times and waiting times. Potential downstream effects of these interventions include improved patient experience, decreased staff burnout, financial savings, and opportunities for expanding clinical capacity.

Machine Learning-Based Prediction Models for Different Clinical Risks in Different Hospitals: Evaluation of Live Performance.

BACKGROUND: Machine learning algorithms are currently used in a wide array of clinical domains to produce models that can predict clinical risk events. Most models are developed and evaluated with retrospective data, very few are evaluated in a clinical workflow, and even fewer report performances in different hospitals. In this study, we provide detailed evaluations of clinical risk prediction models in live clinical workflows for three different use cases in three different hospitals. OBJECTIVE: The main objective of this study was to evaluate clinical risk prediction models in live clinical workflows and compare their performance in these setting with their performance when using retrospective data. We also aimed at generalizing the results by applying our investigation to three different use cases in three different hospitals. METHODS: We trained clinical risk prediction models for three use cases (ie, delirium, sepsis, and acute kidney injury) in three different hospitals with retrospective data. We used machine learning and, specifically, deep learning to train models that were based on the Transformer model. The models were trained using a calibration tool that is common for all hospitals and use cases. The models had a common design but were calibrated using each hospital's specific data. The models were deployed in these three hospitals and used in daily clinical practice. The predictions made by these models were logged and correlated with the diagnosis at discharge. We compared their performance with evaluations on retrospective data and conducted cross-hospital evaluations. RESULTS: The performance of the prediction models with data from live clinical workflows was similar to the performance with retrospective data. The average value of the area under the receiver operating characteristic curve (AUROC) decreased slightly by 0.6 percentage points (from 94.8% to 94.2% at discharge). The cross-hospital evaluations exhibited severely reduced performance: the average AUROC decreased by 8 percentage points (from 94.2% to 86.3% at discharge), which indicates the importance of model calibration with data from the deployment hospital. CONCLUSIONS: Calibrating the prediction model with data from different deployment hospitals led to good performance in live settings. The performance degradation in the cross-hospital evaluation identified limitations in developing a generic model for different hospitals. Designing a generic process for model development to generate specialized prediction models for each hospital guarantees model performance in different hospitals.

Clinical and operational insights from data-driven care pathway mapping: a systematic review.

BACKGROUND: Accumulated electronic data from a wide variety of clinical settings has been processed using a range of informatics methods to determine the sequence of care activities experienced by patients. The "as is" or "de facto" care pathways derived can be analysed together with other data to yield clinical and operational information. It seems likely that the needs of both health systems and patients will lead to increasing application of such analyses. A comprehensive review of the literature is presented, with a focus on the study context, types of analysis undertaken, and the utility of the information gained. METHODS: A systematic review was conducted of literature abstracting sequential patient care activities ("de facto" care pathways) from care records. Broad coverage was achieved by initial screening of a Scopus search term, followed by screening of citations (forward snowball) and references (backwards snowball). Previous reviews of related topics were also considered. Studies were initially classified according to the perspective captured in the derived pathways. Concept matrices were then derived, classifying studies according to additional data used and subsequent analysis undertaken, with regard for the clinical domain examined and the knowledge gleaned. RESULTS: 254 publications were identified. The majority (n = 217) of these studies derived care pathways from data of an administrative/clinical type. 80% (n = 173) applied further analytical techniques, while 60% (n = 131) combined care pathways with enhancing data to gain insight into care processes. DISCUSSION: Classification of the objectives, analyses and complementary data used in data-driven care pathway mapping illustrates areas of greater and lesser focus in the literature. The increasing tendency for these methods to find practical application in service redesign is explored across the variety of contexts and research questions identified. A limitation of our approach is that the topic is broad, limiting discussion of methodological issues. CONCLUSION: This review indicates that methods utilising data-driven determination of de facto patient care pathways can provide empirical information relevant to healthcare planning, management, and practice. It is clear that despite the number of publications found the topic reviewed is still in its infancy.

A Bespoke Electronic Health Record for Epilepsy Care (EpiToMe): Development and Qualitative Evaluation.

BACKGROUND: While electronic health records (EHR) bring various benefits to health care, EHR systems are often criticized as cumbersome to use, failing to fulfill the promise of improved health care delivery with little more than a means of meeting regulatory and billing requirements. EHR has also been recognized as one of the contributing factors for physician burnout. OBJECTIVE: Specialty-specific EHR systems have been suggested as an alternative approach that can potentially address challenges associated with general-purpose EHRs. We introduce the Epilepsy Tracking and optimized Management engine (EpiToMe), an exemplar bespoke EHR system for epilepsy care. EpiToMe uses an agile, physician-centered development strategy to optimize clinical workflow and patient care documentation. We present the design and implementation of EpiToMe and report the initial feedback on its utility for physician burnout. METHODS: Using collaborative, asynchronous data capturing interfaces anchored to a domain ontology, EpiToMe distributes reporting and documentation workload among technicians, clinical fellows, and attending physicians. Results of documentation are transmitted to the parent EHR to meet patient care requirements with a push of a button. An HL7 (version 2.3) messaging engine exchanges information between EpiToMe and the parent EHR to optimize clinical workflow tasks without redundant data entry. EpiToMe also provides live, interactive patient tracking interfaces to ease the burden of care management. RESULTS: Since February 2019, 15,417 electroencephalogram reports, 2635 Epilepsy Monitoring Unit daily reports, and 1369 Epilepsy Monitoring Unit phase reports have been completed in EpiToMe for 6593 unique patients. A 10-question survey was completed by 11 (among 16 invited) senior clinical attending physicians. Consensus was found that EpiToMe eased the burden of care documentation for patient management, a contributing factor to physician burnout. CONCLUSIONS: EpiToMe offers an exemplar bespoke EHR system developed using a physician-centered design and latest advancements in information technology. The bespoke approach has the potential to ease the burden of care management in epilepsy. This approach is applicable to other clinical specialties.

A micro-analytic approach to understanding electronic health record navigation paths.

Clinician task performance is significantly impacted by the navigational efficiency of the system interface. Here we propose and evaluate a navigational complexity framework useful for examining differences in electronic health record (EHR) interface systems and their impact on task performance. The methodological approach includes 1) expert-based methods-specifically, representational analysis (focused on interface elements), keystroke level modeling (KLM), and cognitive walkthrough; and 2) quantitative analysis of interactive behaviors based on video-captured observations. Medication administration record (MAR) tasks completed by nurses during preoperative (PreOp) patient assessment were studied across three Mayo Clinic regional campuses and three different EHR systems. By analyzing the steps executed within the interfaces involved to complete the MAR tasks, we characterized complexities in EHR navigation. These complexities were reflected in time spent on task, click counts, and screen transitions, and were found to potentially influence nurses' performance. Two of the EHR systems, employing a single screen format, required less time to complete (mean 101.5, range 106-97 s), respectively, compared to one system employing multiple screens (176 s, 73% increase). These complexities surfaced through trade-offs in cognitive processes that could potentially influence nurses' performance. Factors such as perceptual-motor activity, visual search, and memory load impacted navigational complexity. An implication of this work is that small tractable changes in interface design can substantially improve EHR navigation, overall usability, and workflow.

EHR audit logs: A new goldmine for health services research?

A byproduct of the transition to electronic health records (EHRs) is the associated observational data that capture EHR users' granular interactions with the medical record. Often referred to as audit log data or event log data, these datasets capture and timestamp user activity while they are logged in to the EHR. These data - alone and in combination with other datasets - offer a new source of insights, which cannot be gleaned from claims data or clinical data, to support health services research and those studying healthcare processes and outcomes. In this commentary, we seek to promote broader awareness of EHR audit log data and to stimulate their use in many contexts. We do so by describing EHR audit log data and offering a framework for their potential uses in quality domains (as defined by the National Academy of Medicine). The framework is illustrated with select examples in the safety and efficiency domains, along with their accompanying methodologies, which serve as a proof of concept. This article also discusses insights and challenges from working with EHR audit log data. Ensuring that researchers are aware of such data, and the new opportunities they offer, is one way to assure that our healthcare system benefits from the digital revolution.

Overlaying multiple sources of data to identify bottlenecks in clinical workflow.

The pursuit of increased efficiency and quality of clinical care based on the analysis of workflow has seen the introduction of several modern technologies into medical environments. Electronic health records (EHRs) remain central to analysis of workflow, owing to their wide-ranging impact on clinical processes. The two most common interventions to facilitate EHR-related workflow analysis are automated location tracking using sensor-based technologies and EHR usage data logs. However, to maximize the potential of these technologies, and especially to facilitate workflow redesign, it is necessary to overlay these quantitative findings on the contextual data from qualitative methods such as ethnography. Such a complementary approach promises to yield more precise measures of clinical workflow that provide insights into how redesign could address inefficiencies. In this paper, we categorize clinical workflow in the Emergency Department (ED) into three types (perceived, real and ideal) to create a structured approach to workflow redesign using the available data. We use diverse data sources: sensor-based location tracking through Radio-Frequency Identification (RFID), summary EHR usage data logs, and data from physician interviews augmented by direct observations (through clinician shadowing). Our goal is to discover inefficiencies and bottlenecks that can be addressed to achieve a more ideal workflow state relative to its real and perceived state. We thereby seek to demonstrate a novel data-driven approach toward iterative workflow redesign that generalizes for use in a variety of settings. We also propose types of targeted support or adjustments to offset some of the inefficiencies we noted.

Impact of Data Presentation on Physician Performance Utilizing Artificial Intelligence-Based Computer-Aided Diagnosis and Decision Support Systems.

Ultrasound (US) is a valuable imaging modality used to detect primary breast malignancy. However, radiologists have a limited ability to distinguish between benign and malignant lesions on US, leading to false-positive and false-negative results, which limit the positive predictive value of lesions sent for biopsy (PPV3) and specificity. A recent study demonstrated that incorporating an AI-based decision support (DS) system into US image analysis could help improve US diagnostic performance. While the DS system is promising, its efficacy in terms of its impact also needs to be measured when integrated into existing clinical workflows. The current study evaluates workflow schemas for DS integration and its impact on diagnostic accuracy. The impact on two different reading methodologies, sequential and independent, was assessed. This study demonstrates significant accuracy differences between the two workflow schemas as measured by area under the receiver operating curve (AUC), as well as inter-operator variability differences as measured by Kendall's tau-b. This evaluation has practical implications on the utilization of such technologies in diagnostic environments as compared to previous studies.

A method for the analysis and visualization of clinical workflow in dynamic environments.

The analysis of clinical workflow offers many challenges, especially in settings characterized by rapid dynamic change. Typically, some combination of approaches drawn from ethnography and grounded theory-based qualitative methods are used to develop relevant metrics. Medical institutions have recently attempted to introduce technological interventions to develop quantifiable quality metrics to supplement existing purely qualitative analyses. These interventions range from automated location tracking to repositories of clinical data (e.g., electronics health record (EHR) data, medical equipment logs). Our goal in this paper is to present a cohesive framework that combines a set of analytic techniques that can potentially complement traditional human observations to derive a deeper understanding of clinical workflow and thereby to enhance the quality, safety, and efficiency of care offered in that environment. We present a series of theoretically-guided techniques to perform analysis and visualization of data developed using location tracking, with illustrations using the Emergency Department (ED) as an example. Our framework is divided into three modules: (i) transformation, (ii) analysis, and (iii) visualization. We describe the methods used in each of these modules, and provide a series of visualizations developed using location-tracking data collected at the Mayo Clinic ED (Phoenix, AZ). Our innovative analytics go beyond qualitative study, and includes user data collected from a relatively modern but increasingly ubiquitous technique of location tracking, with the goal of creating quantitative workflow metrics. Although we believe that the methods we have developed will generalize well to other settings, additional work will be required to demonstrate their broad utility beyond our single study environment.

Data integrity systems for organ contours in radiation therapy planning.

The purpose of this research is to develop effective data integrity models for contoured anatomy in a radiotherapy workflow for both real-time and retrospective analysis. Within this study, two classes of contour integrity models were developed: data driven models and contiguousness models. The data driven models aim to highlight contours which deviate from a gross set of contours from similar disease sites and encompass the following regions of interest (ROI): bladder, femoral heads, spinal cord, and rectum. The contiguousness models, which individually analyze the geometry of contours to detect possible errors, are applied across many different ROI's and are divided into two metrics: Extent and Region Growing over volume. After analysis, we found that 70% of detected bladder contours were verified as suspicious. The spinal cord and rectum models verified that 73% and 80% of contours were suspicious respectively. The contiguousness models were the most accurate models and the Region Growing model was the most accurate submodel. 100% of the detected noncontiguous contours were verified as suspicious, but in the cases of spinal cord, femoral heads, bladder, and rectum, the Region Growing model detected additional two to five suspicious contours that the Extent model failed to detect. When conducting a blind review to detect false negatives, it was found that all the data driven models failed to detect all suspicious contours. The Region Growing contiguousness model produced zero false negatives in all regions of interest other than prostate. With regards to runtime, the contiguousness via extent model took an average of 0.2 s per contour. On the other hand, the region growing method had a longer runtime which was dependent on the number of voxels in the contour. Both contiguousness models have potential for real-time use in clinical radiotherapy while the data driven models are better suited for retrospective use.

A Survey of Perianesthesia Nursing Electronic Documentation.

PURPOSE: Electronic health records have become a common part of the perianesthesia care workflow, particularly for data gathering and documentation. The purpose of this survey of perianesthesia nurses was to examine patterns of adoption of electronic health records and their effect on clinical documentation and patient care. DESIGN: A survey was sent to nurses who are members of the American Society of Perianesthesia Nursing (ASPAN). METHODS: The electronic documentation survey was sent to the e-mail addresses of 13,339 ASPAN members representing various practice environments across the United States. Results were examined through descriptive statistics. FINDINGS: About two thirds (66.02%) of the respondents indicated that they could capture 80% of their clinical interactions with the patient. Few nurses indicated that adoption of the EHR was done using a standardized terminology. Respondents (63.99%) overwhelmingly indicated that they spent less time interacting with patients and families because of electronic documentation demands. CONCLUSIONS: The results pertaining to the impact of the EHR on their practice were fairly mixed with some indication that there was greater access to important patient data, but with a trade-off of less satisfaction and efficiency. Improvements and evaluation of clinical documentation are being done, but ongoing optimization and improvements to the EHR based on the knowledge needs of nurses will help realize the promise of greater quality, safety, and access to data.

Physician activity during outpatient visits and subjective workload.

We describe methods for capturing and analyzing EHR use and clinical workflow of physicians during outpatient encounters and relating activity to physicians' self-reported workload. We collected temporally-resolved activity data including audio, video, EHR activity, and eye-gaze along with post-visit assessments of workload. These data are then analyzed through a combination of manual content analysis and computational techniques to temporally align streams, providing a range of process measures of EHR usage, clinical workflow, and physician-patient communication. Data was collected from primary care and specialty clinics at the Veterans Administration San Diego Healthcare System and UCSD Health, who use Electronic Health Record (EHR) platforms, CPRS and Epic, respectively. Grouping visit activity by physician, site, specialty, and patient status enables rank-ordering activity factors by their correlation to physicians' subjective work-load as captured by NASA Task Load Index survey. We developed a coding scheme that enabled us to compare timing studies between CPRS and Epic and extract patient and visit complexity profiles. We identified similar patterns of EHR use and navigation at the 2 sites despite differences in functions, user interfaces and consequent coded representations. Both sites displayed similar proportions of EHR function use and navigation, and distribution of visit length, proportion of time physicians attended to EHRs (gaze), and subjective work-load as measured by the task load survey. We found that visit activity was highly variable across individual physicians, and the observed activity metrics ranged widely as correlates to subjective workload. We discuss implications of our study for methodology, clinical workflow and EHR redesign.

Contextual Computing: A Bluetooth based approach for tracking healthcare providers in the emergency room.

Hospital Emergency Departments (EDs) frequently experience crowding. One of the factors that contributes to this crowding is the "door to doctor time", which is the time from a patient's registration to when the patient is first seen by a physician. This is also one of the Meaningful Use (MU) performance measures that emergency departments report to the Center for Medicare and Medicaid Services (CMS). Current documentation methods for this measure are inaccurate due to the imprecision in manual data collection. We describe a method for automatically (in real time) and more accurately documenting the door to physician time. Using sensor-based technology, the distance between the physician and the computer is calculated by using the single board computers installed in patient rooms that log each time a Bluetooth signal is seen from a device that the physicians carry. This distance is compared automatically with the accepted room radius to determine if the physicians are present in the room at the time logged to provide greater precision. The logged times, accurate to the second, were compared with physicians' handwritten times, showing automatic recordings to be more precise. This real time automatic method will free the physician from extra cognitive load of manually recording data. This method for evaluation of performance is generic and can be used in any other setting outside the ED, and for purposes other than measuring physician time.

CT and MR Protocol Standardization Across a Large Health System: Providing a Consistent Radiologist, Patient, and Referring Provider Experience.

Building and maintaining a comprehensive yet simple set of standardized protocols for a cross-sectional image can be a daunting task. A single department may have difficulty preventing "protocol creep," which almost inevitably occurs when an organized "playbook" of protocols does not exist and individual radiologists and technologists alter protocols at will and on a case-by-case basis. When multiple departments or groups function in a large health system, the lack of uniformity of protocols can increase exponentially. In 2012, the University of Colorado Hospital formed a large health system (UCHealth) and became a 5-hospital provider network. CT and MR imaging studies are conducted at multiple locations by different radiology groups. To facilitate consistency in ordering, acquisition, and appearance of a given study, regardless of location, we minimized the number of protocols across all scanners and sites of practice with a clinical indication-driven protocol selection and standardization process. Here we review the steps utilized to perform this process improvement task and insure its stability over time. Actions included creation of a standardized protocol template, which allowed for changes in electronic storage and management of protocols, designing a change request form, and formation of a governance structure. We utilized rapid improvement events (1 day for CT, 2 days for MR) and reduced 248 CT protocols into 97 standardized protocols and 168 MR protocols to 66. Additional steps are underway to further standardize output and reporting of imaging interpretation. This will result in an improved, consistent radiologist, patient, and provider experience across the system.

Fully Automated Deep Learning System for Bone Age Assessment.

Skeletal maturity progresses through discrete phases, a fact that is used routinely in pediatrics where bone age assessments (BAAs) are compared to chronological age in the evaluation of endocrine and metabolic disorders. While central to many disease evaluations, little has changed to improve the tedious process since its introduction in 1950. In this study, we propose a fully automated deep learning pipeline to segment a region of interest, standardize and preprocess input radiographs, and perform BAA. Our models use an ImageNet pretrained, fine-tuned convolutional neural network (CNN) to achieve 57.32 and 61.40% accuracies for the female and male cohorts on our held-out test images. Female test radiographs were assigned a BAA within 1 year 90.39% and within 2 years 98.11% of the time. Male test radiographs were assigned 94.18% within 1 year and 99.00% within 2 years. Using the input occlusion method, attention maps were created which reveal what features the trained model uses to perform BAA. These correspond to what human experts look at when manually performing BAA. Finally, the fully automated BAA system was deployed in the clinical environment as a decision supporting system for more accurate and efficient BAAs at much faster interpretation time (<2 s) than the conventional method.

Workflow Challenges of Enterprise Imaging: HIMSS-SIIM Collaborative White Paper.

With the advent of digital cameras, there has been an explosion in the number of medical specialties using images to diagnose or document disease and guide interventions. In many specialties, these images are not added to the patient's electronic medical record and are not distributed so that other providers caring for the patient can view them. As hospitals begin to develop enterprise imaging strategies, they have found that there are multiple challenges preventing the implementation of systems to manage image capture, image upload, and image management. This HIMSS-SIIM white paper will describe the key workflow challenges related to enterprise imaging and offer suggestions for potential solutions to these challenges.

Open-Source Radiation Exposure Extraction Engine (RE3) with Patient-Specific Outlier Detection.

We present an open-source, picture archiving and communication system (PACS)-integrated radiation exposure extraction engine (RE3) that provides study-, series-, and slice-specific data for automated monitoring of computed tomography (CT) radiation exposure. RE3 was built using open-source components and seamlessly integrates with the PACS. RE3 calculations of dose length product (DLP) from the Digital imaging and communications in medicine (DICOM) headers showed high agreement (R (2) = 0.99) with the vendor dose pages. For study-specific outlier detection, RE3 constructs robust, automatically updating multivariable regression models to predict DLP in the context of patient gender and age, scan length, water-equivalent diameter (D w), and scanned body volume (SBV). As proof of concept, the model was trained on 811 CT chest, abdomen + pelvis (CAP) exams and 29 outliers were detected. The continuous variables used in the outlier detection model were scan length (R (2) = 0.45), D w (R (2) = 0.70), SBV (R (2) = 0.80), and age (R (2) = 0.01). The categorical variables were gender (male average 1182.7 ± 26.3 and female 1047.1 ± 26.9 mGy cm) and pediatric status (pediatric average 710.7 ± 73.6 mGy cm and adult 1134.5 ± 19.3 mGy cm).

Using Semantic Components to Represent Dynamics of an Interdisciplinary Healthcare Team in a Multi-Agent Decision Support System.

In healthcare organizations, clinical workflows are executed by interdisciplinary healthcare teams (IHTs) that operate in ways that are difficult to manage. Responding to a need to support such teams, we designed and developed the MET4 multi-agent system that allows IHTs to manage patients according to presentation-specific clinical workflows. In this paper, we describe a significant extension of the MET4 system that allows for supporting rich team dynamics (understood as team formation, management and task-practitioner allocation), including selection and maintenance of the most responsible physician and more complex rules of selecting practitioners for the workflow tasks. In order to develop this extension, we introduced three semantic components: (1) a revised ontology describing concepts and relations pertinent to IHTs, workflows, and managed patients, (2) a set of behavioral rules describing the team dynamics, and (3) an instance base that stores facts corresponding to instances of concepts from the ontology and to relations between these instances. The semantic components are represented in first-order logic and they can be automatically processed using theorem proving and model finding techniques. We employ these techniques to find models that correspond to specific decisions controlling the dynamics of IHT. In the paper, we present the design of extended MET4 with a special focus on the new semantic components. We then describe its proof-of-concept implementation using the WADE multi-agent platform and the Z3 solver (theorem prover/model finder). We illustrate the main ideas discussed in the paper with a clinical scenario of an IHT managing a patient with chronic kidney disease.

Measuring cognitive effort using tabular transformer-based language models of electronic health record-based audit log action sequences.

OBJECTIVES: To develop and validate a novel measure, action entropy, for assessing the cognitive effort associated with electronic health record (EHR)-based work activities. MATERIALS AND METHODS: EHR-based audit logs of attending physicians and advanced practice providers (APPs) from four surgical intensive care units in 2019 were included. Neural language models (LMs) were trained and validated separately for attendings' and APPs' action sequences. Action entropy was calculated as the cross-entropy associated with the predicted probability of the next action, based on prior actions. To validate the measure, a matched pairs study was conducted to assess the difference in action entropy during known high cognitive effort scenarios, namely, attention switching between patients and to or from the EHR inbox. RESULTS: Sixty-five clinicians performing 5 904 429 EHR-based audit log actions on 8956 unique patients were included. All attention switching scenarios were associated with a higher action entropy compared to non-switching scenarios (P < .001), except for the from-inbox switching scenario among APPs. The highest difference among attendings was for the from-inbox attention switching: Action entropy was 1.288 (95% CI, 1.256-1.320) standard deviations (SDs) higher for switching compared to non-switching scenarios. For APPs, the highest difference was for the to-inbox switching, where action entropy was 2.354 (95% CI, 2.311-2.397) SDs higher for switching compared to non-switching scenarios. DISCUSSION: We developed a LM-based metric, action entropy, for assessing cognitive burden associated with EHR-based actions. The metric showed discriminant validity and statistical significance when evaluated against known situations of high cognitive effort (ie, attention switching). With additional validation, this metric can potentially be used as a screening tool for assessing behavioral action phenotypes that are associated with higher cognitive burden. CONCLUSION: An LM-based action entropy metric-relying on sequences of EHR actions-offers opportunities for assessing cognitive effort in EHR-based workflows.