Development and usability testing of a patient digital twin for critical care education: a mixed methods study.

Background: Digital twins are computerized patient replicas that allow clinical interventions testing in silico to minimize preventable patient harm. Our group has developed a novel application software utilizing a digital twin patient model based on electronic health record (EHR) variables to simulate clinical trajectories during the initial 6 h of critical illness. This study aimed to assess the usability, workload, and acceptance of the digital twin application as an educational tool in critical care. Methods: A mixed methods study was conducted during seven user testing sessions of the digital twin application with thirty-five first-year internal medicine residents. Qualitative data were collected using a think-aloud and semi-structured interview format, while quantitative measurements included the System Usability Scale (SUS), NASA Task Load Index (NASA-TLX), and a short survey. Results: Median SUS scores and NASA-TLX were 70 (IQR 62.5-82.5) and 29.2 (IQR 22.5-34.2), consistent with good software usability and low to moderate workload, respectively. Residents expressed interest in using the digital twin application for ICU rotations and identified five themes for software improvement: clinical fidelity, interface organization, learning experience, serious gaming, and implementation strategies. Conclusion: A digital twin application based on EHR clinical variables showed good usability and high acceptance for critical care education.

Development and Verification of a Digital Twin Patient Model to Predict Specific Treatment Response During the First 24 Hours of Sepsis.

To develop and verify a digital twin model of critically ill patient using the causal artificial intelligence approach to predict the response to specific treatment during the first 24 hours of sepsis. DESIGN: Directed acyclic graphs were used to define explicitly the causal relationship among organ systems and specific treatments used. A hybrid approach of agent-based modeling, discrete-event simulation, and Bayesian network was used to simulate treatment effect across multiple stages and interactions of major organ systems (cardiovascular, neurologic, renal, respiratory, gastrointestinal, inflammatory, and hematology). Organ systems were visualized using relevant clinical markers. The application was iteratively revised and debugged by clinical experts and engineers. Agreement statistics was used to test the performance of the model by comparing the observed patient response versus the expected response (primary and secondary) predicted by digital twin. SETTING: Medical ICU of a large quaternary- care academic medical center in the United States. PATIENTS OR SUBJECTS: Adult (> 18 year yr old), medical ICU patients were included in the study. INTERVENTIONS: No additional interventions were made beyond the standard of care for this study. MEASUREMENTS AND MAIN RESULTS: During the verification phase, model performance was prospectively tested on 145 observations in a convenience sample of 29 patients. Median age was 60 years (54-66 d) with a median Sequential Organ Failure Assessment score of 9.5 (interquartile range, 5.0-14.0). The most common source of sepsis was pneumonia, followed by hepatobiliary. The observations were made during the first 24 hours of the ICU admission with one-step interventions, comparing the output in the digital twin with the real patient response. The agreement between the observed versus and the expected response ranged from fair (kappa coefficient of 0.41) for primary response to good (kappa coefficient of 0.65) for secondary response to the intervention. The most common error detected was coding error in 50 observations (35%), followed by expert rule error in 29 observations (20%) and timing error in seven observations (5%). CONCLUSIONS: We confirmed the feasibility of development and prospective testing of causal artificial intelligence model to predict the response to treatment in early stages of critical illness. The availability of qualitative and quantitative data and a relatively short turnaround time makes the ICU an ideal environment for development and testing of digital twin patient models. An accurate digital twin model will allow the effect of an intervention to be tested in a virtual environment prior to use on real patients.

Comparison of Routine and Computer-Guided Glucose Management for Glycemic Control in Critically Ill Patients.

BACKGROUND: Glycemic control is crucial for reducing morbidity and mortality in critically ill patients. A standardized approach to glycemic control using a computer-guided protocol may help maintain blood glucose level within a target range and prevent human-induced medical errors. OBJECTIVE: To determine the effectiveness of a computer-guided glucose management protocol for glycemic control in intensive care patients. METHODS: This controlled, open-label implementation study involved 66 intensive care patients: 33 in the intervention group and 33 in the control group. The blood glucose level target range was established as 120 to 180 mg/dL. The control group received the clinic's routine glycemic monitoring approach, and the intervention group received monitoring using newly developed glycemic control software. At the end of the study, nurse perceptions and satisfaction were determined using a questionnaire. RESULTS: The rates of hyperglycemia and hypoglycemia were lower and the blood glucose level was more successfully maintained in the target range in the intervention group than in the control group (P < .001). The time to achieve the target range was shorter and less insulin was used in the intervention group than in the control group (P < .05). Nurses reported higher levels of satisfaction with the computerized protocol, which they found to be more effective and reliable than routine clinical practice. CONCLUSIONS: The computerized protocol was more effective than routine clinical practice in achieving glycemic control. It was also associated with higher nurse satisfaction levels.

Prompting with electronic checklist improves clinician performance in medical emergencies: a high-fidelity simulation study.

BACKGROUND: Inefficient processes of care delivery during acute resuscitation can compromise the "Golden Hour," the time when quick interventions can rapidly determine the course of the patient's outcome. Checklists have been shown to be an effective tool for standardizing care models. We developed a novel electronic tool, the Checklist for Early Recognition and Treatment of Acute Illness (CERTAIN) to facilitate standardized evaluation and treatment approach for acutely decompensating patients. The checklist was enforced by the use of a "prompter," a team member separate from the leader who records and reviews pertinent CERTAIN algorithms and verbalizes these to the team. Our hypothesis was that the CERTAIN model, with the use of the tool and a prompter, can improve clinician performance and satisfaction in the evaluation of acute decompensating patients in a simulated environment. METHODS: Volunteer clinicians with valid adult cardiac life support (ACLS) certification were invited to test the CERTAIN model in a high-fidelity simulation center. The first session was used to establish a baseline evaluation in a standard clinical resuscitation scenario. Each subject then underwent online training before returning to a simulation center for a live didactic lecture, software knowledge assessment, and practice scenarios. Each subject was then evaluated on a scenario with a similar content to the baseline. All subjects took a post-experience satisfaction survey. Video recordings of the pre-and post-test sessions were evaluated using a validated method by two blinded reviewers. RESULTS: Eighteen clinicians completed baseline and post-education sessions. CERTAIN prompting was associated with reduced omissions of critical tasks (46 to 32%, p < 0.01) and 12 out of 14 general assessment tasks were completed in a more timely manner. The post-test survey indicated that 72% subjects felt better prepared during an emergency scenario using the CERTAIN model and 85% would want to be treated with the CERTAIN if they were critically ill. CONCLUSION: Prompting with electronic checklist improves clinicians' performance and satisfaction when dealing with medical emergencies in high-fidelity simulation environment.

Checklist for Early Recognition and Treatment of Acute Illness (CERTAIN): evolution of a content management system for point-of-care clinical decision support.

BACKGROUND: The Checklist for Early Recognition and Treatment of Acute Illness (CERTAIN) is an international collaborative project with the overall objective of standardizing the approach to the evaluation and treatment of critically ill patients world-wide, in accordance with best-practice principles. One of CERTAIN's key features is clinical decision support providing point-of-care information about common acute illness syndromes, procedures, and medications in an index card format. METHODS: This paper describes 1) the process of developing and validating the content for point-of-care decision support, and 2) the content management system that facilitates frequent peer-review and allows rapid updates of content across different platforms (CERTAIN software, mobile apps, pdf-booklet) and different languages. RESULTS: Content was created based on survey results of acute care providers and validated using an open peer-review process. Over a 3 year period, CERTAIN content expanded to include 67 syndrome cards, 30 procedure cards, and 117 medication cards. 127 (59 %) cards have been peer-reviewed so far. Initially MS Word® and Dropbox® were used to create, store, and share content for peer-review. Recently Google Docs® was used to make the peer-review process more efficient. However, neither of these approaches met our security requirements nor has the capacity to instantly update the different CERTAIN platforms. CONCLUSION: Although we were able to successfully develop and validate a large inventory of clinical decision support cards in a short period of time, commercially available software solutions for content management are suboptimal. Novel custom solutions are necessary for efficient global point of care content system management.

Development and validation of clinical performance assessment in simulated medical emergencies: an observational study.

BACKGROUND: Critical illness is a time-sensitive process which requires practitioners to process vast quantities of data and make decisions rapidly. We have developed a tool, the Checklist for Early Recognition and Treatment of Acute Illness (CERTAIN), aimed at enhancing care delivery in such situations. To determine the efficacy of CERTAIN and similar cognitive aids, we developed rubric for evaluating provider performance in a simulated medical resuscitation environments. METHODS: We recruited 18 clinicians with current valid ACLS certification for evaluation in three simulated medical scenarios designed to mimic typical medical decompensation events routinely experienced in clinical care. Subjects were stratified as experienced or novice based on prior critical care training. A checklist of critical actions was designed using face validity for each scenario to evaluate task completion and performance. Simulation sessions were video recorded and scored by two independent raters. Construct validity was assessed under the assumption that experienced clinicians should perform better than novice clinicians on each task. Reliability was assessed as percentage agreement, kappa statistics and Bland-Altman plots as appropriate. RESULTS: Eleven experts and seven novices completed evaluation. The overall agreement on common checklist item completion was 84.8 %. The overall model achieved face validity and was consistent with our construct, with experienced clinicians trending towards better performance compared to novices for accuracy and speed of task completion. CONCLUSIONS: A standardized video assessment tool has potential to provide a valid and reliable method to assess 12 performances of clinicians facing simulated medical emergencies.

Ultrasound-guided pulsed radio frequency treatment in Morton's neuroma.

BACKGROUND: Morton's neuroma is a perineural fibrosis of an intermetatarsal plantar nerve. Burning, numbness, paresthesia, and tingling down the interspaces of involved toes may also be experienced. Taking into account all of this information, we designed a prospective open-label study to evaluate the efficacy of pulsed radio frequency on Morton's neuroma. METHODS: Twenty patients with Morton's neuroma were experiencing symptomatic neuroma pain in the foot not relieved by routine conservative treatment. All of the patients had been evaluated by a specialized orthopedist and were offered pulsed radio frequency as a last option before having surgery. Initially, pain level (numerical rating scale), successful pain control (a ≥50% pain decrease was accepted as successful pain control), comfort when walking (yes or no), and satisfaction level (satisfied or not satisfied) were evaluated. RESULTS: We found a decrease in the pain level in 18 of 20 patients, successful pain control in 12, and wearing shoes and walking without pain in 16. Overall, satisfaction was rated as excellent or good by 12 patients with Morton's neuroma in this series. CONCLUSIONS: This evidence indicates that ultrasound-guided pulsed radio frequency is a promising treatment modality in the management of Morton's neuroma pain.

The implementation of clinician designed, human-centered electronic medical record viewer in the intensive care unit: a pilot step-wedge cluster randomized trial.

OBJECTIVES: AWARE (Ambient Warning and Response Evaluation) is a novel electronic medical record (EMR) dashboard designed by clinicians to support bedside clinical information management in the ICU. AWARE sits on top of pre-existing, comprehensive EMR systems. The purpose of the study was to test the acceptance and impact of AWARE on data management in live clinical ICU settings. The primary outcome measure was observed efficiency of data utilization as determined by time spent in data gathering before morning rounds. DESIGN: Step wedge cluster randomization trial. SETTING: Four ICUs (surgical, medical, and mixed) at an academic referral center. SUBJECTS: All members of the critical care team participating in morning ICU rounds. INTERVENTION: Pilot implementation of a novel EMR interface with direct observation and survey. MEASUREMENTS AND MAIN RESULTS: The study took place between April and July 2012. A total of 80 and 63 direct observations were made in the pre- and post-implementation study periods respectively. The time spent on pre-round data gathering per patient decreased from 12 (10-15) to 9 (7.3-11) min for pre- and post-implementation phases respectively (p=0.03). Compared to the existing EMR, information management (data presentation format, efficiency of data access) was reported to be better after AWARE implementation. AWARE made the task of gathering data for rounds significantly less difficult and mentally demanding. CONCLUSIONS: The introduction of a novel, patient-centered EMR viewer for the ICU was associated with improved efficiency and ease of clinical data management compared to the standard EMR.

Checklist for early recognition and treatment of acute illness: International collaboration to improve critical care practice.

Processes to ensure world-wide best-practice for critical care delivery are likely to minimize preventable death, disability and costly complications for any healthcare system's sickest patients, but no large-scale efforts have so far been undertaken towards these goals. The advances in medical informatics and human factors engineering have provided possibility for novel and user-friendly clinical decision support tools that can be applied in a complex and busy hospital setting. To facilitate timely and accurate best-practice delivery in critically ill patients international group of intensive care unit (ICU) physicians and researchers developed a simple decision support tool: Checklist for Early Recognition and Treatment of Acute Illness (CERTAIN). The tool has been refined and tested in high fidelity simulated clinical environment and has been shown to improve performance of clinical providers faced with simulated emergencies. The aim of this international educational intervention is to implement CERTAIN into clinical practice in hospital settings with variable resources (included those in low income countries) and evaluate the impact of the tool on the care processes and patient outcomes. To accomplish our aims, CERTAIN will be uniformly available on either mobile or fixed computing devices (as well as a backup paper version) and applied in a standardized manner in the ICUs of diverse hospitals. To ensure the effectiveness of the proposed intervention, access to CERTAIN is coupled with structured training of bedside ICU providers.