Development, deployment and scaling of operating room-ready artificial intelligence for real-time surgical decision support.

Deep learning for computer vision can be leveraged for interpreting surgical scenes and providing surgeons with real-time guidance to avoid complications. However, neither generalizability nor scalability of computer-vision-based surgical guidance systems have been demonstrated, especially to geographic locations that lack hardware and infrastructure necessary for real-time inference. We propose a new equipment-agnostic framework for real-time use in operating suites. Using laparoscopic cholecystectomy and semantic segmentation models for predicting safe/dangerous ("Go"/"No-Go") zones of dissection as an example use case, this study aimed to develop and test the performance of a novel data pipeline linked to a web-platform that enables real-time deployment from any edge device. To test this infrastructure and demonstrate its scalability and generalizability, lightweight U-Net and SegFormer models were trained on annotated frames from a large and diverse multicenter dataset from 136 institutions, and then tested on a separate prospectively collected dataset. A web-platform was created to enable real-time inference on any surgical video stream, and performance was tested on and optimized for a range of network speeds. The U-Net and SegFormer models respectively achieved mean Dice scores of 57% and 60%, precision 45% and 53%, and recall 82% and 75% for predicting the Go zone, and mean Dice scores of 76% and 76%, precision 68% and 68%, and recall 92% and 92% for predicting the No-Go zone. After optimization of the client-server interaction over the network, we deliver a prediction stream of at least 60 fps and with a maximum round-trip delay of 70 ms for speeds above 8 Mbps. Clinical deployment of machine learning models for surgical guidance is feasible and cost-effective using a generalizable, scalable and equipment-agnostic framework that lacks dependency on hardware with high computing performance or ultra-fast internet connection speed.

Usability, Ergonomics, and Educational Value of a Novel Telestration Tool for Surgical Coaching: Usability Study.

BACKGROUND: Telementoring studies found technical challenges in achieving accurate and stable annotations during live surgery using commercially available telestration software intraoperatively. To address the gap, a wireless handheld telestration device was developed to facilitate dynamic user interaction with live video streams. OBJECTIVE: This study aims to find the perceived usability, ergonomics, and educational value of a first-generation handheld wireless telestration platform. METHODS: A prototype was developed with four core hand-held functions: (1) free-hand annotation, (2) cursor navigation, (3) overlay and manipulation (rotation) of ghost (avatar) instrumentation, and (4) hand-held video feed navigation on a remote monitor. This device uses a proprietary augmented reality platform. Surgeons and trainees were invited to test the core functions of the platform by performing standardized tasks. Usability and ergonomics were evaluated with a validated system usability scale and a 5-point Likert scale survey, which also evaluated the perceived educational value of the device. RESULTS: In total, 10 people (9 surgeons and 1 senior resident; 5 male and 5 female) participated. Participants strongly agreed or agreed (SA/A) that it was easy to perform annotations (SA/A 9, 90% and neutral 0, 0%), video feed navigation (SA/A 8, 80% and neutral 1, 10%), and manipulation of ghost (avatar) instruments on the monitor (SA/A 6, 60% and neutral 3, 30%). Regarding ergonomics, 40% (4) of participants agreed or strongly agreed (neutral 4, 40%) that the device was physically comfortable to use and hold. These results are consistent with open-ended comments on the device's size and weight. The average system usability scale was 70 (SD 12.5; median 75, IQR 63-84) indicating an above average usability score. Participants responded favorably to the device's perceived educational value, particularly for postoperative coaching (agree 6, 60%, strongly agree 4, 40%). CONCLUSIONS: This study presents the preliminary usability results of a novel first-generation telestration tool customized for use in surgical coaching. Favorable usability and perceived educational value were reported. Future iterations of the device should focus on incorporating user feedback and additional studies should be conducted to evaluate its effectiveness for improving surgical education. Ultimately, such tools can be incorporated into pedagogical models of surgical coaching to optimize feedback and training.

LapBot-Safe Chole: validation of an artificial intelligence-powered mobile game app to teach safe cholecystectomy.

BACKGROUND: Gaming can serve as an educational tool to allow trainees to practice surgical decision-making in a low-stakes environment. LapBot is a novel free interactive mobile game application that uses artificial intelligence (AI) to provide players with feedback on safe dissection during laparoscopic cholecystectomy (LC). This study aims to provide validity evidence for this mobile game. METHODS: Trainees and surgeons participated by downloading and playing LapBot on their smartphone. Players were presented with intraoperative LC scenes and required to locate their preferred location of dissection of the hepatocystic triangle. They received immediate accuracy scores and personalized feedback using an AI algorithm ("GoNoGoNet") that identifies safe/dangerous zones of dissection. Player scores were assessed globally and across training experience using non-parametric ANOVA. Three-month questionnaires were administered to assess the educational value of LapBot. RESULTS: A total of 903 participants from 64 countries played LapBot. As game difficulty increased, average scores (p < 0.0001) and confidence levels (p < 0.0001) decreased significantly. Scores were significantly positively correlated with players' case volume (p = 0.0002) and training level (p = 0.0003). Most agreed that LapBot should be incorporated as an adjunct into training programs (64.1%), as it improved their ability to reflect critically on feedback they receive during LC (47.5%) or while watching others perform LC (57.5%). CONCLUSIONS: Serious games, such as LapBot, can be effective educational tools for deliberate practice and surgical coaching by promoting learner engagement and experiential learning. Our study demonstrates that players' scores were correlated to their level of expertise, and that after playing the game, most players perceived a significant educational value.

Education in Laparoscopic Cholecystectomy: Design and Feasibility Study of the LapBot Safe Chole Mobile Game.

BACKGROUND:  Major bile duct injuries during laparoscopic cholecystectomy (LC), often stemming from errors in surgical judgment and visual misperception of critical anatomy, significantly impact morbidity, mortality, disability, and health care costs. OBJECTIVE:  To enhance safe LC learning, we developed an educational mobile game, LapBot Safe Chole, which uses an artificial intelligence (AI) model to provide real-time coaching and feedback, improving intraoperative decision-making. METHODS:  LapBot Safe Chole offers a free, accessible simulated learning experience with real-time AI feedback. Players engage with intraoperative LC scenarios (short video clips) and identify ideal dissection zones. After the response, users receive an accuracy score from a validated AI algorithm. The game consists of 5 levels of increasing difficulty based on the Parkland grading scale for cholecystitis. RESULTS:  Beta testing (n=29) showed score improvements with each round, with attendings and senior trainees achieving top scores faster than junior residents. Learning curves and progression distinguished candidates, with a significant association between user level and scores (P=.003). Players found LapBot enjoyable and educational. CONCLUSIONS:  LapBot Safe Chole effectively integrates safe LC principles into a fun, accessible, and educational game using AI-generated feedback. Initial beta testing supports the validity of the assessment scores and suggests high adoption and engagement potential among surgical trainees.

Educational value of a novel telestration device for surgical coaching-a randomized controlled trial.

INTRODUCTION: Communication is fundamental to effective surgical coaching. This can be challenging for training during image-guided procedures where coaches and trainees need to articulate technical details on a monitor. Telestration devices that annotate on monitors remotely could potentially overcome these limitations and enhance the coaching experience. This study aims to evaluate the value of a novel telestration device in surgical coaching. METHODS: A randomized-controlled trial was designed. All participants watched a video demonstrating the task followed by a baseline performance assessment and randomization into either control group (conventional verbal coaching without telestration) or telestration group (verbal coaching with telestration). Coaching for a simulated laparoscopic small bowel anastomosis on a dry lab model was done by a faculty surgeon. Following the coaching session, participants underwent a post-coaching performance assessment of the same task. Assessments were recorded and rated by blinded reviewers using a modified Global Rating Scale of the Objective Structured Assessment of Technical Skills (OSATS). Coaching sessions were also recorded and compared in terms of mentoring moments; guidance misinterpretations, questions/clarifications by trainees, and task completion time. A 5-point Likert scale was administered to obtain feedback. RESULTS: Twenty-four residents participated (control group 13, telestration group 11). Improvements in some elements of the OSATS scale were noted in the Telestration arm but there was no statistical significance in the overall score between the two groups. Mentoring moments were more in the telestration Group. Amongst the telestration Group, 55% felt comfortable that they could perform this task independently, compared to only 8% amongst the control group and 82% would recommend the use of telestration tools here. CONCLUSION: There is demonstrated educational value of this novel telestration device mainly in the non-technical aspects of the interaction by enhancing the coaching experience with improvement in communication and greater mentoring moments between coach and trainee.

Training for excellence: using a multimodal videoconferencing platform to coach surgeons and improve intraoperative performance.

INTRODUCTION: Continuing Professional Development opportunities for lifelong learning are fundamental to the acquisition of surgical expertise. However, few opportunities exist for longitudinal and structured learning to support the educational needs of surgeons in practice. While peer-to-peer coaching has been proposed as a potential solution, there remains significant logistical constraints and a lack of evidence to support its effectiveness. The purpose of this study is to determine whether the use of remote videoconferencing for video-based coaching improves operative performance. METHODS: Early career surgeon mentees participated in a remote coaching intervention with a surgeon coach of their choice and using a virtual telestration platform (Zoom Video Communications, San Jose, CA). Feedback was articulated through annotating videos. The coach evaluated mentee performance using a modified Intraoperative Performance Assessment Tool (IPAT). Participants completed a 5-point Likert scale on the educational value of the coaching program. RESULTS: Eight surgeons were enrolled in the study, six of whom completed a total of two coaching sessions (baseline, 6-month). Subspecialties included endocrine, hepatopancreatobiliary, and surgical oncology. Mean age of participants was 39 (SD 3.3), with mean 5 (SD 4.1) years in independent practice. Total IPAT scores increased significantly from the first session (mean 47.0, SD 1.9) to the second session (mean 51.8, SD 2.1), p = 0.03. Sub-category analysis showed a significant improvement in the Advanced Cognitive Skills domain with a mean of 33.2 (SD 2.5) versus a mean of 37.0 (SD 2.4), p < 0.01. There was no improvement in the psychomotor skills category. Participants agreed or strongly agreed that the coaching programs can improve surgical performance and decision-making (coaches 85%; mentees 100%). CONCLUSION: Remote surgical coaching is feasible and has educational value using ubiquitous commercially available virtual platforms. Logistical issues with scheduling and finding cases aligned with learning objectives continue to challenge program adoption and widespread dissemination.

Use of artificial intelligence for decision-support to avoid high-risk behaviors during laparoscopic cholecystectomy.

INTRODUCTION: Bile duct injuries (BDIs) are a significant source of morbidity among patients undergoing laparoscopic cholecystectomy (LC). GoNoGoNet is an artificial intelligence (AI) algorithm that has been developed and validated to identify safe ("Go") and dangerous ("No-Go") zones of dissection during LC, with the potential to prevent BDIs through real-time intraoperative decision-support. This study evaluates GoNoGoNet's ability to predict Go/No-Go zones during LCs with BDIs. METHODS AND PROCEDURES: Eleven LC videos with BDI (BDI group) were annotated by GoNoGoNet. All tool-tissue interactions, including the one that caused the BDI, were characterized in relation to the algorithm's predicted location of Go/No-Go zones. These were compared to another 11 LC videos with cholecystitis (control group) deemed to represent "safe cholecystectomy" by experts. The probability threshold of GoNoGoNet annotations were then modulated to determine its relationship to Go/No-Go predictions. Data is shown as % difference [99% confidence interval]. RESULTS: Compared to control, the BDI group showed significantly greater proportion of sharp dissection (+ 23.5% [20.0-27.0]), blunt dissection (+ 32.1% [27.2-37.0]), and total interactions (+ 33.6% [31.0-36.2]) outside of the Go zone. Among injury-causing interactions, 4 (36%) were in the No-Go zone, 2 (18%) were in the Go zone, and 5 (45%) were outside both zones, after maximizing the probability threshold of the Go algorithm. CONCLUSION: AI has potential to detect unsafe dissection and prevent BDIs through real-time intraoperative decision-support. More work is needed to determine how to optimize integration of this technology into the operating room workflow and adoption by end-users.

What is the educational value and clinical utility of artificial intelligence for intraoperative and postoperative video analysis? A survey of surgeons and trainees.

INTRODUCTION: Surgical complications often occur due to lapses in judgment and decision-making. Advances in artificial intelligence (AI) have made it possible to train algorithms that identify anatomy and interpret the surgical field. These algorithms can potentially be used for intraoperative decision-support and postoperative video analysis and feedback. Despite the very early success of proof-of-concept algorithms, it remains unknown whether this innovation meets the needs of end-users or how best to deploy it. This study explores users' opinion on the value, usability and design for adapting AI in operating rooms. METHODS: A device-agnostic web-accessible software was developed to provide AI inference either (1) intraoperatively on a live video stream (synchronous mode), or (2) on an uploaded video or image file (asynchronous mode) postoperatively for feedback. A validated AI model (GoNoGoNet), which identifies safe and dangerous zones of dissection during laparoscopic cholecystectomy, was used as the use case. Surgeons and trainees performing laparoscopic cholecystectomy interacted with the AI platform and completed a 5-point Likert scale survey to evaluate the educational value, usability and design of the platform. RESULTS: Twenty participants (11 surgeons and 9 trainees) evaluated the platform intraoperatively (n = 10) and postoperatively (n = 11). The majority agreed or strongly agreed that AI is an effective adjunct to surgical training (81%; neutral = 10%), effective for providing real-time feedback (70%; neutral = 20%), postoperative feedback (73%; neutral = 27%), and capable of improving surgeon confidence (67%; neutral = 29%). Only 40% (neutral = 50%) and 57% (neutral = 43%) believe that the tool is effective in improving intraoperative decisions and performance, or beneficial for patient care, respectively. Overall, 38% (neutral = 43%) reported they would use this platform consistently if available. The majority agreed or strongly agreed that the platform was easy to use (81%; neutral = 14%) and has acceptable resolution (62%; neutral = 24%), while 30% (neutral = 20%) reported that it disrupted the OR workflow, and 20% (neutral = 0%) reported significant time lag. All respondents reported that such a system should be available "on-demand" to turn on/off at their discretion. CONCLUSIONS: Most found AI to be a useful tool for providing support and feedback to surgeons, despite several implementation obstacles. The study findings will inform the future design and usability of this technology in order to optimize its clinical impact and adoption by end-users.

Perceptions of the multidisciplinary operative team on intraoperative telecoaching among surgeons.

Telecoaching, intraoperative coaching through videoconference, has been suggested as a tool to overcome logistical barriers with in-person coaching. However, little is known about the operative team's perception of telecoaching and its unique set of challenges. This qualitative study explores the perceptions of the multidisciplinary operative team on surgical telecoaching. A telecoaching program between peer surgeons was implemented using the Karl Storz Visitor1 remote presence system (Karl Storz, Germany). Semistructured interviews were conducted with the 12 operative team members present during 2 telecoaching sessions completed during the study period. Twelve participants were interviewed. The 4 central themes that emerged from the data were effective communication and collaboration, improving performance, operating room workflow, and culture and optics. Participating surgeon mentees reported that the session met expectations and learning goals and revealed concerns about negative perception of their autonomy and expertise by colleagues and patients. Conversely, team members unanimously reported a positive impression of surgeon mentees for taking additional measures to improve their performance and for patient outcomes. The operative team members reported that telecoaching was conducive to their own learning and relevant for complex cases. Considerations for future implementation of telecoaching include robust privacy standards for patients and staff, strong internet connectivity, coordinating with the operative team, and space constraints. Operative team participants viewed the intervention favorably and identified practical considerations for its continued use in an operating room environment. However, more work is needed on surgical culture as a contributor to low adoption and its impact on coaching programming activity.

Establishing validity evidence for device-assisted advanced laparoscopic suturing tasks using simulation.

BACKGROUND: Laparoscopic suturing (LS) is an essential technique required for a wide range of procedures, and it is one of the most challenging for surgical trainees to master. We designed and collected validity evidence for advanced LS tasks using an automated suturing device and evaluated the perceived educational value of the tasks. METHODS: This project was a multicentre prospective study involving McGill University, University of Toronto (UofT), and Louisiana State University (LSU) Health New Orleans. Novice (NS) and experienced (ES) surgeons performed suturing under tension (UT) and continuous suturing (CS) tasks. ES performed the tasks twice to establish proficiency benchmarks, and they were interviewed to develop formative feedback tools (FFT). Participants were assessed on completion time, error, Global Operative Assessment of Laparoscopic Skills (GOALS), and FFT. Data were analyzed using descriptive and inferential statistical methods. RESULTS: Twenty-seven participants (13 ES, 14 NS, median age 34 years; 85% male) completed the study. Eight were attending surgeons, 7 fellows, 4 PGY5, 5 PGY4, and 3 PGY3 (18 from McGill, 5 UofT, and 4 LSU). Comparing ES and NS, for UT task, ES had significantly greater task scores (570 [563-648] vs 323 [130-464], p value 0.00036) and GOALS scores (14 [13-16] vs 10 [8-12], p value 0.0038). Similarly, for CS, ES had significantly greater task scores (976 [959-1010] vs 785 [626-856], p value 0.00009) and GOALS scores (16 [12-17] vs 12.5 [8.25-15], p value 0.028). After FFTs were developed, comparing ES and NS, for both UT and CS tasks, ES had significantly greater FFT scores (UT 25 [24-26] vs 17 [14-20], p value 0.0016 and CS 30 [27-32] vs 22[17.2-25.8], p value 0.00061). CONCLUSION: In conclusion, preliminary validity evidence was provided for the tasks. Once further validity evidence is established, incorporating the tasks into the training curricula could improve trainee skills and help to meet the need for better advanced suturing models.

Development of a simulation curriculum to teach and assess advanced laparoscopic suturing skills using telesimulation: a feasibility study.

BACKGROUND: Telesimulation helps overcome limitations in time and local expertise by eliminating the need for the learner and educator to be physically co-located, especially important during COVID-19. We investigated whether teaching advanced laparoscopic suturing (ALS) through telesimulation is feasible, effective, and leads to improved suturing in the operating room (OR). METHODS: In this prospective feasibility study, three previously developed 3D-printed ALS tasks were used: needle handling (NH), suturing under tension (UT), and continuous suturing (CS). General surgery residents (PGY4-5) underwent 1-month of telesimulation training, during which an expert educator at one site remotely trained residents at the other site over 2-3 teaching sessions. Trainees were assessed in the three tasks and in the OR at three time points: baseline(A1), control period(A2), and post-intervention(A3) and completed questionnaires regarding educational value and usability of telesimulation. Paired t-test was used to compare scores between the three assessment points. RESULTS: Six residents were included. Scores for UT improved significantly post-intervention A3(568 ± 60) when compared to baseline A1(416 ± 133) (p < 0.019). Similarly, scores for CS improved significantly post-intervention A3(756 ± 113) vs. baseline A1(539 ± 211) (p < 0.02). For intraoperative assessments, scores improved significantly post-intervention A3(21 ± 3) when compared to both A1(17 ± 4) (p < 0.018) and A2(18 ± 4) (p < 0.0008). All residents agreed that tasks were relevant to practice, helped improve technical competence, and adequately measured suturing skill. All residents found telesimulation easy to use, had strong educational value, and want the system to be incorporated into their training. CONCLUSION: The use of telesimulation for remotely training residents using ALS tasks was feasible and effective. Residents found value in training using the tasks and telesimulation system, and improved ALS skills in the OR. As the pandemic has caused a major structural shift in resident education, telesimulation can be an effective alternative to on-site simulation programs. Future research should focus on how telesimulation can be effectively incorporated into training programs.

Impact of a Mobile App to Support Home Recovery of Patients Undergoing Bariatric Surgery.

BACKGROUND: Perioperative patient education and engagement are critical components of care in patients undergoing bariatric surgery, given the short length of stay and the requirements to adhere to various instructions. The use of patient engagement mobile technology may promote adherence to perioperative protocols and improve care by potentially identifying complications earlier and reducing associated health care costs. MATERIALS AND METHODS: We introduced a mobile app that provides bariatric patients with access to educational materials and the ability to report on their symptoms. Using the data from the app and linking the data to patient outcomes collected in the Metabolic and Bariatric Surgery Accreditation and Quality Improvement Program database, we examined the effects of the app on readmission, length of stay, visits to the emergency department (ED), and patient satisfaction. RESULTS: A total of 505 patients were enrolled in the app between July 2017 and March 2019. Among them, 396 patients who met the inclusion criteria for the study were compared with 458 patients in the Metabolic and Bariatric Surgery Accreditation and Quality Improvement Program database who were not enrolled in the app for the same study period. While the use of the app was not associated with the rates of prolonged length of stay, ED visits, and readmission, patients who completed a survey at 30 d after discharge reported that the app helped them avoid phone calls to the hospital (48.5%) and ED visits (13.0%). Furthermore, 94.8% of these patients reported that they would recommend the app to other patients undergoing the same surgery. CONCLUSIONS: Additional features, such as the ability for patients to directly communicate with the health care providers within the app, may be effective in decreasing unnecessary health care utilization.

Choice of rating scale labels: implication for minimizing patient satisfaction response ceiling effect in telemedicine surveys.

BACKGROUND: Lack of response variability is problematic in surveys because of its detrimental effects on sensitivity and consequently reliability of the responses. In satisfaction surveys, this problem is caused by the ceiling effect resulting from high satisfaction ratings. A potential solution strategy is to manipulate the labels of the rating scale to create greater discrimination of responses on the high end of the response continuum. This study examined the effects of a positive-centered scale on the distribution and reliability of telemedicine satisfaction responses in a highly positive respondent population. MATERIALS AND METHODS: In total, 216 telemedicine participants were randomly assigned to one of three experimental conditions as defined by the form of Likert scale: (1) 5-point Balanced Equal-Interval, (2) 5-point Positive-Packed, and (3) 5-point Positive-Centered Equal-Interval. RESULTS: Although the study findings were not statistically significant, partially because of sample size, the distribution and internal consistency reliability of responses occurred in the direction hypothesized. Loading the rating scale with more positive labels appears to be a useful strategy for reducing the ceiling effect and increases the discrimination ability of survey responses. CONCLUSIONS: The current research provides a survey design strategy to minimize ceiling effects. Although the findings provide some evidence suggesting the benefit of using rating scales loaded with positive labels, more research is needed to confirm this, as well as extend it to examine other types of rating scales and the interaction between rating scale formats and respondent characteristics.

Multiple Intravenous Infusions Phase 2a: Ontario Survey.

BACKGROUND: Research conducted in earlier phases of this study prospectively identified a number of concerns related to the safe administration of multiple intravenous (IV) infusions in Ontario hospitals. OBJECTIVE: To investigate the potential prevalence of practices or policies that may contribute to the patient safety risks identified in Phase 1b of this study. DATA SOURCES AND REVIEW METHODS: Sixty-four survey responses were analyzed from clinical units where multiple IV infusions may occur (e.g., adult intensive care units). Survey questions were organized according to the topics identified in Phase 1b as potential contributors to patient harm (e.g., labelling practices, patient transfer practices, secondary infusion policies). RESULTS: Survey results indicated suboptimal practices and policies in some clinical units, and variability in a number of infusion practices. Key areas of concern included the following: use of primary IV tubing without back check valves when administering secondary infusions, administration of secondary infusions with/as high-alert continuous IV medications, potential confusion about how IV tubing should be labelled to reflect replacement date and time, interruptions to IV therapy due to IV pump and/or tubing changes when patients are transferred between clinical units, coadministration of continuous or intermittent infusions on central venous pressure monitoring ports, variability in respondents' awareness of the infusion pump's bolus capabilities. LIMITATIONS: Due to the limited sample size, survey responses may not be representative of infusion practices across Ontario. Answers to some questions indicated that the intent of the questions might have been misunderstood. Due to a design error, 1 question about bolus administration methods was not shown to as many respondents as appropriate. CONCLUSIONS: The Ontario survey revealed variability in IV infusion practice across the province and potential opportunities to improve safety.

Multiple Intravenous Infusions Phase 2b: Laboratory Study.

BACKGROUND: Administering multiple intravenous (IV) infusions to a single patient via infusion pump occurs routinely in health care, but there has been little empirical research examining the risks associated with this practice or ways to mitigate those risks. OBJECTIVES: To identify the risks associated with multiple IV infusions and assess the impact of interventions on nurses' ability to safely administer them. DATA SOURCES AND REVIEW METHODS: Forty nurses completed infusion-related tasks in a simulated adult intensive care unit, with and without interventions (i.e., repeated-measures design). RESULTS: Errors were observed in completing common tasks associated with the administration of multiple IV infusions, including the following (all values from baseline, which was current practice): setting up and programming multiple primary continuous IV infusions (e.g., 11.7% programming errors)identifying IV infusions (e.g., 7.7% line-tracing errors)managing dead volume (e.g., 96.0% flush rate errors following IV syringe dose administration)setting up a secondary intermittent IV infusion (e.g., 11.3% secondary clamp errors)administering an IV pump bolus (e.g., 11.5% programming errors)Of 10 interventions tested, 6 (1 practice, 3 technology, and 2 educational) significantly decreased or even eliminated errors compared to baseline. LIMITATIONS: The simulation of an adult intensive care unit at 1 hospital limited the ability to generalize results. The study results were representative of nurses who received training in the interventions but had little experience using them. The longitudinal effects of the interventions were not studied. CONCLUSIONS: Administering and managing multiple IV infusions is a complex and risk-prone activity. However, when a patient requires multiple IV infusions, targeted interventions can reduce identified risks. A combination of standardized practice, technology improvements, and targeted education is required.

Developing healthcare rule-based expert systems: case study of a heart failure telemonitoring system.

BACKGROUND: The use of expert systems to generate automated alerts and patient instructions based on telemonitoring data could enable increased self-care and improve clinical management. However, of great importance is the development of the rule set to ensure safe and clinically relevant alerts and instructions are sent. The purpose of this work was to develop a rule-based expert system for a heart failure mobile phone-based telemonitoring system, to evaluate the expert system, and to generalize the lessons learned from the development process for use in other healthcare applications. METHODS: Semi-structured interviews were conducted with 10 heart failure clinicians to inform the development of a draft heart failure rule set for alerts and patient instructions. The draft rule set was validated and refined with 9 additional interviews with heart failure clinicians. Finally, the clinical champion of the project vetted the rule set. The concerns voiced by the clinicians during the interviews were noted, and methods to mitigate these concerns were employed. The rule set was then evaluated as part of a 6-month randomized controlled trial of a mobile phone-based heart failure telemonitoring system (n=50 for each of the telemonitoring and control groups). RESULTS: The developed expert system generated alerts and instructions based on the patient's weight, blood pressure, heart rate, and symptoms. During the trial, 1620 alerts were generated, which led to various clinical actions including 105 medication changes/instructions. The findings from the trial indicated the rule set was associated with improved quality of life and self-care. CONCLUSIONS: A rule set was developed with extensive input by heart failure clinicians. The results from the trial indicated the rule set was associated with significantly increased self-care and improved the clinical management of heart failure. The developed rule set can be used as a basis for other heart failure telemonitoring systems, but should be validated and modified as necessary. In addition, the process used to develop the rule set can be generalized and applied to create robust and complete rule sets for other healthcare expert systems.

Mobile phone-based telemonitoring for heart failure management: a randomized controlled trial.

BACKGROUND: Previous trials of telemonitoring for heart failure management have reported inconsistent results, largely due to diverse intervention and study designs. Mobile phones are becoming ubiquitous and economical, but the feasibility and efficacy of a mobile phone-based telemonitoring system have not been determined. OBJECTIVE: The objective of this trial was to investigate the effects of a mobile phone-based telemonitoring system on heart failure management and outcomes. METHODS: One hundred patients were recruited from a heart function clinic and randomized into telemonitoring and control groups. The telemonitoring group (N = 50) took daily weight and blood pressure readings and weekly single-lead ECGs, and answered daily symptom questions on a mobile phone over 6 months. Readings were automatically transmitted wirelessly to the mobile phone and then to data servers. Instructions were sent to the patients' mobile phones and alerts to a cardiologist's mobile phone as required. RESULTS: Baseline questionnaires were completed and returned by 94 patients, and 84 patients returned post-study questionnaires. About 70% of telemonitoring patients completed at least 80% of their possible daily readings. The change in quality of life from baseline to post-study, as measured with the Minnesota Living with Heart Failure Questionnaire, was significantly greater for the telemonitoring group compared to the control group (P = .05). A between-group analysis also found greater post-study self-care maintenance (measured with the Self-Care of Heart Failure Index) for the telemonitoring group (P = .03). Brain natriuretic peptide (BNP) levels, self-care management, and left ventricular ejection fraction (LVEF) improved significantly for both groups from baseline to post-study, but did not show a between-group difference. However, a subgroup within-group analysis using the data from the 63 patients who had attended the heart function clinic for more than 6 months revealed the telemonitoring group had significant improvements from baseline to post-study in BNP (decreased by 150 pg/mL, P = .02), LVEF (increased by 7.4%, P = .005) and self-care maintenance (increased by 7 points, P = .05) and management (increased by 14 points, P = .03), while the control group did not. No differences were found between the telemonitoring and control groups in terms of hospitalization, mortality, or emergency department visits, but the trial was underpowered to detect differences in these metrics. CONCLUSIONS: Our findings provide evidence of improved quality of life through improved self-care and clinical management from a mobile phone-based telemonitoring system. The use of the mobile phone-based system had high adherence and was feasible for patients, including the elderly and those with no experience with mobile phones. TRIAL REGISTRATION: ClinicalTrials.gov NCT00778986.

Perceptions and experiences of heart failure patients and clinicians on the use of mobile phone-based telemonitoring.

BACKGROUND: Previous trials of heart failure telemonitoring systems have produced inconsistent findings, largely due to diverse interventions and study designs. OBJECTIVES: The objectives of this study are (1) to provide in-depth insight into the effects of telemonitoring on self-care and clinical management, and (2) to determine the features that enable successful heart failure telemonitoring. METHODS: Semi-structured interviews were conducted with 22 heart failure patients attending a heart function clinic who had used a mobile phone-based telemonitoring system for 6 months. The telemonitoring system required the patients to take daily weight and blood pressure readings, weekly single-lead ECGs, and to answer daily symptom questions on a mobile phone. Instructions were sent to the patient's mobile phone based on their physiological values. Alerts were also sent to a cardiologist's mobile phone, as required. All clinicians involved in the study were also interviewed post-trial (N = 5). The interviews were recorded, transcribed, and then analyzed using a conventional content analysis approach. RESULTS: The telemonitoring system improved patient self-care by instructing the patients in real-time how to appropriately modify their lifestyle behaviors. Patients felt more aware of their heart failure condition, less anxiety, and more empowered. Many were willing to partially fund the use of the system. The clinicians were able to manage their patients' heart failure conditions more effectively, because they had physiological data reported to them frequently to help in their decision-making (eg, for medication titration) and were alerted at the earliest sign of decompensation. Essential characteristics of the telemonitoring system that contributed to improved heart failure management included immediate self-care and clinical feedback (ie, teachable moments), how the system was easy and quick to use, and how the patients and clinicians perceived tangible benefits from telemonitoring. Some clinical concerns included ongoing costs of the telemonitoring system and increased clinical workload. A few patients did not want to be watched long-term while some were concerned they might become dependent on the system. CONCLUSIONS: The success of a telemonitoring system is highly dependent on its features and design. The essential system characteristics identified in this study should be considered when developing telemonitoring solutions.

Self-care and quality of life of heart failure patients at a multidisciplinary heart function clinic.

BACKGROUND: Multidisciplinary heart function clinics aim to improve self-care through patient education and to provide clinical management. OBJECTIVE: The objectives of the present study were to investigate the self-care and quality of life of patients attending a multidisciplinary heart function clinic and to explore the relationship between self-care and quality of life. METHODS: One hundred outpatients attending a multidisciplinary heart function clinic were asked to complete a questionnaire. The questionnaire included the Self-care of Heart Failure Index (SCHFI) and the Minnesota Living With Heart Failure Questionnaire, which were used to assess self-care behavior and quality of life, respectively. Self-care practices and perceived barriers were also assessed through semistructured interviews with each patient. RESULTS: : The returned questionnaires (n = 94) were used to compute the following SCHFI maintenance, management, and confidence scores: 60.8 (SD, 19.3), 62.0 (SD, 20.7), and 55.9 (SD, 19.7), respectively. Higher SCHFI scores indicate better self-care. None of the self-care dimensions reached the self-care adequacy cut point of 70. The average score on the Minnesota Living With Heart Failure Questionnaire was 49.9 (SD, 25.4), indicating a moderate health-related quality of life. Lower ejection fraction, older age, and better quality of life were associated with better self-care. Determinants of better quality of life were older age, better functional capacity, higher self-care confidence, and fewer comorbidities. The patient interviews revealed that better quality of life is associated with higher self-care confidence and barriers to self-care caused anxiety to the patients. The self-care barriers were found to include lack of self-care education, financial constraints, lack of perceived benefit, and low self-efficacy. CONCLUSIONS: Patients attending a large multidisciplinary Canadian heart failure clinic do not perform adequate self-care as measured with the SCHFI and report only a moderate quality of life. Increasing self-care through education and tools that target self-care barriers are required and may help improve quality of life.

Attitudes of heart failure patients and health care providers towards mobile phone-based remote monitoring.

BACKGROUND: Mobile phone-based remote patient monitoring systems have been proposed for heart failure management because they are relatively inexpensive and enable patients to be monitored anywhere. However, little is known about whether patients and their health care providers are willing and able to use this technology. OBJECTIVE: The objective of our study was to assess the attitudes of heart failure patients and their health care providers from a heart function clinic in a large urban teaching hospital toward the use of mobile phone-based remote monitoring. METHODS: A questionnaire regarding attitudes toward home monitoring and technology was administered to 100 heart failure patients (94/100 returned a completed questionnaire). Semi-structured interviews were also conducted with 20 heart failure patients and 16 clinicians to determine the perceived benefits and barriers to using mobile phone-based remote monitoring, as well as their willingness and ability to use the technology. RESULTS: The survey results indicated that the patients were very comfortable using mobile phones (mean rating 4.5, SD 0.6, on a five-point Likert scale), even more so than with using computers (mean 4.1, SD 1.1). The difference in comfort level between mobile phones and computers was statistically significant (P< .001). Patients were also confident in using mobile phones to view health information (mean 4.4, SD 0.9). Patients and clinicians were willing to use the system as long as several conditions were met, including providing a system that was easy to use with clear tangible benefits, maintaining good patient-provider communication, and not increasing clinical workload. Clinicians cited several barriers to implementation of such a system, including lack of remuneration for telephone interactions with patients and medicolegal implications. CONCLUSIONS: Patients and clinicians want to use mobile phone-based remote monitoring and believe that they would be able to use the technology. However, they have several reservations, such as potential increased clinical workload, medicolegal issues, and difficulty of use for some patients due to lack of visual acuity or manual dexterity.