Crowd-sourced and expert video assessment in minimally invasive esophagectomy.

BACKGROUND: Video-based assessment by experts may structurally measure surgical performance using procedure-specific competency assessment tools (CATs). A CAT for minimally invasive esophagectomy (MIE-CAT) was developed and validated previously. However, surgeon's time is scarce and video assessment is time-consuming and labor intensive. This study investigated non-procedure-specific assessment of MIE video clips by MIE experts and crowdsourcing, collective surgical performance evaluation by anonymous and untrained laypeople, to assist procedure-specific expert review. METHODS: Two surgical performance scoring frameworks were used to assess eight MIE videos. First, global performance was assessed with the non-procedure-specific Global Operative Assessment of Laparoscopic Skills (GOALS) of 64 procedural phase-based video clips < 10 min. Each clip was assessed by two MIE experts and > 30 crowd workers. Second, the same experts assessed procedure-specific performance with the MIE-CAT of the corresponding full-length video. Reliability and convergent validity of GOALS for MIE were investigated using hypothesis testing with correlations (experience, blood loss, operative time, and MIE-CAT). RESULTS: Less than 75% of hypothesized correlations between GOALS scores and experience of the surgical team (r < 0.3), blood loss (r = - 0.82 to 0.02), operative time (r = - 0.42 to 0.07), and the MIE-CAT scores (r = - 0.04 to 0.76) were met for both crowd workers and experts. Interestingly, experts' GOALS and MIE-CAT scores correlated strongly (r = 0.40 to 0.79), while crowd workers' GOALS and experts' MIE-CAT scores correlations were weak (r = - 0.04 to 0.49). Expert and crowd worker GOALS scores correlated poorly (ICC ≤ 0.42). CONCLUSION: GOALS assessments by crowd workers lacked convergent validity and showed poor reliability. It is likely that MIE is technically too difficult to assess for laypeople. Convergent validity of GOALS assessments by experts could also not be established. GOALS might not be comprehensive enough to assess detailed MIE performance. However, expert's GOALS and MIE-CAT scores strongly correlated indicating video clip (instead of full-length video) assessments could be useful to shorten assessment time.

Comparing Continuous with Periodic Vital Sign Scoring for Clinical Deterioration Using a Patient Data Model.

To evaluate a minute-by-minute monitoring algorithm against a periodic early warning score (EWS) in detecting clinical deterioration and workload. Periodic EWSs suffer from large measurement intervals, causing late detection of deterioration. This might be prevented by continuous vital sign monitoring with a real-time algorithm such as the Visensia Safety Index (VSI). This prospective comparative data modeling cohort study (NCT04189653) compares continuous algorithmic alerts against periodic EWS in continuous monitored medical and surgical inpatients. We evaluated sensitivity, frequency, number of warnings needed to evaluate (NNE) and time of initial alert till escalation of care (EOC): Rapid Response Team activation, unplanned ICU admission, emergency surgery, or death. Also, the percentage of VSI alerting minutes was compared between patients with or without EOC. In 1529 admissions continuous VSI warned for 55% of EOC (95% CI: 45-64%) versus 51% (95% CI: 41-61%) by periodic EWS. NNE for VSI was 152 alerts per detected EOC (95% CI: 114-190) compared to 21 (95% CI: 17-28). It generated 0.99 warnings per day per patient compared to 0.13. Time from detection score till escalation was 8.3 hours (IQR: 2.6-24.8) with VSI versus 5.2 (IQR: 2.7-12.3) hours with EWS (P=0.074). The percentage of warning VSI minutes was higher in patients with EOC than in stable patients (2.36% vs 0.81%, P<0.001). Although sensitivity of detection was not significantly improved continuous vital sign monitoring shows potential for earlier alerts for deterioration compared to periodic EWS. A higher percentage of alerting minutes may indicate risk for deterioration.

The Effect of Continuous Versus Periodic Vital Sign Monitoring on Disease Severity of Patients with an Unplanned ICU Transfer.

Continuous vital sign monitoring (CM) may detect ward patient's deterioration earlier than periodic monitoring. This could result in timely ICU transfers or in a transfer delay due to misperceived higher level of care on the ward. The primary objective of this study was to compare patient's disease severity upon unplanned ICU transfer, before and after CM implementation. We included a one-year period before and after CM implementation between August 1, 2017 - July 31, 2019. Before implementation, surgical and internal medicine patients' vital signs were periodically monitored, compared to continuous monitoring with wireless linkage to hospital systems after implementation. In both periods the same early warning score (EWS) protocol was in place. Primary outcome was disease severity scores upon ICU transfer. Secondary outcomes were ICU and hospital length of stay, incidence of mechanical ventilation and ICU mortality. In the two one-year periods 93 and 59 unplanned ICU transfer episodes were included, respectively. Median SOFA (3 (2-6) vs 4 (2-7), p = .574), APACHE II (17 (14-20) vs 16 (14-21), p = .824) and APACHE IV (59 (46-67) vs 50 (36-65), p = .187) were comparable between both periods, as were the median ICU LOS (3.0 (1.7-5.8) vs 3.1 (1.6-6.1), p.962), hospital LOS (23.6 (11.5-38.0) vs 19 (13.9-39.2), p = .880), incidence of mechanical ventilation (28 (47%) vs 22 (54%), p.490), and ICU mortality (11 (13%) vs 10 (19%), p.420). This study shows no difference in disease severity upon unplanned ICU transfer after CM implementation for patients who have deteriorated on the ward.

Effect of continuous wireless vital sign monitoring on unplanned ICU admissions and rapid response team calls: a before-and-after study.

BACKGROUND: Continuous vital sign monitoring may potentially be improved through the use of wearable monitors linked wirelessly to hospital electronic patient records. By improving early detection of physiological deterioration this approach may save lives. METHODS: We performed a single-centre before-and-after study including surgical and medical patients at a university hospital in The Netherlands. The study intervention was continuous vital sign monitoring using wearable monitors linked wirelessly to hospital systems. The co-primary outcomes were unplanned ICU admission and rapid response team calls. Secondary outcomes were length of hospital stay and in-patient death. RESULTS: Our baseline cohort included 2466 admissions and our intervention cohort included 2303 admissions recruited from August 2017 to July 2019. Patients in the intervention cohort experienced fewer unplanned ICU admissions (84 [3.4%] vs 54 [2.3%]; P=0.03) and fewer rapid response team calls (107 [4.3%] vs 71 [3.1%]; P=0.02). The number of rapid response team calls that did not result in ICU admission also declined (70 [2.8%] vs 45 [2.0%]; P=0.05). The number of rapid response team calls that did result in ICU admission was not significantly different (52 [2.1%] vs 36 [1.6%]; P=0.16). There were no differences in hospital stay or in-patient deaths between the two study periods. CONCLUSIONS: Continuous monitoring of patient vital signs using wearable monitoring technology linked wirelessly to hospital systems was associated with a reduction in unplanned ICU admissions and rapid response team calls. Further research is necessary to confirm the impact of this approach on patient survival.

European consensus on essential steps of Minimally Invasive Ivor Lewis and McKeown Esophagectomy through Delphi methodology.

BACKGROUND: Minimally invasive esophagectomy (MIE) is a complex and technically demanding procedure with a long learning curve, which is associated with increased morbidity and mortality. To master MIE, training in essential steps is crucial. Yet, no consensus on essential steps of MIE is available. The aim of this study was to achieve expert consensus on essential steps in Ivor Lewis and McKeown MIE through Delphi methodology. METHODS: Based on expert opinion and peer-reviewed literature, essential steps were defined for Ivor Lewis (IL) and McKeown (McK) MIE. In a round table discussion, experts finalized the lists of steps and an online Delphi questionnaire was sent to an international expert panel (7 European countries) of minimally invasive upper GI surgeons. Based on replies and comments, steps were adjusted and rephrased and sent in iterative fashion until consensus was achieved. RESULTS: Two Delphi rounds were conducted and response rates were 74% (23 out of 31 experts) for the first and 81% (27 out of 33 experts) for the second round. Consensus was achieved on 106 essential steps for both the IL and McK approach. Cronbach's alpha in the first round was 0.78 (IL) and 0.78 (McK) and in the second round 0.92 (IL) and 0.88 (McK). CONCLUSIONS: Consensus among European experts was achieved on essential surgical steps for both Ivor Lewis and McKeown minimally invasive esophagectomy.

A Video-Based Procedure-Specific Competency Assessment Tool for Minimally Invasive Esophagectomy.

Importance: Minimally invasive esophagectomy (MIE) is a complex procedure with substantial learning curves. In other complex minimally invasive procedures, suboptimal surgical performance has convincingly been associated with less favorable patient outcomes as assessed by peer review of the surgical procedure. Objective: To develop and validate a procedure-specific competency assessment tool (CAT) for MIE. Design, Setting, and Participants: In this international quality improvement study, a procedure-specific MIE-CAT was developed and validated. The MIE-CAT contains 8 procedural phases, and 4 quality components per phase are scored with a Likert scale ranging from 1 to 4. For evaluation of the MIE-CAT, intraoperative MIE videos performed by a single surgical team in the Esophageal Center East Netherlands were peer reviewed by 18 independent international MIE experts (with more than 120 MIEs performed). Each video was assessed by 2 or 3 blinded experts to evaluate feasibility, content validity, reliability, and construct validity. MIE-CAT version 2 was composed with refined content aimed at improving interrater reliability. A total of 32 full-length MIE videos from patients who underwent MIE between 2011 and 2020 were analyzed. Data were analyzed from January 2021 to January 2023. Exposure: Performance assessment of transthoracic MIE with an intrathoracic anastomosis. Main Outcomes and Measures: Feasibility, content validity, interrater and intrarater reliability, and construct validity, including correlations with both experience of the surgical team and clinical parameters, of the developed MIE-CAT. Results: Experts found the MIE-CAT easy to understand and easy to use to grade surgical performance. The MIE-CAT demonstrated good intrarater reliability (range of intraclass correlation coefficients [ICCs], 0.807 [95% CI, 0.656 to 0.892] for quality component score to 0.898 [95% CI, 0.846 to 0.932] for phase score). Interrater reliability was moderate (range of ICCs, 0.536 [95% CI, -0.220 to 0.994] for total MIE-CAT score to 0.705 [95% CI, 0.473 to 0.846] for quality component score), and most discrepancies originated in the lymphadenectomy phases. Hypothesis testing for construct validity showed more than 75% of hypotheses correct: MIE-CAT performance scores correlated with experience of the surgical team (r = 0.288 to 0.622), blood loss (r = -0.034 to -0.545), operative time (r = -0.309 to -0.611), intraoperative complications (r = -0.052 to -0.319), and severe postoperative complications (r = -0.207 to -0.395). MIE-CAT version 2 increased usability. Interrater reliability improved but remained moderate (range of ICCs, 0.666 to 0.743), and most discrepancies between raters remained in the lymphadenectomy phases. Conclusions and Relevance: The MIE-CAT was developed and its feasibility, content validity, reliability, and construct validity were demonstrated. By providing insight into surgical performance of MIE, the MIE-CAT might be used for clinical, training, and research purposes.

Do Simulated Hospital Admissions Reflect Reality? A Qualitative Study of Volunteer Well-Being During a 24-Hr Simulated Hospitalization.

OBJECTIVES: This study aims to delineate if and how healthy volunteers admitted to simulated care can aid in understanding real well-being experiences of in-hospital surgical patients. BACKGROUND: Scientific research is necessary to understand the mediating effect of healthcare design on patient outcomes. Studies with patients are, however, difficult to conduct as they require substantial funding, time, and research capacity, and recovering patients are often not willing or able to participate. If studies conducted with volunteers provide similar findings, such studies might serve as fruitful alternatives for future research. METHOD: A multimethod study was conducted between July 2017 and December 2017 with 17 volunteers who underwent a 24-hr simulated inpatient postsurgical care protocol. Data on value experiences, norms, and design requirements for an optimal healing environment were collected via diaries and semi-structured value-oriented interviews, focused on the values of spatial comfort, privacy, autonomy, sensory comfort, safety and security, and social comfort. Volunteers' outcomes were compared to prior literature on similar patients' outcomes. RESULTS: Volunteers seem to experience their healing environment similarly to patients with regard to the values of spatial comfort, privacy, autonomy, sensory comfort, and social comfort related to contact with personnel and relatives. Less valuable insights were gained on the values of safety and security, and social comfort related to interaction with other patients, most probably due to the study design and because the participants did not truly experience a diseased bodily state. CONCLUSION: Simulated hospital admissions with volunteers provide a satisfactory alternative for studying real patient outcomes.

Algorithms for Prediction of Clinical Deterioration on the General Wards: A Scoping Review.

OBJECTIVE: The primary objective of this scoping review was to identify and describe state-of-the-art models that use vital sign monitoring to predict clinical deterioration on the general ward. The secondary objective was to identify facilitators, barriers, and effects of implementing these models. DATA SOURCES: PubMed, Embase, and CINAHL databases until November 2020. STUDY SELECTION: We selected studies that compared vital signs-based automated real-time predictive algorithms to current track-and-trace protocols in regard to the outcome of clinical deterioration in a general ward population. DATA EXTRACTION: Study characteristics, predictive characteristics and barriers, facilitators, and effects. RESULTS: We identified 1741 publications, 21 of which were included in our review. Two of the these were clinical trials, 2 were prospective observational studies, and the remaining 17 were retrospective studies. All of the studies focused on hospitalized adult patients. The reported area under the receiver operating characteristic curves ranged between 0.65 and 0.95 for the outcome of clinical deterioration. Positive predictive value and sensitivity ranged between 0.223 and 0.773 and from 7.2% to 84.0%, respectively. Input variables differed widely, and predicted endpoints were inconsistently defined. We identified 57 facilitators and 48 barriers to the implementation of these models. We found 68 reported effects, 57 of which were positive. CONCLUSION: Predictive algorithms can detect clinical deterioration on the general ward earlier and more accurately than conventional protocols, which in one recent study led to lower mortality. Consensus is needed on input variables, predictive time horizons, and definitions of endpoints to better facilitate comparative research.

Low compliance to a vital sign safety protocol on general hospital wards: A retrospective cohort study.

BACKGROUND: Periodic vital sign monitoring is commonly used to detect clinical deterioration. Incomplete assessment of Early Warning Score (EWS) and poor protocol compliance may negatively impact nurses' responsiveness to critical situations. OBJECTIVE: This study assesses the quality of recorded EWSs and the degree of overall protocol compliance and protocol compliance in adverse event patients, in a centre that is an early adopter of an EWS safety protocol. DESIGN: Retrospective single-centre cohort study. SETTING: General hospital wards of a tertiary referral university medical centre. PARTICIPANTS: 48,864 patients admitted to general wards between 2015 and 2018. METHODS: Vital sign and Modified EWS (MEWS) data were collected from the electronic health record. MEWS completeness was evaluated by determining the proportion of missing values for each vital sign. To evaluate protocol compliance, we assessed whether the elapsed time between consecutive MEWSs was within the protocollary time as dictated by the MEWS protocol. Outcome measures were overall MEWS completeness, and both overall protocol compliance and protocol compliance specifically in the 24 h before adverse events. RESULTS: All required items to calculate a MEWS were present in 76 percent of the 1,663,743 vital sign measurements. Overall protocol compliance was 62 percent. For the low, mid-range and high MEWS groups, protocol compliances were 67, 47 and 30 percent, respectively. Time-to-next-MEWS exceeded protocol-recommended timing by up to 9 h for a substantial amount of MEWSs in the mid- and high-range MEWS groups. In patients with adverse events, compliance to the MEWS protocol during the 24 h prior to the events was 66 percent in aggregate, ranging from 80% in low MEWS groups, 74% in the mid-range, and 49% in the high MEWS groups. In the high MEWS group, improvements in protocol compliance were primarily noted in five hours immediately preceding the event. CONCLUSIONS: EWS assessments were incomplete in one-quarter of measurements. Compliance to a MEWS safety protocol was generally low, specifically when the protocol mandated more frequent MEWS assessments. Although more MEWSs were recorded in patients with adverse events, the increase in vital sign measurements' frequency mostly occurred shortly before the event manifested. This finding suggests missed opportunities to detect clinical deterioration.