The augmented value of using clinical notes in semi-automated surveillance of deep surgical site infections after colorectal surgery.

BACKGROUND: In patients who underwent colorectal surgery, an existing semi-automated surveillance algorithm based on structured data achieves high sensitivity in detecting deep surgical site infections (SSI), however, generates a significant number of false positives. The inclusion of unstructured, clinical narratives to the algorithm may decrease the number of patients requiring manual chart review. The aim of this study was to investigate the performance of this semi-automated surveillance algorithm augmented with a natural language processing (NLP) component to improve positive predictive value (PPV) and thus workload reduction (WR). METHODS: Retrospective, observational cohort study in patients who underwent colorectal surgery from January 1, 2015, through September 30, 2020. NLP was used to detect keyword counts in clinical notes. Several NLP-algorithms were developed with different count input types and classifiers, and added as component to the original semi-automated algorithm. Traditional manual surveillance was compared with the NLP-augmented surveillance algorithms and sensitivity, specificity, PPV and WR were calculated. RESULTS: From the NLP-augmented models, the decision tree models with discretized counts or binary counts had the best performance (sensitivity 95.1% (95%CI 83.5-99.4%), WR 60.9%) and improved PPV and WR by only 2.6% and 3.6%, respectively, compared to the original algorithm. CONCLUSIONS: The addition of an NLP component to the existing algorithm had modest effect on WR (decrease of 1.4-12.5%), at the cost of sensitivity. For future implementation it will be a trade-off between optimal case-finding techniques versus practical considerations such as acceptability and availability of resources.

External validation of semi-automated surveillance algorithms for deep surgical site infections after colorectal surgery in an independent country.

BACKGROUND: Automated surveillance methods that re-use electronic health record data are considered an attractive alternative to traditional manual surveillance. However, surveillance algorithms need to be thoroughly validated before being implemented in a clinical setting. With semi-automated surveillance patients are classified as low or high probability of having developed infection, and only high probability patients subsequently undergo manual record review. The aim of this study was to externally validate two existing semi-automated surveillance algorithms for deep SSI after colorectal surgery, developed on Spanish and Dutch data, in a Swedish setting. METHODS: The algorithms were validated in 225 randomly selected surgeries from Karolinska University Hospital from the period January 1, 2015 until August 31, 2020. Both algorithms were based on (re)admission and discharge data, mortality, reoperations, radiology orders, and antibiotic prescriptions, while one additionally used microbiology cultures. SSI was based on ECDC definitions. Sensitivity, specificity, positive predictive value, negative predictive value, and workload reduction were assessed compared to manual surveillance. RESULTS: Both algorithms performed well, yet the algorithm not relying on microbiological culture data had highest sensitivity (97.6, 95%CI: 87.4-99.6), which was comparable to previously published results. The latter algorithm aligned best with clinical practice and would lead to 57% records less to review. CONCLUSIONS: The results highlight the importance of thorough validation before implementation in other clinical settings than in which algorithms were originally developed: the algorithm excluding microbiology cultures had highest sensitivity in this new setting and has the potential to support large-scale semi-automated surveillance of SSI after colorectal surgery.

Semiautomated surveillance of deep surgical site infections after colorectal surgeries: A multicenter external validation of two surveillance algorithms.

OBJECTIVE: Automated surveillance methods increasingly replace or support conventional (manual) surveillance; the latter is labor intensive and vulnerable to subjective interpretation. We sought to validate 2 previously developed semiautomated surveillance algorithms to identify deep surgical site infections (SSIs) in patients undergoing colorectal surgeries in Dutch hospitals. DESIGN: Multicenter retrospective cohort study. METHODS: From 4 hospitals, we selected colorectal surgery patients between 2018 and 2019 based on procedure codes, and we extracted routine care data from electronic health records. Per hospital, a classification model and a regression model were applied independently to classify patients into low- or high probability of having developed deep SSI. High-probability patients need manual SSI confirmation; low-probability records are classified as no deep SSI. Sensitivity, positive predictive value (PPV), and workload reduction were calculated compared to conventional surveillance. RESULTS: In total, 672 colorectal surgery patients were included, of whom 28 (4.1%) developed deep SSI. Both surveillance models achieved good performance. After adaptation to clinical practice, the classification model had 100% sensitivity and PPV ranged from 11.1% to 45.8% between hospitals. The regression model had 100% sensitivity and 9.0%-14.9% PPV. With both models, <25% of records needed review to confirm SSI. The regression model requires more complex data management skills, partly due to incomplete data. CONCLUSIONS: In this independent external validation, both surveillance models performed well. The classification model is preferred above the regression model because of source-data availability and less complex data-management requirements. The next step is implementation in infection prevention practices and workflow processes.

Reliability and validity of multicentre surveillance of surgical site infections after colorectal surgery.

BACKGROUND: Surveillance is the cornerstone of surgical site infection prevention programs. The validity of the data collection and awareness of vulnerability to inter-rater variation is crucial for correct interpretation and use of surveillance data. The aim of this study was to investigate the reliability and validity of surgical site infection (SSI) surveillance after colorectal surgery in the Netherlands. METHODS: In this multicentre prospective observational study, seven Dutch hospitals performed SSI surveillance after colorectal surgeries performed in 2018 and/or 2019. When executing the surveillance, a local case assessment was performed to calculate the overall percentage agreement between raters within hospitals. Additionally, two case-vignette assessments were performed to estimate intra-rater and inter-rater reliability by calculating a weighted Cohen's Kappa and Fleiss' Kappa coefficient. To estimate the validity, answers of the two case-vignettes questionnaires were compared with the answers of an external medical panel. RESULTS: 1111 colorectal surgeries were included in this study with an overall SSI incidence of 8.8% (n = 98). From the local case assessment it was estimated that the overall percent agreement between raters within a hospital was good (mean 95%, range 90-100%). The Cohen's Kappa estimated for the intra-rater reliability of case-vignette review varied from 0.73 to 1.00, indicating substantial to perfect agreement. The inter-rater reliability within hospitals showed more variation, with Kappa estimates ranging between 0.61 and 0.94. In total, 87.9% of the answers given by the raters were in accordance with the medical panel. CONCLUSIONS: This study showed that raters were consistent in their SSI-ascertainment (good reliability), but improvements can be made regarding the accuracy (moderate validity). Accuracy of surveillance may be improved by providing regular training, adapting definitions to reduce subjectivity, and by supporting surveillance through automation.

A framework to develop semiautomated surveillance of surgical site infections: An international multicenter study.

OBJECTIVE: Automated surveillance of healthcare-associated infections reduces workload and improves standardization, but it has not yet been adopted widely. In this study, we assessed the performance and feasibility of an easy implementable framework to develop algorithms for semiautomated surveillance of deep incisional and organ-space surgical site infections (SSIs) after orthopedic, cardiac, and colon surgeries. DESIGN: Retrospective cohort study in multiple countries. METHODS: European hospitals were recruited and selected based on the availability of manual SSI surveillance data from 2012 onward (reference standard) and on the ability to extract relevant data from electronic health records. A questionnaire on local manual surveillance and clinical practices was administered to participating hospitals, and the information collected was used to pre-emptively design semiautomated surveillance algorithms standardized for multiple hospitals and for center-specific application. Algorithm sensitivity, positive predictive value, and reduction of manual charts requiring review were calculated. Reasons for misclassification were explored using discrepancy analyses. RESULTS: The study included 3 hospitals, in the Netherlands, France, and Spain. Classification algorithms were developed to indicate procedures with a high probability of SSI. Components concerned microbiology, prolonged length of stay or readmission, and reinterventions. Antibiotics and radiology ordering were optional. In total, 4,770 orthopedic procedures, 5,047 cardiac procedures, and 3,906 colon procedures were analyzed. Across hospitals, standardized algorithm sensitivity ranged between 82% and 100% for orthopedic surgery, between 67% and 100% for cardiac surgery, and between 84% and 100% for colon surgery, with 72%-98% workload reduction. Center-specific algorithms had lower sensitivity. CONCLUSIONS: Using this framework, algorithms for semiautomated surveillance of SSI can be successfully developed. The high performance of standardized algorithms holds promise for large-scale standardization.

A diagnostic algorithm for the surveillance of deep surgical site infections after colorectal surgery.

OBJECTIVE: Surveillance of surgical site infections (SSIs) is important for infection control and is usually performed through retrospective manual chart review. The aim of this study was to develop an algorithm for the surveillance of deep SSIs based on clinical variables to enhance efficiency of surveillance. DESIGN: Retrospective cohort study (2012-2015). SETTING: A Dutch teaching hospital. PARTICIPANTS: We included all consecutive patients who underwent colorectal surgery excluding those with contaminated wounds at the time of surgery. All patients were evaluated for deep SSIs through manual chart review, using the Centers for Disease Control and Prevention (CDC) criteria as the reference standard. ANALYSIS: We used logistic regression modeling to identify predictors that contributed to the estimation of diagnostic probability. Bootstrapping was applied to increase generalizability, followed by assessment of statistical performance and clinical implications. RESULTS: In total, 1,606 patients were included, of whom 129 (8.0%) acquired a deep SSI. The final model included postoperative length of stay, wound class, readmission, reoperation, and 30-day mortality. The model achieved 68.7% specificity and 98.5% sensitivity and an area under the receiver operator characteristic (ROC) curve (AUC) of 0.950 (95% CI, 0.932-0.969). Positive and negative predictive values were 21.5% and 99.8%, respectively. Applying the algorithm resulted in a 63.4% reduction in the number of records requiring full manual review (from 1,606 to 590). CONCLUSIONS: This 5-parameter model identified 98.5% of patients with a deep SSI. The model can be used to develop semiautomatic surveillance of deep SSIs after colorectal surgery, which may further improve efficiency and quality of SSI surveillance.

Preoperative Oral Antibiotic Prophylaxis Reduces Surgical Site Infections After Elective Colorectal Surgery: Results From a Before-After Study.

BACKGROUND: Surgical site infections (SSIs) are common complications after colorectal procedures and remain an important source of morbidity and costs. Preoperative oral antibiotic prophylaxis is a potential infection control strategy, but its effectiveness without simultaneous use of mechanical bowel preparation (MBP) is unclear. In this study, we aimed to determine whether preoperative oral antibiotics reduce the risk of deep SSIs in elective colorectal surgery. METHODS: We performed a before-after analysis in a teaching hospital in the Netherlands. Patients who underwent surgery between January 2012 and December 2015 were included. On 1 January 2013, oral antibiotic prophylaxis with tobramycin and colistin was implemented as standard of care prior to colorectal surgery. The year before implementation was used as the control period. The primary outcome was a composite of deep SSI and/or mortality within 30 days after surgery. RESULTS: Of the 1410 patients, 352 underwent colorectal surgery in the control period and 1058 in the period after implementation of the antibiotic prophylaxis. We observed a decrease in incidence of the primary endpoint of 6.2% after prophylaxis implementation. When adjusted for confounders, the risk ratio for development of the primary outcome was 0.58 (95% confidence interval, 0.40-0.79). Other findings included a decreased risk of anastomotic leakage and a reduction in the length of postoperative stay. CONCLUSIONS: Preoperative oral antibiotic prophylaxis prior to colorectal surgery is associated with a significant decrease in SSI and/or mortality in a setting without MBP. Preoperative oral antibiotics can therefore be considered without MBP for patients who undergo colorectal surgery.

Embryonic implantation: cytokines, adhesion molecules, and immune cells in establishing an implantation environment.

Successful implantation is an absolute requirement for the reproduction of species, including humans. The process by which a foreign blastocyst is accepted by the maternal endometrium is complex and requires interplay of many systems. Implantation occurs during the putative implantation window, in which the maternal endometrium is ready to accept the blastocyst, which on the other hand, also plays a specific role. It produces cytokines and chemokines and expresses adhesion molecules and certain classes of MHC molecules. We review the most important players in implantation. Concerning the cytokines, the establishment of controlled aggression is key; an excess of pro- or anti-inflammation is detrimental to pregnancy outcome. Chemokines control the orientation of the embryo. The adhesion molecules are necessary to establish the required physical interaction between mother and blastocyst. Finally, immune cells and in particular, uterine NK and regulatory T cells are pivotal in inducing tolerance to the blastocyst. The aim of this review is to discuss mechanisms at play and their relative importance to the establishment of pregnancy.