Training and experience of coding with the World Health Organization's International Classification of Diseases, Eleventh Revision.

BACKGROUND: The new International Classification of Diseases, Eleventh Revision for Mortality and Morbidity Statistics (ICD-11) was developed and released by the World Health Organization (WHO) in June 2018. Because ICD-11 incorporates new codes and features, training materials for coding with ICD-11 are urgently needed prior to its implementation. OBJECTIVE: This study outlines the development of ICD-11 training materials, training processes and experiences of clinical coders while learning to code using ICD-11. METHOD: Six certified clinical coders were recruited to code inpatient charts using ICD-11. Training materials were developed with input from experts from the Canadian Institute for Health Information and the WHO, and the clinical coders were trained to use the new classification. Monthly team meetings were conducted to enable discussions on coding issues and to select the correct ICD-11 codes. The training experience was evaluated using qualitative interviews, a questionnaire and a coding quiz. RESULTS: total of 3011 charts were coded using ICD-11. In general, clinical coders provided positive feedback regarding the training program. The average score for the coding quiz (multiple choice, True/False) was 84%, suggesting that the training program was effective. Feedback from the coders enabled the ICD-11 code content, electronic tooling and terminologies to be updated. CONCLUSION: This study provides a detailed account of the processes involved with training clinical coders to use ICD-11. Important findings from the interviews were reported at the annual WHO conferences, and these findings helped improve the ICD-11 browser and reference guide.

Performance of machine learning algorithms for surgical site infection case detection and prediction: A systematic review and meta-analysis.

Background: Medical researchers and clinicians have shown much interest in developing machine learning (ML) algorithms to detect/predict surgical site infections (SSIs). However, little is known about the overall performance of ML algorithms in predicting SSIs and how to improve the algorithm's robustness. We conducted a systematic review and meta-analysis to summarize the performance of ML algorithms in SSIs case detection and prediction and to describe the impact of using unstructured and textual data in the development of ML algorithms. Methods: MEDLINE, EMBASE, CINAHL, CENTRAL and Web of Science were searched from inception to March 25, 2021. Study characteristics and algorithm development information were extracted. Performance statistics (e.g., sensitivity, area under the receiver operating characteristic curve [AUC]) were pooled using a random effect model. Stratified analysis was applied to different study characteristic levels. Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Diagnostic Test Accuracy Studies (PRISMA-DTA) was followed. Results: Of 945 articles identified, 108 algorithms from 32 articles were included in this review. The overall pooled estimate of the SSI incidence rate was 3.67%, 95% CI: 3.58-3.76. Mixed-use of structured and textual data-based algorithms (pooled estimates of sensitivity 0.83, 95% CI: 0.78-0.87, specificity 0.92, 95% CI: 0.86-0.95, AUC 0.92, 95% CI: 0.89-0.94) outperformed algorithms solely based on structured data (sensitivity 0.56, 95% CI:0.43-0.69, specificity 0.95, 95% CI:0.91-0.97, AUC = 0.90, 95% CI: 0.87-0.92). Conclusions: ML algorithms developed with structured and textual data provided optimal performance. External validation of ML algorithms is needed to translate current knowledge into clinical practice.

Field testing a new ICD coding system: methods and early experiences with ICD-11 Beta Version 2018.

OBJECTIVE: A beta version (2018) of International Classification of Diseases, 11th Revision for MMS (ICD-11), needed testing. Field-testing involves real-world application of the new codes to examine usability. We describe creating a dataset and characterizing the usability of ICD-11 code set by coders. We compare ICD-11 against ICD-10-CA (Canadian modification) and a reference standard dataset of diagnoses. Real-world usability encompasses code selection and time to code a complete inpatient chart using ICD-11 compared with ICD-10-CA. METHODS AND RESULTS: A random sample of inpatient records previously coded using ICD-10-CA was selected from hospitals in Calgary, Alberta (N = 2896). Nurses examined these charts for conditions and healthcare-related harms. Clinical coders re-coded the same charts using ICD-11 codes. Inter-rater reliability (IRR) and coding time improved with ICD-11 coding experience (23.6 to 9.9 min average per chart). Code structure comparisons and challenges encountered are described. Overall, 86.3% of main condition codes matched. Coder comments regarding duplicate codes, missing codes, code finding issues enabled improvements to the ICD-11 Browser, Coding Tool, and Reference Guide. Training is essential for solid IRR with 17,000 diagnostic categories in the new ICD-11. As countries transition to ICD-11, our coding experiences and methods can inform users for implementation or field testing.

Decision algorithm for when to use the ICD-11 3-part model for healthcare harms.

Accurate data collection of healthcare-related adverse events provides a foundation for quality and health system improvement. The International Classification of Diseases for Mortality and Morbidity Statistics, 11th revision (ICD-11 MMS) includes new codes to identify harm or injury and the events or actions leading to the adverse events. However, it is difficult to choose the correct codes without in-depth understanding of which event may be classified as an injury or harm. A 3-part model will be available in the ICD-11 MMS to cluster the codes for the harm or injury that occurred, the causal factors, and the mode (mechanism) involved. While field testing coding of adverse events, our team developed a decision tree (algorithm), which guides when to use the 3-part model. The decision tree now resides in the ICD-11 Reference Guide. This paper is part of a special ICD-11 paper series and outlines the steps used in the decision-tree (algorithm) and provides examples to help understand the process.While it may take coders some time to gain experience to use the 3-part model and decision-tree, the ICD-11 Reference Guide and this paper can be helpful resources to help clarify the process.

Validated algorithms for identifying timing of second event of oropharyngeal squamous cell carcinoma using real-world data.

BACKGROUND: Understanding occurrence and timing of second events (recurrence and second primary cancer) is essential for cancer specific survival analysis. However, this information is not readily available in administrative data. METHODS: Alberta Cancer Registry, physician claims, and other administrative data were used. Timing of second event was estimated based on our developed algorithm. For validation, the difference, in days between the algorithm estimated and the chart-reviewed timing of second event. Further, the result of Cox-regression modeling cancer-free survival was compared to chart review data. RESULTS: Majority (74.3%) of the patients had a difference between the chart-reviewed and algorithm-estimated timing of second event falling within the 0-60 days window. Kaplan-Meier curves generated from the estimated data and chart review data were comparable with a 5-year second-event-free survival rate of 75.4% versus 72.5%. CONCLUSION: The algorithm provided an estimated timing of second event similar to that of the chart review.

Supervised consumption site enables cost savings by avoiding emergency services: a cost analysis study.

BACKGROUND AND AIMS: We report on a cost analysis study, using population level data to determine the emergency service costs avoided from emergency overdose management at supervised consumption services (SCS). DESIGN: We completed a cost analysis from a payer's perspective. In this setting, there is a single-payer model of service delivery. SETTING: In Calgary, Canada, 'Safeworks Harm Reduction Program' was established in late 2017 and offers 24/7 access to SCS. The facility is a nurse-led service, available for client drop-in. We conducted a cost analysis for the entire duration of the program from November 2017 to January 2020, a period of 2 years and 3 months. METHODS: We assessed costs using the following factors from government health databases: monthly operational costs of providing services for drug consumption, cost of providing ambulance pre-hospital care for clients with overdoses who could not be revived at the facility, cost of initial treatment in an emergency department, and benefit of costs averted from overdoses that were successfully managed at the SCS. RESULTS: The proportion of clients who have overdosed at the SCS has decreased steadily for the duration of the program. The number of overdoses that can be managed on site at the SCS has trended upward, currently 98%. Each overdose that is managed at the SCS produces approximately $1600 CAD in cost savings, with a savings of over $2.3 million for the lifetime of the program. CONCLUSION: Overdose management at an SCS creates cost savings by offsetting costs required for managing overdoses using emergency department and pre-hospital ambulance services.