Characterizing the limitations of using diagnosis codes in the context of machine learning for healthcare.

BACKGROUND: Diagnostic codes are commonly used as inputs for clinical prediction models, to create labels for prediction tasks, and to identify cohorts for multicenter network studies. However, the coverage rates of diagnostic codes and their variability across institutions are underexplored. The primary objective was to describe lab- and diagnosis-based labels for 7 selected outcomes at three institutions. Secondary objectives were to describe agreement, sensitivity, and specificity of diagnosis-based labels against lab-based labels. METHODS: This study included three cohorts: SickKids from The Hospital for Sick Children, and StanfordPeds and StanfordAdults from Stanford Medicine. We included seven clinical outcomes with lab-based definitions: acute kidney injury, hyperkalemia, hypoglycemia, hyponatremia, anemia, neutropenia and thrombocytopenia. For each outcome, we created four lab-based labels (abnormal, mild, moderate and severe) based on test result and one diagnosis-based label. Proportion of admissions with a positive label were presented for each outcome stratified by cohort. Using lab-based labels as the gold standard, agreement using Cohen's Kappa, sensitivity and specificity were calculated for each lab-based severity level. RESULTS: The number of admissions included were: SickKids (n = 59,298), StanfordPeds (n = 24,639) and StanfordAdults (n = 159,985). The proportion of admissions with a positive diagnosis-based label was significantly higher for StanfordPeds compared to SickKids across all outcomes, with odds ratio (99.9% confidence interval) for abnormal diagnosis-based label ranging from 2.2 (1.7-2.7) for neutropenia to 18.4 (10.1-33.4) for hyperkalemia. Lab-based labels were more similar by institution. When using lab-based labels as the gold standard, Cohen's Kappa and sensitivity were lower at SickKids for all severity levels compared to StanfordPeds. CONCLUSIONS: Across multiple outcomes, diagnosis codes were consistently different between the two pediatric institutions. This difference was not explained by differences in test results. These results may have implications for machine learning model development and deployment.

Opportunities and challenges to enhance the value and uptake of Chief Nursing Informatics Officer (CNIO) Roles in Canada: A Qualitative Study.

Clinician informatics leadership has been identified as an essential component of addressing the 'implementation to benefits realization gap' that exists for many digital health technologies. Chief Medical Informatics Officers (CMIOs), and Chief Nursing Informatics Officers (CNIOs) are well-positioned to ensure the success of these initiatives. However, while the CMIO role is fairly well-established in Canada, there is limited uptake of CNIO roles in the country. The main objective of this work is to build on the current progress of the CMIO role and explore how the CNIO role can be best positioned for uptake and value across healthcare organizations in Canada. A qualitative study was conducted. Ten clinician leaders in CMIO, CNIO, and related roles in Canada were interviewed about the value of these roles and strategies for supporting the uptake of the role. This study provides the foundation for future initiatives for supporting and showcasing the value of the CNIO in a digitally enabled healthcare organization.