Transportability of bacterial infection prediction models for critically ill patients.

OBJECTIVE: Bacterial infections (BIs) are common, costly, and potentially life-threatening in critically ill patients. Patients with suspected BIs may require empiric multidrug antibiotic regimens and therefore potentially be exposed to prolonged and unnecessary antibiotics. We previously developed a BI risk model to augment practices and help shorten the duration of unnecessary antibiotics to improve patient outcomes. Here, we have performed a transportability assessment of this BI risk model in 2 tertiary intensive care unit (ICU) settings and a community ICU setting. We additionally explored how simple multisite learning techniques impacted model transportability. METHODS: Patients suspected of having a community-acquired BI were identified in 3 datasets: Medical Information Mart for Intensive Care III (MIMIC), Northwestern Medicine Tertiary (NM-T) ICUs, and NM "community-based" ICUs. ICU encounters from MIMIC and NM-T datasets were split into 70/30 train and test sets. Models developed on training data were evaluated against the NM-T and MIMIC test sets, as well as NM community validation data. RESULTS: During internal validations, models achieved AUROCs of 0.78 (MIMIC) and 0.81 (NM-T) and were well calibrated. In the external community ICU validation, the NM-T model had robust transportability (AUROC 0.81) while the MIMIC model transported less favorably (AUROC 0.74), likely due to case-mix differences. Multisite learning provided no significant discrimination benefit in internal validation studies but offered more stability during transport across all evaluation datasets. DISCUSSION: These results suggest that our BI risk models maintain predictive utility when transported to external cohorts. CONCLUSION: Our findings highlight the importance of performing external model validation on myriad clinically relevant populations prior to implementation.

Evaluation of the portability of computable phenotypes with natural language processing in the eMERGE network.

The electronic Medical Records and Genomics (eMERGE) Network assessed the feasibility of deploying portable phenotype rule-based algorithms with natural language processing (NLP) components added to improve performance of existing algorithms using electronic health records (EHRs). Based on scientific merit and predicted difficulty, eMERGE selected six existing phenotypes to enhance with NLP. We assessed performance, portability, and ease of use. We summarized lessons learned by: (1) challenges; (2) best practices to address challenges based on existing evidence and/or eMERGE experience; and (3) opportunities for future research. Adding NLP resulted in improved, or the same, precision and/or recall for all but one algorithm. Portability, phenotyping workflow/process, and technology were major themes. With NLP, development and validation took longer. Besides portability of NLP technology and algorithm replicability, factors to ensure success include privacy protection, technical infrastructure setup, intellectual property agreement, and efficient communication. Workflow improvements can improve communication and reduce implementation time. NLP performance varied mainly due to clinical document heterogeneity; therefore, we suggest using semi-structured notes, comprehensive documentation, and customization options. NLP portability is possible with improved phenotype algorithm performance, but careful planning and architecture of the algorithms is essential to support local customizations.

Comprehensive molecular characterization of a rare case of Philadelphia chromosome-positive acute myeloid leukemia.

The Philadelphia chromosome (Ph) resulting from the t(9;22) translocation generates the oncogenic BCR::ABL1 fusion protein that is most commonly associated with chronic myeloid leukemia (CML) and Ph-positive (Ph+) acute lymphoblastic leukemia (ALL). There are also rare instances of patients (≤1%) with newly diagnosed acute myeloid leukemia (AML) that harbor this translocation (Paietta et al., Leukemia 12: 1881 [1998]; Keung et al., Leuk Res 28: 579 [2004]; Soupir et al., Am J Clin Pathol 127: 642 [2007]). AML with BCR::ABL has only recently been provisionally classified by the World Health Organization as a diagnostically distinct subtype of AML. Discernment from the extremely close differential diagnosis of myeloid blast crisis CML is challenging, largely relying on medical history rather than clinical characteristics (Arber et al., Blood 127: 2391 [2016]). To gain insight into the genomic features underlying the evolution of AML with BCR::ABL, we identified a patient presenting with a high-risk myelodysplastic syndrome that acquired a BCR::ABL alteration after a peripheral blood stem cell transplant. Serial samples were collected and analyzed using whole-exome sequencing, RNA-seq, and ex vivo functional drug screens. Persistent subclones were identified, both at diagnosis and at relapse, including an SF3B1p.Lys700Glu mutation that later cooccurred with an NRASp.Gly12Cys mutation. Functional ex vivo drug screening performed on primary patient cells suggested that combination therapies of ABL1 with RAS or PI3K pathway inhibitors could have augmented the patient's response throughout the course of disease. Together, our findings argue for the importance of genomic profiling and the potential value of ABL1 inhibitor-inclusive combination treatment strategies in patients with this rare disease.

Development and validation of MicrobEx: an open-source package for microbiology culture concept extraction.

Objective: Microbiology culture reports contain critical information for important clinical and public health applications. However, microbiology reports often have complex, semistructured, free-text data that present a barrier for secondary use. Here we present the development and validation of an open-source package designed to ingest free-text microbiology reports, determine whether the culture is positive, and return a list of Systemized Nomenclature of Medicine (SNOMED)-CT mapped bacteria. Materials and Methods: Our concept extraction Python package, MicrobEx, is built upon a rule-based natural language processing algorithm and was developed using microbiology reports from 2 different electronic health record systems in a large healthcare organization, and then externally validated on the reports of 2 other institutions with manually reviewed results as a benchmark. Results: MicrobEx achieved F1 scores >0.95 on all classification tasks across 2 independent validation sets with minimal customization. Additionally, MicrobEx matched or surpassed our MetaMap-based benchmark algorithm performance across positive culture classification and species capture classification tasks. Discussion: Our results suggest that MicrobEx can be used to reliably estimate binary bacterial culture status, extract bacterial species, and map these to SNOMED organism observations when applied to semistructured, free-text microbiology reports from different institutions with relatively low customization. Conclusion: MicrobEx offers an open-source software solution (available on both GitHub and PyPI) for bacterial culture status estimation and bacterial species extraction from free-text microbiology reports. The package was designed to be reused and adapted to individual institutions as an upstream process for other clinical applications such as: machine learning, clinical decision support, and disease surveillance systems.

Predictive modeling of bacterial infections and antibiotic therapy needs in critically ill adults.

Unnecessary antibiotic regimens in the intensive care unit (ICU) are associated with adverse patient outcomes and antimicrobial resistance. Bacterial infections (BI) are both common and deadly in ICUs, and as a result, patients with a suspected BI are routinely started on broad-spectrum antibiotics prior to having confirmatory microbiologic culture results or when an occult BI is suspected, a practice known as empiric antibiotic therapy (EAT). However, EAT guidelines lack consensus and existing methods to quantify patient-level BI risk rely largely on clinical judgement and inaccurate biomarkers or expensive diagnostic tests. As a consequence, patients with low risk of BI often are continued on EAT, exposing them to unnecessary side effects. Augmenting current intuition-based practices with data-driven predictions of BI risk could help inform clinical decisions to shorten the duration of unnecessary EAT and improve patient outcomes. We propose a novel framework to identify ICU patients with low risk of BI as candidates for earlier EAT discontinuation. For this study, patients suspected of having a community-acquired BI were identified in the Medical Information Mart for Intensive Care III (MIMIC-III) dataset and categorized based on microbiologic culture results and EAT duration. Using structured longitudinal data collected up to 24-, 48-, and 72-hours after starting EAT, our best models identified patients at low risk of BI with AUROCs up to 0.8 and negative predictive values >93%. Overall, these results demonstrate the feasibility of forecasting BI risk in a critical care setting using patient features found in the electronic health record and call for more extensive research in this promising, yet relatively understudied, area.

Next-generation sequencing-defined minimal residual disease before stem cell transplantation predicts acute myeloid leukemia relapse.

In acute myeloid leukemia (AML), the assessment of post-treatment minimal residual disease (MRD) may inform a more effective management approach. We investigated the prognostic utility of next-generation sequencing (NGS)-based MRD detection undertaken before hematopoietic stem cell transplantation (HSCT). Forty-two AML subjects underwent serial disease monitoring both by standard methods, and a targeted 42-gene NGS assay, able to detect leukemia-specific mutant alleles (with >0.5% VAF) (mean 5.1 samples per subject). The prognostic relevance of any persisting diagnostic mutation before transplant (≤27 days) was assessed during 22.1 months (median) of post-transplant follow-up. The sensitivity of the NGS assay (27 MRD-positive subjects) exceeded that of the non-molecular methods (morphology, FISH, and flow cytometry) (11 positive subjects). Only one of the 13 subjects who relapsed after HSCT was NGS MRD-negative (92% assay sensitivity). The cumulative incidence of post-transplant leukemic relapse was significantly higher in the pre-transplant NGS MRD-positive (vs MRD-negative) subjects (P = .014). After adjusting for TP53 mutation and transplant conditioning regimen, NGS MRD-positivity retained independent prognostic significance for leukemic relapse (subdistribution hazard ratio = 7.3; P = .05). The pre-transplant NGS MRD-positive subjects also had significantly shortened progression-free survival (P = .038), and marginally shortened overall survival (P = .068). In patients with AML undergoing HSCT, the pre-transplant persistence of NGS-defined MRD imparts a significant, sensitive, strong, and independent increased risk for subsequent leukemic relapse and death. Given that NGS can simultaneously detect multiple leukemia-associated mutations, it can be used in the majority of AML patients to monitor disease burdens and inform treatment decisions.

CRISPR-Cas9-mediated saturated mutagenesis screen predicts clinical drug resistance with improved accuracy.

Developing tools to accurately predict the clinical prevalence of drug-resistant mutations is a key step toward generating more effective therapeutics. Here we describe a high-throughput CRISPR-Cas9-based saturated mutagenesis approach to generate comprehensive libraries of point mutations at a defined genomic location and systematically study their effect on cell growth. As proof of concept, we mutagenized a selected region within the leukemic oncogene BCR-ABL1 Using bulk competitions with a deep-sequencing readout, we analyzed hundreds of mutations under multiple drug conditions and found that the effects of mutations on growth in the presence or absence of drug were critical for predicting clinically relevant resistant mutations, many of which were cancer adaptive in the absence of drug pressure. Using this approach, we identified all clinically isolated BCR-ABL1 mutations and achieved a prediction score that correlated highly with their clinical prevalence. The strategy described here can be broadly applied to a variety of oncogenes to predict patient mutations and evaluate resistance susceptibility in the development of new therapeutics.

Highly accurate molecular genetic testing for HFE hereditary hemochromatosis: results from 10 years of blinded proficiency surveys by the College of American Pathologists.

PURPOSE: The College of American Pathologists offers blinded proficiency testing (PT) for laboratories performing HFE genetic tests for hereditary hemochromatosis (common C282Y and H63D variants). This study used 10 years of PT data to determine laboratory performance for HFE analytical genotyping and clinical interpretation. METHODS: Laboratories were graded for accuracy of genotype determination (six possible C282Y/H63D genotypes) and clinical interpretation regarding whether the genotype was likely to have contributed to iron overload in a hypothetical patient. RESULTS: The analytical genotyping error rate was low (0.73%) in 7,663 results (from 257 unique laboratories). Genotyping errors were significantly higher in C282Y heterozygous, H63D homozygous, and C282Y homozygous samples, in non-American laboratories, and in laboratories with lower testing volume. Analytical sensitivity and specificity were >98.5 and >99.5%. The interpretive error rate (4.3%) was higher than the genotyping error rate, with two problematic genotypes (C282Y heterozygous and H63D homozygous) accounting for 77% of total interpretive errors. There was a time-dependent improvement in the interpretation of the clinical significance of HFE genotypes. CONCLUSIONS: HFE molecular genetic testing, performed by non-US Food and Drug Administration-approved laboratory-developed tests, demonstrated excellent accuracy, sensitivity, and specificity. Clinical interpretations were more heterogeneous, probably owing to the low clinical penetrance of some common HFE genotypes.Genet Med 18 12, 1206-1213.

Environmental regulation of plant gene expression: an RT-qPCR laboratory project for an upper-level undergraduate biochemistry or molecular biology course.

We present a novel laboratory project employing "real-time" RT-qPCR to measure the effect of environment on the expression of the FLOWERING LOCUS C gene, a key regulator of floral timing in Arabidopsis thaliana plants. The project requires four 3-hr laboratory sessions and is aimed at upper-level undergraduate students in biochemistry or molecular biology courses. The project provides students with hands-on experience with RT-qPCR, the current "gold standard" for gene expression analysis, including detailed data analysis using the common 2-ΔΔCT method. Moreover, it provides a convenient starting point for many inquiry-driven projects addressing diverse questions concerning ecological biochemistry, naturally occurring genetic variation, developmental biology, and the regulation of gene expression in nature.