Development and Validation of a Natural Language Processing Algorithm for Extracting Clinical and Pathological Features of Breast Cancer From Pathology Reports.

PURPOSE: Electronic health records (EHRs) are valuable information repositories that offer insights for enhancing clinical research on breast cancer (BC) using real-world data. The objective of this study was to develop a natural language processing (NLP) model specifically designed to extract structured data from BC pathology reports written in natural language. METHODS: During the initial phase, the algorithm's development cohort comprised 193 pathology reports from 116 patients with BC from 2012 to 2016. A rule-based NLP algorithm was applied to extract 26 variables for analysis and was compared with the manual extraction of data performed by both a data entry specialist and an oncologist. Following the first approach, the data set was expanded to include 513 reports, and a Named Entity Recognition (NER)-NLP model was trained and evaluated using K-fold cross-validation. RESULTS: The first approach led to a concordance analysis, which revealed an 82.9% agreement between the algorithm and the oncologist, whereas the concordance between the data entry specialist and the oncologist was 90.8%. The second training approach introduced the definition of an NER-NLP model, in which the accuracy showed remarkable potential (97.8%). Notably, the model demonstrated remarkable performance, especially for parameters such as estrogen receptor, progesterone receptor, human epidermal growth factor receptor 2, and Ki-67 (F1-score 1.0). CONCLUSION: The present study aligns with the rapidly evolving field of artificial intelligence (AI) applications in oncology, seeking to expedite the development of complex cancer databases and registries. The results of the model are currently undergoing postprocessing procedures to organize the data into tabular structures, facilitating their utilization in real-world clinical and research endeavors.

Bayesian Networks in the Management of Hospital Admissions: A Comparison between Explainable AI and Black Box AI during the Pandemic.

Artificial Intelligence (AI) and Machine Learning (ML) approaches that could learn from large data sources have been identified as useful tools to support clinicians in their decisional process; AI and ML implementations have had a rapid acceleration during the recent COVID-19 pandemic. However, many ML classifiers are "black box" to the final user, since their underlying reasoning process is often obscure. Additionally, the performance of such models suffers from poor generalization ability in the presence of dataset shifts. Here, we present a comparison between an explainable-by-design ("white box") model (Bayesian Network (BN)) versus a black box model (Random Forest), both studied with the aim of supporting clinicians of Policlinico San Matteo University Hospital in Pavia (Italy) during the triage of COVID-19 patients. Our aim is to evaluate whether the BN predictive performances are comparable with those of a widely used but less explainable ML model such as Random Forest and to test the generalization ability of the ML models across different waves of the pandemic.

Performance of an AI algorithm during the different phases of the COVID pandemics: what can we learn from the AI and vice versa.

Background: Artificial intelligence (AI) has proved to be of great value in diagnosing and managing Sars-Cov-2 infection. ALFABETO (ALL-FAster-BEtter-TOgether) is a tool created to support healthcare professionals in the triage, mainly in optimizing hospital admissions. Methods: The AI was trained during the pandemic's "first wave" (February-April 2020). Our aim was to assess the performance during the "third wave" of the pandemics (February-April 2021) and evaluate its evolution. The neural network proposed behavior (hospitalization vs home care) was compared with what was actually done. If there were discrepancies between ALFABETO's predictions and clinicians' decisions, the disease's progression was monitored. Clinical course was defined as "favorable/mild" if patients could be managed at home or in spoke centers and "unfavorable/severe" if patients need to be managed in a hub center. Results: ALFABETO showed accuracy of 76%, AUROC of 83%; specificity was 78% and recall 74%. ALFABETO also showed high precision (88%). 81 hospitalized patients were incorrectly predicted to be in "home care" class. Among those "home-cared" by the AI and "hospitalized" by the clinicians, 3 out of 4 misclassified patients (76.5%) showed a favorable/mild clinical course. ALFABETO's performance matched the reports in literature. Conclusions: The discrepancies mostly occurred when the AI predicted patients could stay at home but clinicians hospitalized them; these cases could be handled in spoke centers rather than hubs, and the discrepancies may aid clinicians in patient selection. The interaction between AI and human experience has the potential to improve both AI performance and our comprehension of pandemic management.