Transfer learning may explain pigeons' ability to detect cancer in histopathology.

Pigeons' unexpected competence in learning to categorize unseen histopathological images has remained an unexplained discovery for almost a decade (Levensonet al2015PLoS One10e0141357). Could it be that knowledge transferred from their bird's-eye views of the earth's surface gleaned during flight contributes to this ability? Employing a simulation-based verification strategy, we recapitulate this biological phenomenon with a machine-learning analog. We model pigeons' visual experience during flight with the self-supervised pre-training of a deep neural network on BirdsEyeViewNet; our large-scale aerial imagery dataset. As an analog of the differential food reinforcement performed in Levensonet al's study 2015PLoS One10e0141357), we apply transfer learning from this pre-trained model to the same Hematoxylin and Eosin (H&E) histopathology and radiology images and tasks that the pigeons were trained and tested on. The study demonstrates that pre-training neural networks with bird's-eye view data results in close agreement with pigeons' performance. These results support transfer learning as a reasonable computational model of pigeon representation learning. This is further validated with six large-scale downstream classification tasks using H&E stained whole slide image datasets representing diverse cancer types.

Histopathology and proteomics are synergistic for High-Grade Serous Ovarian Cancer platinum response prediction.

Patients with High-Grade Serous Ovarian Cancer (HGSOC) exhibit varied responses to treatment, with 20-30% showing de novo resistance to platinum-based chemotherapy. While hematoxylin-eosin (H&E) pathological slides are used for routine diagnosis of cancer type, they may also contain diagnostically useful information about treatment response. Our study demonstrates that combining H&E-stained Whole Slide Images (WSIs) with proteomic signatures using a multimodal deep learning framework significantly improves the prediction of platinum response in both discovery and validation cohorts. This method outperforms the Homologous Recombination Deficiency (HRD) score in predicting platinum response and overall patient survival. The study sets new performance benchmarks and explores the intersection of histology and proteomics, highlighting phenotypes related to treatment response pathways, including homologous recombination, DNA damage response, nucleotide synthesis, apoptosis, and ER stress. This integrative approach has the potential to improve personalized treatment and provide insights into the therapeutic vulnerabilities of HGSOC.

Automated prediction of COVID-19 severity upon admission by chest X-ray images and clinical metadata aiming at accuracy and explainability.

In the past few years COVID-19 posed a huge threat to healthcare systems around the world. One of the first waves of the pandemic hit Northern Italy severely resulting in high casualties and in the near breakdown of primary care. Due to these facts, the Covid CXR Hackathon-Artificial Intelligence for Covid-19 prognosis: aiming at accuracy and explainability challenge had been launched at the beginning of February 2022, releasing a new imaging dataset with additional clinical metadata for each accompanying chest X-ray (CXR). In this article we summarize our techniques at correctly diagnosing chest X-ray images collected upon admission for severity of COVID-19 outcome. In addition to X-ray imagery, clinical metadata was provided and the challenge also aimed at creating an explainable model. We created a best-performing, as well as, an explainable model that makes an effort to map clinical metadata to image features whilst predicting the prognosis. We also did many ablation studies in order to identify crucial parts of the models and the predictive power of each feature in the datasets. We conclude that CXRs at admission do not help the predicting power of the metadata significantly by itself and contain mostly information that is also mutually present in the blood samples and other clinical factors collected at admission.