Using histopathology latent diffusion models as privacy-preserving dataset augmenters improves downstream classification performance.

Latent diffusion models (LDMs) have emerged as a state-of-the-art image generation method, outperforming previous Generative Adversarial Networks (GANs) in terms of training stability and image quality. In computational pathology, generative models are valuable for data sharing and data augmentation. However, the impact of LDM-generated images on histopathology tasks compared to traditional GANs has not been systematically studied. We trained three LDMs and a styleGAN2 model on histology tiles from nine colorectal cancer (CRC) tissue classes. The LDMs include 1) a fine-tuned version of stable diffusion v1.4, 2) a Kullback-Leibler (KL)-autoencoder (KLF8-DM), and 3) a vector quantized (VQ)-autoencoder deploying LDM (VQF8-DM). We assessed image quality through expert ratings, dimensional reduction methods, distribution similarity measures, and their impact on training a multiclass tissue classifier. Additionally, we investigated image memorization in the KLF8-DM and styleGAN2 models. All models provided a high image quality, with the KLF8-DM achieving the best Frechet Inception Distance (FID) and expert rating scores for complex tissue classes. For simpler classes, the VQF8-DM and styleGAN2 models performed better. Image memorization was negligible for both styleGAN2 and KLF8-DM models. Classifiers trained on a mix of KLF8-DM generated and real images achieved a 4% improvement in overall classification accuracy, highlighting the usefulness of these images for dataset augmentation. Our systematic study of generative methods showed that KLF8-DM produces the highest quality images with negligible image memorization. The higher classifier performance in the generatively augmented dataset suggests that this augmentation technique can be employed to enhance histopathology classifiers for various tasks.

Prediction models for hormone receptor status in female breast cancer do not extend to males: further evidence of sex-based disparity in breast cancer.

Breast cancer prognosis and management for both men and women are reliant upon estrogen receptor alpha (ERα) and progesterone receptor (PR) expression to inform therapy. Previous studies have shown that there are sex-specific binding characteristics of ERα and PR in breast cancer and, counterintuitively, ERα expression is more common in male than female breast cancer. We hypothesized that these differences could have morphological manifestations that are undetectable to human observers but could be elucidated computationally. To investigate this, we trained attention-based multiple instance learning prediction models for ERα and PR using H&E-stained images of female breast cancer from the Cancer Genome Atlas (TCGA) (n = 1085) and deployed them on external female (n = 192) and male breast cancer images (n = 245). Both targets were predicted in the internal (AUROC for ERα prediction: 0.86 ± 0.02, p < 0.001; AUROC for PR prediction = 0.76 ± 0.03, p < 0.001) and external female cohorts (AUROC for ERα prediction: 0.78 ± 0.03, p < 0.001; AUROC for PR prediction = 0.80 ± 0.04, p < 0.001) but not the male cohort (AUROC for ERα prediction: 0.66 ± 0.14, p = 0.43; AUROC for PR prediction = 0.63 ± 0.04, p = 0.05). This suggests that subtle morphological differences invisible upon visual inspection may exist between the sexes, supporting previous immunohistochemical, genomic, and transcriptomic analyses.

Transformer-based biomarker prediction from colorectal cancer histology: A large-scale multicentric study.

Deep learning (DL) can accelerate the prediction of prognostic biomarkers from routine pathology slides in colorectal cancer (CRC). However, current approaches rely on convolutional neural networks (CNNs) and have mostly been validated on small patient cohorts. Here, we develop a new transformer-based pipeline for end-to-end biomarker prediction from pathology slides by combining a pre-trained transformer encoder with a transformer network for patch aggregation. Our transformer-based approach substantially improves the performance, generalizability, data efficiency, and interpretability as compared with current state-of-the-art algorithms. After training and evaluating on a large multicenter cohort of over 13,000 patients from 16 colorectal cancer cohorts, we achieve a sensitivity of 0.99 with a negative predictive value of over 0.99 for prediction of microsatellite instability (MSI) on surgical resection specimens. We demonstrate that resection specimen-only training reaches clinical-grade performance on endoscopic biopsy tissue, solving a long-standing diagnostic problem.

A multimodal comparison of latent denoising diffusion probabilistic models and generative adversarial networks for medical image synthesis.

Although generative adversarial networks (GANs) can produce large datasets, their limited diversity and fidelity have been recently addressed by denoising diffusion probabilistic models, which have demonstrated superiority in natural image synthesis. In this study, we introduce Medfusion, a conditional latent DDPM designed for medical image generation, and evaluate its performance against GANs, which currently represent the state-of-the-art. Medfusion was trained and compared with StyleGAN-3 using fundoscopy images from the AIROGS dataset, radiographs from the CheXpert dataset, and histopathology images from the CRCDX dataset. Based on previous studies, Progressively Growing GAN (ProGAN) and Conditional GAN (cGAN) were used as additional baselines on the CheXpert and CRCDX datasets, respectively. Medfusion exceeded GANs in terms of diversity (recall), achieving better scores of 0.40 compared to 0.19 in the AIROGS dataset, 0.41 compared to 0.02 (cGAN) and 0.24 (StyleGAN-3) in the CRMDX dataset, and 0.32 compared to 0.17 (ProGAN) and 0.08 (StyleGAN-3) in the CheXpert dataset. Furthermore, Medfusion exhibited equal or higher fidelity (precision) across all three datasets. Our study shows that Medfusion constitutes a promising alternative to GAN-based models for generating high-quality medical images, leading to improved diversity and less artifacts in the generated images.

Self-supervised attention-based deep learning for pan-cancer mutation prediction from histopathology.

The histopathological phenotype of tumors reflects the underlying genetic makeup. Deep learning can predict genetic alterations from pathology slides, but it is unclear how well these predictions generalize to external datasets. We performed a systematic study on Deep-Learning-based prediction of genetic alterations from histology, using two large datasets of multiple tumor types. We show that an analysis pipeline that integrates self-supervised feature extraction and attention-based multiple instance learning achieves a robust predictability and generalizability.

Generalizable biomarker prediction from cancer pathology slides with self-supervised deep learning: A retrospective multi-centric study.

Deep learning (DL) can predict microsatellite instability (MSI) from routine histopathology slides of colorectal cancer (CRC). However, it is unclear whether DL can also predict other biomarkers with high performance and whether DL predictions generalize to external patient populations. Here, we acquire CRC tissue samples from two large multi-centric studies. We systematically compare six different state-of-the-art DL architectures to predict biomarkers from pathology slides, including MSI and mutations in BRAF, KRAS, NRAS, and PIK3CA. Using a large external validation cohort to provide a realistic evaluation setting, we show that models using self-supervised, attention-based multiple-instance learning consistently outperform previous approaches while offering explainable visualizations of the indicative regions and morphologies. While the prediction of MSI and BRAF mutations reaches a clinical-grade performance, mutation prediction of PIK3CA, KRAS, and NRAS was clinically insufficient.

Nerve Fibers in the Tumor Microenvironment Are Co-Localized with Lymphoid Aggregates in Pancreatic Cancer.

B cells and tertiary lymphoid structures (TLS) are reported to be important in survival in cancer. Pancreatic Cancer (PDAC) is one of the most lethal cancer types, and currently, it is the seventh leading cause of cancer-related death worldwide. A better understanding of tumor biology is pivotal to improve clinical outcome. The desmoplastic stroma is a complex system in which crosstalk takes place between cancer-associated fibroblasts, immune cells and cancer cells. Indirect and direct cellular interactions within the tumor microenvironment (TME) drive key processes such as tumor progression, metastasis formation and treatment resistance. In order to understand the aggressiveness of PDAC and its resistance to therapeutics, the TME needs to be further unraveled. There are some limited data about the influence of nerve fibers on cancer progression. Here we show that small nerve fibers are located at lymphoid aggregates in PDAC. This unravels future pathways and has potential to improve clinical outcome by a rational development of new therapeutic strategies.

Nerve fibers in the tumor microenvironment in neurotropic cancer-pancreatic cancer and cholangiocarcinoma.

Pancreatic ductal adenocarcinoma (PDAC) and cholangiocarcinoma (CCA) are both deadly cancers and they share many biological features besides their close anatomical location. One of the main histological features is neurotropism, which results in frequent perineural invasion. The underlying mechanism of cancer cells favoring growth by and through the nerve fibers is not fully understood. In this review, we provide knowledge of these cancers with frequent perineural invasion. We discuss nerve fiber crosstalk with the main different components of the tumor microenvironment (TME), the immune cells, and the fibroblasts. Also, we discuss the crosstalk between the nerve fibers and the cancer. We highlight the shared signaling pathways of the mechanisms behind perineural invasion in PDAC and CCA. Hereby we have focussed on signaling neurotransmitters and neuropeptides which may be a target for future therapies. Furthermore, we have summarized retrospective results of the previous literature about nerve fibers in PDAC and CCA patients. We provide our point of view in the potential for nerve fibers to be used as powerful biomarker for prognosis, as a tool to stratify patients for therapy or as a target in a (combination) therapy. Taking the presence of nerves into account can potentially change the field of personalized care in these neurotropic cancers.

Pan-cancer image-based detection of clinically actionable genetic alterations.

Molecular alterations in cancer can cause phenotypic changes in tumor cells and their micro-environment. Routine histopathology tissue slides - which are ubiquitously available - can reflect such morphological changes. Here, we show that deep learning can consistently infer a wide range of genetic mutations, molecular tumor subtypes, gene expression signatures and standard pathology biomarkers directly from routine histology. We developed, optimized, validated and publicly released a one-stop-shop workflow and applied it to tissue slides of more than 5000 patients across multiple solid tumors. Our findings show that a single deep learning algorithm can be trained to predict a wide range of molecular alterations from routine, paraffin-embedded histology slides stained with hematoxylin and eosin. These predictions generalize to other populations and are spatially resolved. Our method can be implemented on mobile hardware, potentially enabling point-of-care diagnostics for personalized cancer treatment. More generally, this approach could elucidate and quantify genotype-phenotype links in cancer.

Precise QCD Predictions for the Production of Dijet Final States in Deep Inelastic Scattering.

The production of two-jet final states in deep inelastic scattering is an important QCD precision observable. We compute it for the first time to next-to-next-to-leading order (NNLO) in perturbative QCD. Our calculation is fully differential in the lepton and jet variables and allows one to impose cuts on the jets in both the laboratory and the Breit frame. We observe that the NNLO corrections are moderate in size, except at kinematical edges, and that their inclusion leads to a substantial reduction of the scale variation uncertainty on the predictions. Our results will enable the inclusion of deep inelastic dijet data in precision phenomenology studies.

Signal modeling in low coherence interference microscopy on example of rectangular grating.

Besides the illumination wavelength also the numerical aperture (NA) of a microscope objective affects the fringe spacing in interference microscopy. Therefore, at high NA values an effective wavelength should be obtained by calibration. At step height structures both, the effective wavelength and the batwing effect strongly depend on the height-to-wavelength-ratio (HWR). Therefore, changes of the effective wavelength considering temporal and spatial coherence enable us to estimate the batwing effect in measurement results. For high NA systems and broadband illumination two different theoretical approaches for signal modeling are introduced to study the influence of the center wavelength, the temporal, and the spatial coherence of the illuminating light on measurement results of a rectangular grating. In both models diffraction is considered. While the first simulation model (Kirchhoff) is mostly analytical the second one (Richards-Wolf) is primarily numerical. Simulation results of both models show a good agreement with experimental measurement results.

Lateral resolution and transfer characteristics of vertical scanning white-light interferometers.

White-light interferometers are widely used for high-accuracy topography measurement in industrial and scientific applications. A common way to characterize a white-light interferometer is to assume small surface amplitudes resulting in linear transfer characteristics described by the instrument transfer function (ITF). However, the well-known batwing effect gives rise to systematic errors, causing extra nonlinearity to the ITF. In this paper a model to simulate an interference pattern in the image plane as it is obtained by a vertical scanning white-light interferometer is introduced in order to overcome the limitation of small surface amplitudes. Repeating the simulation procedure for different height positions of the object results in an image stack that can be analyzed by the same algorithms as real measurement data. The simulation results agree with experimental observations: the batwing effect occurs in certain situations and the correct amplitude of a rectangular grating structure can be obtained as long as the structure is optically resolved. Both simulation, as well as experimental results, provide transfer characteristics of broader bandwidth than predicted by theoretical approaches based on linear system behavior.

Transfer characteristics of rectangular phase gratings in interference microscopy.

In this Letter, the transfer characteristics of rectangular periodic phase objects are studied. It turns out that there are significant differences compared to amplitude objects. The imaging of an amplitude object can be understood as a linear process, whereas phase objects behave nonlinearly. It is shown that under certain conditions the correct shape of a rectangular phase grating can be obtained by an interference microscope as long as the first order diffraction component passes the optical imaging system. This result is in a good agreement with experimental observations and computer simulation results.

Dual-wavelength vertical scanning low-coherence interferometric microscope.

White-light interferometry has turned into a standard tool in the field of high-accuracy topography measurements. Nevertheless, surfaces with relatively large local surface tilts or height steps often give rise to systematic measuring errors. The reasons are diffraction and dispersion effects, which cause deviations between height values obtained from the envelope maximum of the white-light interference signal and those obtained from the signal's phase. In certain cases this may result in ghost steps appearing in the measured topography. To identify and eliminate these ghost steps we use a second LED emitting light at a different mean wavelength. This now allows the measurement of curved or structured specular surfaces with high resolution, which up to now was restricted by the mentioned effects.