MERF: A Practical HDR-Like Image Generator via Mutual-Guided Learning Between Multi-Exposure Registration and Fusion.

In this paper, we present a novel high dynamic range (HDR)-like image generator that utilizes mutual-guided learning between multi-exposure registration and fusion, leading to promising dynamic multi-exposure image fusion. The method consists of three main components: the registration network, the fusion network, and the dual attention network which seamlessly integrates registration and fusion processes. Initially, within the registration network, the estimation of deformation fields among multi-exposure image sequences is conducted following an exposure-invariant feature extraction phase. This leads to enhanced accuracy by mitigating discrepancies across domains. Subsequently, the fusion network utilizes a progressive frequency fusion module in two distinct stages, addressing color correction and detail preservation within low and high-frequency domains, respectively. To facilitate the mutual enhancement of the registration and fusion networks, we undertake a mutual-guided learning strategy encompassing their physical connection and constraint paradigm. Firstly, a dual attention network bridges the registration and fusion networks, addressing ghosting, which is beyond the scope of registration and facilitates information exchange between input images. Secondly, a meticulously designed generative adversarial network-like iterative training schema guides the overall network framework, thereby yielding high-quality HDR-like images through mutual enhancement. Comprehensive experiments on publicly available datasets validate the superiority of our method over existing state-of-the-art approaches.

Multimodal data analysis reveals that pancreatobiliary-type ampullary adenocarcinoma resembles pancreatic adenocarcinoma and differs from cholangiocarcinoma.

BACKGROUND: Ampullary adenocarcinoma (AAC) arises from the ampulla of Vater where the pancreatic duct and bile duct join and empty into the duodenum. It can be classified into intestinal and pancreatobiliary types based on histopathology or immunohistochemistry. However, there are no biomarkers for further classification of pancreatobiliary-type AAC which has important implications for its treatment. We aimed to identify the tumor origin of pancreatobiliary-type AAC by systematically analyzing whole-slide images (WSIs), survival data, and genome sequencing data collected from multiple centers. METHODS: This study involved three experiments. First, we extracted quantitative and highly interpretable features from the tumor region in WSIs and constructed a histologic classifier to differentiate between pancreatic adenocarcinoma (PAC) and cholangiocarcinoma. The histologic classifier was then applied to patients with pancreatobiliary-type AAC to infer the tumor origin. Secondly, we compared the overall survival of patients with pancreatobiliary-type AAC stratified by the adjuvant chemotherapy regimens designed for PAC or cholangiocarcinoma. Finally, we compared the mutation landscape of pancreatobiliary-type AAC with those of PAC and cholangiocarcinoma. RESULTS: The histologic classifier accurately classified PAC and cholangiocarcinoma in both the internal and external validation sets (AUC > 0.99). All pancreatobiliary-type AACs (n = 45) were classified as PAC. The patients with pancreatobiliary-type AAC receiving regimens designed for PAC showed more favorable overall survival than those receiving regimens designed for cholangiocarcinoma in a multivariable Cox regression (hazard ratio = 7.24, 95% confidence interval: 1.28-40.78, P = 0.025). The results of mutation analysis showed that the mutation landscape of AAC was very similar to that of PAC but distinct from that of cholangiocarcinoma. CONCLUSIONS: This multi-center study provides compelling evidence that pancreatobiliary-type AAC resembles PAC instead of cholangiocarcinoma in different aspects, which can guide the treatment selection and clinical trials planning for pancreatobiliary-type AAC.

Differentiation between immune checkpoint inhibitor-related and radiation pneumonitis in lung cancer by CT radiomics and machine learning.

PURPOSE: Consolidation immunotherapy after completion of chemoradiotherapy has become the standard of care for unresectable locally advanced non-small cell lung cancer and can induce potentially severe and life-threatening adverse events, including both immune checkpoint inhibitor-related pneumonitis (CIP) and radiation pneumonitis (RP), which are very challenging for radiologists to diagnose. Differentiating between CIP and RP has significant implications for clinical management such as the treatments for pneumonitis and the decision to continue or restart immunotherapy. The purpose of this study is to differentiate between CIP and RP by a CT radiomics approach. METHODS: We retrospectively collected the CT images and clinical information of patients with pneumonitis who received immune checkpoint inhibitor (ICI) only (n = 28), radiotherapy (RT) only (n = 31), and ICI+RT (n = 14). Three kinds of radiomic features (intensity histogram, gray-level co-occurrence matrix [GLCM] based, and bag-of-words [BoW] features) were extracted from CT images, which characterize tissue texture at different scales. Classification models, including logistic regression, random forest, and linear SVM, were first developed and tested in patients who received ICI or RT only with 10-fold cross-validation and further tested in patients who received ICI+RT using clinicians' diagnosis as a reference. RESULTS: Using 10-fold cross-validation, the classification models built on the intensity histogram features, GLCM-based features, and BoW features achieved an area under curve (AUC) of 0.765, 0.848, and 0.937, respectively. The best model was then applied to the patients receiving combination treatment, achieving an AUC of 0.896. CONCLUSIONS: This study demonstrates the promising potential of radiomic analysis of CT images for differentiating between CIP and RP in lung cancer, which could be a useful tool to attribute the cause of pneumonitis in patients who receive both ICI and RT.

Computational Image Analysis Identifies Histopathological Image Features Associated With Somatic Mutations and Patient Survival in Gastric Adenocarcinoma.

Computational analysis of histopathological images can identify sub-visual objective image features that may not be visually distinguishable by human eyes, and hence provides better modeling of disease phenotypes. This study aims to investigate whether specific image features are associated with somatic mutations and patient survival in gastric adenocarcinoma (sample size = 310). An automated image analysis pipeline was developed to extract quantitative morphological features from H&E stained whole-slide images. We found that four frequently somatically mutated genes (TP53, ARID1A, OBSCN, and PIK3CA) were significantly associated with tumor morphological changes. A prognostic model built on the image features significantly stratified patients into low-risk and high-risk groups (log-rank test p-value = 2.6e-4). Multivariable Cox regression showed the model predicted risk index was an additional prognostic factor besides tumor grade and stage. Gene ontology enrichment analysis showed that the genes whose expressions mostly correlated with the contributing features in the prognostic model were enriched on biological processes such as cell cycle and muscle contraction. These results demonstrate that histopathological image features can reflect underlying somatic mutations and identify high-risk patients that may benefit from more precise treatment regimens. Both the image features and pipeline are highly interpretable to enable translational applications.