Understanding metric-related pitfalls in image analysis validation.

Validation metrics are key for tracking scientific progress and bridging the current chasm between artificial intelligence research and its translation into practice. However, increasing evidence shows that, particularly in image analysis, metrics are often chosen inadequately. Although taking into account the individual strengths, weaknesses and limitations of validation metrics is a critical prerequisite to making educated choices, the relevant knowledge is currently scattered and poorly accessible to individual researchers. Based on a multistage Delphi process conducted by a multidisciplinary expert consortium as well as extensive community feedback, the present work provides a reliable and comprehensive common point of access to information on pitfalls related to validation metrics in image analysis. Although focused on biomedical image analysis, the addressed pitfalls generalize across application domains and are categorized according to a newly created, domain-agnostic taxonomy. The work serves to enhance global comprehension of a key topic in image analysis validation.

Metrics reloaded: recommendations for image analysis validation.

Increasing evidence shows that flaws in machine learning (ML) algorithm validation are an underestimated global problem. In biomedical image analysis, chosen performance metrics often do not reflect the domain interest, and thus fail to adequately measure scientific progress and hinder translation of ML techniques into practice. To overcome this, we created Metrics Reloaded, a comprehensive framework guiding researchers in the problem-aware selection of metrics. Developed by a large international consortium in a multistage Delphi process, it is based on the novel concept of a problem fingerprint-a structured representation of the given problem that captures all aspects that are relevant for metric selection, from the domain interest to the properties of the target structure(s), dataset and algorithm output. On the basis of the problem fingerprint, users are guided through the process of choosing and applying appropriate validation metrics while being made aware of potential pitfalls. Metrics Reloaded targets image analysis problems that can be interpreted as classification tasks at image, object or pixel level, namely image-level classification, object detection, semantic segmentation and instance segmentation tasks. To improve the user experience, we implemented the framework in the Metrics Reloaded online tool. Following the convergence of ML methodology across application domains, Metrics Reloaded fosters the convergence of validation methodology. Its applicability is demonstrated for various biomedical use cases.

Artificial Intelligence in Bone Marrow Histological Diagnostics: Potential Applications and Challenges.

The expanding digitalization of routine diagnostic histological slides holds a potential to apply artificial intelligence (AI) to pathology, including bone marrow (BM) histology. In this perspective, we describe potential tasks in diagnostics that can be supported, investigations that can be guided, and questions that can be answered by the future application of AI on whole-slide images of BM biopsies. These range from characterization of cell lineages and quantification of cells and stromal structures to disease prediction. First glimpses show an exciting potential to detect subtle phenotypic changes with AI that are due to specific genotypes. The discussion is illustrated by examples of current AI research using BM biopsy slides. In addition, we briefly discuss current challenges for implementation of AI-supported diagnostics.

Artificial intelligence assistance significantly improves Gleason grading of prostate biopsies by pathologists.

The Gleason score is the most important prognostic marker for prostate cancer patients, but it suffers from significant observer variability. Artificial intelligence (AI) systems based on deep learning can achieve pathologist-level performance at Gleason grading. However, the performance of such systems can degrade in the presence of artifacts, foreign tissue, or other anomalies. Pathologists integrating their expertise with feedback from an AI system could result in a synergy that outperforms both the individual pathologist and the system. Despite the hype around AI assistance, existing literature on this topic within the pathology domain is limited. We investigated the value of AI assistance for grading prostate biopsies. A panel of 14 observers graded 160 biopsies with and without AI assistance. Using AI, the agreement of the panel with an expert reference standard increased significantly (quadratically weighted Cohen's kappa, 0.799 vs. 0.872; p = 0.019). On an external validation set of 87 cases, the panel showed a significant increase in agreement with a panel of international experts in prostate pathology (quadratically weighted Cohen's kappa, 0.733 vs. 0.786; p = 0.003). In both experiments, on a group-level, AI-assisted pathologists outperformed the unassisted pathologists and the standalone AI system. Our results show the potential of AI systems for Gleason grading, but more importantly, show the benefits of pathologist-AI synergy.

Mini Review: The Last Mile-Opportunities and Challenges for Machine Learning in Digital Toxicologic Pathology.

The 2019 manuscript by the Special Interest Group on Digital Pathology and Image Analysis of the Society of Toxicologic pathology suggested that a synergism between artificial intelligence (AI) and machine learning (ML) technologies and digital toxicologic pathology would improve the daily workflow and future impact of toxicologic pathologists globally. Now 2 years later, the authors of this review consider whether, in their opinion, there is any evidence that supports that thesis. Specifically, we consider the opportunities and challenges for applying ML (the study of computer algorithms that are able to learn from example data and extrapolate the learned information to unseen data) algorithms in toxicologic pathology and how regulatory bodies are navigating this rapidly evolving field. Although we see similarities with the "Last Mile" metaphor, the weight of evidence suggests that toxicologic pathologists should approach ML with an equal dose of skepticism and enthusiasm. There are increasing opportunities for impact in our field that leave the authors cautiously excited and optimistic. Toxicologic pathologists have the opportunity to critically evaluate ML applications with a "call-to-arms" mentality. Why should we be late adopters? There is ample evidence to encourage engagement, growth, and leadership in this field.

Automated deep-learning system for Gleason grading of prostate cancer using biopsies: a diagnostic study.

BACKGROUND: The Gleason score is the strongest correlating predictor of recurrence for prostate cancer, but has substantial inter-observer variability, limiting its usefulness for individual patients. Specialised urological pathologists have greater concordance; however, such expertise is not widely available. Prostate cancer diagnostics could thus benefit from robust, reproducible Gleason grading. We aimed to investigate the potential of deep learning to perform automated Gleason grading of prostate biopsies. METHODS: In this retrospective study, we developed a deep-learning system to grade prostate biopsies following the Gleason grading standard. The system was developed using randomly selected biopsies, sampled by the biopsy Gleason score, from patients at the Radboud University Medical Center (pathology report dated between Jan 1, 2012, and Dec 31, 2017). A semi-automatic labelling technique was used to circumvent the need for manual annotations by pathologists, using pathologists' reports as the reference standard during training. The system was developed to delineate individual glands, assign Gleason growth patterns, and determine the biopsy-level grade. For validation of the method, a consensus reference standard was set by three expert urological pathologists on an independent test set of 550 biopsies. Of these 550, 100 were used in an observer experiment, in which the system, 13 pathologists, and two pathologists in training were compared with respect to the reference standard. The system was also compared to an external test dataset of 886 cores, which contained 245 cores from a different centre that were independently graded by two pathologists. FINDINGS: We collected 5759 biopsies from 1243 patients. The developed system achieved a high agreement with the reference standard (quadratic Cohen's kappa 0·918, 95% CI 0·891-0·941) and scored highly at clinical decision thresholds: benign versus malignant (area under the curve 0·990, 95% CI 0·982-0·996), grade group of 2 or more (0·978, 0·966-0·988), and grade group of 3 or more (0·974, 0·962-0·984). In an observer experiment, the deep-learning system scored higher (kappa 0·854) than the panel (median kappa 0·819), outperforming 10 of 15 pathologist observers. On the external test dataset, the system obtained a high agreement with the reference standard set independently by two pathologists (quadratic Cohen's kappa 0·723 and 0·707) and within inter-observer variability (kappa 0·71). INTERPRETATION: Our automated deep-learning system achieved a performance similar to pathologists for Gleason grading and could potentially contribute to prostate cancer diagnosis. The system could potentially assist pathologists by screening biopsies, providing second opinions on grade group, and presenting quantitative measurements of volume percentages. FUNDING: Dutch Cancer Society.

Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer.

Importance: Application of deep learning algorithms to whole-slide pathology images can potentially improve diagnostic accuracy and efficiency. Objective: Assess the performance of automated deep learning algorithms at detecting metastases in hematoxylin and eosin-stained tissue sections of lymph nodes of women with breast cancer and compare it with pathologists' diagnoses in a diagnostic setting. Design, Setting, and Participants: Researcher challenge competition (CAMELYON16) to develop automated solutions for detecting lymph node metastases (November 2015-November 2016). A training data set of whole-slide images from 2 centers in the Netherlands with (n = 110) and without (n = 160) nodal metastases verified by immunohistochemical staining were provided to challenge participants to build algorithms. Algorithm performance was evaluated in an independent test set of 129 whole-slide images (49 with and 80 without metastases). The same test set of corresponding glass slides was also evaluated by a panel of 11 pathologists with time constraint (WTC) from the Netherlands to ascertain likelihood of nodal metastases for each slide in a flexible 2-hour session, simulating routine pathology workflow, and by 1 pathologist without time constraint (WOTC). Exposures: Deep learning algorithms submitted as part of a challenge competition or pathologist interpretation. Main Outcomes and Measures: The presence of specific metastatic foci and the absence vs presence of lymph node metastasis in a slide or image using receiver operating characteristic curve analysis. The 11 pathologists participating in the simulation exercise rated their diagnostic confidence as definitely normal, probably normal, equivocal, probably tumor, or definitely tumor. Results: The area under the receiver operating characteristic curve (AUC) for the algorithms ranged from 0.556 to 0.994. The top-performing algorithm achieved a lesion-level, true-positive fraction comparable with that of the pathologist WOTC (72.4% [95% CI, 64.3%-80.4%]) at a mean of 0.0125 false-positives per normal whole-slide image. For the whole-slide image classification task, the best algorithm (AUC, 0.994 [95% CI, 0.983-0.999]) performed significantly better than the pathologists WTC in a diagnostic simulation (mean AUC, 0.810 [range, 0.738-0.884]; P < .001). The top 5 algorithms had a mean AUC that was comparable with the pathologist interpreting the slides in the absence of time constraints (mean AUC, 0.960 [range, 0.923-0.994] for the top 5 algorithms vs 0.966 [95% CI, 0.927-0.998] for the pathologist WOTC). Conclusions and Relevance: In the setting of a challenge competition, some deep learning algorithms achieved better diagnostic performance than a panel of 11 pathologists participating in a simulation exercise designed to mimic routine pathology workflow; algorithm performance was comparable with an expert pathologist interpreting whole-slide images without time constraints. Whether this approach has clinical utility will require evaluation in a clinical setting.

Computer-extracted Features Can Distinguish Noncancerous Confounding Disease from Prostatic Adenocarcinoma at Multiparametric MR Imaging.

PURPOSE: To determine the best features to discriminate prostate cancer from benign disease and its relationship to benign disease class and cancer grade. MATERIALS AND METHODS: The institutional review board approved this study and waived the need for informed consent. A retrospective cohort of 70 patients (age range, 48-70 years; median, 62 years), all of whom were scheduled to undergo radical prostatectomy and underwent preoperative 3-T multiparametric magnetic resonance (MR) imaging, including T2-weighted, diffusion-weighted, and dynamic contrast material-enhanced imaging, were included. The digitized prostatectomy slides were annotated for cancer and noncancerous disease and coregistered to MR imaging with an interactive deformable coregistration scheme. Computer-identified features for each of the noncancerous disease categories (eg, benign prostatic hyperplasia [BPH], prostatic intraepithelial neoplasia [PIN], inflammation, and atrophy) and prostate cancer were extracted. Feature selection was performed to identify the features with the highest discriminatory power. The performance of these five features was evaluated by using the area under the receiver operating characteristic curve (AUC). RESULTS: High-b-value diffusion-weighted images were more discriminative in distinguishing BPH from prostate cancer than apparent diffusion coefficient, which was most suitable for distinguishing PIN from prostate cancer. The focal appearance of lesions on dynamic contrast-enhanced images may help discriminate atrophy and inflammation from cancer. Which imaging features are discriminative for different benign lesions is influenced by cancer grade. The apparent diffusion coefficient appeared to be the most discriminative feature in identifying high-grade cancer. Classification results showed increased performance by taking into account specific benign types (AUC = 0.70) compared with grouping all noncancerous findings together (AUC = 0.62). CONCLUSION: The best features with which to discriminate prostate cancer from noncancerous benign disease depend on the type of benign disease and cancer grade. Use of the best features may result in better diagnostic performance.

Prostate Cancer: The European Society of Urogenital Radiology Prostate Imaging Reporting and Data System Criteria for Predicting Extraprostatic Extension by Using 3-T Multiparametric MR Imaging.

PURPOSE: To retrospectively assess the use of the European Society of Urogenital Radiology (ESUR) Prostate Imaging Reporting and Data System (PI-RADS) criteria and 3-T multiparametric magnetic resonance (MR) imaging for detection of extraprostatic extension (EPE) of prostate cancer. MATERIALS AND METHODS: The institutional review board approval requirement was waived. Consecutive patients with prostate cancer (n = 133) underwent 3-T multiparametric MR imaging before prostatectomy. Lesions were assessed by using ESUR/PI-RADS criteria for T2-weighted, diffusion-weighted, and dynamic contrast material-enhanced imaging, and by using the sum of these scores. Zonal dominant parameters corresponding to the score of diffusion-weighted imaging for peripheral zone lesions and to T2-weighted imaging scores for transitional zone lesions were calculated. In addition, the presence of EPE in each patient was evaluated on the basis of subjective multiparametric MR imaging features. Histopathologic examination of whole-mount radical prostatectomy specimens was used as the reference standard. Sensitivity, specificity, positive predictive, and negative predictive values; likelihood ratios; and areas under the receiver operating characteristic curve were calculated for each parameter on the basis of its usefulness for prediction of EPE. RESULTS: EPE was found in 60 of 133 (45%) patients. Receiver operating characteristic curve analysis for the prediction of EPE revealed an area under the curve of 0.72 for T2-weighted, 0.67 for diffusion-weighted, and 0.64 for dynamic contrast-enhanced imaging; 0.74 for the dominant parameter; and 0.74 for the sum of the PI-RADS scores, and a score of 5 was defined as the best threshold for the individual parameters, with a score greater than or equal to 13 as the threshold for the sum of the PI-RADS scores. By applying these thresholds, sensitivity, negative predictive value, and negative likelihood ratio (ruling out EPE) were 77%, 77%, and 0.36, respectively, and specificity, positive predictive value, and positive likelihood ratio (ruling in EPE) were 64%, 64%, and 2.15, respectively, for the dominant parameter. Feature analysis showed an area under the curve of 0.72; sensitivity, negative predictive value, and negative likelihood ratio of 63%, 72%, and 0.56, respectively, and specificity, positive predictive value, and positive likelihood ratio of 78%, 70%, and 3.77, respectively. CONCLUSION: ESUR/PI-RADS criteria showed moderate overall accuracy for use in the prediction of EPE, and these results were similar to those of multiparametric MR imaging assessment of features in this study sample.

Clinical evaluation of a computer-aided diagnosis system for determining cancer aggressiveness in prostate MRI.

OBJECTIVES: To investigate the added value of computer-aided diagnosis (CAD) on the diagnostic accuracy of PIRADS reporting and the assessment of cancer aggressiveness. METHODS: Multi-parametric MRI and histopathological outcome of MR-guided biopsies of a consecutive set of 130 patients were included. All cases were prospectively PIRADS reported and the reported lesions underwent CAD analysis. Logistic regression combined the CAD prediction and radiologist PIRADS score into a combination score. Receiver-operating characteristic (ROC) analysis and Spearman's correlation coefficient were used to assess the diagnostic accuracy and correlation to cancer grade. Evaluation was performed for discriminating benign lesions from cancer and for discriminating indolent from aggressive lesions. RESULTS: In total 141 lesions (107 patients) were included for final analysis. The area-under-the-ROC-curve of the combination score was higher than for the PIRADS score of the radiologist (benign vs. cancer, 0.88 vs. 0.81, p = 0.013 and indolent vs. aggressive, 0.88 vs. 0.78, p < 0.01). The combination score correlated significantly stronger with cancer grade (0.69, p = 0.0014) than the individual CAD system or radiologist (0.54 and 0.58). CONCLUSIONS: Combining CAD prediction and PIRADS into a combination score has the potential to improve diagnostic accuracy. Furthermore, such a combination score has a strong correlation with cancer grade. KEY POINTS: • Computer-aided diagnosis helps radiologists discriminate benign findings from cancer in prostate MRI. • Combining PIRADS and computer-aided diagnosis improves differentiation between indolent and aggressive cancer. • Adding computer-aided diagnosis to PIRADS increases the correlation coefficient with respect to cancer grade.

Automatic data augmentation to improve generalization of deep learning in H&E stained histopathology.

In histopathology practice, scanners, tissue processing, staining, and image acquisition protocols vary from center to center, resulting in subtle variations in images. Vanilla convolutional neural networks are sensitive to such domain shifts. Data augmentation is a popular way to improve domain generalization. Currently, state-of-the-art domain generalization in computational pathology is achieved using a manually curated set of augmentation transforms. However, manual tuning of augmentation parameters is time-consuming and can lead to sub-optimal generalization performance. Meta-learning frameworks can provide efficient ways to find optimal training hyper-parameters, including data augmentation. In this study, we hypothesize that an automated search of augmentation hyper-parameters can provide superior generalization performance and reduce experimental optimization time. We select four state-of-the-art automatic augmentation methods from general computer vision and investigate their capacity to improve domain generalization in histopathology. We analyze their performance on data from 25 centers across two different tasks: tumor metastasis detection in lymph nodes and breast cancer tissue type classification. On tumor metastasis detection, most automatic augmentation methods achieve comparable performance to state-of-the-art manual augmentation. On breast cancer tissue type classification, the leading automatic augmentation method significantly outperforms state-of-the-art manual data augmentation.

Diffusion models for out-of-distribution detection in digital pathology.

The ability to detect anomalies, i.e. anything not seen during training or out-of-distribution (OOD), in medical imaging applications is essential for successfully deploying machine learning systems. Filtering out OOD data using unsupervised learning is especially promising because it does not require costly annotations. A new class of models called AnoDDPMs, based on denoising diffusion probabilistic models (DDPMs), has recently achieved significant progress in unsupervised OOD detection. This work provides a benchmark for unsupervised OOD detection methods in digital pathology. By leveraging fast sampling techniques, we apply AnoDDPM on a large enough scale for whole-slide image analysis on the complete test set of the Camelyon16 challenge. Based on ROC analysis, we show that AnoDDPMs can detect OOD data with an AUC of up to 94.13 and 86.93 on two patch-level OOD detection tasks, outperforming the other unsupervised methods. We observe that AnoDDPMs alter the semantic properties of inputs, replacing anomalous data with more benign-looking tissue. Furthermore, we highlight the flexibility of AnoDDPM towards different information bottlenecks by evaluating reconstruction errors for inputs with different signal-to-noise ratios. While there is still a significant performance gap with fully supervised learning, AnoDDPMs show considerable promise in the field of OOD detection in digital pathology.

Full resolution reconstruction of whole-mount sections from digitized individual tissue fragments.

Whole-mount sectioning is a technique in histopathology where a full slice of tissue, such as a transversal cross-section of a prostate specimen, is prepared on a large microscope slide without further sectioning into smaller fragments. Although this technique can offer improved correlation with pre-operative imaging and is paramount for multimodal research, it is not commonly employed due to its technical difficulty, associated cost and cumbersome integration in (digital) pathology workflows. In this work, we present a computational tool named PythoStitcher which reconstructs artificial whole-mount sections from digitized tissue fragments, thereby bringing the benefits of whole-mount sections to pathology labs currently unable to employ this technique. Our proposed algorithm consists of a multi-step approach where it (i) automatically determines how fragments need to be reassembled, (ii) iteratively optimizes the stitch using a genetic algorithm and (iii) efficiently reconstructs the final artificial whole-mount section on full resolution (0.25 µm/pixel). PythoStitcher was validated on a total of 198 cases spanning five datasets with a varying number of tissue fragments originating from different organs from multiple centers. PythoStitcher successfully reconstructed the whole-mount section in 86-100% of cases for a given dataset with a residual registration mismatch of 0.65-2.76 mm on automatically selected landmarks. It is expected that our algorithm can aid pathology labs unable to employ whole-mount sectioning through faster clinical case evaluation and improved radiology-pathology correlation workflows.

Detection and subtyping of basal cell carcinoma in whole-slide histopathology using weakly-supervised learning.

The frequency of basal cell carcinoma (BCC) cases is putting an increasing strain on dermatopathologists. BCC is the most common type of skin cancer, and its incidence is increasing rapidly worldwide. AI can play a significant role in reducing the time and effort required for BCC diagnostics and thus improve the overall efficiency of the process. To train such an AI system in a fully-supervised fashion however, would require a large amount of pixel-level annotation by already strained dermatopathologists. Therefore, in this study, our primary objective was to develop a weakly-supervised for the identification of basal cell carcinoma (BCC) and the stratification of BCC into low-risk and high-risk categories within histopathology whole-slide images (WSI). We compared Clustering-constrained Attention Multiple instance learning (CLAM) with StreamingCLAM and hypothesized that the latter would be the superior approach. A total of 5147 images were used to train and validate the models, which were subsequently tested on an internal set of 949 images and an external set of 183 images. The labels for training were automatically extracted from free-text pathology reports using a rule-based approach. All data has been made available through the COBRA dataset. The results showed that both the CLAM and StreamingCLAM models achieved high performance for the detection of BCC, with an area under the ROC curve (AUC) of 0.994 and 0.997, respectively, on the internal test set and 0.983 and 0.993 on the external dataset. Furthermore, the models performed well on risk stratification, with AUC values of 0.912 and 0.931, respectively, on the internal set, and 0.851 and 0.883 on the external set. In every single metric the StreamingCLAM model outperformed the CLAM model or is on par. The performance of both models was comparable to that of two pathologists who scored 240 BCC positive slides. Additionally, in the public test set, StreamingCLAM demonstrated a comparable AUC of 0.958, markedly superior to CLAM's 0.803. This difference was statistically significant and emphasized the strength and better adaptability of the StreamingCLAM approach.

Transformer-Based Weakly Supervised Learning for Whole Slide Lung Cancer Image Classification.

Image analysis can play an important role in supporting histopathological diagnoses of lung cancer, with deep learning methods already achieving remarkable results. However, due to the large scale of whole-slide images (WSIs), creating manual pixel-wise annotations from expert pathologists is expensive and time-consuming. In addition, the heterogeneity of tumors and similarities in the morphological phenotype of tumor subtypes have caused inter-observer variability in annotations, which limits optimal performance. Effective use of weak labels could potentially alleviate these issues. In this paper, we propose a two-stage transformer-based weakly supervised learning framework called Simple Shuffle-Remix Vision Transformer (SSRViT). Firstly, we introduce a Shuffle-Remix Vision Transformer (SRViT) to retrieve discriminative local tokens and extract effective representative features. Then, the token features are selected and aggregated to generate sparse representations of WSIs, which are fed into a simple transformer-based classifier (SViT) for slide-level prediction. Experimental results demonstrate that the performance of our proposed SSRViT is significantly improved compared with other state-of-the-art methods in discriminating between adenocarcinoma, pulmonary sclerosing pneumocytoma and normal lung tissue (accuracy of 96.9% and AUC of 99.6%).

Three-dimensional identification of microvascular pathology and neurovascular inflammation in severe white matter hyperintensity: a case report.

White matter hyperintensities (WMH) are the most prevalent markers of cerebral small vessel disease (SVD), which is the major vascular risk factor for dementia. Microvascular pathology and neuroinflammation are suggested to drive the transition from normal-appearing white matter (NAWM) to WMH, particularly in individuals with hypertension. However, current imaging techniques cannot capture ongoing NAWM changes. The transition from NAWM into WMH is a continuous process, yet white matter lesions are often examined dichotomously, which may explain their underlying heterogeneity. Therefore, we examined microvascular and neurovascular inflammation pathology in NAWM and severe WMH three-dimensionally, along with gradual magnetic resonance imaging (MRI) fluid-attenuated inversion recovery (FLAIR) signal (sub-)segmentation. In WMH, the vascular network exhibited reduced length and complexity compared to NAWM. Neuroinflammation was more severe in WMH. Vascular inflammation was more pronounced in NAWM, suggesting its potential significance in converting NAWM into WMH. Moreover, the (sub-)segmentation of FLAIR signal displayed varying degrees of vascular pathology, particularly within WMH regions. These findings highlight the intricate interplay between microvascular pathology and neuroinflammation in the transition from NAWM to WMH. Further examination of neurovascular inflammation across MRI-visible alterations could aid deepening our understanding on WMH conversion, and therewith how to improve the prognosis of SVD.

Automatic quantification of tumor-stroma ratio as a prognostic marker for pancreatic cancer.

PURPOSE: This study aims to introduce an innovative multi-step pipeline for automatic tumor-stroma ratio (TSR) quantification as a potential prognostic marker for pancreatic cancer, addressing the limitations of existing staging systems and the lack of commonly used prognostic biomarkers. METHODS: The proposed approach involves a deep-learning-based method for the automatic segmentation of tumor epithelial cells, tumor bulk, and stroma from whole-slide images (WSIs). Models were trained using five-fold cross-validation and evaluated on an independent external test set. TSR was computed based on the segmented components. Additionally, TSR's predictive value for six-month survival on the independent external dataset was assessed. RESULTS: Median Dice (inter-quartile range (IQR)) of 0.751(0.15) and 0.726(0.25) for tumor epithelium segmentation on internal and external test sets, respectively. Median Dice of 0.76(0.11) and 0.863(0.17) for tumor bulk segmentation on internal and external test sets, respectively. TSR was evaluated as an independent prognostic marker, demonstrating a cross-validation AUC of 0.61±0.12 for predicting six-month survival on the external dataset. CONCLUSION: Our pipeline for automatic TSR quantification offers promising potential as a prognostic marker for pancreatic cancer. The results underscore the feasibility of computational biomarker discovery in enhancing patient outcome prediction, thus contributing to personalized patient management.

Understanding metric-related pitfalls in image analysis validation.

Validation metrics are key for the reliable tracking of scientific progress and for bridging the current chasm between artificial intelligence (AI) research and its translation into practice. However, increasing evidence shows that particularly in image analysis, metrics are often chosen inadequately in relation to the underlying research problem. This could be attributed to a lack of accessibility of metric-related knowledge: While taking into account the individual strengths, weaknesses, and limitations of validation metrics is a critical prerequisite to making educated choices, the relevant knowledge is currently scattered and poorly accessible to individual researchers. Based on a multi-stage Delphi process conducted by a multidisciplinary expert consortium as well as extensive community feedback, the present work provides the first reliable and comprehensive common point of access to information on pitfalls related to validation metrics in image analysis. Focusing on biomedical image analysis but with the potential of transfer to other fields, the addressed pitfalls generalize across application domains and are categorized according to a newly created, domain-agnostic taxonomy. To facilitate comprehension, illustrations and specific examples accompany each pitfall. As a structured body of information accessible to researchers of all levels of expertise, this work enhances global comprehension of a key topic in image analysis validation.

Differentiation of prostatitis and prostate cancer by using diffusion-weighted MR imaging and MR-guided biopsy at 3 T.

PURPOSE: To determine if prostatitis and prostate cancer (PCa) can be distinguished by using apparent diffusion coefficients (ADCs) on magnetic resonance (MR) images, with specimens obtained at MR-guided biopsy as the standard of reference. MATERIALS AND METHODS: The need for institutional review board approval and informed consent was waived. MR-guided biopsies were performed in 130 consecutive patients with cancer-suspicious regions (CSRs) on multiparametric MR images obtained at 3 T. In this retrospective study, 88 patients met the inclusion criteria. During the biopsy procedure, an axial diffusion-weighted sequence was performed and ADC maps were generated (repetition time msec/echo time msec, 2000/67; section thickness, 4 mm; in-plane resolution, 1.8 × 1.8 mm; and b values of 0, 100, 500, and 800 sec/mm(2)). Subsequently, a confirmation image with the needle left in situ was acquired and projected on the ADC map. The corresponding ADCs at the biopsy location were compared with the histopathologic outcomes of the biopsy specimens. Linear mixed-model regression analyses were used to test for ADC differences between the histopathologic groups. RESULTS: The study included 116 biopsy specimens. Median ADCs of normal prostate tissue, prostatitis, low-grade PCa (Gleason grade components 2 or 3), and high-grade PCa (Gleason grade components 4 or 5) were 1.22 × 10(-3) mm(2)/sec (standard deviation, ± 0.21), 1.08 × 10(-3) mm(2)/sec (± 0.18), 0.88 × 10(-3) mm(2)/sec (± 0.15), and 0.88 × 10(-3) mm(2)/sec (± 0.13), respectively. Although the median ADCs of biopsy specimens with prostatitis were significantly higher compared with low- and high-grade PCa (P < .001), there is a considerable overlap between the tissue types. CONCLUSION: Diffusion-weighted imaging is a noninvasive technique that shows differences between prostatitis and PCa in both the peripheral zone and central gland, although its usability in clinical practice is limited as a result of significant overlap in ADCs.