Metrics reloaded: recommendations for image analysis validation.

Increasing evidence shows that flaws in machine learning (ML) algorithm validation are an underestimated global problem. In biomedical image analysis, chosen performance metrics often do not reflect the domain interest, and thus fail to adequately measure scientific progress and hinder translation of ML techniques into practice. To overcome this, we created Metrics Reloaded, a comprehensive framework guiding researchers in the problem-aware selection of metrics. Developed by a large international consortium in a multistage Delphi process, it is based on the novel concept of a problem fingerprint-a structured representation of the given problem that captures all aspects that are relevant for metric selection, from the domain interest to the properties of the target structure(s), dataset and algorithm output. On the basis of the problem fingerprint, users are guided through the process of choosing and applying appropriate validation metrics while being made aware of potential pitfalls. Metrics Reloaded targets image analysis problems that can be interpreted as classification tasks at image, object or pixel level, namely image-level classification, object detection, semantic segmentation and instance segmentation tasks. To improve the user experience, we implemented the framework in the Metrics Reloaded online tool. Following the convergence of ML methodology across application domains, Metrics Reloaded fosters the convergence of validation methodology. Its applicability is demonstrated for various biomedical use cases.

Addition of inflammation-related biomarkers to the CAIDE model for risk prediction of all-cause dementia, Alzheimer's disease and vascular dementia in a prospective study.

BACKGROUND: It is of interest whether inflammatory biomarkers can improve dementia prediction models, such as the widely used Cardiovascular Risk Factors, Aging and Dementia (CAIDE) model. METHODS: The Olink Target 96 Inflammation panel was assessed in a nested case-cohort design within a large, population-based German cohort study (n = 9940; age-range: 50-75 years). All study participants who developed dementia over 20 years of follow-up and had complete CAIDE variable data (n = 562, including 173 Alzheimer's disease (AD) and 199 vascular dementia (VD) cases) as well as n = 1,356 controls were selected for measurements. 69 inflammation-related biomarkers were eligible for use. LASSO logistic regression and bootstrapping were utilized to select relevant biomarkers and determine areas under the curve (AUCs). RESULTS: The CAIDE model 2 (including Apolipoprotein E (APOE) ε4 carrier status) predicted all-cause dementia, AD, and VD better than CAIDE model 1 (without APOE ε4) with AUCs of 0.725, 0.752 and 0.707, respectively. Although 20, 7, and 4 inflammation-related biomarkers were selected by LASSO regression to improve CAIDE model 2, the AUCs did not increase markedly. CAIDE models 1 and 2 generally performed better in mid-life (50-64 years) than in late-life (65-75 years) sub-samples of our cohort, but again, inflammation-related biomarkers did not improve their predictive abilities. CONCLUSIONS: Despite a lack of improvement in dementia risk prediction, the selected inflammation-related biomarkers were significantly associated with dementia outcomes and may serve as a starting point to further elucidate the pathogenesis of dementia.

Simulating and reporting frequentist operating characteristics of clinical trials that borrow external information: Towards a fair comparison in case of one-arm and hybrid control two-arm trials.

Borrowing information from historical or external data to inform inference in a current trial is an expanding field in the era of precision medicine, where trials are often performed in small patient cohorts for practical or ethical reasons. Even though methods proposed for borrowing from external data are mainly based on Bayesian approaches that incorporate external information into the prior for the current analysis, frequentist operating characteristics of the analysis strategy are often of interest. In particular, type I error rate and power at a prespecified point alternative are the focus. We propose a procedure to investigate and report the frequentist operating characteristics in this context. The approach evaluates type I error rate of the test with borrowing from external data and calibrates the test without borrowing to this type I error rate. On this basis, a fair comparison of power between the test with and without borrowing is achieved. We show that no power gains are possible in one-sided one-arm and two-arm hybrid control trials with normal endpoint, a finding proven in general before. We prove that in one-arm fixed-borrowing situations, unconditional power (i.e., when external data is random) is reduced. The Empirical Bayes power prior approach that dynamically borrows information according to the similarity of current and external data avoids the exorbitant type I error inflation occurring with fixed borrowing. In the hybrid control two-arm trial we observe power reductions as compared to the test calibrated to borrowing that increase when considering unconditional power.

Robust incorporation of historical information with known type I error rate inflation.

Bayesian clinical trials can benefit from available historical information through the specification of informative prior distributions. Concerns are however often raised about the potential for prior-data conflict and the impact of Bayes test decisions on frequentist operating characteristics, with particular attention being assigned to inflation of type I error (TIE) rates. This motivates the development of principled borrowing mechanisms, that strike a balance between frequentist and Bayesian decisions. Ideally, the trust assigned to historical information defines the degree of robustness to prior-data conflict one is willing to sacrifice. However, such relationship is often not directly available when explicitly considering inflation of TIE rates. We build on available literature relating frequentist and Bayesian test decisions, and investigate a rationale for inflation of TIE rate which explicitly and linearly relates the amount of borrowing and the amount of TIE rate inflation in one-arm studies. A novel dynamic borrowing mechanism tailored to hypothesis testing is additionally proposed. We show that, while dynamic borrowing prevents the possibility to obtain a simple closed-form TIE rate computation, an explicit upper bound can still be enforced. Connections with the robust mixture prior approach, particularly in relation to the choice of the mixture weight and robust component, are made. Simulations are performed to show the properties of the approach for normal and binomial outcomes, and an exemplary application is demonstrated in a case study.

A decision-theoretic approach to Bayesian clinical trial design and evaluation of robustness to prior-data conflict.

Bayesian clinical trials allow taking advantage of relevant external information through the elicitation of prior distributions, which influence Bayesian posterior parameter estimates and test decisions. However, incorporation of historical information can have harmful consequences on the trial's frequentist (conditional) operating characteristics in case of inconsistency between prior information and the newly collected data. A compromise between meaningful incorporation of historical information and strict control of frequentist error rates is therefore often sought. Our aim is thus to review and investigate the rationale and consequences of different approaches to relaxing strict frequentist control of error rates from a Bayesian decision-theoretic viewpoint. In particular, we define an integrated risk which incorporates losses arising from testing, estimation, and sampling. A weighted combination of the integrated risk addends arising from testing and estimation allows moving smoothly between these two targets. Furthermore, we explore different possible elicitations of the test error costs, leading to test decisions based either on posterior probabilities, or solely on Bayes factors. Sensitivity analyses are performed following the convention which makes a distinction between the prior of the data-generating process, and the analysis prior adopted to fit the data. Simulation in the case of normal and binomial outcomes and an application to a one-arm proof-of-concept trial, exemplify how such analysis can be conducted to explore sensitivity of the integrated risk, the operating characteristics, and the optimal sample size, to prior-data conflict. Robust analysis prior specifications, which gradually discount potentially conflicting prior information, are also included for comparison. Guidance with respect to cost elicitation, particularly in the context of a Phase II proof-of-concept trial, is provided.

On the feasibility of pediatric dose-finding trials in small samples with information from a preceding trial in adults.

We consider the case of pediatric dose-finding trials with extremely limited sample size. The operating characteristics of the standard design, the Continual Reassessment Method (CRM), are only well described for sample sizes of about 20 patients or more. In this simulation study, we assume the situation of a pediatric trial with only 10 patients and a preceding dose-finding trial in adults. Based on the adult data, we reduce the set of pediatric doses and formulate (partially) informative prior distributions for the pediatric trial. Our simulations show that such small pediatric dose-finding trials with robustified priors may provide sufficient operating characteristics.

Simulated clinical deployment of fully automatic deep learning for clinical prostate MRI assessment.

OBJECTIVES: To simulate clinical deployment, evaluate performance, and establish quality assurance of a deep learning algorithm (U-Net) for detection, localization, and segmentation of clinically significant prostate cancer (sPC), ISUP grade group ≥ 2, using bi-parametric MRI. METHODS: In 2017, 284 consecutive men in active surveillance, biopsy-naïve or pre-biopsied, received targeted and extended systematic MRI/transrectal US-fusion biopsy, after examination on a single MRI scanner (3 T). A prospective adjustment scheme was evaluated comparing the performance of the Prostate Imaging Reporting and Data System (PI-RADS) and U-Net using sensitivity, specificity, predictive values, and the Dice coefficient. RESULTS: In the 259 eligible men (median 64 [IQR 61-72] years), PI-RADS had a sensitivity of 98% [106/108]/84% [91/108] with a specificity of 17% [25/151]/58% [88/151], for thresholds at ≥ 3/≥ 4 respectively. U-Net using dynamic threshold adjustment had a sensitivity of 99% [107/108]/83% [90/108] (p > 0.99/> 0.99) with a specificity of 24% [36/151]/55% [83/151] (p > 0.99/> 0.99) for probability thresholds d3 and d4 emulating PI-RADS ≥ 3 and ≥ 4 decisions respectively, not statistically different from PI-RADS. Co-occurrence of a radiological PI-RADS ≥ 4 examination and U-Net ≥ d3 assessment significantly improved the positive predictive value from 59 to 63% (p = 0.03), on a per-patient basis. CONCLUSIONS: U-Net has similar performance to PI-RADS in simulated continued clinical use. Regular quality assurance should be implemented to ensure desired performance. KEY POINTS: • U-Net maintained similar diagnostic performance compared to radiological assessment of PI-RADS ≥ 4 when applied in a simulated clinical deployment. • Application of our proposed prospective dynamic calibration method successfully adjusted U-Net performance within acceptable limits of the PI-RADS reference over time, while not being limited to PI-RADS as a reference. • Simultaneous detection by U-Net and radiological assessment significantly improved the positive predictive value on a per-patient and per-lesion basis, while the negative predictive value remained unchanged.

External validation of novel magnetic resonance imaging-based models for prostate cancer prediction.

OBJECTIVES: To validate, in an external cohort, three novel risk models, including the recently updated European Randomized Study of Screening for Prostate Cancer (ERSPC) risk calculator, that combine multiparametric magnetic resonance imaging (mpMRI) and clinical variables to predict clinically significant prostate cancer (PCa). PATIENTS AND METHODS: We retrospectively analysed 307 men who underwent mpMRI prior to transperineal ultrasound fusion biopsy between October 2015 and July 2018 at two German centres. mpMRI was rated by Prostate Imaging Reporting and Data System (PI-RADS) v2.0 and clinically significant PCa was defined as International Society of Urological Pathology Gleason grade group ≥2. The prediction performance of the three models (MRI-ERSPC-3/4, and two risk models published by Radtke et al. and Distler et al., ModRad and ModDis) were compared using receiver-operating characteristic (ROC) curve analyses, with area under the ROC curve (AUC), calibration curve analyses and decision curves used to assess net benefit. RESULTS: The AUCs of the three novel models (MRI-ERSPC-3/4, ModRad and ModDis) were 0.82, 0.85 and 0.83, respectively. Calibration curve analyses showed the best intercept for MRI-ERSPC-3 and -4 of 0.35 and 0.76. Net benefit analyses indicated clear benefit of the MRI-ERSPC-3/4 risk models compared with the other two validated models. The MRI-ERSPC-3/4 risk models demonstrated a discrimination benefit for a risk threshold of up to 15% for clinically significant PCa as compared to the other risk models. CONCLUSION: In our external validation of three novel prostate cancer risk models, which incorporate mpMRI findings, a head-to-head comparison indicated that the MRI-ERSPC-3/4 risk model in particular could help to reduce unnecessary biopsies.

Quantification of prior impact in terms of effective current sample size.

Bayesian methods allow borrowing of historical information through prior distributions. The concept of prior effective sample size (prior ESS) facilitates quantification and communication of such prior information by equating it to a sample size. Prior information can arise from historical observations; thus, the traditional approach identifies the ESS with such a historical sample size. However, this measure is independent of newly observed data, and thus would not capture an actual "loss of information" induced by the prior in case of prior-data conflict. We build on a recent work to relate prior impact to the number of (virtual) samples from the current data model and introduce the effective current sample size (ECSS) of a prior, tailored to the application in Bayesian clinical trial designs. Special emphasis is put on robust mixture, power, and commensurate priors. We apply the approach to an adaptive design in which the number of recruited patients is adjusted depending on the effective sample size at an interim analysis. We argue that the ECSS is the appropriate measure in this case, as the aim is to save current (as opposed to historical) patients from recruitment. Furthermore, the ECSS can help overcome lack of consensus in the ESS assessment of mixture priors and can, more broadly, provide further insights into the impact of priors. An R package accompanies the paper.

Power gains by using external information in clinical trials are typically not possible when requiring strict type I error control.

In the era of precision medicine, novel designs are developed to deal with flexible clinical trials that incorporate many treatment strategies for multiple diseases in one trial setting. This situation often leads to small sample sizes in disease-treatment combinations and has fostered the discussion about the benefits of borrowing of external or historical information for decision-making in these trials. Several methods have been proposed that dynamically discount the amount of information borrowed from historical data based on the conformity between historical and current data. Specifically, Bayesian methods have been recommended and numerous investigations have been performed to characterize the properties of the various borrowing mechanisms with respect to the gain to be expected in the trials. However, there is common understanding that the risk of type I error inflation exists when information is borrowed and many simulation studies are carried out to quantify this effect. To add transparency to the debate, we show that if prior information is conditioned upon and a uniformly most powerful test exists, strict control of type I error implies that no power gain is possible under any mechanism of incorporation of prior information, including dynamic borrowing. The basis of the argument is to consider the test decision function as a function of the current data even when external information is included. We exemplify this finding in the case of a pediatric arm appended to an adult trial and dichotomous outcome for various methods of dynamic borrowing from adult information to the pediatric arm. In conclusion, if use of relevant external data is desired, the requirement of strict type I error control has to be replaced by more appropriate metrics.

Classification of Cancer at Prostate MRI: Deep Learning versus Clinical PI-RADS Assessment.

Background Men suspected of having clinically significant prostate cancer (sPC) increasingly undergo prostate MRI. The potential of deep learning to provide diagnostic support for human interpretation requires further evaluation. Purpose To compare the performance of clinical assessment to a deep learning system optimized for segmentation trained with T2-weighted and diffusion MRI in the task of detection and segmentation of lesions suspicious for sPC. Materials and Methods In this retrospective study, T2-weighted and diffusion prostate MRI sequences from consecutive men examined with a single 3.0-T MRI system between 2015 and 2016 were manually segmented. Ground truth was provided by combined targeted and extended systematic MRI-transrectal US fusion biopsy, with sPC defined as International Society of Urological Pathology Gleason grade group greater than or equal to 2. By using split-sample validation, U-Net was internally validated on the training set (80% of the data) through cross validation and subsequently externally validated on the test set (20% of the data). U-Net-derived sPC probability maps were calibrated by matching sextant-based cross-validation performance to clinical performance of Prostate Imaging Reporting and Data System (PI-RADS). Performance of PI-RADS and U-Net were compared by using sensitivities, specificities, predictive values, and Dice coefficient. Results A total of 312 men (median age, 64 years; interquartile range [IQR], 58-71 years) were evaluated. The training set consisted of 250 men (median age, 64 years; IQR, 58-71 years) and the test set of 62 men (median age, 64 years; IQR, 60-69 years). In the test set, PI-RADS cutoffs greater than or equal to 3 versus cutoffs greater than or equal to 4 on a per-patient basis had sensitivity of 96% (25 of 26) versus 88% (23 of 26) at specificity of 22% (eight of 36) versus 50% (18 of 36). U-Net at probability thresholds of greater than or equal to 0.22 versus greater than or equal to 0.33 had sensitivity of 96% (25 of 26) versus 92% (24 of 26) (both P > .99) with specificity of 31% (11 of 36) versus 47% (17 of 36) (both P > .99), not statistically different from PI-RADS. Dice coefficients were 0.89 for prostate and 0.35 for MRI lesion segmentation. In the test set, coincidence of PI-RADS greater than or equal to 4 with U-Net lesions improved the positive predictive value from 48% (28 of 58) to 67% (24 of 36) for U-Net probability thresholds greater than or equal to 0.33 (P = .01), while the negative predictive value remained unchanged (83% [25 of 30] vs 83% [43 of 52]; P > .99). Conclusion U-Net trained with T2-weighted and diffusion MRI achieves similar performance to clinical Prostate Imaging Reporting and Data System assessment. © RSNA, 2019 Online supplemental material is available for this article. See also the editorial by Padhani and Turkbey in this issue.

Histopathological to multiparametric MRI spatial mapping of extended systematic sextant and MR/TRUS-fusion-targeted biopsy of the prostate.

PURPOSE: MRI has limited ability to detect multifocal disease or the full extent of prostate involvement with clinically significant prostate cancer (sPC). We compare the spatial co-localization at sextant resolution of MRI lesions and histopathological mapping by combined targeted and extended systematic biopsies. MATERIALS AND METHODS: Sextants were mapped for sPC (ISUP group ≥ 2) by 24-core transperineal systematic biopsies in 316 patients with suspicion for sPC and by MR lesions of PI-RADS score of ≥ 3. The gold standard is combined systematic (median 23 cores) and targeted biopsies. RESULTS: Of 316 men, 121 (38%) harbored sPC. Of these 121 patients, 4 (3%) had a negative MRI. MRI correctly identified 117/121 (97%) patients with sPC. In these patients, mpMRI missed no additional sPC in 96 (82%), while MRI-negative sPC lesions were present in 21 patients (18%). Of 1896 sextants, 379 (20%) harbored sPC. MR-positive sextants contained sPC in 26% (337/1275), compared to 7% (42/621) in MR-negative sextants. On a patient basis, sensitivity was 0.97, specificity 0.22, positive predictive value 0.43, and negative predictive value 0.91. On a sextant basis, sensitivity was 0.73, specificity 0.38, positive predictive value 0.26, and negative predictive value 0.93. CONCLUSION: MpMRI mapping agreed well with histopathology with, at the observed sPC prevalence and on a patient basis, excellent sensitivity and negative predictive value, and acceptable specificity and positive predictive value for sPC. However, 18% of sPC was outside the mpMRI mapped region, quantifying limitations of MRI for complete localization of disease extent. KEY POINTS: • Currently, exclusive MRI mapping of the prostate for focal treatment planning cannot be recommended, as significant prostate cancer may remain untreated in a substantial number of cases. • At the observed sPC prevalence and on a patient basis, mpMRI has excellent sensitivity and NPV, and acceptable specificity and PPV for detection of prostate cancer, supporting its use to detect suspicious lesions before biopsy. • Despite the excellent global performance, 18% of sPC was outside the mpMRI mapped region even when a security margin of 10 mm was considered, indicating that prostate MRI has limited ability to completely map all cancer foci within the prostate.

Monitoring futility and efficacy in phase II trials with Bayesian posterior distributions-A calibration approach.

A multistage single arm phase II trial with binary endpoint is considered. Bayesian posterior probabilities are used to monitor futility in interim analyses and efficacy in the final analysis. For a beta-binomial model, decision rules based on Bayesian posterior probabilities are converted to "traditional" decision rules in terms of number of responders among patients observed so far. Analytical derivations are given for the probability of stopping for futility and for the probability to declare efficacy. A workflow is presented on how to select the parameters specifying the Bayesian design, and the operating characteristics of the design are investigated. It is outlined how the presented approach can be transferred to statistical models other than the beta-binomial model.

Antibodies against human papillomaviruses as diagnostic and prognostic biomarker in patients with neck squamous cell carcinoma from unknown primary tumor.

Treatment of patients with neck lymph node metastasis of squamous cell carcinoma (SCC) from unknown primary tumor (NSCCUP) is challenging due to the risk of missing occult tumors or inducing toxicity to unaffected sites. Human papillomavirus (HPV) is a promising biomarker given its causal link to oropharyngeal SCC and superior survival of patients with HPV-driven oropharyngeal SCC and NSCCUP. Identification of HPV-driven NSCCUP could focus diagnostic work-up and treatment on the oropharynx. For the first time, we assessed HPV antibodies and their prognostic value in NSCCUP patients. Antibodies against E6 and E7 (HPV16/18/31/33/35), E1 and E2 (HPV16/18) were assessed in 46 NSCCUP patients in sera collected at diagnosis, and in follow-up sera from five patients. In 28 patients, HPV tumor status was determined using molecular markers (HPV DNA, mRNA and cellular p16INK4a ). Thirteen (28%) NSCCUP patients were HPV-seropositive for HPV16, 18, 31, or 33. Of eleven patients with HPV-driven NSCCUP, ten were HPV-seropositive, while all 17 patients with non-HPV-driven NSCCUP were HPV-seronegative, resulting in 91% sensitivity (95% CI: 59-100%) and 100% specificity (95% CI: 80-100%). HPV antibody levels decreased after curative treatment. Recurrence was associated with increasing levels in an individual case. HPV-seropositive patients had a better overall and progression-free survival with hazard ratios of 0.09 (95% CI: 0.01-0.42) and 0.03 (95% CI: 0.002-0.18), respectively. For the first time, seropositivity to HPV proteins is described in NSCCUP patients, and high sensitivity and specificity for HPV-driven NSCCUP are demonstrated. HPV seropositivity appears to be a reliable diagnostic and prognostic biomarker for patients with HPV-driven NSCCUP.

Radiomic Machine Learning for Characterization of Prostate Lesions with MRI: Comparison to ADC Values.

Purpose To compare biparametric contrast-free radiomic machine learning (RML), mean apparent diffusion coefficient (ADC), and radiologist assessment for characterization of prostate lesions detected during prospective MRI interpretation. Materials and Methods This single-institution study included 316 men (mean age ± standard deviation, 64.0 years ± 7.8) with an indication for MRI-transrectal US fusion biopsy between May 2015 and September 2016 (training cohort, 183 patients; test cohort, 133 patients). Lesions identified by prospective clinical readings were manually segmented for mean ADC and radiomics analysis. Global and zone-specific random forest RML and mean ADC models for classification of clinically significant prostate cancer (Gleason grade group ≥ 2) were developed on the training set and the fixed models tested on an independent test set. Clinical readings, mean ADC, and radiomics were compared by using the McNemar test and receiver operating characteristic (ROC) analysis. Results In the test set, radiologist interpretation had a per-lesion sensitivity of 88% (53 of 60) and specificity of 50% (79 of 159). Quantitative measurement of the mean ADC (cut-off 732 mm2/sec) significantly reduced false-positive (FP) lesions from 80 to 60 (specificity 62% [99 of 159]) and false-negative (FN) lesions from seven to six (sensitivity 90% [54 of 60]) (P = .048). Radiologist interpretation had a per-patient sensitivity of 89% (40 of 45) and specificity of 43% (38 of 88). Quantitative measurement of the mean ADC reduced the number of patients with FP lesions from 50 to 43 (specificity 51% [45 of 88]) and the number of patients with FN lesions from five to three (sensitivity 93% [42 of 45]) (P = .496). Comparison of the area under the ROC curve (AUC) for the mean ADC (AUCglobal = 0.84; AUCzone-specific ≤ 0.87) vs the RML (AUCglobal = 0.88, P = .176; AUCzone-specific ≤ 0.89, P ≥ .493) showed no significantly different performance. Conclusion Quantitative measurement of the mean apparent diffusion coefficient (ADC) improved differentiation of benign versus malignant prostate lesions, compared with clinical assessment. Radiomic machine learning had comparable but not better performance than mean ADC assessment. © RSNA, 2018 Online supplemental material is available for this article.

Sensitivity and specificity of antibodies against HPV16 E6 and other early proteins for the detection of HPV16-driven oropharyngeal squamous cell carcinoma.

To determine the sensitivity and specificity of HPV16 serology as diagnostic marker for HPV16-driven oropharyngeal squamous cell carcinoma (OPSCC), 214 HNSCC patients from Germany and Italy with fresh-frozen tumor tissues and sera collected before treatment were included in this study. Hundred and twenty cancer cases were from the oropharynx and 94 were from head and neck cancer regions outside the oropharynx (45 oral cavity, 12 hypopharynx and 35 larynx). Serum antibodies to early (E1, E2, E6 and E7) and late (L1) HPV16 proteins were analyzed by multiplex serology and were compared to tumor HPV RNA status as the gold standard. A tumor was defined as HPV-driven in the presence of HPV16 DNA and HPV16 transformation-specific RNA transcript patterns (E6*I, E1∧ E4 and E1C). Of 120 OPSCC, 66 (55%) were HPV16-driven. HPV16 E6 seropositivity was the best predictor of HPV16-driven OPSCC (diagnostic accuracy 97% [95%CI 92-99%], Cohen's kappa 0.93 [95%CI 0.8-1.0]). Of the 66 HPV-driven OPSCC, 63 were HPV16 E6 seropositive, compared to only one (1.8%) among the 54 non-HPV-driven OPSCC, resulting in a sensitivity of 96% (95%CI 88-98) and a specificity of 98% (95%CI 90-100). Of 94 HNSCC outside the oropharynx, six (6%) were HPV16-driven. In these patients, HPV16 E6 seropositivity had lower sensitivity (50%, 95%CI 19-81), but was highly specific (100%, 95%CI 96-100). In conclusion, HPV16 E6 seropositivity appears to be a highly reliable diagnostic marker for HPV16-driven OPSCC with very high sensitivity and specificity, but might be less sensitive for HPV16-driven HNSCC outside the oropharynx.

Prediction of malignancy by a radiomic signature from contrast agent-free diffusion MRI in suspicious breast lesions found on screening mammography.

PURPOSE: To assess radiomics as a tool to determine how well lesions found suspicious on breast cancer screening X-ray mammography can be categorized into malignant and benign with unenhanced magnetic resonance (MR) mammography with diffusion-weighted imaging and T2 -weighted sequences. MATERIALS AND METHODS: From an asymptomatic screening cohort, 50 women with mammographically suspicious findings were examined with contrast-enhanced breast MRI (ceMRI) at 1.5T. Out of this protocol an unenhanced, abbreviated diffusion-weighted imaging protocol (ueMRI) including T2 -weighted, (T2 w), diffusion-weighted imaging (DWI), and DWI with background suppression (DWIBS) sequences and corresponding apparent diffusion coefficient (ADC) maps were extracted. From ueMRI-derived radiomic features, three Lasso-supervised machine-learning classifiers were constructed and compared with the clinical performance of a highly experienced radiologist: 1) univariate mean ADC model, 2) unconstrained radiomic model, 3) constrained radiomic model with mandatory inclusion of mean ADC. RESULTS: The unconstrained and constrained radiomic classifiers consisted of 11 parameters each and achieved differentiation of malignant from benign lesions with a .632 + bootstrap receiver operating characteristics (ROC) area under the curve (AUC) of 84.2%/85.1%, compared to 77.4% for mean ADC and 95.9%/95.9% for the experienced radiologist using ceMRI/ueMRI. CONCLUSION: In this pilot study we identified two ueMRI radiomics classifiers that performed well in the differentiation of malignant from benign lesions and achieved higher performance than the mean ADC parameter alone. Classification was lower than the almost perfect performance of a highly experienced breast radiologist. The potential of radiomics to provide a training-independent diagnostic decision tool is indicated. A performance reaching the human expert would be highly desirable and based on our results is considered possible when the concept is extended in larger cohorts with further development and validation of the technique. LEVEL OF EVIDENCE: 1 Technical Efficacy: Stage 2 J. MAGN. RESON. IMAGING 2017;46:604-616.

Bovine meat and milk factor protein expression in tumor-free mucosa of colorectal cancer patients coincides with macrophages and might interfere with patient survival.

Bovine milk and meat factors (BMMFs) are plasmid-like DNA molecules isolated from bovine milk and serum, as well as the peritumor of colorectal cancer (CRC) patients. BMMFs have been proposed as zoonotic infectious agents and drivers of indirect carcinogenesis of CRC, inducing chronic tissue inflammation, radical formation and increased levels of DNA damage. Data on expression of BMMFs in large clinical cohorts to test an association with co-markers and clinical parameters were not previously available and were therefore assessed in this study. Tissue sections with paired tumor-adjacent mucosa and tumor tissues of CRC patients [individual cohorts and tissue microarrays (TMAs) (n = 246)], low-/high-grade dysplasia (LGD/HGD) and mucosa of healthy donors were used for immunohistochemical quantification of the expression of BMMF replication protein (Rep) and CD68/CD163 (macrophages) by co-immunofluorescence microscopy and immunohistochemical scoring (TMA). Rep was expressed in the tumor-adjacent mucosa of 99% of CRC patients (TMA), was histologically associated with CD68+ /CD163+ macrophages and was increased in CRC patients when compared to healthy controls. Tumor tissues showed only low stromal Rep expression. Rep was expressed in LGD and less in HGD but was strongly expressed in LGD/HGD-adjacent tissues. Albeit not reaching statistical significance, incidence curves for CRC-specific death were increased for higher Rep expression (TMA), with high tumor-adjacent Rep expression being linked to the highest incidence of death. BMMF Rep expression might represent a marker and early risk factor for CRC. The correlation between Rep and CD68 expression supports a previous hypothesis that BMMF-specific inflammatory regulations, including macrophages, are involved in the pathogenesis of CRC.

Detection of Clinically Significant Prostate Cancer Using Targeted Biopsy with Four Cores Versus Target Saturation Biopsy with Nine Cores in Transperineal Prostate Fusion Biopsy: A Prospective Randomized Trial.

BACKGROUND: Multiparametric magnetic resonance imaging (mpMRI) and targeted biopsy (TB) facilitate accurate detection of clinically significant prostate cancer (csPC). However, it remains unclear how targeted cores should be applied for accurate diagnosis of csPC. OBJECTIVE: To assess csPC detection rates for two target-directed MRI/transrectal ultrasonography (TRUS) fusion biopsy approaches, conventional TB and target saturation biopsy (TS). DESIGN, SETTING, AND PARTICIPANTS: This was a prospective single-center study of outcomes for transperineal MRI/TRUS fusion biopsies for 170 men. Half of the men (n = 85) were randomized to conventional TB with four cores per lesion and half (n = 85) to TS with nine cores. Biopsies were performed by three experienced board-certified urologists. OUTCOME MEASUREMENTS AND STATISTICAL ANALYSIS: PC and csPC (International Society of Urological Pathology grade group ≥2) detection rates for systematic biopsy (SB), TB, and TS were analyzed using McNemar's test for intrapatient comparisons and Fisher's exact test for TS versus TB. A combination of targeted biopsy (TS or TB) and SB served as the reference. RESULTS AND LIMITATIONS: According to the reference, csPC was diagnosed for 57 men in the TS group and 36 men in the TB group. Of these, TS detected 57/57 csPC cases and TB detected 33/36 csPC cases (p = 0.058). Detection of Gleason grade group 1 disease was 10/12 cases with TS and 8/17 cases with TB (p = 0.055). In addition, TS detected 97% of 63 csPC lesions, compared to 86% with TB (p = 0.1). Limitations include the single-center design, the limited generalizability owing to the transperineal biopsy route, the lack of central review of pathology and radical prostatectomy correlation, and uneven distributions of csPC prevalence, Prostate Imaging-Reporting and Data System (PI-RADS) 5 lesions, men with two or more PI-RADS ≥3 lesions, and prostate-specific antigen density between the groups, which may have affected the results. CONCLUSIONS: In our study, rates of csPC detection did not significantly differ between TS and TB. PATIENT SUMMARY: In this study, we investigated two targeted approaches for taking prostate biopsy samples after observation of suspicious lesions on prostate scans. We found that the rates of detection of prostate cancer did not significantly differ between the two approaches.

Beyond rankings: Learning (more) from algorithm validation.

Challenges have become the state-of-the-art approach to benchmark image analysis algorithms in a comparative manner. While the validation on identical data sets was a great step forward, results analysis is often restricted to pure ranking tables, leaving relevant questions unanswered. Specifically, little effort has been put into the systematic investigation on what characterizes images in which state-of-the-art algorithms fail. To address this gap in the literature, we (1) present a statistical framework for learning from challenges and (2) instantiate it for the specific task of instrument instance segmentation in laparoscopic videos. Our framework relies on the semantic meta data annotation of images, which serves as foundation for a General Linear Mixed Models (GLMM) analysis. Based on 51,542 meta data annotations performed on 2,728 images, we applied our approach to the results of the Robust Medical Instrument Segmentation Challenge (ROBUST-MIS) challenge 2019 and revealed underexposure, motion and occlusion of instruments as well as the presence of smoke or other objects in the background as major sources of algorithm failure. Our subsequent method development, tailored to the specific remaining issues, yielded a deep learning model with state-of-the-art overall performance and specific strengths in the processing of images in which previous methods tended to fail. Due to the objectivity and generic applicability of our approach, it could become a valuable tool for validation in the field of medical image analysis and beyond.