Evaluating the risk of ovarian cancer before surgery using the ADNEX model to differentiate between benign, borderline, early and advanced stage invasive, and secondary metastatic tumours: prospective multicentre diagnostic study.

OBJECTIVES: To develop a risk prediction model to preoperatively discriminate between benign, borderline, stage I invasive, stage II-IV invasive, and secondary metastatic ovarian tumours. DESIGN: Observational diagnostic study using prospectively collected clinical and ultrasound data. SETTING: 24 ultrasound centres in 10 countries. PARTICIPANTS: Women with an ovarian (including para-ovarian and tubal) mass and who underwent a standardised ultrasound examination before surgery. The model was developed on 3506 patients recruited between 1999 and 2007, temporally validated on 2403 patients recruited between 2009 and 2012, and then updated on all 5909 patients. MAIN OUTCOME MEASURES: Histological classification and surgical staging of the mass. RESULTS: The Assessment of Different NEoplasias in the adneXa (ADNEX) model contains three clinical and six ultrasound predictors: age, serum CA-125 level, type of centre (oncology centres v other hospitals), maximum diameter of lesion, proportion of solid tissue, more than 10 cyst locules, number of papillary projections, acoustic shadows, and ascites. The area under the receiver operating characteristic curve (AUC) for the classic discrimination between benign and malignant tumours was 0.94 (0.93 to 0.95) on temporal validation. The AUC was 0.85 for benign versus borderline, 0.92 for benign versus stage I cancer, 0.99 for benign versus stage II-IV cancer, and 0.95 for benign versus secondary metastatic. AUCs between malignant subtypes varied between 0.71 and 0.95, with an AUC of 0.75 for borderline versus stage I cancer and 0.82 for stage II-IV versus secondary metastatic. Calibration curves showed that the estimated risks were accurate. CONCLUSIONS: The ADNEX model discriminates well between benign and malignant tumours and offers fair to excellent discrimination between four types of ovarian malignancy. The use of ADNEX has the potential to improve triage and management decisions and so reduce morbidity and mortality associated with adnexal pathology.

Prognostic value of serum HE4 levels and risk of ovarian malignancy algorithm scores at the time of ovarian cancer diagnosis.

OBJECTIVES: The aim of this study is to assess whether the pretreatment serum HE4 levels or the Risk of Ovarian Malignancy Algorithm (ROMA) scores at the time of initial diagnosis are associated with progression-free survival (PFS) and disease-specific survival (DSS) in patients with ovarian cancer receiving either primary debulking surgery or neoadjuvant chemotherapy followed by interval debulking surgery. METHODS: A survival analysis of 101 cases of invasive ovarian cancer recruited in a previous diagnostic accuracy study was conducted from 2005 to 2009 at the University Hospital KU Leuven, Belgium. Serum HE4 levels (pM) and ROMA scores (%) were obtained before primary treatment. Dates of death were obtained by record linkage with patient hospital files. Progression was evaluated according to the Response Evaluation Criteria in Solid Tumors. Adjusted hazards ratios (HRs) were estimated using multivariable Cox regression. RESULTS: Eighty patients (79%) with invasive ovarian cancer underwent primary debulking surgery, whereas 21 (21%) received neoadjuvant chemotherapy. The median DSS was 3.72 years (95% confidence interval [CI], 3.19-4.07). Fifty-two patients (51%) died of disease, and 74 patients (73%) had progressive disease during follow-up. On univariable analysis, elevated pretreatment HE4 levels and ROMA scores were related to worse prognosis. However, after the adjustment for classic prognostic variables, HE4 levels (log2-transformed) and ROMA scores were unrelated to DSS (log-2 HE4: adjusted HR, 1.01; 95% CI, 0.84-1.21 and ROMA: adjusted HR per 10% increase, 0.96; 95% CI, 0.84-1.12) and PFS (log-2 HE4: adjusted HR, 0.98; 95% CI, 0.84-1.13 and ROMA: adjusted HR per 10% increase, 0.98; 95% CI, 0.88-1.11). CONCLUSIONS: Pretreatment HE4 levels and ROMA scores are not independent prognostic factors for DSS and PFS after multivariable adjustment in patients with ovarian cancer.

White box radial basis function classifiers with component selection for clinical prediction models.

OBJECTIVE: To propose a new flexible and sparse classifier that results in interpretable decision support systems. METHODS: Support vector machines (SVMs) for classification are very powerful methods to obtain classifiers for complex problems. Although the performance of these methods is consistently high and non-linearities and interactions between variables can be handled efficiently when using non-linear kernels such as the radial basis function (RBF) kernel, their use in domains where interpretability is an issue is hampered by their lack of transparency. Many feature selection algorithms have been developed to allow for some interpretation but the impact of the different input variables on the prediction still remains unclear. Alternative models using additive kernels are restricted to main effects, reducing their usefulness in many applications. This paper proposes a new approach to expand the RBF kernel into interpretable and visualizable components, including main and two-way interaction effects. In order to obtain a sparse model representation, an iterative l1-regularized parametric model using the interpretable components as inputs is proposed. RESULTS: Results on toy problems illustrate the ability of the method to select the correct contributions and an improved performance over standard RBF classifiers in the presence of irrelevant input variables. For a 10-dimensional x-or problem, an SVM using the standard RBF kernel obtains an area under the receiver operating characteristic curve (AUC) of 0.947, whereas the proposed method achieves an AUC of 0.997. The latter additionally identifies the relevant components. In a second 10-dimensional artificial problem, the underlying class probability follows a logistic regression model. An SVM with the RBF kernel results in an AUC of 0.975, as apposed to 0.994 for the presented method. The proposed method is applied to two benchmark datasets: the Pima Indian diabetes and the Wisconsin Breast Cancer dataset. The AUC is in both cases comparable to those of the standard method (0.826 versus 0.826 and 0.990 versus 0.996) and those reported in the literature. The selected components are consistent with different approaches reported in other work. However, this method is able to visualize the effect of each of the components, allowing for interpretation of the learned logic by experts in the application domain. CONCLUSIONS: This work proposes a new method to obtain flexible and sparse risk prediction models. The proposed method performs as well as a support vector machine using the standard RBF kernel, but has the additional advantage that the resulting model can be interpreted by experts in the application domain.

Accurate prediction of pregnancy viability by means of a simple scoring system.

STUDY QUESTION: What is the performance of a simple scoring system to predict whether women will have an ongoing viable intrauterine pregnancy beyond the first trimester? SUMMARY ANSWER: A simple scoring system using demographic and initial ultrasound variables accurately predicts pregnancy viability beyond the first trimester with an area under the curve (AUC) in a receiver operating characteristic curve of 0.924 [95% confidence interval (CI) 0.900-0.947] on an independent test set. WHAT IS KNOWN ALREADY: Individual demographic and ultrasound factors, such as maternal age, vaginal bleeding and gestational sac size, are strong predictors of miscarriage. Previous mathematical models have combined individual risk factors with reasonable performance. A simple scoring system derived from a mathematical model that can be easily implemented in clinical practice has not previously been described for the prediction of ongoing viability. STUDY DESIGN, SIZE AND DURATION: This was a prospective observational study in a single early pregnancy assessment centre during a 9-month period. PARTICIPANTS/MATERIALS, SETTING AND METHODS: A cohort of 1881 consecutive women undergoing transvaginal ultrasound scan at a gestational age <84 days were included. Women were excluded if the first trimester outcome was not known. Demographic features, symptoms and ultrasound variables were tested for their influence on ongoing viability. Logistic regression was used to determine the influence on first trimester viability from demographics and symptoms alone, ultrasound findings alone and then from all the variables combined. Each model was developed on a training data set, and a simple scoring system was derived from this. This scoring system was tested on an independent test data set. MAIN RESULTS AND THE ROLE OF CHANCE: The final outcome based on a total of 1435 participants was an ongoing viable pregnancy in 885 (61.7%) and early pregnancy loss in 550 (38.3%) women. The scoring system using significant demographic variables alone (maternal age and amount of bleeding) to predict ongoing viability gave an AUC of 0.724 (95% CI = 0.692-0.756) in the training set and 0.729 (95% CI = 0.684-0.774) in the test set. The scoring system using significant ultrasound variables alone (mean gestation sac diameter, mean yolk sac diameter and the presence of fetal heart beat) gave an AUC of 0.873 (95% CI = 0.850-0.897) and 0.900 (95% CI = 0.871-0.928) in the training and the test sets, respectively. The final scoring system using demographic and ultrasound variables together gave an AUC of 0.901 (95% CI = 0.881-0.920) and 0.924 (CI = 0.900-0.947) in the training and the test sets, respectively. After defining the cut-off at which the sensitivity is 0.90 on the training set, this model performed with a sensitivity of 0.92, specificity of 0.73, positive predictive value of 84.7% and negative predictive value of 85.4% in the test set. LIMITATIONS, REASONS FOR CAUTION: BMI and smoking variables were a potential omission in the data collection and might further improve the model performance if included. A further limitation is the absence of information on either bleeding or pain in 18% of women. Caution should be exercised before implementation of this scoring system prior to further external validation studies WIDER IMPLICATIONS OF THE FINDINGS: This simple scoring system incorporates readily available data that are routinely collected in clinical practice and does not rely on complex data entry. As such it could, unlike most mathematical models, be easily incorporated into normal early pregnancy care, where women may appreciate an individualized calculation of the likelihood of ongoing pregnancy viability. STUDY FUNDING/COMPETING INTEREST(S): Research by V.V.B. supported by Research Council KUL: GOA MaNet, PFV/10/002 (OPTEC), several PhD/postdoc & fellow grants; IWT: TBM070706-IOTA3, PhD Grants; IBBT; Belgian Federal Science Policy Office: IUAP P7/(DYSCO, `Dynamical systems, control and optimization', 2012-2017). T.B. is supported by the Imperial Healthcare NHS Trust NIHR Biomedical Research Centre. TRIAL REGISTRATION NUMBER: Not applicable.

Assessing the discriminative ability of risk models for more than two outcome categories.

The discriminative ability of risk models for dichotomous outcomes is often evaluated with the concordance index (c-index). However, many medical prediction problems are polytomous, meaning that more than two outcome categories need to be predicted. Unfortunately such problems are often dichotomized in prediction research. We present a perspective on the evaluation of discriminative ability of polytomous risk models, which may instigate researchers to consider polytomous prediction models more often. First, we suggest a "discrimination plot" as a tool to visualize the model's discriminative ability. Second, we discuss the use of one overall polytomous c-index versus a set of dichotomous measures to summarize the performance of the model. Third, we address several aspects to consider when constructing a polytomous c-index. These involve the assessment of concordance in pairs versus sets of patients, weighting by outcome prevalence, the value related to models with random performance, the reduction to the dichotomous c-index for dichotomous problems, and interpretation. We illustrate these issues on case studies dealing with ovarian cancer (four outcome categories) and testicular cancer (three categories). We recommend the use of a discrimination plot together with an overall c-index such as the Polytomous Discrimination Index. If the overall c-index suggests that the model has relevant discriminative ability, pairwise c-indexes for each pair of outcome categories are informative. For pairwise c-indexes we recommend the 'conditional-risk' method which is consistent with the analytical approach of the multinomial logistic regression used to develop polytomous risk models.

Extending the c-statistic to nominal polytomous outcomes: the Polytomous Discrimination Index.

Diagnostic problems in medicine are sometimes polytomous, meaning that the outcome has more than two distinct categories. For example, ovarian tumors can be benign, borderline, primary invasive, or metastatic. Extending the main measure of binary discrimination, the c-statistic or area under the ROC curve, to nominal polytomous settings is not straightforward. This paper reviews existing measures and presents the polytomous discrimination index (PDI) as an alternative. The PDI assesses all sets of k cases consisting of one case from each outcome category. For each category i (i = 1, … ,k), it is assessed whether the risk of category i is highest for the case from category i. A score of 1∕k is given per category for which this holds, yielding a set score between 0 and 1 to indicate the level of discrimination. The PDI is the average set score and is interpreted as the probability to correctly identify a case from a randomly selected category within a set of k cases. This probability can be split up by outcome category, yielding k category-specific values that result in the PDI when averaged. We demonstrate the measures on two diagnostic problems (residual mass histology after chemotherapy for testicular cancer; diagnosis of ovarian tumors). We compare the behavior of the measures on theoretical data, showing that PDI is more strongly influenced by simultaneous discrimination between all categories than by partial discrimination between pairs of categories. In conclusion, the PDI is attractive because it better matches the requirements of a measure to summarize polytomous discrimination.

A mathematical model for interpretable clinical decision support with applications in gynecology.

BACKGROUND: Over time, methods for the development of clinical decision support (CDS) systems have evolved from interpretable and easy-to-use scoring systems to very complex and non-interpretable mathematical models. In order to accomplish effective decision support, CDS systems should provide information on how the model arrives at a certain decision. To address the issue of incompatibility between performance, interpretability and applicability of CDS systems, this paper proposes an innovative model structure, automatically leading to interpretable and easily applicable models. The resulting models can be used to guide clinicians when deciding upon the appropriate treatment, estimating patient-specific risks and to improve communication with patients. METHODS AND FINDINGS: We propose the interval coded scoring (ICS) system, which imposes that the effect of each variable on the estimated risk is constant within consecutive intervals. The number and position of the intervals are automatically obtained by solving an optimization problem, which additionally performs variable selection. The resulting model can be visualised by means of appealing scoring tables and color bars. ICS models can be used within software packages, in smartphone applications, or on paper, which is particularly useful for bedside medicine and home-monitoring. The ICS approach is illustrated on two gynecological problems: diagnosis of malignancy of ovarian tumors using a dataset containing 3,511 patients, and prediction of first trimester viability of pregnancies using a dataset of 1,435 women. Comparison of the performance of the ICS approach with a range of prediction models proposed in the literature illustrates the ability of ICS to combine optimal performance with the interpretability of simple scoring systems. CONCLUSIONS: The ICS approach can improve patient-clinician communication and will provide additional insights in the importance and influence of available variables. Future challenges include extensions of the proposed methodology towards automated detection of interaction effects, multi-class decision support systems, prognosis and high-dimensional data.

Support vector methods for survival analysis: a comparison between ranking and regression approaches.

OBJECTIVE: To compare and evaluate ranking, regression and combined machine learning approaches for the analysis of survival data. METHODS: The literature describes two approaches based on support vector machines to deal with censored observations. In the first approach the key idea is to rephrase the task as a ranking problem via the concordance index, a problem which can be solved efficiently in a context of structural risk minimization and convex optimization techniques. In a second approach, one uses a regression approach, dealing with censoring by means of inequality constraints. The goal of this paper is then twofold: (i) introducing a new model combining the ranking and regression strategy, which retains the link with existing survival models such as the proportional hazards model via transformation models; and (ii) comparison of the three techniques on 6 clinical and 3 high-dimensional datasets and discussing the relevance of these techniques over classical approaches fur survival data. RESULTS: We compare svm-based survival models based on ranking constraints, based on regression constraints and models based on both ranking and regression constraints. The performance of the models is compared by means of three different measures: (i) the concordance index, measuring the model's discriminating ability; (ii) the logrank test statistic, indicating whether patients with a prognostic index lower than the median prognostic index have a significant different survival than patients with a prognostic index higher than the median; and (iii) the hazard ratio after normalization to restrict the prognostic index between 0 and 1. Our results indicate a significantly better performance for models including regression constraints above models only based on ranking constraints. CONCLUSIONS: This work gives empirical evidence that svm-based models using regression constraints perform significantly better than svm-based models based on ranking constraints. Our experiments show a comparable performance for methods including only regression or both regression and ranking constraints on clinical data. On high dimensional data, the former model performs better. However, this approach does not have a theoretical link with standard statistical models for survival data. This link can be made by means of transformation models when ranking constraints are included.

Qualitative assessment of the progesterone receptor and HER2 improves the Nottingham Prognostic Index up to 5 years after breast cancer diagnosis.

PURPOSE: To investigate whether the estrogen receptor (ER), progesterone receptor (PR), and human epidermal growth factor receptor 2 (HER2) can improve the Nottingham Prognostic Index (NPI) in the classification of patients with primary operable breast cancer for disease-free survival (DFS). PATIENTS AND METHODS: The analysis is based on 1,927 patients with breast cancer treated between 2000 and 2005 at the University Hospitals, Leuven. We compared performances of NPI with and without ER, PR and/or HER2. Validation was done on two external data sets containing 862 and 2,805 patients from Oslo (Norway) and Auckland (New Zealand), respectively. RESULTS: In the Leuven cohort, median follow-up was 66 months, and 13.7% of patients experienced a breast cancer-related event. Positive staining for ER, PR, and HER2 was detected, respectively, in 86.9%, 75.5%, and 11.9% of patients. Based on multivariate Cox regression modeling, the improved NPI (iNPI) was derived as NPI - PR positivity + HER2 positivity. Validation results showed a risk group reclassification of 20% to 30% of patients when using iNPI with its optimal risk boundaries versus NPI, in a majority of patients to more appropriate risk groups. An additional 10% of patients were classified into the extreme risk groups, where clinical actions are less ambiguous. Survival curves of reclassified patients resembled more closely those for patients in the same iNPI group than those for patients in the same NPI group. CONCLUSION: The addition of PR and HER2 to NPI increases its 5-year prognostic accuracy. The iNPI can be considered as a clinically useful tool for stratification of patients with breast cancer receiving standard of care.

Short-term outcome of primary operated early breast cancer by hormone and HER-2 receptors.

INTRODUCTION: Prognostic subgroup classification of operable breast cancers using cDNA clustering of breast cancer-related genes resembles the classification based on the combined immunohistochemical (IHC) expression of the hormone and HER-2 receptors. We here report the short-term disease-free interval (DFI) of operable breast cancers by their joint hormone receptor/HER-2 phenotype. PATIENTS AND METHODS: Short-term follow-up (FU) of a prospective cohort of 1,958 breast-cancer patients primary operated at our institution between 2000 and 2005. Receptors were evaluated using IHC. Steroid receptors were considered positive for any nuclear staining; HER-2 for strong (3+) membrane staining or positive fluorescence in situ hybridization (FISH). Kaplan-Meier (KM) DFI curves were calculated for any relapse defined as a local, regional, contralateral, or distant breast cancer event for the six predefined breast cancer subgroups: ER + PR + HER-2 - (PPN), ER + PR - HER-2 - (PNN), ER + PR + HER-2 + (PPP), ER - PR - HER-2 - (NNN), ER - PR - HER-2 + (NNP), and ER + PR - HER-2 + (PNP). P-values were calculated for comparison of the six different survival curves using two possible adaptations for multiple testing. A multivariate model for the receptors predicting DFI did incorporate local and systemic adjuvant therapy. RESULTS: Median patient age was 57 years (ranges 26-96) and median FU was 3.35 years. Overall, DFI at median FU was 91%; 94% for PPN, 89% for PNN, 86% for NNN, 81% for PPP, 80% for PNP, and 76% for NNP cases. Some receptor subgroups had a significantly better DFI than others based on multiple testing, especially when the PPN group was compared against the four most frequent subtypes. The multivariate model with local and systemic adjuvant therapy confirmed the prognostic value of ER, PR, and HER-2 for short-term DFI. CONCLUSION: It is possible to distinguish short-term prognostic breast cancer subgroups only on the basis of ER, PR, and HER-2 even when stratified for local and systemic adjuvant therapy. While gene expression profiles based on microarray data of over hundreds of genes will probably teach us much about breast cancer biology, heterogeneity, and prognosis, we emphasize the important short-term prognostic value of currently used IHC markers for ER, PR, and HER-2.

Plasma MMP1 and MMP8 expression in breast cancer: protective role of MMP8 against lymph node metastasis.

BACKGROUND: Elevated levels of matrix metalloproteinases have been found to associate with poor prognosis in various carcinomas. This study aimed at evaluating plasma levels of MMP1, MMP8 and MMP13 as diagnostic and prognostic markers of breast cancer. METHODS: A total of 208 breast cancer patients, of which 21 with inflammatory breast cancer, and 42 healthy controls were included. Plasma MMP1, MMP8 and MMP13 levels were measured using ELISA and correlated with clinicopathological characteristics. RESULTS: Median plasma MMP1 levels were higher in controls than in breast cancer patients (3.45 vs. 2.01 ng/ml), while no difference was found for MMP8 (10.74 vs. 10.49 ng/ml). ROC analysis for MMP1 revealed an AUC of 0.67, sensitivity of 80% and specificity of 24% at a cut-off value of 4.24 ng/ml. Plasma MMP13 expression could not be detected. No correlation was found between MMP1 and MMP8 levels. We found a trend of lower MMP1 levels with increasing tumour size (p = 0.07); and higher MMP8 levels with premenopausal status (p = 0.06) and NPI (p = 0.04). The median plasma MMP1 (p = 0.02) and MMP8 (p = 0.007) levels in the non-inflammatory breast cancer patients were almost twice as high as those found in the inflammatory breast cancer patients. Intriguingly, plasma MMP8 levels were positively associated with lymph node involvement but showed a negative correlation with the risk of distant metastasis. Both controls and lymph node negative patients (pN0) had lower MMP8 levels than patients with moderate lymph node involvement (pN1, pN2) (p = 0.001); and showed a trend for higher MMP8 levels compared to patients with extensive lymph node involvement (pN3) and a strong predisposition to distant metastasis (p = 0.11). Based on the hypothesis that blood and tissue protein levels are in reverse association, these results suggest that MMP8 in the tumour may have a protective effect against lymph node metastasis. CONCLUSION: In summary, we observed differences in MMP1 and MMP8 plasma levels between healthy controls and breast cancer patients as well as between breast cancer patients. Interestingly, our results suggest that MMP8 may affect the metastatic behaviour of breast cancer cells through protection against lymph node metastasis, underlining the importance of anti-target identification in drug development.