Estimating risk of endometrial malignancy and other intracavitary uterine pathology in women without abnormal uterine bleeding using IETA-1 multinomial regression model: validation study.

OBJECTIVES: To assess the ability of the International Endometrial Tumor Analysis (IETA)-1 polynomial regression model to estimate the risk of endometrial cancer (EC) and other intracavitary uterine pathology in women without abnormal uterine bleeding. METHODS: This was a retrospective study, in which we validated the IETA-1 model on the IETA-3 study cohort (n = 1745). The IETA-3 study is a prospective observational multicenter study. It includes women without vaginal bleeding who underwent a standardized transvaginal ultrasound examination in one of seven ultrasound centers between January 2011 and December 2018. The ultrasonography was performed either as part of a routine gynecological examination, during follow-up of non-endometrial pathology, in the work-up before fertility treatment or before treatment for uterine prolapse or ovarian pathology. Ultrasonographic findings were described using IETA terminology and were compared with histology, or with results of clinical and ultrasound follow-up of at least 1 year if endometrial sampling was not performed. The IETA-1 model, which was created using data from patients with abnormal uterine bleeding, predicts four histological outcomes: (1) EC or endometrial intraepithelial neoplasia (EIN); (2) endometrial polyp or intracavitary myoma; (3) proliferative or secretory endometrium, endometritis, or endometrial hyperplasia without atypia; and (4) endometrial atrophy. The predictors in the model are age, body mass index and seven ultrasound variables (visibility of the endometrium, endometrial thickness, color score, cysts in the endometrium, non-uniform echogenicity of the endometrium, presence of a bright edge, presence of a single dominant vessel). We analyzed the discriminative ability of the model (area under the receiver-operating-characteristics curve (AUC); polytomous discrimination index (PDI)) and evaluated calibration of its risk estimates (observed/expected ratio). RESULTS: The median age of the women in the IETA-3 cohort was 51 (range, 20-85) years and 51% (887/1745) of the women were postmenopausal. Histology showed EC or EIN in 29 (2%) women, endometrial polyps or intracavitary myomas in 1094 (63%), proliferative or secretory endometrium, endometritis, or hyperplasia without atypia in 144 (8%) and endometrial atrophy in 265 (15%) women. The endometrial sample had insufficient material in five (0.3%) cases. In 208 (12%) women who did not undergo endometrial sampling but were followed up for at least 1 year without clinical or ultrasound signs of endometrial malignancy, the outcome was classified as benign. The IETA-1 model had an AUC of 0.81 (95% CI, 0.73-0.89, n = 1745) for discrimination between malignant (EC or EIN) and benign endometrium, and the observed/expected ratio for EC or EIN was 0.51 (95% CI, 0.32-0.82). The model was able to categorize the four histological outcomes with considerable accuracy: the PDI of the model was 0.68 (95% CI, 0.62-0.73) (n = 1532). The IETA-1 model discriminated very well between endometrial atrophy and all other intracavitary uterine conditions, with an AUC of 0.96 (95% CI, 0.95-0.98). Including only patients in whom the endometrium was measurable (n = 1689), the model's AUC was 0.83 (95% CI, 0.75-0.91), compared with 0.62 (95% CI, 0.52-0.73) when using endometrial thickness alone to predict malignancy (difference in AUC, 0.21; 95% CI, 0.08-0.32). In postmenopausal women with measurable endometrial thickness (n = 848), the IETA-1 model gave an AUC of 0.81 (95% CI, 0.71-0.91), while endometrial thickness alone gave an AUC of 0.70 (95% CI, 0.60-0.81) (difference in AUC, 0.11; 95% CI, 0.01-0.20). CONCLUSION: The IETA-1 model discriminates well between benign and malignant conditions in the uterine cavity in patients without abnormal bleeding, but it overestimates the risk of malignancy. It also discriminates well between the four histological outcome categories. © 2023 International Society of Ultrasound in Obstetrics and Gynecology.

Benign descriptors and ADNEX in two-step strategy to estimate risk of malignancy in ovarian tumors: retrospective validation in IOTA5 multicenter cohort.

OBJECTIVE: Previous work has suggested that the ultrasound-based benign simple descriptors (BDs) can reliably exclude malignancy in a large proportion of women presenting with an adnexal mass. This study aimed to validate a modified version of the BDs and to validate a two-step strategy to estimate the risk of malignancy, in which the modified BDs are followed by the Assessment of Different NEoplasias in the adneXa (ADNEX) model if modified BDs do not apply. METHODS: This was a retrospective analysis using data from the 2-year interim analysis of the International Ovarian Tumor Analysis (IOTA) Phase-5 study, in which consecutive patients with at least one adnexal mass were recruited irrespective of subsequent management (conservative or surgery). The main outcome was classification of tumors as benign or malignant, based on histology or on clinical and ultrasound information during 1 year of follow-up. Multiple imputation was used when outcome based on follow-up was uncertain according to predefined criteria. RESULTS: A total of 8519 patients were recruited at 36 centers between 2012 and 2015. We excluded patients who were already in follow-up at recruitment and all patients from 19 centers that did not fulfil our criteria for good-quality surgical and follow-up data, leaving 4905 patients across 17 centers for statistical analysis. Overall, 3441 (70%) tumors were benign, 978 (20%) malignant and 486 (10%) uncertain. The modified BDs were applicable in 1798/4905 (37%) tumors, of which 1786 (99.3%) were benign. The two-step strategy based on ADNEX without CA125 had an area under the receiver-operating-characteristics curve (AUC) of 0.94 (95% CI, 0.92-0.96). The risk of malignancy was slightly underestimated, but calibration varied between centers. A sensitivity analysis in which we expanded the definition of uncertain outcome resulted in 1419 (29%) tumors with uncertain outcome and an AUC of the two-step strategy without CA125 of 0.93 (95% CI, 0.91-0.95). CONCLUSION: A large proportion of adnexal masses can be classified as benign by the modified BDs. For the remaining masses, the ADNEX model can be used to estimate the risk of malignancy. This two-step strategy is convenient for clinical use. © 2022 The Authors. Ultrasound in Obstetrics & Gynecology published by John Wiley & Sons Ltd on behalf of International Society of Ultrasound in Obstetrics and Gynecology.

Ultrasound features of endometrial pathology in women without abnormal uterine bleeding: results from the International Endometrial Tumor Analysis study (IETA3).

OBJECTIVES: The primary aim of this study was to describe the ultrasound features of various endometrial and other intracavitary pathologies in women without abnormal uterine bleeding (AUB) using the International Endometrial Tumor Analysis (IETA) terminology. The secondary aim was to compare our findings with published data on women with AUB. METHODS: This was a prospective observational study of women presenting at one of seven centers specialized in gynecological ultrasonography, from 2011 until 2018, for indications unrelated to AUB. All patients underwent transvaginal ultrasound using the IETA examination and measurement techniques. Ultrasonography was performed as part of routine gynecological examination or follow-up of non-endometrial pathology, or as part of the work-up before undergoing treatment for infertility, uterine prolapse or ovarian pathology. Ultrasound findings were described using the IETA terminology. Endometrial sampling was performed after the ultrasound scan. The histological endpoints were endometrial atrophy, proliferative or secretory endometrium, endometrial hyperplasia without atypia, endometrial polyp, intracavitary leiomyoma, endometrial intraepithelial neoplasia (EIN), endometrial cancer (EC) and insufficient tissue. The findings in our cohort of women without AUB were compared with those in a published cohort of women with AUB who were examined with transvaginal ultrasound between 2012 and 2015 using the same IETA examination technique and terminology. RESULTS: In this study (IETA3), we included 1745 women without AUB who underwent a standardized transvaginal ultrasound examination followed by either endometrial sampling with histological diagnosis (n = 1537) or at least 1 year of clinical and ultrasound follow-up (n = 208). Of these, 858 (49.2%) women were premenopausal and 887 (50.8%) were postmenopausal. Histology showed the presence of EC and/or EIN in 29 (1.7%) women, endometrial polyps in 1028 (58.9%), intracavitary myomas in 66 (3.8%), proliferative or secretory changes or hyperplasia without atypia in 144 (8.3%), endometrial atrophy in 265 (15.2%) and insufficient tissue in five (0.3%). Most cases of EC or EIN (25/29 (86.2%)) were diagnosed after menopause. The mean endometrial thickness in women with EC or EIN was 11.2 mm (95% CI, 8.9-13.6 mm), being on average 2.4 mm (95% CI, 0.3-4.6 mm) thicker than their benign counterparts. Women with malignant endometrial pathology manifested more frequently non-uniform echogenicity (22/29 (75.9%)) than did those with benign endometrial pathology (929/1716 (54.1%)) (difference, +21.8% (95% CI, +4.2% to +39.2%)). Moderate to abundant vascularization (color score 3-4) was seen in 31.0% (9/29) of cases with EC or EIN compared with 12.8% (220/1716) of those with a benign outcome (difference, +18.2% (95% CI, -0.5% to +36.9%)). Multiple multifocal vessels were recorded in 24.1% (7/29) women with EC or EIN vs 4.0% (68/1716) of those with a benign outcome (difference, +20.2% (95% CI, +4.6% to +35.7%)). A regular endometrial-myometrial junction was seen less frequently in women with EC or EIN (19/29 (65.5%)) vs those with a benign outcome (1412/1716 (82.3%)) (difference, -16.8% (95% CI, -34.2% to +0.6%)). In women with endometrial polyps without AUB, a single dominant vessel was the most frequent vascular pattern (666/1028 (64.8%)). In women with EC, both in those with and those without AUB, the endometrium usually manifested heterogeneous echogenicity, but the endometrium was on average 8.6 mm (95% CI, 5.2-12.0 mm) thinner and less intensely vascularized (color score 3-4: difference, -26.8% (95% CI, -52.2% to -1.3%)) in women without compared to those with AUB. In both pre- and postmenopausal women, asymptomatic endometrial polyps were associated with a thinner endometrium, and they manifested more frequently a bright edge, a regular endometrial-myometrial junction and a single dominant vessel than did polyps in symptomatic women, and they were less intensely vascularized. CONCLUSIONS: We describe the typical ultrasound features of EC, polyps and other intracavitary histologies using IETA terminology in women without AUB. Our findings suggest that the presence of asymptomatic polyps or endometrial malignancy may be accompanied by thinner and less intensely vascularized endometria than their symptomatic counterparts. © 2022 International Society of Ultrasound in Obstetrics and Gynecology.

External validation of models to predict the outcome of pregnancies of unknown location: a multicentre cohort study.

OBJECTIVE: To validate externally five approaches to predict ectopic pregnancy (EP) in pregnancies of unknown location (PUL): the M6P and M6NP risk models, the two-step triage strategy (2ST, which incorporates M6P), the M4 risk model, and beta human chorionic gonadotropin ratio cut-offs (BhCG-RC). DESIGN: Secondary analysis of a prospective cohort study. SETTING: Eight UK early pregnancy assessment units. POPULATION: Women presenting with a PUL and BhCG >25 IU/l. METHODS: Women were managed using the 2ST protocol: PUL were classified as low risk of EP if presenting progesterone ≤2 nmol/l; the remaining cases returned 2 days later for triage based on M6P. EP risk ≥5% was used to classify PUL as high risk. Missing values were imputed, and predictions for the five approaches were calculated post hoc. We meta-analysed centre-specific results. MAIN OUTCOME MEASURES: Discrimination, calibration and clinical utility (decision curve analysis) for predicting EP. RESULTS: Of 2899 eligible women, the primary analysis excluded 297 (10%) women who were lost to follow up. The area under the ROC curve for EP was 0.89 (95% CI 0.86-0.91) for M6P, 0.88 (0.86-0.90) for 2ST, 0.86 (0.83-0.88) for M6NP and 0.82 (0.78-0.85) for M4. Sensitivities for EP were 96% (M6P), 94% (2ST), 92% (N6NP), 80% (M4) and 58% (BhCG-RC); false-positive rates were 35%, 33%, 39%, 24% and 13%. M6P and 2ST had the best clinical utility and good overall calibration, with modest variability between centres. CONCLUSIONS: 2ST and M6P performed best for prediction and triage in PUL. TWEETABLE ABSTRACT: The M6 model, as part of a two-step triage strategy, is the best approach to characterise and triage PULs.

Differences in post-traumatic stress, anxiety and depression following miscarriage or ectopic pregnancy between women and their partners: multicenter prospective cohort study.

OBJECTIVES: To investigate and compare post-traumatic stress (PTS), depression and anxiety in women and their partners over a 9-month period following miscarriage or ectopic pregnancy. METHODS: This was a prospective cohort study. Consecutive women and their partners were approached in the early pregnancy units of three hospitals in central London. At 1, 3 and 9 months after early pregnancy loss, recruits were e-mailed links to surveys containing the Hospital Anxiety and Depression Scale and the Post-traumatic Stress Diagnostic Scale. The proportion of participants meeting the screening criteria for moderate or severe anxiety or depression and PTS was assessed. Mixed-effects logistic regression was used to analyze differences between women and their partners and their evolution over time. RESULTS: In total, 386 partners were approached after the woman in whom the early pregnancy loss had been diagnosed consented to participate, and 192 couples were recruited. All partners were male. Response rates were 60%, 48% and 39% for partners and 78%, 70% and 59% for women, at 1, 3 and 9 months, respectively. Of the partners, 7% met the criteria for PTS at 1 month, 8% at 3 months and 4% at 9 months, compared with 34%, 26% and 21% of women, respectively. Partners also experienced lower rates of moderate/severe anxiety (6% vs 30% at 1 month, 9% vs 25% at 3 months and 6% vs 22% at 9 months) and moderate/severe depression (2% vs 10% at 1 month, 5% vs 8% at 3 months and 1% vs 7% at 9 months). The odds ratios for psychological morbidity in partners vs women after 1 month were 0.02 (95% CI, 0.004-0.12) for PTS, 0.05 (95% CI, 0.01-0.19) for moderate/severe anxiety and 0.15 (95% CI, 0.02-0.96) for moderate/severe depression. Morbidity for each outcome decreased modestly over time, without strong evidence of a different evolution between women and their partners. CONCLUSIONS: Some partners report clinically relevant levels of PTS, anxiety and depression after pregnancy loss, though to a far lesser extent than women physically experiencing the loss. © 2020 International Society of Ultrasound in Obstetrics and Gynecology.

Typical ultrasound features of various endometrial pathologies described using International Endometrial Tumor Analysis (IETA) terminology in women with abnormal uterine bleeding.

OBJECTIVE: To describe the ultrasound features of different endometrial and other intracavitary pathologies inpre- and postmenopausal women presenting with abnormal uterine bleeding, using the International Endometrial Tumor Analysis (IETA) terminology. METHODS: This was a prospective observational multicenter study of consecutive women presenting with abnormal uterine bleeding. Unenhanced sonography with color Doppler and fluid-instillation sonography were performed. Endometrial sampling was performed according to each center's local protocol. The histological endpoints were cancer, atypical endometrial hyperplasia/endometrioid intraepithelial neoplasia (EIN), endometrial atrophy, proliferative or secretory endometrium, endometrial hyperplasia without atypia, endometrial polyp, intracavitary leiomyoma and other. For fluid-instillation sonography, the histological endpoints were endometrial polyp, intracavitary leiomyoma and cancer. For each histological endpoint, we report typical ultrasound features using the IETA terminology. RESULTS: The database consisted of 2856 consecutive women presenting with abnormal uterine bleeding. Unenhanced sonography with color Doppler was performed in all cases and fluid-instillation sonography in 1857. In 2216 women, endometrial histology was available, and these comprised the study population. Median age was 49 years (range, 19-92 years), median parity was 2 (range, 0-10) and median body mass index was 24.9 kg/m2 (range, 16.0-72.1 kg/m2 ). Of the study population, 843 (38.0%) women were postmenopausal. Endometrial polyps were diagnosed in 751 (33.9%) women, intracavitary leiomyomas in 223 (10.1%) and endometrial cancer in 137 (6.2%). None (0% (95% CI, 0.0-5.5%)) of the 66 women with endometrial thickness < 3 mm had endometrial cancer or atypical hyperplasia/EIN. Endometrial cancer or atypical hyperplasia/EIN was found in three of 283 (1.1% (95% CI, 0.4-3.1%)) endometria with a three-layer pattern, in three of 459 (0.7% (95% CI, 0.2-1.9%)) endometria with a linear endometrial midline and in five of 337 (1.5% (95% CI, 0.6-3.4%)) cases with a single vessel without branching on unenhanced ultrasound. CONCLUSIONS: The typical ultrasound features of endometrial cancer, polyps, hyperplasia and atrophy and intracavitary leiomyomas, are described using the IETA terminology. The detection of some easy-to-assess IETA features (i.e. endometrial thickness < 3 mm, three-layer pattern, linear midline and single vessel without branching) makes endometrial cancer unlikely. Copyright © 2020 ISUOG. Published by John Wiley & Sons Ltd.

Validation of ultrasound strategies to assess tumor extension and to predict high-risk endometrial cancer in women from the prospective IETA (International Endometrial Tumor Analysis)-4 cohort.

OBJECTIVES: To compare the performance of ultrasound measurements and subjective ultrasound assessment (SA) in detecting deep myometrial invasion (MI) and cervical stromal invasion (CSI) in women with endometrial cancer, overall and according to whether they had low- or high-grade disease separately, and to validate published measurement cut-offs and prediction models to identify MI, CSI and high-risk disease (Grade-3 endometrioid or non-endometrioid cancer and/or deep MI and/or CSI). METHODS: The study comprised 1538 patients with endometrial cancer from the International Endometrial Tumor Analysis (IETA)-4 prospective multicenter study, who underwent standardized expert transvaginal ultrasound examination. SA and ultrasound measurements were used to predict deep MI and CSI. We assessed the diagnostic accuracy of the tumor/uterine anteroposterior (AP) diameter ratio for detecting deep MI and that of the distance from the lower margin of the tumor to the outer cervical os (Dist-OCO) for detecting CSI. We also validated two two-step strategies for the prediction of high-risk cancer; in the first step, biopsy-confirmed Grade-3 endometrioid or mucinous or non-endometrioid cancers were classified as high-risk cancer, while the second step encompassed the application of a mathematical model to classify the remaining tumors. The 'subjective prediction model' included biopsy grade (Grade 1 vs Grade 2) and subjective assessment of deep MI or CSI (presence or absence) as variables, while the 'objective prediction model' included biopsy grade (Grade 1 vs Grade 2) and minimal tumor-free margin. The predictive performance of the two two-step strategies was compared with that of simply classifying patients as high risk if either deep MI or CSI was suspected based on SA or if biopsy showed Grade-3 endometrioid or mucinous or non-endometrioid histotype (i.e. combining SA with biopsy grade). Histological assessment from hysterectomy was considered the reference standard. RESULTS: In 1275 patients with measurable lesions, the sensitivity and specificity of SA for detecting deep MI was 70% and 80%, respectively, in patients with a Grade-1 or -2 endometrioid or mucinous tumor vs 76% and 64% in patients with a Grade-3 endometrioid or mucinous or a non-endometrioid tumor. The corresponding values for the detection of CSI were 51% and 94% vs 50% and 91%. Tumor AP diameter and tumor/uterine AP diameter ratio showed the best performance for predicting deep MI (area under the receiver-operating characteristics curve (AUC) of 0.76 and 0.77, respectively), and Dist-OCO had the best performance for predicting CSI (AUC, 0.72). The proportion of patients classified correctly as having high-risk cancer was 80% when simply combining SA with biopsy grade vs 80% and 74% when using the subjective and objective two-step strategies, respectively. The subjective and objective models had an AUC of 0.76 and 0.75, respectively, when applied to Grade-1 and -2 endometrioid tumors. CONCLUSIONS: In the hands of experienced ultrasound examiners, SA was superior to ultrasound measurements for the prediction of deep MI and CSI of endometrial cancer, especially in patients with a Grade-1 or -2 tumor. The mathematical models for the prediction of high-risk cancer performed as expected. The best strategies for predicting high-risk endometrial cancer were combining SA with biopsy grade and the subjective two-step strategy, both having an accuracy of 80%. Copyright © 2019 ISUOG. Published by John Wiley & Sons Ltd.

Ultrasound-based risk model for preoperative prediction of lymph-node metastases in women with endometrial cancer: model-development study.

OBJECTIVE: To develop a preoperative risk model, using endometrial biopsy results and clinical and ultrasound variables, to predict the individual risk of lymph-node metastases in women with endometrial cancer. METHODS: A mixed-effects logistic regression model for prediction of lymph-node metastases was developed in 1501 prospectively included women with endometrial cancer undergoing transvaginal ultrasound examination before surgery, from 16 European centers. Missing data, including missing lymph-node status, were imputed. Discrimination, calibration and clinical utility of the model were evaluated using leave-center-out cross validation. The predictive performance of the model was compared with that of risk classification from endometrial biopsy alone (high-risk defined as endometrioid cancer Grade 3/non-endometrioid cancer) or combined endometrial biopsy and ultrasound (high-risk defined as endometrioid cancer Grade 3/non-endometrioid cancer/deep myometrial invasion/cervical stromal invasion/extrauterine spread). RESULTS: Lymphadenectomy was performed in 691 women, of whom 127 had lymph-node metastases. The model for prediction of lymph-node metastases included the predictors age, duration of abnormal bleeding, endometrial biopsy result, tumor extension and tumor size according to ultrasound and undefined tumor with an unmeasurable endometrium. The model's area under the curve was 0.73 (95% CI, 0.68-0.78), the calibration slope was 1.06 (95% CI, 0.79-1.34) and the calibration intercept was 0.06 (95% CI, -0.15 to 0.27). Using a risk threshold for lymph-node metastases of 5% compared with 20%, the model had, respectively, a sensitivity of 98% vs 48% and specificity of 11% vs 80%. The model had higher sensitivity and specificity than did classification as high-risk, according to endometrial biopsy alone (50% vs 35% and 80% vs 77%, respectively) or combined endometrial biopsy and ultrasound (80% vs 75% and 53% vs 52%, respectively). The model's clinical utility was higher than that of endometrial biopsy alone or combined endometrial biopsy and ultrasound at any given risk threshold. CONCLUSIONS: Based on endometrial biopsy results and clinical and ultrasound characteristics, the individual risk of lymph-node metastases in women with endometrial cancer can be estimated reliably before surgery. The model is superior to risk classification by endometrial biopsy alone or in combination with ultrasound. Copyright © 2019 ISUOG. Published by John Wiley & Sons Ltd.

Untapped potential of multicenter studies: a review of cardiovascular risk prediction models revealed inappropriate analyses and wide variation in reporting.

BACKGROUND: Clinical prediction models are often constructed using multicenter databases. Such a data structure poses additional challenges for statistical analysis (clustered data) but offers opportunities for model generalizability to a broad range of centers. The purpose of this study was to describe properties, analysis, and reporting of multicenter studies in the Tufts PACE Clinical Prediction Model Registry and to illustrate consequences of common design and analyses choices. METHODS: Fifty randomly selected studies that are included in the Tufts registry as multicenter and published after 2000 underwent full-text screening. Simulated examples illustrate some key concepts relevant to multicenter prediction research. RESULTS: Multicenter studies differed widely in the number of participating centers (range 2 to 5473). Thirty-nine of 50 studies ignored the multicenter nature of data in the statistical analysis. In the others, clustering was resolved by developing the model on only one center, using mixed effects or stratified regression, or by using center-level characteristics as predictors. Twenty-three of 50 studies did not describe the clinical settings or type of centers from which data was obtained. Four of 50 studies discussed neither generalizability nor external validity of the developed model. CONCLUSIONS: Regression methods and validation strategies tailored to multicenter studies are underutilized. Reporting on generalizability and potential external validity of the model lacks transparency. Hence, multicenter prediction research has untapped potential. REGISTRATION: This review was not registered.

Random-effects meta-analysis of the clinical utility of tests and prediction models.

The use of data from multiple studies or centers for the validation of a clinical test or a multivariable prediction model allows researchers to investigate the test's/model's performance in multiple settings and populations. Recently, meta-analytic techniques have been proposed to summarize discrimination and calibration across study populations. Here, we rather consider performance in terms of net benefit, which is a measure of clinical utility that weighs the benefits of true positive classifications against the harms of false positives. We posit that it is important to examine clinical utility across multiple settings of interest. This requires a suitable meta-analysis method, and we propose a Bayesian trivariate random-effects meta-analysis of sensitivity, specificity, and prevalence. Across a range of chosen harm-to-benefit ratios, this provides a summary measure of net benefit, a prediction interval, and an estimate of the probability that the test/model is clinically useful in a new setting. In addition, the prediction interval and probability of usefulness can be calculated conditional on the known prevalence in a new setting. The proposed methods are illustrated by 2 case studies: one on the meta-analysis of published studies on ear thermometry to diagnose fever in children and one on the validation of a multivariable clinical risk prediction model for the diagnosis of ovarian cancer in a multicenter dataset. Crucially, in both case studies the clinical utility of the test/model was heterogeneous across settings, limiting its usefulness in practice. This emphasizes that heterogeneity in clinical utility should be assessed before a test/model is routinely implemented.

Does ignoring clustering in multicenter data influence the performance of prediction models? A simulation study.

Clinical risk prediction models are increasingly being developed and validated on multicenter datasets. In this article, we present a comprehensive framework for the evaluation of the predictive performance of prediction models at the center level and the population level, considering population-averaged predictions, center-specific predictions, and predictions assuming an average random center effect. We demonstrated in a simulation study that calibration slopes do not only deviate from one because of over- or underfitting of patterns in the development dataset, but also as a result of the choice of the model (standard versus mixed effects logistic regression), the type of predictions (marginal versus conditional versus assuming an average random effect), and the level of model validation (center versus population). In particular, when data is heavily clustered (ICC 20%), center-specific predictions offer the best predictive performance at the population level and the center level. We recommend that models should reflect the data structure, while the level of model validation should reflect the research question.

Key steps and common pitfalls in developing and validating risk models.

Models for estimating an individual's risk of having or developing a disease are abundant in the medical literature, yet many do not meet the methodological standards that have been set to maximise generalisability and utility. This paper presents an overview of ten steps from the conception of the study to the implementation of the risk model and discusses common pitfalls. We discuss crucial aspects of study design, data collection, model development and performance evaluation, and discuss how to bring the model to clinical practice. TWEETABLE ABSTRACT: We present an overview of ten key steps for the development of risk models and discuss common pitfalls.

Practical guidance for applying the ADNEX model from the IOTA group to discriminate between different subtypes of adnexal tumors.

All gynecologists are faced with ovarian tumors on a regular basis, and the accurate preoperative diagnosis of these masses is important because appropriate management depends on the type of tumor. Recently, the International Ovarian Tumor Analysis (IOTA) consortium published the Assessment of Different NEoplasias in the adneXa (ADNEX) model, the first risk model that differentiates between benign and four types of malignant ovarian tumors: borderline, stage I cancer, stage II-IV cancer, and secondary metastatic cancer. This approach is novel compared to existing tools that only differentiate between benign and malignant tumors, and therefore questions may arise on how ADNEX can be used in clinical practice. In the present paper, we first provide an in-depth discussion about the predictors used in ADNEX and the ability for risk prediction with different tumor histologies. Furthermore, we formulate suggestions about the selection and interpretation of risk cut-offs for patient stratification and choice of appropriate clinical management. This is illustrated with a few example patients. We cannot propose a generally applicable algorithm with fixed cut-offs, because (as with any risk model) this depends on the specific clinical setting in which the model will be used. Nevertheless, this paper provides a guidance on how the ADNEX model may be adopted into clinical practice.

A simulation study of sample size demonstrated the importance of the number of events per variable to develop prediction models in clustered data.

OBJECTIVES: This study aims to investigate the influence of the amount of clustering [intraclass correlation (ICC) = 0%, 5%, or 20%], the number of events per variable (EPV) or candidate predictor (EPV = 5, 10, 20, or 50), and backward variable selection on the performance of prediction models. STUDY DESIGN AND SETTING: Researchers frequently combine data from several centers to develop clinical prediction models. In our simulation study, we developed models from clustered training data using multilevel logistic regression and validated them in external data. RESULTS: The amount of clustering was not meaningfully associated with the models' predictive performance. The median calibration slope of models built in samples with EPV = 5 and strong clustering (ICC = 20%) was 0.71. With EPV = 5 and ICC = 0%, it was 0.72. A higher EPV related to an increased performance: the calibration slope was 0.85 at EPV = 10 and ICC = 20% and 0.96 at EPV = 50 and ICC = 20%. Variable selection sometimes led to a substantial relative bias in the estimated predictor effects (up to 118% at EPV = 5), but this had little influence on the model's performance in our simulations. CONCLUSION: We recommend at least 10 EPV to fit prediction models in clustered data using logistic regression. Up to 50 EPV may be needed when variable selection is performed.

Strategies to diagnose ovarian cancer: new evidence from phase 3 of the multicentre international IOTA study.

BACKGROUND: To compare different ultrasound-based international ovarian tumour analysis (IOTA) strategies and risk of malignancy index (RMI) for ovarian cancer diagnosis using a meta-analysis approach of centre-specific data from IOTA3. METHODS: This prospective multicentre diagnostic accuracy study included 2403 patients with 1423 benign and 980 malignant adnexal masses from 2009 until 2012. All patients underwent standardised transvaginal ultrasonography. Test performance of RMI, subjective assessment (SA) of ultrasound findings, two IOTA risk models (LR1 and LR2), and strategies involving combinations of IOTA simple rules (SRs), simple descriptors (SDs) and LR2 with and without SA was estimated using a meta-analysis approach. Reference standard was histology after surgery. RESULTS: The areas under the receiver operator characteristic curves of LR1, LR2, SA and RMI were 0.930 (0.917-0.942), 0.918 (0.905-0.930), 0.914 (0.886-0.936) and 0.875 (0.853-0.894). Diagnostic one-step and two-step strategies using LR1, LR2, SR and SD achieved summary estimates for sensitivity 90-96%, specificity 74-79% and diagnostic odds ratio (DOR) 32.8-50.5. Adding SA when IOTA methods yielded equivocal results improved performance (DOR 57.6-75.7). Risk of Malignancy Index had sensitivity 67%, specificity 91% and DOR 17.5. CONCLUSIONS: This study shows all IOTA strategies had excellent diagnostic performance in comparison with RMI. The IOTA strategy chosen may be determined by clinical preference.

Multicentre external validation of IOTA prediction models and RMI by operators with varied training.

BACKGROUND: Correct characterisation of ovarian tumours is critical to optimise patient care. The purpose of this study is to evaluate the diagnostic performance of the International Ovarian Tumour Analysis (IOTA) logistic regression model (LR2), ultrasound Simple Rules (SR), the Risk of Malignancy Index (RMI) and subjective assessment (SA) for preoperative characterisation of adnexal masses, when ultrasonography is performed by examiners with different background training and experience. METHODS: A 2-year prospective multicentre cross-sectional study. Thirty-five level II ultrasound examiners contributed in three UK hospitals. Transvaginal ultrasonography was performed using a standardised approach. The final outcome was the surgical findings and histological diagnosis. To characterise the adnexal masses, the six-variable prediction model (LR2) with a cutoff of 0.1, the RMI with cutoff of 200, ten SR (five rules for malignancy and five rules for benignity) and SA were applied. The area under the curves (AUCs) for performance of LR2 and RMI were calculated. Diagnostic performance measures for all models assessed were sensitivity, specificity, positive and negative likelihood ratios (LR+ and LR-), and the diagnostic odds ratio (DOR). RESULTS: Nine-hundred and sixty-two women with adnexal masses underwent transvaginal ultrasonography, whereas 255 had surgery. Prevalence of malignancy was 29% (49 primary invasive epithelial ovarian cancers, 18 borderline ovarian tumours, and 7 metastatic tumours). The AUCs for LR2 and RMI for all masses were 0.94 (95% confidence interval (CI): 0.89-0.97) and 0.90 (95% CI: 0.83-0.94), respectively. In premenopausal women, LR2-RMI difference was 0.09 (95% CI: 0.03-0.15) compared with -0.02 (95% CI: -0.08 to 0.04) in postmenopausal women. For all masses, the DORs for LR2, RMI, SR+SA (using SA when SR inapplicable), SR+MA (assuming malignancy when SR inapplicable), and SA were 62 (95% CI: 27-142), 43 (95% CI: 19-97), 109 (95% CI: 44-274), 66 (95% CI: 27-158), and 70 (95% CI: 30-163), respectively. CONCLUSION: Overall, the test performance of IOTA prediction models and rules as well as the RMI was maintained in examiners with varying levels of training and experience.

Does the presence of a Caesarean section scar affect implantation site and early pregnancy outcome in women attending an early pregnancy assessment unit?

STUDY QUESTION: Are there any differences in the location and distance to the internal cervical ostium of the implantation site of the intrauterine gestation sacs, early pregnancy symptoms and pregnancy outcome at 12 weeks gestation between women with and without a previous Caesarean section (CS)? SUMMARY ANSWER: The presence of a CS scar affects the site of implantation, and the distance between implantation site and the scar is related to the risk of spontaneous abortion. WHAT IS KNOWN ALREADY?: Little is known about the impact of a CS scar on implantation other than the risk of Caesarean scar pregnancy (CSP). Furthermore, there is a paucity of information on how the proximity of implantation to the scar impacts on pregnancy outcome in the first trimester. STUDY DESIGN, SIZE, AND DURATION: A prospective cohort study conducted over 15 months in the early pregnancy unit of a London Teaching Hospital. Three hundred and eighty women underwent a transvaginal scan at 6-11 weeks of gestation. A total of 170 women had undergone ≥1 CS, and 210 women had no history of CS. PARTICIPANTS/MATERIALS, SETTING, METHODS: The 380 women were recruited as consecutive non-selected cases. The relationship between the implanted sac and the CS scar was assessed by quantifiable measures and by subjective impression. Logistic regression analysis was used to determine the influence of the presence of a CS scar on pregnancy outcome. The final outcome of the study was the viability of the pregnancy at 12 weeks. MAIN RESULTS AND THE ROLE OF CHANCE: Implantation was most frequently posterior (53%) in the CS group and fundal in the non-CS group (42%). Gestation sac implantation was 8.7 mm lower in the CS group (95% confidence interval (CI) 6.7-10.7, P < 0.0001). Presenting complaints differed in women with and without a previous CS (P = 0.0009). More frequent vaginal bleeding [73 versus 55%, difference -18, 95% CI (-27 to -8%] yet no clearly increased spontaneous abortion rates were noted in the CS group compared with the non-CS group (adjusted odds ratio = 1.1, 95% CI 0.6-1.9, P = 0.74). Subjective impression showed that in eight cases the implantation site crossed the scar, seven of which resulted in spontaneous abortion, while the remaining case survived to term complicated by placenta praevia and post-partum haemorrhage. The subjective impression of the examiner was supported by the measurements of distance between implantation site and CS scar. LIMITATIONS, REASONS FOR CAUTION: A weakness of the study is the lack of a reference technique to verify the location of implantation. WIDER IMPLICATIONS OF THE FINDINGS: This study adds further support to the hypothesis that the presence of a CS on the uterus impacts on the implantation site of a future pregnancy. The possibility that the CS scar has an impact on the risk of spontaneous abortion should be further studied. Caution must be exercised when implantation occurs near to, and crosses, a CS scar as this is not always associated with the diagnosis of CSP. A potential limitation of the study is that we did not examine scar dimensions and morphology.

Predicting successful vaginal birth after Cesarean section using a model based on Cesarean scar features examined by transvaginal sonography.

OBJECTIVE: To develop a model to predict the success of a trial of vaginal birth after Cesarean section (VBAC) based on sonographic measurements of Cesarean section (CS) scar features, demographic variables and previous obstetric history. METHODS: We used transvaginal sonography (TVS) to examine the CS scar of 320 consecutive pregnant women. TVS was carried out at 11-13, 19-21 and 34-36 weeks' gestation and prospective measurements of the scar were recorded at each visit according to a defined protocol. A logistic regression model to predict success of VBAC was developed for those patients with a visible scar on ultrasound and only one previous CS. The model was evaluated using bootstrap validation. RESULTS: There were 131 women with one previous CS and a visible scar, of whom 10 underwent CS prior to labor and were excluded from analysis. Successful VBAC was achieved in 74/121 (61%) of the remaining cases. The prediction model developed was based on patient age, previous history of VBAC, residual myometrial thickness (RMT) and the change in RMT from the first to the second trimester (ΔRMT). The internally validated area under the receiver-operating characteristics curve was 0.62 when measurements of RMT and ΔRMT were excluded, but 0.94 when scar information was incorporated into the model. CONCLUSION: Ultrasound measurements of CS scar, namely RMT and the change in RMT from the first to the second trimester of pregnancy, when incorporated into a mathematical model, can predict accurately a successful trial of labor in patients with one previous CS.