Validation of a deep neural network-based algorithm supporting clinical management of adnexal mass.

Background: Conservative management of adnexal mass is warranted when there is imaging-based and clinical evidence of benign characteristics. Malignancy risk is, however, a concern due to the mortality rate of ovarian cancer. Malignancy occurs in 10-15% of adnexal masses that go to surgery, whereas the rate of malignancy is much lower in masses clinically characterized as benign or indeterminate. Additional diagnostic tests could assist conservative management of these patients. Here we report the clinical validation of OvaWatch, a multivariate index assay, with real-world evidence of performance that supports conservative management of adnexal masses. Methods: OvaWatch utilizes a previously characterized neural network-based algorithm combining serum biomarkers and clinical covariates and was used to examine malignancy risk in prospective and retrospective samples of patients with an adnexal mass. Retrospective data sets were assembled from previous studies using patients who had adnexal mass and were scheduled for surgery. The prospective study was a multi-center trial of women with adnexal mass as identified on clinical examination and indeterminate or asymptomatic by imaging. The performance to detect ovarian malignancy was evaluated at a previously validated score threshold. Results: In retrospective, low prevalence (N = 1,453, 1.5% malignancy rate) data from patients that received an independent physician assessment of benign, OvaWatch has a sensitivity of 81.8% [95% confidence interval (CI) 65.1-92.7] for identifying a histologically confirmed malignancy, and a negative predictive value (NPV) of 99.7%. OvaWatch identified 18/22 malignancies missed by physician assessment. A prospective data set had 501 patients where 106 patients with adnexal mass went for surgery. The prevalence was 2% (10 malignancies). The sensitivity of OvaWatch for malignancy was 40% (95% CI: 16.8-68.7%), and the specificity was 87% (95% CI: 83.7-89.7) when patients were included in the analysis who did not go to surgery and were evaluated as benign. The NPV remained 98.6% (95% CI: 97.0-99.4%). An independent analysis set with a high prevalence (45.8%) the NPV value was 87.8% (95% CI: 95% CI: 75.8-94.3%). Conclusion: OvaWatch demonstrated high NPV across diverse data sets and promises utility as an effective diagnostic test supporting management of suspected benign or indeterminate mass to safely decrease or delay unnecessary surgeries.

A reflex testing protocol using two multivariate index assays improves the risk assessment for ovarian cancer in patients with an adnexal mass.

OBJECTIVES: Patients with adnexal masses suspicious for malignancy benefit from referral to oncology specialists during presurgical assessment of the mass. OVA1 is a multivariate assay using a five-biomarker panel which offers high overall and early-stage sensitivity. However, OVA1 has a high false-positive rate for benign masses. Overa, a second-generation multivariate index assay was developed to reduce the false-positive rate. The aim of the present study was to use Overa as a reflex for OVA1 and increase specificity. METHODS: OVA1 cut-off scores were established to place patients into three categories: low, intermediate, and high cancer risk. Samples with intermediate-risk OVA1 scores were reflexed to the Overa and defined as high or low risk. This protocol was tested with 1035 prospectively collected serum samples and validated with an independent prospectively collected sample set (N = 207). RESULTS: Thirty-five per cent (359) of samples had intermediate OVA1 scores. Reflexing these to Overa eliminated 58% of the false-positives and improved the overall specificity from 50% to 72%. This finding was confirmed in the independent dataset, in which the specificity increased from 56% to 73%. CONCLUSIONS: Reflexing samples with intermediate OVA1 scores significantly decreases the false-positive rate, thereby reducing unnecessary surgical referrals.

Analytical Validation of a Deep Neural Network Algorithm for the Detection of Ovarian Cancer.

PURPOSE: Early detection of ovarian cancer, the deadliest gynecologic cancer, is crucial for reducing mortality. Current noninvasive risk assessment measures include protein biomarkers in combination with other clinical factors, which vary in their accuracy. Machine learning can be applied to optimizing the combination of these features, leading to more accurate assessment of malignancy. However, the low prevalence of the disease can make rigorous validation of these tests challenging and can result in unbalanced performance. METHODS: MIA3G is a deep feedforward neural network for ovarian cancer risk assessment, using seven protein biomarkers along with age and menopausal status as input features. The algorithm was developed on a heterogenous data set of 1,067 serum specimens from women with adnexal masses (prevalence = 31.8%). It was subsequently validated on a cohort almost twice that size (N = 2,000). RESULTS: In the analytical validation data set (prevalence = 4.9%), MIA3G demonstrated a sensitivity of 89.8% and a specificity of 84.02%. The positive predictive value was 22.45%, and the negative predictive value was 99.38%. When stratified by cancer type and stage, MIA3G achieved sensitivities of 94.94% for epithelial ovarian cancer, 76.92% for early-stage cancer, and 98.04% for late-stage cancer. CONCLUSION: The balanced performance of MIA3G leads to a high sensitivity and high specificity, a combination that may be clinically useful for providers in evaluating the appropriate management strategy for their patients. Limitations of this work include the largely retrospective nature of the data set and the unequal, albeit random, assignment of histologic subtypes between the training and validation data sets. Future directions may include the addition of new biomarkers or other modalities to strengthen the performance of the algorithm.

Salvaging Detection of Early-Stage Ovarian Malignancies When CA125 Is Not Informative.

BACKGROUND: Ovarian cancer is the deadliest gynecologic cancer, with no recommended screening test to assist with early detection. Cancer antigen 125 (CA125) is a serum biomarker commonly used by clinicians to assess preoperative cancer risk, but it underperforms in premenopausal women, early-stage malignancies, and several histologic subtypes. OVA1 is a multivariate index assay that combines CA125 and four other serum proteins to assess the malignant risk of an adnexal mass. OBJECTIVE: To evaluate the performance of OVA1 in a cohort of patients with low-risk serum CA125 values. STUDY DESIGN: We analyzed patient data from previous collections (N = 2305, prevalence = 4.5%) where CA125 levels were at or below 67 units/milliliter (U/mL) for pre-menopausal women and 35 U/mL for post-menopausal women. We compare the performance of OVA1 to CA125 in classifying the risk of malignancy in this cohort, including sensitivity, specificity, positive and negative predictive values. RESULTS: The overall sensitivity of OVA1 in patients with a low-risk serum CA125 was 59% with a false-positive rate of 30%. OVA1 detected over 50% of ovarian malignancies in premenopausal women despite a low-risk serum CA125. OVA1 also correctly identified 63% of early-stage cancers missed by CA125. The most common epithelial ovarian cancer subtypes in the study population were mucinous (25%) and serous (23%) carcinomas. Despite a low-risk CA125, OVA1 successfully detected 83% of serous, 58% of mucinous, and 50% of clear cell ovarian cancers. CONCLUSIONS: As a standalone test, CA125 misses a significant number of ovarian malignancies that can be detected by OVA1. This is particularly important for premenopausal women and early-stage cancers, which have a much better long-term survival than late-stage malignancies. Using OVA1 in the setting of a normal serum CA125 can help identify at-risk ovarian tumors for referral to a gynecologic oncologist, potentially improving overall survival.

Low-risk multivariate index assay scores, physician referral and surgical choices in women with adnexal masses.

OBJECTIVE: To assess the use of Multivariate Index Assay (MIA OVA1) by gynecologists and determine referral practices and surgical decision making for women with adnexal masses and low-risk MIA OVA1 scores. METHODS: Information on patients who received an OVA1 test was collected retrospectively from 22 gynecologic practices through a chart review. Referral patterns were examined for patients with low-risk OVA1 results prior to first surgical intervention. Chart reviews were from a variety of practice and hospital settings representing major geographic regions within the United States. RESULTS: A total of 282 independent patient charts were reviewed. Low-risk results were found for 146 patients (52%). Surgery was performed on 82 (56%) patients with low-risk scores. The referral rate to specialty care was 21% (17/82) for low-risk OVA1 patients. Three low-malignant potential tumors were identified in the low-risk patients, with no cases of invasive malignancy. Eighty-six percent of the surgeries performed on low-risk OVA1 patients were minimally invasive. In 44% of the low-risk OVA1 patients, no surgical intervention was performed. CONCLUSIONS: A high proportion of low-risk OVA1 patients were not referred to a gynecologic oncologist prior to surgery, indicating gynecologists may use MIA OVA1 along with clinical and radiographic findings to appropriately retain patients for their care. This practice is safe and may be cost-saving, with patient satisfaction implications.

Adnexal mass risk assessment: a multivariate index assay for malignancy risk stratification.

Aims: Adnexal mass risk assessment (AMRA) stratifies patients with adnexal masses, identifying the relatively small number of malignancies from benigns which might take a 'watchful waiting' approach. Methods: AMRA uses seven biomarkers and derived from women with adnexal masses scheduled for surgery. Estimated clinical performance was calculated using fixed prevalence. Results: At 5% prevalence, the high-risk group, 7.9% total, captured 75.9% of invasive malignancies at a positive predictive value of 35.8%. High risk/intermediate risk combined had a sensitivity of 89.7 and 95.6% for pre- and post-menopausal cancers, respectively. The low-risk group, 67.8% total, had an negative predictive value of 99.0%. Conclusion: With highly differentiating risk stratification capability across histological subtypes and stages, AMRA is potentially applicable to patients with adnexal masses to assist deciding whether immediate surgery is recommended.

Ethnic disparity in clinical performance between multivariate index assay and CA125 in detection of ovarian malignancy.

Aim: Based on evidence that African-American (AA) women have lower CA125 values than Caucasian (C) women, we investigated this to see if this disparity would have an impact on ovarian cancer detection using CA125 and multivariate index assay (MIA). Materials & methods: Serum from two prospective trials of 1029 (274 malignancies [250 C/24AA]) were analyzed for CA125 and MIA results. Clinical performance was calculated. Results: Sensitivity of MIA in Caucasian women was 93.2%, 74.4% for CA125 at the ACOG approved cut-off level of 200 U/ml cutoff, and 80.4% using the 2007, Dearking 67 U/ml cutoff. In AA American women, MIA sensitivity was 79.2%, 33.3% for CA125 at the ACOG approved cut-off levels and 62.5% at the 2007, Dearking 67 U/ml cutoff. Conclusion: Our results support that CA125 in AA women with adnexal masses has lower sensitivity than MIA no matter what the cutoff value is. Implementation of MIA in evaluation of adnexal masses should increase sensitivity of detection of malignancy compared with CA125, particularly in AA women.

Multivariate Index Assay Is Superior to CA125 and HE4 Testing in Detection of Ovarian Malignancy in African-American Women.

OBJECTIVE: To review and analyze the serum values of risk of ovarian malignancy algorithm (ROMA) and multivariate index assay (MIA) in subgroups of women who underwent surgery for adnexal masses to determine sensitivity, specificity, and positive and negative predictive values for the detection of malignancy in different ethnic populations. METHODS: Serum samples from 2 prospective trials of 1029 women in which 274 women diagnosed with malignancy were analyzed for ROMA scores and MIA results. Biomarker data were obtained from the previous prospective studies that validated the MIA test. Of these, 250 women were Caucasian (C) and 24 were African-American (AA). Sensitivity, specificity, positive and negative predictive values, and confidence intervals for preoperative test results were calculated using DTComPair package of the R programming language. In premenopausal women, a ROMA value equal to or greater than 1.14 indicates a high risk of finding epithelial ovarian cancer. In premenopausal women, MIA values greater than 5.0 are associated with a greater risk of malignancy. In postmenopausal women, a ROMA value equal to or greater than 2.99 indicates a high risk of finding epithelial ovarian cancer. In postmenopausal women, MIA values greater than 4.4 are associated with a greater risk of malignancy. RESULTS: Primary ovarian malignancy was diagnosed in 179 cases (167 C/12 AA) and metastatic disease to the ovary in an additional 27 cases (22 C/5 AA). Overall results are shown below. CONCLUSIONS: Our results demonstrate that ROMA in AA women with adnexal masses have lower sensitivity for the detection of malignancy than does MIA. Implementation of MIA in the evaluation of adnexal masses will increase the sensitivity of the detection of malignancy compared with ROMA, with the most marked results in AA women.

Combined symptom index and second-generation multivariate biomarker test for prediction of ovarian cancer in patients with an adnexal mass.

OBJECTIVE: To assess the performance of a symptom index (SI) and multivariate biomarker panel in the identification of ovarian cancer in women presenting for surgery with an adnexal mass. STUDY DESIGN: Prospective study of patients seen at a tertiary medical center. Following consent, patients completed an SI and preoperative serum was collected for individual markers (CA 125) and a second-generation FDA-cleared biomarker test (MIA2G). Results for the SI and MIA2G were correlated with operative findings and surgical pathology. Logistic regression modeling was performed to assess the interaction of the SI with MIA2G to determine the risk of malignancy (ROM). RESULTS: Of the 218 patients enrolled, the mean age was 53.6 years (range 18-86). One-hundred and forty-seven patients (67.4%) were postmenopausal. Sixty-four patients (29.4%) had epithelial ovarian cancer or fallopian tube cancer (EOC/FTC) and 17 (7.8%) had borderline ovarian tumors. A positive SI or MIA2G correctly identified 96.1% of patients with EOC/FTC. Using logistic regression, we found that both SI and MIA2G score were significantly associated with ROM (p < 0.001). In a simulation with disease prevalence set at 5%, patients with a negative SI and a MIA2G score of 6 had a ROM of 1.8% whereas patients with the same MIA2G and positive SI had a 10.5% ROM, nearly a 6-fold higher risk. CONCLUSIONS: The combination of a patient-reported symptom index and refined biomarker panel allows for improved accuracy in the assessment for ovarian cancer in patients with an adnexal mass. This strategy could offer a personalized approach to addressing ROM to triage patients with an adnexal mass to appropriate care.