Internal validation of Automated Visual Evaluation (AVE) on smartphone images for cervical cancer screening in a prospective study in Zambia.

OBJECTIVES: Visual inspection with acetic acid (VIA) is a low-cost approach for cervical cancer screening used in most low- and middle-income countries (LMICs) but, similar to other visual tests, is subjective and requires sustained training and quality assurance. We developed, trained, and validated an artificial-intelligence-based "Automated Visual Evaluation" (AVE) tool that can be adapted to run on smartphones to assess smartphone-captured images of the cervix and identify precancerous lesions, helping augment VIA performance. DESIGN: Prospective study. SETTING: Eight public health facilities in Zambia. PARTICIPANTS: A total of 8204 women aged 25-55. INTERVENTIONS: Cervical images captured on commonly used low-cost smartphone models were matched with key clinical information including human immunodeficiency virus (HIV) and human papillomavirus (HPV) status, plus histopathology analysis (where applicable), to develop and train an AVE algorithm and evaluate its performance for use as a primary screen and triage test for women who are HPV positive. MAIN OUTCOME MEASURES: Area under the receiver operating curve (AUC); sensitivity; specificity. RESULTS: As a general population screening tool for cervical precancerous lesions, AVE identified cases of cervical precancerous and cancerous (CIN2+) lesions with high performance (AUC = 0.91, 95% confidence interval [CI] = 0.89-0.93), which translates to a sensitivity of 85% (95% CI = 81%-90%) and specificity of 86% (95% CI = 84%-88%) based on maximizing the Youden's index. This represents a considerable improvement over naked eye VIA, which as per a meta-analysis by the World Health Organization (WHO) has a sensitivity of 66% and specificity of 87%. For women living with HIV, the AUC of AVE was 0.91 (95% CI = 0.88-0.93), and among those testing positive for high-risk HPV types, the AUC was 0.87 (95% CI = 0.83-0.91). CONCLUSIONS: These results demonstrate the feasibility of utilizing AVE on images captured using a commonly available smartphone by nurses in a screening program, and support our ongoing efforts for moving to more broadly evaluate AVE for its clinical sensitivity, specificity, feasibility, and acceptability across a wider range of settings. Limitations of this study include potential inflation of performance estimates due to verification bias (as biopsies were only obtained from participants with visible aceto-white cervical lesions) and due to this being an internal validation (the test data, while independent from that used to develop the algorithm was drawn from the same study).

Internal Validation of Automated Visual Evaluation (AVE) on Smartphone Images for Cervical Cancer Screening in a Prospective Study in Zambia.

Objectives: Visual inspection with acetic acid (VIA) is a low-cost approach for cervical cancer screening used in most low- and middle-income countries (LMICs) but, similar to other visual tests like histopathology, is subjective and requires sustained training and quality assurance. We developed, trained, and validated an artificial-intelligence-based "Automated Visual Evaluation" (AVE) tool that can be adapted to run on smartphones to assess smartphone-captured images of the cervix and identify precancerous lesions, helping augment performance of VIA. Design: Prospective study. Setting: Eight public health facilities in Zambia. Participants: 8,204 women aged 25-55. Interventions: Cervical images captured on commonly used low-cost smartphone models were matched with key clinical information including human immunodeficiency virus (HIV) and human papillomavirus (HPV) status, plus histopathology analysis (where applicable), to develop and train an AVE algorithm and evaluate its performance for use as a primary screen and triage test for women who are HPV positive. Main outcome measures: Area under the receiver operating curve (AUC); sensitivity; specificity. Results: As a general population screening for cervical precancerous lesions, AVE identified cases of cervical precancerous and cancerous (CIN2+) lesions with high performance (AUC = 0.91, 95% confidence interval [CI] = 0.89 to 0.93), which translates to a sensitivity of 85% (95% CI = 81% to 90%) and specificity of 86% (95% CI = 84% to 88%) based on maximizing the Youden's index. This represents a considerable improvement over VIA, which a meta-analysis by the World Health Organization (WHO) estimates to have sensitivity of 66% and specificity of 87%. For women living with HIV, the AUC of AVE was 0.91 (95% CI = 0.88 to 0.93), and among those testing positive for high-risk HPV types, the AUC was 0.87 (95% CI = 0.83 to 0.91). Conclusions: These results demonstrate the feasibility of utilizing AVE on images captured using a commonly available smartphone by screening nurses and support our transition to clinical evaluation of AVE's sensitivity, specificity, feasibility, and acceptability across a broader range of settings. The performance of the algorithm as reported may be inflated, as biopsies were obtained only from study participants with visible aceto-white cervical lesions, which can lead to verification bias; and the images and data sets used for testing of the model, although "unseen" by the algorithm during training, were acquired from the same set of patients and devices, limiting the study to that of an internal validation of the AVE algorithm.