Internal Validation of Automated Visual Evaluation (AVE) on Smartphone Images for Cervical Cancer Screening in a Prospective Study in Zambia.

Objectives: Visual inspection with acetic acid (VIA) is a low-cost approach for cervical cancer screening used in most low- and middle-income countries (LMICs) but, similar to other visual tests like histopathology, is subjective and requires sustained training and quality assurance. We developed, trained, and validated an artificial-intelligence-based "Automated Visual Evaluation" (AVE) tool that can be adapted to run on smartphones to assess smartphone-captured images of the cervix and identify precancerous lesions, helping augment performance of VIA. Design: Prospective study. Setting: Eight public health facilities in Zambia. Participants: 8,204 women aged 25-55. Interventions: Cervical images captured on commonly used low-cost smartphone models were matched with key clinical information including human immunodeficiency virus (HIV) and human papillomavirus (HPV) status, plus histopathology analysis (where applicable), to develop and train an AVE algorithm and evaluate its performance for use as a primary screen and triage test for women who are HPV positive. Main outcome measures: Area under the receiver operating curve (AUC); sensitivity; specificity. Results: As a general population screening for cervical precancerous lesions, AVE identified cases of cervical precancerous and cancerous (CIN2+) lesions with high performance (AUC = 0.91, 95% confidence interval [CI] = 0.89 to 0.93), which translates to a sensitivity of 85% (95% CI = 81% to 90%) and specificity of 86% (95% CI = 84% to 88%) based on maximizing the Youden's index. This represents a considerable improvement over VIA, which a meta-analysis by the World Health Organization (WHO) estimates to have sensitivity of 66% and specificity of 87%. For women living with HIV, the AUC of AVE was 0.91 (95% CI = 0.88 to 0.93), and among those testing positive for high-risk HPV types, the AUC was 0.87 (95% CI = 0.83 to 0.91). Conclusions: These results demonstrate the feasibility of utilizing AVE on images captured using a commonly available smartphone by screening nurses and support our transition to clinical evaluation of AVE's sensitivity, specificity, feasibility, and acceptability across a broader range of settings. The performance of the algorithm as reported may be inflated, as biopsies were obtained only from study participants with visible aceto-white cervical lesions, which can lead to verification bias; and the images and data sets used for testing of the model, although "unseen" by the algorithm during training, were acquired from the same set of patients and devices, limiting the study to that of an internal validation of the AVE algorithm.

Internal validation of Automated Visual Evaluation (AVE) on smartphone images for cervical cancer screening in a prospective study in Zambia.

OBJECTIVES: Visual inspection with acetic acid (VIA) is a low-cost approach for cervical cancer screening used in most low- and middle-income countries (LMICs) but, similar to other visual tests, is subjective and requires sustained training and quality assurance. We developed, trained, and validated an artificial-intelligence-based "Automated Visual Evaluation" (AVE) tool that can be adapted to run on smartphones to assess smartphone-captured images of the cervix and identify precancerous lesions, helping augment VIA performance. DESIGN: Prospective study. SETTING: Eight public health facilities in Zambia. PARTICIPANTS: A total of 8204 women aged 25-55. INTERVENTIONS: Cervical images captured on commonly used low-cost smartphone models were matched with key clinical information including human immunodeficiency virus (HIV) and human papillomavirus (HPV) status, plus histopathology analysis (where applicable), to develop and train an AVE algorithm and evaluate its performance for use as a primary screen and triage test for women who are HPV positive. MAIN OUTCOME MEASURES: Area under the receiver operating curve (AUC); sensitivity; specificity. RESULTS: As a general population screening tool for cervical precancerous lesions, AVE identified cases of cervical precancerous and cancerous (CIN2+) lesions with high performance (AUC = 0.91, 95% confidence interval [CI] = 0.89-0.93), which translates to a sensitivity of 85% (95% CI = 81%-90%) and specificity of 86% (95% CI = 84%-88%) based on maximizing the Youden's index. This represents a considerable improvement over naked eye VIA, which as per a meta-analysis by the World Health Organization (WHO) has a sensitivity of 66% and specificity of 87%. For women living with HIV, the AUC of AVE was 0.91 (95% CI = 0.88-0.93), and among those testing positive for high-risk HPV types, the AUC was 0.87 (95% CI = 0.83-0.91). CONCLUSIONS: These results demonstrate the feasibility of utilizing AVE on images captured using a commonly available smartphone by nurses in a screening program, and support our ongoing efforts for moving to more broadly evaluate AVE for its clinical sensitivity, specificity, feasibility, and acceptability across a wider range of settings. Limitations of this study include potential inflation of performance estimates due to verification bias (as biopsies were only obtained from participants with visible aceto-white cervical lesions) and due to this being an internal validation (the test data, while independent from that used to develop the algorithm was drawn from the same study).

Clinical validation of an open-access SARS-COV-2 antigen detection lateral flow assay, compared to commercially available assays.

Rapid tests for SARS-COV-2 infection are important tools for pandemic control, but current rapid tests are based on proprietary designs and reagents. We report clinical validation results of an open-access lateral flow assay (OA-LFA) design using commercially available materials and reagents, along with RT-qPCR and commercially available comparators (BinaxNOW® and Sofia®). Adult patients with suspected COVID-19 based on clinical signs and symptoms, and with symptoms ≤7 days duration, underwent anterior nares (AN) sampling for the OA-LFA, Sofia®, BinaxNOW ™, and RT-qPCR, along with nasopharyngeal (NP) RT-qPCR. Results indicate a positive predictive agreement with NP sampling as 69% (60% -78%) OA-LFA, 74% (64% - 82%) Sofia®, and 82% (73% - 88%) BinaxNOW™. The implication for these results is that we provide an open-access LFA design that meets the minimum WHO target product profile for a rapid test, that virtually any diagnostic manufacturer could produce.

Deep learning-based image evaluation for cervical precancer screening with a smartphone targeting low resource settings - Engineering approach.

Cervical cancer is the fourth most common cancer among women and still one of the major causes of women's death around the world. Early screening of high grade Cervical Intraepithelial Neoplasia (CIN), precursors to cervical cancer, is vital to efforts aimed at improving survival rate and eventually eliminating cervical cancer. Visual Inspection with Acetic acid (VIA) is an assessment method which can inspect the cervix and potentially detect lesions caused by human papillomavirus (HPV), which is a major cause of cervical cancer. VIA has the potential to be an effective screening method in low resource settings when triaged with HPV test, but it has the drawback that it depends on the subjective evaluation of health workers with varying levels of training. A new deep learning algorithm called Automated Visual Evaluation (AVE) for analyzing cervigram images has been recently reported that can automatically detect cervical precancer better than human experts. In this paper, we address the question of whether mobile phone-based cervical cancer screening is feasible. We consider the capabilities of two key components of a mobile phone platform for cervical cancer screening: (1) the core AVE algorithm and (2) an image quality algorithm. We consider both accuracy and speed in our assessment. We show that the core AVE algorithm, by refactoring to a new deep learning detection framework, can run in ~30 seconds on a low-end smartphone (i.e. Samsung J8), with equivalent accuracy. We developed an image quality algorithm that can localize the cervix and assess image quality in ~1 second on a low-end smartphone, achieving an area under the ROC curve (AUC) of 0.95. Field validation of the mobile phone platform for cervical cancer screening is in progress.