Internal Validation of Automated Visual Evaluation (AVE) on Smartphone Images for Cervical Cancer Screening in a Prospective Study in Zambia.

Objectives: Visual inspection with acetic acid (VIA) is a low-cost approach for cervical cancer screening used in most low- and middle-income countries (LMICs) but, similar to other visual tests like histopathology, is subjective and requires sustained training and quality assurance. We developed, trained, and validated an artificial-intelligence-based "Automated Visual Evaluation" (AVE) tool that can be adapted to run on smartphones to assess smartphone-captured images of the cervix and identify precancerous lesions, helping augment performance of VIA. Design: Prospective study. Setting: Eight public health facilities in Zambia. Participants: 8,204 women aged 25-55. Interventions: Cervical images captured on commonly used low-cost smartphone models were matched with key clinical information including human immunodeficiency virus (HIV) and human papillomavirus (HPV) status, plus histopathology analysis (where applicable), to develop and train an AVE algorithm and evaluate its performance for use as a primary screen and triage test for women who are HPV positive. Main outcome measures: Area under the receiver operating curve (AUC); sensitivity; specificity. Results: As a general population screening for cervical precancerous lesions, AVE identified cases of cervical precancerous and cancerous (CIN2+) lesions with high performance (AUC = 0.91, 95% confidence interval [CI] = 0.89 to 0.93), which translates to a sensitivity of 85% (95% CI = 81% to 90%) and specificity of 86% (95% CI = 84% to 88%) based on maximizing the Youden's index. This represents a considerable improvement over VIA, which a meta-analysis by the World Health Organization (WHO) estimates to have sensitivity of 66% and specificity of 87%. For women living with HIV, the AUC of AVE was 0.91 (95% CI = 0.88 to 0.93), and among those testing positive for high-risk HPV types, the AUC was 0.87 (95% CI = 0.83 to 0.91). Conclusions: These results demonstrate the feasibility of utilizing AVE on images captured using a commonly available smartphone by screening nurses and support our transition to clinical evaluation of AVE's sensitivity, specificity, feasibility, and acceptability across a broader range of settings. The performance of the algorithm as reported may be inflated, as biopsies were obtained only from study participants with visible aceto-white cervical lesions, which can lead to verification bias; and the images and data sets used for testing of the model, although "unseen" by the algorithm during training, were acquired from the same set of patients and devices, limiting the study to that of an internal validation of the AVE algorithm.

Field evaluation of the diagnostic performance of EasyScan GO: a digital malaria microscopy device based on machine-learning.

BACKGROUND: Microscopic examination of Giemsa-stained blood films remains the reference standard for malaria parasite detection and quantification, but is undermined by difficulties in ensuring high-quality manual reading and inter-reader reliability. Automated parasite detection and quantification may address this issue. METHODS: A multi-centre, observational study was conducted during 2018 and 2019 at 11 sites to assess the performance of the EasyScan Go, a microscopy device employing machine-learning-based image analysis. Sensitivity, specificity, accuracy of species detection and parasite density estimation were assessed with expert microscopy as the reference. Intra- and inter-device reliability of the device was also evaluated by comparing results from repeat reads on the same and two different devices. This study has been reported in accordance with the Standards for Reporting Diagnostic accuracy studies (STARD) checklist. RESULTS: In total, 2250 Giemsa-stained blood films were prepared and read independently by expert microscopists and the EasyScan Go device. The diagnostic sensitivity of EasyScan Go was 91.1% (95% CI 88.9-92.7), and specificity 75.6% (95% CI 73.1-78.0). With good quality slides sensitivity was similar (89.1%, 95%CI 86.2-91.5), but specificity increased to 85.1% (95%CI 82.6-87.4). Sensitivity increased with parasitaemia rising from 57% at < 200 parasite/µL, to ≥ 90% at > 200-200,000 parasite/µL. Species were identified accurately in 93% of Plasmodium falciparum samples (kappa = 0.76, 95% CI 0.69-0.83), and in 92% of Plasmodium vivax samples (kappa = 0.73, 95% CI 0.66-0.80). Parasite density estimates by the EasyScan Go were within ± 25% of the microscopic reference counts in 23% of slides. CONCLUSIONS: The performance of the EasyScan Go in parasite detection and species identification accuracy fulfil WHO-TDR Research Malaria Microscopy competence level 2 criteria. In terms of parasite quantification and false positive rate, it meets the level 4 WHO-TDR Research Malaria Microscopy criteria. All performance parameters were significantly affected by slide quality. Further software improvement is required to improve sensitivity at low parasitaemia and parasite density estimations. Trial registration ClinicalTrials.gov number NCT03512678.

Performance of a fully-automated system on a WHO malaria microscopy evaluation slide set.

BACKGROUND: Manual microscopy remains a widely-used tool for malaria diagnosis and clinical studies, but it has inconsistent quality in the field due to variability in training and field practices. Automated diagnostic systems based on machine learning hold promise to improve quality and reproducibility of field microscopy. The World Health Organization (WHO) has designed a 55-slide set (WHO 55) for their External Competence Assessment of Malaria Microscopists (ECAMM) programme, which can also serve as a valuable benchmark for automated systems. The performance of a fully-automated malaria diagnostic system, EasyScan GO, on a WHO 55 slide set was evaluated. METHODS: The WHO 55 slide set is designed to evaluate microscopist competence in three areas of malaria diagnosis using Giemsa-stained blood films, focused on crucial field needs: malaria parasite detection, malaria parasite species identification (ID), and malaria parasite quantitation. The EasyScan GO is a fully-automated system that combines scanning of Giemsa-stained blood films with assessment algorithms to deliver malaria diagnoses. This system was tested on a WHO 55 slide set. RESULTS: The EasyScan GO achieved 94.3 % detection accuracy, 82.9 % species ID accuracy, and 50 % quantitation accuracy, corresponding to WHO microscopy competence Levels 1, 2, and 1, respectively. This is, to our knowledge, the best performance of a fully-automated system on a WHO 55 set. CONCLUSIONS: EasyScan GO's expert ratings in detection and quantitation on the WHO 55 slide set point towards its potential value in drug efficacy use-cases, as well as in some case management situations with less stringent species ID needs. Improved runtime may enable use in general case management settings.

Deep learning-based image evaluation for cervical precancer screening with a smartphone targeting low resource settings - Engineering approach.

Cervical cancer is the fourth most common cancer among women and still one of the major causes of women's death around the world. Early screening of high grade Cervical Intraepithelial Neoplasia (CIN), precursors to cervical cancer, is vital to efforts aimed at improving survival rate and eventually eliminating cervical cancer. Visual Inspection with Acetic acid (VIA) is an assessment method which can inspect the cervix and potentially detect lesions caused by human papillomavirus (HPV), which is a major cause of cervical cancer. VIA has the potential to be an effective screening method in low resource settings when triaged with HPV test, but it has the drawback that it depends on the subjective evaluation of health workers with varying levels of training. A new deep learning algorithm called Automated Visual Evaluation (AVE) for analyzing cervigram images has been recently reported that can automatically detect cervical precancer better than human experts. In this paper, we address the question of whether mobile phone-based cervical cancer screening is feasible. We consider the capabilities of two key components of a mobile phone platform for cervical cancer screening: (1) the core AVE algorithm and (2) an image quality algorithm. We consider both accuracy and speed in our assessment. We show that the core AVE algorithm, by refactoring to a new deep learning detection framework, can run in ~30 seconds on a low-end smartphone (i.e. Samsung J8), with equivalent accuracy. We developed an image quality algorithm that can localize the cervix and assess image quality in ~1 second on a low-end smartphone, achieving an area under the ROC curve (AUC) of 0.95. Field validation of the mobile phone platform for cervical cancer screening is in progress.

An Observational Study of Deep Learning and Automated Evaluation of Cervical Images for Cancer Screening.

BACKGROUND: Human papillomavirus vaccination and cervical screening are lacking in most lower resource settings, where approximately 80% of more than 500 000 cancer cases occur annually. Visual inspection of the cervix following acetic acid application is practical but not reproducible or accurate. The objective of this study was to develop a "deep learning"-based visual evaluation algorithm that automatically recognizes cervical precancer/cancer. METHODS: A population-based longitudinal cohort of 9406 women ages 18-94 years in Guanacaste, Costa Rica was followed for 7 years (1993-2000), incorporating multiple cervical screening methods and histopathologic confirmation of precancers. Tumor registry linkage identified cancers up to 18 years. Archived, digitized cervical images from screening, taken with a fixed-focus camera ("cervicography"), were used for training/validation of the deep learning-based algorithm. The resultant image prediction score (0-1) could be categorized to balance sensitivity and specificity for detection of precancer/cancer. All statistical tests were two-sided. RESULTS: Automated visual evaluation of enrollment cervigrams identified cumulative precancer/cancer cases with greater accuracy (area under the curve [AUC] = 0.91, 95% confidence interval [CI] = 0.89 to 0.93) than original cervigram interpretation (AUC = 0.69, 95% CI = 0.63 to 0.74; P < .001) or conventional cytology (AUC = 0.71, 95% CI = 0.65 to 0.77; P < .001). A single visual screening round restricted to women at the prime screening ages of 25-49 years could identify 127 (55.7%) of 228 precancers (cervical intraepithelial neoplasia 2/cervical intraepithelial neoplasia 3/adenocarcinoma in situ [AIS]) diagnosed cumulatively in the entire adult population (ages 18-94 years) while referring 11.0% for management. CONCLUSIONS: The results support consideration of automated visual evaluation of cervical images from contemporary digital cameras. If achieved, this might permit dissemination of effective point-of-care cervical screening.

A paper microfluidic cartridge for automated staining of malaria parasites with an optically transparent microscopy window.

A paper microfluidic cartridge for the automated staining of malaria parasites (Plasmodium) with acridine orange prior to microscopy is presented. The cartridge enables simultaneous, sub-minute generation of both thin and thick smears of acridine orange stained parasites. Parasites are stained in a cellulose matrix, after which the parasites are ejected via capillary forces into an optically transparent chamber. The unique slanted design of the chamber ensures that a high percentage of the stained blood will be of the required thickness for a thin smear, without resorting to spacers or other methods that can increase production cost or require tight quality controls. A hydrophobic snorkel facilitates the removal of air bubbles during filling. The cartridge contains both a thin smear region, where a single layer of cells is presented unobstructed, for ease of species identification, and a thick smear region, containing multiple cell layers, for enhanced limit of detection.

Limitations of haemozoin-based diagnosis of Plasmodium falciparum using dark-field microscopy.

BACKGROUND: The haemozoin crystal continues to be investigated extensively for its potential as a biomarker for malaria diagnostics. In order for haemozoin to be a valuable biomarker, it must be present in detectable quantities in the peripheral blood and distinguishable from false positives. Here, dark-field microscopy coupled with sophisticated image processing algorithms is used to characterize the abundance of detectable haemozoin within infected erythrocytes from field samples in order to determine the window of detection in peripheral blood. METHODS: Thin smears from Plasmodium falciparum-infected and uninfected patients were imaged in both dark field (DF) unstained and bright field (BF) Giemsa-stained modes. The images were co-registered such that each parasite had thumbnails in both BF and DF modes, providing an accurate map between parasites and DF objects. This map was used to find the abundance of haemozoin as a function of parasite stage through careful parasite staging and correlation with DF objects. An automated image-processing and classification algorithm classified the bright spots in the DF images as either haemozoin or non-haemozoin objects. RESULTS: The algorithm distinguishes haemozoin from non-haemozoin objects in DF images with an object-level sensitivity of 95% and specificity of 97%. Ring stages older than about 6 hours begin to show detectable haemozoin, and rings between 10-16 hours reliably contain detectable haemozoin. However, DF microscopy coupled with the image-processing algorithm detect no haemozoin in rings younger than six hours. DISCUSSION: Although this method demonstrates the most sensitive detection of haemozoin in field samples reported to date, it does not detect haemozoin in ring-stage parasites younger than six hours. Thus, haemozoin is a poor biomarker for field samples primarily composed of young ring-stage parasites because the crystal is not present in detectable quantities by the methods described here. Based on these results, the implications for patient-level diagnosis and recommendations for future work are discussed.

Detection of malarial byproduct hemozoin utilizing its unique scattering properties.

The scattering characteristics of the malaria byproduct hemozoin, including its scattering distribution and depolarization, are modeled using Discrete Dipole Approximation (DDA) and compared to those of healthy red blood cells. Scattering (or dark-field) spectroscopy and imaging are used to identify hemozoin in fresh rodent blood samples. A new detection method is proposed and demonstrated using dark-field in conjunction with cross-polarization imaging and spectroscopy. SNRs greater than 50:1 are achieved for hemozoin in fresh blood without the addition of stains or reagents. The potential of such a detection system is discussed.