Field evaluation of the diagnostic performance of EasyScan GO: a digital malaria microscopy device based on machine-learning.

BACKGROUND: Microscopic examination of Giemsa-stained blood films remains the reference standard for malaria parasite detection and quantification, but is undermined by difficulties in ensuring high-quality manual reading and inter-reader reliability. Automated parasite detection and quantification may address this issue. METHODS: A multi-centre, observational study was conducted during 2018 and 2019 at 11 sites to assess the performance of the EasyScan Go, a microscopy device employing machine-learning-based image analysis. Sensitivity, specificity, accuracy of species detection and parasite density estimation were assessed with expert microscopy as the reference. Intra- and inter-device reliability of the device was also evaluated by comparing results from repeat reads on the same and two different devices. This study has been reported in accordance with the Standards for Reporting Diagnostic accuracy studies (STARD) checklist. RESULTS: In total, 2250 Giemsa-stained blood films were prepared and read independently by expert microscopists and the EasyScan Go device. The diagnostic sensitivity of EasyScan Go was 91.1% (95% CI 88.9-92.7), and specificity 75.6% (95% CI 73.1-78.0). With good quality slides sensitivity was similar (89.1%, 95%CI 86.2-91.5), but specificity increased to 85.1% (95%CI 82.6-87.4). Sensitivity increased with parasitaemia rising from 57% at < 200 parasite/µL, to ≥ 90% at > 200-200,000 parasite/µL. Species were identified accurately in 93% of Plasmodium falciparum samples (kappa = 0.76, 95% CI 0.69-0.83), and in 92% of Plasmodium vivax samples (kappa = 0.73, 95% CI 0.66-0.80). Parasite density estimates by the EasyScan Go were within ± 25% of the microscopic reference counts in 23% of slides. CONCLUSIONS: The performance of the EasyScan Go in parasite detection and species identification accuracy fulfil WHO-TDR Research Malaria Microscopy competence level 2 criteria. In terms of parasite quantification and false positive rate, it meets the level 4 WHO-TDR Research Malaria Microscopy criteria. All performance parameters were significantly affected by slide quality. Further software improvement is required to improve sensitivity at low parasitaemia and parasite density estimations. Trial registration ClinicalTrials.gov number NCT03512678.

Performance of a fully-automated system on a WHO malaria microscopy evaluation slide set.

BACKGROUND: Manual microscopy remains a widely-used tool for malaria diagnosis and clinical studies, but it has inconsistent quality in the field due to variability in training and field practices. Automated diagnostic systems based on machine learning hold promise to improve quality and reproducibility of field microscopy. The World Health Organization (WHO) has designed a 55-slide set (WHO 55) for their External Competence Assessment of Malaria Microscopists (ECAMM) programme, which can also serve as a valuable benchmark for automated systems. The performance of a fully-automated malaria diagnostic system, EasyScan GO, on a WHO 55 slide set was evaluated. METHODS: The WHO 55 slide set is designed to evaluate microscopist competence in three areas of malaria diagnosis using Giemsa-stained blood films, focused on crucial field needs: malaria parasite detection, malaria parasite species identification (ID), and malaria parasite quantitation. The EasyScan GO is a fully-automated system that combines scanning of Giemsa-stained blood films with assessment algorithms to deliver malaria diagnoses. This system was tested on a WHO 55 slide set. RESULTS: The EasyScan GO achieved 94.3 % detection accuracy, 82.9 % species ID accuracy, and 50 % quantitation accuracy, corresponding to WHO microscopy competence Levels 1, 2, and 1, respectively. This is, to our knowledge, the best performance of a fully-automated system on a WHO 55 set. CONCLUSIONS: EasyScan GO's expert ratings in detection and quantitation on the WHO 55 slide set point towards its potential value in drug efficacy use-cases, as well as in some case management situations with less stringent species ID needs. Improved runtime may enable use in general case management settings.

Deep learning-based image evaluation for cervical precancer screening with a smartphone targeting low resource settings - Engineering approach.

Cervical cancer is the fourth most common cancer among women and still one of the major causes of women's death around the world. Early screening of high grade Cervical Intraepithelial Neoplasia (CIN), precursors to cervical cancer, is vital to efforts aimed at improving survival rate and eventually eliminating cervical cancer. Visual Inspection with Acetic acid (VIA) is an assessment method which can inspect the cervix and potentially detect lesions caused by human papillomavirus (HPV), which is a major cause of cervical cancer. VIA has the potential to be an effective screening method in low resource settings when triaged with HPV test, but it has the drawback that it depends on the subjective evaluation of health workers with varying levels of training. A new deep learning algorithm called Automated Visual Evaluation (AVE) for analyzing cervigram images has been recently reported that can automatically detect cervical precancer better than human experts. In this paper, we address the question of whether mobile phone-based cervical cancer screening is feasible. We consider the capabilities of two key components of a mobile phone platform for cervical cancer screening: (1) the core AVE algorithm and (2) an image quality algorithm. We consider both accuracy and speed in our assessment. We show that the core AVE algorithm, by refactoring to a new deep learning detection framework, can run in ~30 seconds on a low-end smartphone (i.e. Samsung J8), with equivalent accuracy. We developed an image quality algorithm that can localize the cervix and assess image quality in ~1 second on a low-end smartphone, achieving an area under the ROC curve (AUC) of 0.95. Field validation of the mobile phone platform for cervical cancer screening is in progress.

Design and feasibility of a novel program of cervical screening in Nigeria: self-sampled HPV testing paired with visual triage.

BACKGROUND: Accelerated global control of cervical cancer would require primary prevention with human papillomavirus (HPV) vaccination in addition to novel screening program strategies that are simple, inexpensive, and effective. We present the feasibility and outcome of a community-based HPV self-sampled screening program. METHODS: In Ile Ife, Nigeria, 9406 women aged 30-49 years collected vaginal self-samples, which were tested for HPV in the local study laboratory using Hybrid Capture-2 (HC2) (Qiagen). HPV-positive women were referred to the colposcopy clinic. Gynecologist colposcopic impression dictated immediate management; biopsies were taken when definite acetowhitening was present to produce a histopathologic reference standard of precancer (and to determine final clinical management). Retrospective linkage to the medical records identified 442 of 9406 women living with HIV (WLWH). RESULTS: With self-sampling, it was possible to screen more than 100 women per day per clinic. Following an audio-visual presentation and in-person instructions, overall acceptability of self-sampling was very high (81.2% women preferring self-sampling over clinician collection). HPV positivity was found in 17.3% of women. Intensive follow-up contributed to 85.9% attendance at the colposcopy clinic. Of those referred, 8.2% were initially treated with thermal ablation and 5.6% with large loop excision of transformation zone (LLETZ). Full visibility of the squamocolumnar junction, necessary for optimal visual triage and ablation, declined from 68.5% at age 30 to 35.4% at age 49. CIN2+ and CIN3+ (CIN- Cervical intraepithelial neoplasia), including five cancers, were identified by histology in 5.9 and 3.2% of the HPV-positive women, respectively (0.9 and 0.5% of the total screening population), leading to additional treatment as indicated. The prevalences of HPV infection and CIN2+ were substantially higher (40.5 and 2.5%, respectively) among WLWH. Colposcopic impression led to over- and under-treatment compared to the histopathology reference standard. CONCLUSION: A cervical cancer screening program using self-sampled HPV testing, with colposcopic immediate management of women positive for HPV, proved feasible in Nigeria. Based on the collected specimens and images, we are now evaluating the use of a combination of partial HPV typing and automated visual evaluation (AVE) of cervical images to improve the accuracy of the screening program.

Automated microscopy for routine malaria diagnosis: a field comparison on Giemsa-stained blood films in Peru.

BACKGROUND: Microscopic examination of Giemsa-stained blood films remains a major form of diagnosis in malaria case management, and is a reference standard for research. However, as with other visualization-based diagnoses, accuracy depends on individual technician performance, making standardization difficult and reliability poor. Automated image recognition based on machine-learning, utilizing convolutional neural networks, offers potential to overcome these drawbacks. A prototype digital microscope device employing an algorithm based on machine-learning, the Autoscope, was assessed for its potential in malaria microscopy. Autoscope was tested in the Iquitos region of Peru in 2016 at two peripheral health facilities, with routine microscopy and PCR as reference standards. The main outcome measures include sensitivity and specificity of diagnosis of malaria from Giemsa-stained blood films, using PCR as reference. METHODS: A cross-sectional, observational trial was conducted at two peripheral primary health facilities in Peru. 700 participants were enrolled with the criteria: (1) age between 5 and 75 years, (2) history of fever in the last 3 days or elevated temperature on admission, (3) informed consent. The main outcome measures included sensitivity and specificity of diagnosis of malaria from Giemsa-stained blood films, using PCR as reference. RESULTS: At the San Juan clinic, sensitivity of Autoscope for diagnosing malaria was 72% (95% CI 64-80%), and specificity was 85% (95% CI 79-90%). Microscopy performance was similar to Autoscope, with sensitivity 68% (95% CI 59-76%) and specificity 100% (95% CI 98-100%). At San Juan, 85% of prepared slides had a minimum of 600 WBCs imaged, thus meeting Autoscope's design assumptions. At the second clinic, Santa Clara, the sensitivity of Autoscope was 52% (95% CI 44-60%) and specificity was 70% (95% CI 64-76%). Microscopy performance at Santa Clara was 42% (95% CI 34-51) and specificity was 97% (95% CI 94-99). Only 39% of slides from Santa Clara met Autoscope's design assumptions regarding WBCs imaged. CONCLUSIONS: Autoscope's diagnostic performance was on par with routine microscopy when slides had adequate blood volume to meet its design assumptions, as represented by results from the San Juan clinic. Autoscope's diagnostic performance was poorer than routine microscopy on slides from the Santa Clara clinic, which generated slides with lower blood volumes. Results of the study reflect both the potential for artificial intelligence to perform tasks currently conducted by highly-trained experts, and the challenges of replicating the adaptiveness of human thought processes.