Validation of a deep learning system for the detection of diabetic retinopathy in Indigenous Australians.

BACKGROUND/AIMS: Deep learning systems (DLSs) for diabetic retinopathy (DR) detection show promising results but can underperform in racial and ethnic minority groups, therefore external validation within these populations is critical for health equity. This study evaluates the performance of a DLS for DR detection among Indigenous Australians, an understudied ethnic group who suffer disproportionately from DR-related blindness. METHODS: We performed a retrospective external validation study comparing the performance of a DLS against a retinal specialist for the detection of more-than-mild DR (mtmDR), vision-threatening DR (vtDR) and all-cause referable DR. The validation set consisted of 1682 consecutive, single-field, macula-centred retinal photographs from 864 patients with diabetes (mean age 54.9 years, 52.4% women) at an Indigenous primary care service in Perth, Australia. Three-person adjudication by a panel of specialists served as the reference standard. RESULTS: For mtmDR detection, sensitivity of the DLS was superior to the retina specialist (98.0% (95% CI, 96.5 to 99.4) vs 87.1% (95% CI, 83.6 to 90.6), McNemar's test p<0.001) with a small reduction in specificity (95.1% (95% CI, 93.6 to 96.4) vs 97.0% (95% CI, 95.9 to 98.0), p=0.006). For vtDR, the DLS's sensitivity was again superior to the human grader (96.2% (95% CI, 93.4 to 98.6) vs 84.4% (95% CI, 79.7 to 89.2), p<0.001) with a slight drop in specificity (95.8% (95% CI, 94.6 to 96.9) vs 97.8% (95% CI, 96.9 to 98.6), p=0.002). For all-cause referable DR, there was a substantial increase in sensitivity (93.7% (95% CI, 91.8 to 95.5) vs 74.4% (95% CI, 71.1 to 77.5), p<0.001) and a smaller reduction in specificity (91.7% (95% CI, 90.0 to 93.3) vs 96.3% (95% CI, 95.2 to 97.4), p<0.001). CONCLUSION: The DLS showed improved sensitivity and similar specificity compared with a retina specialist for DR detection. This demonstrates its potential to support DR screening among Indigenous Australians, an underserved population with a high burden of diabetic eye disease.

Enhancing care quality and accessibility through digital technology-supported decentralisation of hypertension and diabetes management: a proof-of-concept study in rural Bangladesh.

OBJECTIVE: The critical shortage of healthcare workers, particularly in rural areas, is a major barrier to quality care for non-communicable diseases (NCD) in low-income and middle-income countries. In this proof-of-concept study, we aimed to test a decentralised model for integrated diabetes and hypertension management in rural Bangladesh to improve accessibility and quality of care. DESIGN AND SETTING: The study is a single-cohort proof-of-concept study. The key interventions comprised shifting screening, routine monitoring and dispensing of medication refills from a doctor-managed subdistrict NCD clinic to non-physician health worker-managed village-level community clinics; a digital care coordination platform was developed for electronic health records, point-of-care support, referral and routine patient follow-up. The study was conducted in the Parbatipur subdistrict, Rangpur Division, Bangladesh. PARTICIPANTS: A total of 624 participants were enrolled in the study (mean (SD) age, 59.5 (12.0); 65.1% female). OUTCOMES: Changes in blood pressure and blood glucose control, patient retention and patient-visit volume at the NCD clinic and community clinics. RESULTS: The proportion of patients with uncontrolled blood pressure reduced from 60% at baseline to 26% at the third month of follow-up, a 56% (incidence rate ratio 0.44; 95% CI 0.33 to 0.57) reduction after adjustment for covariates. The proportion of patients with uncontrolled blood glucose decreased from 74% to 43% at the third month of follow-up. Attrition rates immediately after baseline and during the entire study period were 29.1% and 36.2%, respectively. CONCLUSION: The proof-of-concept study highlights the potential for involving lower-level primary care facilities and non-physician health workers to rapidly expand much-needed services to patients with hypertension and diabetes in Bangladesh and in similar global settings. Further investigations are needed to evaluate the effectiveness of decentralised hypertension and diabetes care.

Cost-Utility Analysis of Deep Learning and Trained Human Graders for Diabetic Retinopathy Screening in a Nationwide Program.

INTRODUCTION: Deep learning (DL) for screening diabetic retinopathy (DR) has the potential to address limited healthcare resources by enabling expanded access to healthcare. However, there is still limited health economic evaluation, particularly in low- and middle-income countries, on this subject to aid decision-making for DL adoption. METHODS: In the context of a middle-income country (MIC), using Thailand as a model, we constructed a decision tree-Markov hybrid model to estimate lifetime costs and outcomes of Thailand's national DR screening program via DL and trained human graders (HG). We calculated the incremental cost-effectiveness ratio (ICER) between the two strategies. Sensitivity analyses were performed to probe the influence of modeling parameters. RESULTS: From a societal perspective, screening with DL was associated with a reduction in costs of ~ US$ 2.70, similar quality-adjusted life-years (QALY) of + 0.0043, and an incremental net monetary benefit of ~ US$ 24.10 in the base case. In sensitivity analysis, DL remained cost-effective even with a price increase from US$ 1.00 to US$ 4.00 per patient at a Thai willingness-to-pay threshold of ~ US$ 4.997 per QALY gained. When further incorporating recent findings suggesting improved compliance to treatment referral with DL, our analysis models effectiveness benefits of ~ US$ 20 to US$ 50 depending on compliance. CONCLUSION: DR screening using DL in an MIC using Thailand as a model may result in societal cost-savings and similar health outcomes compared with HG. This study may provide an economic rationale to expand DL-based DR screening in MICs as an alternative solution for limited availability of skilled human resources for primary screening, particularly in MICs with similar prevalence of diabetes and low compliance to referrals for treatment.

Real-time diabetic retinopathy screening by deep learning in a multisite national screening programme: a prospective interventional cohort study.

BACKGROUND: Diabetic retinopathy is a leading cause of preventable blindness, especially in low-income and middle-income countries (LMICs). Deep-learning systems have the potential to enhance diabetic retinopathy screenings in these settings, yet prospective studies assessing their usability and performance are scarce. METHODS: We did a prospective interventional cohort study to evaluate the real-world performance and feasibility of deploying a deep-learning system into the health-care system of Thailand. Patients with diabetes and listed on the national diabetes registry, aged 18 years or older, able to have their fundus photograph taken for at least one eye, and due for screening as per the Thai Ministry of Public Health guidelines were eligible for inclusion. Eligible patients were screened with the deep-learning system at nine primary care sites under Thailand's national diabetic retinopathy screening programme. Patients with a previous diagnosis of diabetic macular oedema, severe non-proliferative diabetic retinopathy, or proliferative diabetic retinopathy; previous laser treatment of the retina or retinal surgery; other non-diabetic retinopathy eye disease requiring referral to an ophthalmologist; or inability to have fundus photograph taken of both eyes for any reason were excluded. Deep-learning system-based interpretations of patient fundus images and referral recommendations were provided in real time. As a safety mechanism, regional retina specialists over-read each image. Performance of the deep-learning system (accuracy, sensitivity, specificity, positive predictive value [PPV], and negative predictive value [NPV]) were measured against an adjudicated reference standard, provided by fellowship-trained retina specialists. This study is registered with the Thai national clinical trials registry, TCRT20190902002. FINDINGS: Between Dec 12, 2018, and March 29, 2020, 7940 patients were screened for inclusion. 7651 (96·3%) patients were eligible for study analysis, and 2412 (31·5%) patients were referred for diabetic retinopathy, diabetic macular oedema, ungradable images, or low visual acuity. For vision-threatening diabetic retinopathy, the deep-learning system had an accuracy of 94·7% (95% CI 93·0-96·2), sensitivity of 91·4% (87·1-95·0), and specificity of 95·4% (94·1-96·7). The retina specialist over-readers had an accuracy of 93·5 (91·7-95·0; p=0·17), a sensitivity of 84·8% (79·4-90·0; p=0·024), and specificity of 95·5% (94·1-96·7; p=0·98). The PPV for the deep-learning system was 79·2 (95% CI 73·8-84·3) compared with 75·6 (69·8-81·1) for the over-readers. The NPV for the deep-learning system was 95·5 (92·8-97·9) compared with 92·4 (89·3-95·5) for the over-readers. INTERPRETATION: A deep-learning system can deliver real-time diabetic retinopathy detection capability similar to retina specialists in community-based screening settings. Socioenvironmental factors and workflows must be taken into consideration when implementing a deep-learning system within a large-scale screening programme in LMICs. FUNDING: Google and Rajavithi Hospital, Bangkok, Thailand. TRANSLATION: For the Thai translation of the abstract see Supplementary Materials section.

Longitudinal Screening for Diabetic Retinopathy in a Nationwide Screening Program: Comparing Deep Learning and Human Graders.

OBJECTIVE: To evaluate diabetic retinopathy (DR) screening via deep learning (DL) and trained human graders (HG) in a longitudinal cohort, as case spectrum shifts based on treatment referral and new-onset DR. METHODS: We randomly selected patients with diabetes screened twice, two years apart within a nationwide screening program. The reference standard was established via adjudication by retina specialists. Each patient's color fundus photographs were graded, and a patient was considered as having sight-threatening DR (STDR) if the worse eye had severe nonproliferative DR, proliferative DR, or diabetic macular edema. We compared DR screening via two modalities: DL and HG. For each modality, we simulated treatment referral by excluding patients with detected STDR from the second screening using that modality. RESULTS: There were 5,738 patients (12.3% STDR) in the first screening. DL and HG captured different numbers of STDR cases, and after simulated referral and excluding ungradable cases, 4,148 and 4,263 patients remained in the second screening, respectively. The STDR prevalence at the second screening was 5.1% and 6.8% for DL- and HG-based screening, respectively. Along with the prevalence decrease, the sensitivity for both modalities decreased from the first to the second screening (DL: from 95% to 90%, p = 0.008; HG: from 74% to 57%, p < 0.001). At both the first and second screenings, the rate of false negatives for the DL was a fifth that of HG (0.5-0.6% vs. 2.9-3.2%). CONCLUSION: On 2-year longitudinal follow-up of a DR screening cohort, STDR prevalence decreased for both DL- and HG-based screening. Follow-up screenings in longitudinal DR screening can be more difficult and induce lower sensitivity for both DL and HG, though the false negative rate was substantially lower for DL. Our data may be useful for health-economics analyses of longitudinal screening settings.

HealthNavigator: a mobile application for chronic disease screening and linkage to services at an urban Primary Health Network.

Mobile applications (apps) are promising tools to support chronic disease screening and linkage to health services. They have the potential to increase healthcare access for vulnerable populations. The HealthNavigator app was developed to provide chronic disease risk assessments, linkage to local general practitioners (GPs) and lifestyle programs, and a personalised health report for discussion with a GP. Assessments were either self-administered or facilitated by community health workers through a Primary Health Network (PHN) initiative targeting ethnically diverse communities. In total, 1492 assessments (80.4% self-administered, 19.6% facilitated) were conducted over a 12-month period in Queensland, Australia. Of these, 26% of people screened came from postcodes representing the lowest quartile of socioeconomic disadvantage. When compared against self-administered assessments, subjects screened by the facilitated program were more likely to be born outside Australia (80.5 v. 33.2%, P<0.001), and to fall within a high risk category based on cardiovascular risk scores (19.8 v. 13.7%, P<0.01) and type 2 diabetes mellitus risk scores (58.0 v. 40.1%, P<0.001). Mobile apps embedded into PHN programs may be a useful adjunct for the implementation of community screening programs. Further research is needed to determine their effect on health service access and health outcomes.

Supporting Better Evidence Generation and Use within Social Innovation in Health in Low- and Middle-Income Countries: A Qualitative Study.

BACKGROUND: While several papers have highlighted a lack of evidence to scale social innovations in health, fewer have explored decision-maker understandings of the relative merit of different types of evidence, how such data are interpreted and applied, and what practical support is required to improve evidence generation. The objectives of this paper are to understand (1) beliefs and attitudes towards the value of and types of evidence in scaling social innovations for health, (2) approaches to evidence generation and evaluation used in systems and policy change, and (3) how better evidence-generation can be undertaken and supported within social innovation in health. METHODS: Thirty-two one-on-one interviews were conducted between July and November 2015 with purposively selected practitioners, policymakers, and funders from low- and middle- income countries (LMICs). Data were analysed using a Framework Analysis Approach. RESULTS: While practitioners, funders, and policymakers said they held outcome evidence in high regard, their practices only bear out this assertion to varying degrees. Few have given systematic consideration to potential unintended consequences, in particular harm, of the programs they implement, fund, or adopt. Stakeholders suggest that better evidence-generation can be undertaken and supported within social innovation in health by supporting the research efforts of emerging community organizations; creating links between practitioners and academia; altering the funding landscape for evidence-generation; providing responsive technical education; and creating accountability for funders, practitioners, and policymakers. CONCLUSION: How better evidence-generation can be undertaken and supported within social innovation in health is a previously under-operationalised aspect of the policy-making process that remains essential in order to refrain from causing harm, enable the optimization of existing interventions, and ultimately, to scale and fund what works.

An integrated general practice and pharmacy-based intervention to promote the use of appropriate preventive medications among individuals at high cardiovascular disease risk: protocol for a cluster randomized controlled trial.

BACKGROUND: Cardiovascular diseases (CVD) are responsible for significant morbidity, premature mortality, and economic burden. Despite established evidence that supports the use of preventive medications among patients at high CVD risk, treatment gaps remain. Building on prior evidence and a theoretical framework, a complex intervention has been designed to address these gaps among high-risk, under-treated patients in the Australian primary care setting. This intervention comprises a general practice quality improvement tool incorporating clinical decision support and audit/feedback capabilities; availability of a range of CVD polypills (fixed-dose combinations of two blood pressure lowering agents, a statin ± aspirin) for prescription when appropriate; and access to a pharmacy-based program to support long-term medication adherence and lifestyle modification. METHODS: Following a systematic development process, the intervention will be evaluated in a pragmatic cluster randomized controlled trial including 70 general practices for a median period of 18 months. The 35 general practices in the intervention group will work with a nominated partner pharmacy, whereas those in the control group will provide usual care without access to the intervention tools. The primary outcome is the proportion of patients at high CVD risk who were inadequately treated at baseline who achieve target blood pressure (BP) and low-density lipoprotein cholesterol (LDL-C) levels at the study end. The outcomes will be analyzed using data from electronic medical records, utilizing a validated extraction tool. Detailed process and economic evaluations will also be performed. DISCUSSION: The study intends to establish evidence about an intervention that combines technological innovation with team collaboration between patients, pharmacists, and general practitioners (GPs) for CVD prevention. TRIAL REGISTRATION: Australian New Zealand Clinical Trials Registry ACTRN12616000233426.

Development of an integrated e-health tool for people with, or at high risk of, cardiovascular disease: The Consumer Navigation of Electronic Cardiovascular Tools (CONNECT) web application.

BACKGROUND: Cardiovascular disease is the leading killer globally and secondary prevention substantially reduces risk. Uptake of, and adherence to, face-to-face preventive programs is often low. Alternative models of care are exploiting the prominence of technology in daily life to facilitate lifestyle behavior change. OBJECTIVE: To inform the development of a web-based application integrated with the primary care electronic health record, we undertook a collaborative user-centered design process to develop a consumer-focused e-health tool for cardiovascular disease risk reduction. METHODS: A four-phase iterative process involved ten multidisciplinary clinicians and academics (primary care physician, nurses and allied health professionals), two design consultants, one graphic designer, three software developers and fourteen proposed end-users. This 18-month process involved, (1) defining the target audience and needs, (2) pilot testing and refinement, (3) software development including validation and testing the algorithm, (4) user acceptance testing and beta testing. From this process, researchers were able to better understand end-user needs and preferences, thereby improving and enriching the increasingly detailed system designs and prototypes for a mobile responsive web application. RESULTS: We reviewed 14 relevant applications/websites and sixteen observational and interventional studies to derive a set of core components and ideal features for the system. These included the need for interactivity, visual appeal, credible health information, virtual rewards, and emotional and physical support. The features identified as essential were: (i) both mobile and web-enabled 'apps', (ii) an emphasis on medication management, (iii) a strong psychosocial support component. Subsequent workshops (n=6; 2×1.5h) informed the development of functionality and lo-fidelity sketches of application interfaces. These ideas were next tested in consumer focus groups (n=9; 3×1.5h). Specifications for the application were refined from this feedback and a graphic designer iteratively developed the interface. Concurrently, the electronic health record was linked to the consumer portal. A written description of the final algorithms for all decisions and outputs was provided to software programmers. These algorithmic outputs to the app were first validated against those obtained from an independently programmed version in STATA 11. User acceptance testing (n=5, 2×1.0h) and beta testing revealed technical bugs and interface concerns across commonly-used web browsers and smartphones. These were resolved and re-tested until functionality was optimized. CONCLUSION: End-users of a cardiovascular disease prevention program have complex needs. A user-centered design approach aided the integration of these needs into the concept, specifications, development and refinement of a responsive web application for risk factor reduction and disease prevention.

A user-centred home monitoring and self-management system for patients with heart failure: a multicentre cohort study.

AIMS: Previous generations of home monitoring systems have had limited usability. We aimed to develop and evaluate a user-centred and adaptive system for health monitoring and self-management support in patients with heart failure. METHODS AND RESULTS: Patients with heart failure were recruited from three UK centres and provided with Internet-enabled tablet computers that were wirelessly linked with sensor devices for blood pressure, heart rate, and weight monitoring. Patient observations, interviews, and concurrent analyses of the automatically collected data from their monitoring devices were used to increase the usability of the system. Of the 52 participants (median age 77 years, median follow-up 6 months [interquartile range, IQR, 3.6-9.2]), 24 (46%) had no, or very limited prior, experience with digital technologies. It took participants about 1.5 min to complete the daily monitoring tasks, and the rate of failed attempts in completing tasks was <5%. After 45 weeks of observation, participants still used the system on 4.5 days per week (confidence interval 3.2-5.7 days). Of the 46 patients who could complete the final survey, 93% considered the monitoring system as easy to use and 38% asked to keep the system for self-management support after the study was completed. CONCLUSION: We developed a user-centred home monitoring system that enabled a wide range of heart failure patients, with differing degrees of IT literacy, to monitor their health status regularly. Despite no active medical intervention, patients felt that they benefited from the reassurance and sense of connectivity that the monitoring system provided.

Global health training and postgraduate medical education in Australia: the case for greater integration.

Global health (GH) training is well established overseas (particularly in North America) and reflects an increasing focus on social accountability in medical education. Despite significant interest among trainees, GH is poorly integrated with specialty training programs in Australia. While there are numerous benefits from international rotations in resource-poor settings, there are also risks to the host community, trainee and training provider. Safe and effective placements rely on firm ethical foundations as well as strong and durable partnerships between Australian and overseas health services, educational institutions and GH agencies. More formal systems of GH training in Australia have the potential to produce fellows with the skills and knowledge necessary to engage in regional health challenges in a global context.

Remote frozen section examination of breast sentinel lymph nodes by telepathology.

BACKGROUND: Intraoperative sentinel lymph node (SLN) examination during breast cancer surgery guides the need for immediate axillary clearance. This may be difficult to implement when surgery is performed distant to the centres where pathological examination is undertaken. We aimed to implement and validate a telepathology service for the remote examination of breast SLN by frozen section (FS). METHODS: We tested an Internet-based remote microscopy system to report intraoperative FS in real time from two district hospitals without on-site anatomical pathology services. RESULTS: FS was performed remotely on 52 patients. Seventeen out of 52 patients had metastases, of which there were six false-negative diagnoses comprising four of micrometastatic disease and two of isolated tumour cells (ITCs). There were no false-negative diagnoses for macrometastatic disease and no false-positive diagnoses. As a control, we audited our experience with 239 consecutive SLN FS examined by on-site pathologists. Sixty out of 239 patients had metastases, of which there were 24 false-negative diagnoses comprising 12 cases of ITC, 5 of micrometastases and 7 of macrometastases. The accuracy of remote FS was equivalent to that of in-house FS (88.2% versus 89.9%). CONCLUSION: Remote FS for breast SLN is an accurate procedure ,which is not inferior to FS performed on site.

Telemedicine in the Solomon Islands: 2006 to 2009.

Telemedicine has been used in the Solomon Islands since 2000. We used quantitative and qualitative methods to examine telemedicine use in the Solomon Islands from January 2006 to June 2009. During the study period 66 telemedicine cases were submitted to the store and forward telemedicine system being used there. These included orthopaedic, oncology, cardiothoracic, infectious, congenital, gastroenterology and dermatology cases. Most cases (52%) were submitted by doctors at the National Referral Hospital (NRH) in Honiara. The majority of responses came from the NRH (27%). A final, firm recommendation regarding patient diagnosis and/or care was given for 46% of the cases. Interviews were conducted with 23 stakeholders in the Solomon Islands and in Australia to better understand the current and future use of telemedicine. The interviews identified the fragility of the Solomon Islands infrastructure, including the lack of training, as the largest barrier to the future use of telemedicine. The best use of telemedicine appears to be case sharing within the Solomon Islands, with connections to clinicians in other countries as a secondary benefit when particular expertise is required.

A Web-based approach to low-cost telemedicine.

We have developed a Web-based application for managing e-consultations. This solves some of the problems inherent in peer-to-peer email communication. Referrals were from three hospitals in Vietnam. Each hospital was provided with Internet access for up to 4 hours a day for an initial 12-month period. In the first six months, six doctors from the Vietnamese hospitals submitted a total of 30 cases. Specialists in Sydney provided their expertise on a voluntary basis. Preliminary data suggest that the Vietnamese doctors found the system benefited their diagnostic and management decision making. Challenges have included equipment failures, language barriers and the difficulty of obtaining feedback. Successes have included the relationship building between doctors in the two countries. Preliminary results were encouraging and most of the Vietnamese doctors (five of the six) reported that they were slowly incorporating use of the system into their daily practice.