Assessment of a Digital Symptom Checker Tool's Accuracy in Suggesting Reproductive Health Conditions: Clinical Vignettes Study.

BACKGROUND: Reproductive health conditions such as endometriosis, uterine fibroids, and polycystic ovary syndrome (PCOS) affect a large proportion of women and people who menstruate worldwide. Prevalence estimates for these conditions range from 5% to 40% of women of reproductive age. Long diagnostic delays, up to 12 years, are common and contribute to health complications and increased health care costs. Symptom checker apps provide users with information and tools to better understand their symptoms and thus have the potential to reduce the time to diagnosis for reproductive health conditions. OBJECTIVE: This study aimed to evaluate the agreement between clinicians and 3 symptom checkers (developed by Flo Health UK Limited) in assessing symptoms of endometriosis, uterine fibroids, and PCOS using vignettes. We also aimed to present a robust example of vignette case creation, review, and classification in the context of predeployment testing and validation of digital health symptom checker tools. METHODS: Independent general practitioners were recruited to create clinical case vignettes of simulated users for the purpose of testing each condition symptom checker; vignettes created for each condition contained a mixture of condition-positive and condition-negative outcomes. A second panel of general practitioners then reviewed, approved, and modified (if necessary) each vignette. A third group of general practitioners reviewed each vignette case and designated a final classification. Vignettes were then entered into the symptom checkers by a fourth, different group of general practitioners. The outcomes of each symptom checker were then compared with the final classification of each vignette to produce accuracy metrics including percent agreement, sensitivity, specificity, positive predictive value, and negative predictive value. RESULTS: A total of 24 cases were created per condition. Overall, exact matches between the vignette general practitioner classification and the symptom checker outcome were 83% (n=20) for endometriosis, 83% (n=20) for uterine fibroids, and 88% (n=21) for PCOS. For each symptom checker, sensitivity was reported as 81.8% for endometriosis, 84.6% for uterine fibroids, and 100% for PCOS; specificity was reported as 84.6% for endometriosis, 81.8% for uterine fibroids, and 75% for PCOS; positive predictive value was reported as 81.8% for endometriosis, 84.6% for uterine fibroids, 80% for PCOS; and negative predictive value was reported as 84.6% for endometriosis, 81.8% for uterine fibroids, and 100% for PCOS. CONCLUSIONS: The single-condition symptom checkers have high levels of agreement with general practitioner classification for endometriosis, uterine fibroids, and PCOS. Given long delays in diagnosis for many reproductive health conditions, which lead to increased medical costs and potential health complications for individuals and health care providers, innovative health apps and symptom checkers hold the potential to improve care pathways.

Crossing the Digital Divide in Online Self-Management Support: Analysis of Usage Data From HeLP-Diabetes.

BACKGROUND: Digital health is increasingly recognized as a cost-effective means to support patient self-care. However, there are concerns about whether the "digital divide," defined as the gap between those who do and do not make regular use of digital technologies, will lead to increased health inequalities. Access to the internet, computer literacy, motivation to use digital health interventions, and fears about internet security are barriers to use of digital health interventions. Some of these barriers disproportionately affect people of older age, black or minority ethnic background, and low socioeconomic status. HeLP-Diabetes (Healthy Living for People with type 2 Diabetes), a theoretically informed online self-management program for adults with type 2 diabetes, was developed to meet the needs of people from a broad demographic background. OBJECTIVE: This study aimed to determine whether there was evidence of a digital divide when HeLP-Diabetes was integrated into routine care. This was achieved by (1) comparing the characteristics of people who registered for the program against the target population (people with type 2 diabetes in inner London), (2) comparing the characteristics of people who registered for the program and used it with those who did not use it, and (3) comparing sections of the website visited by different demographic groups. METHODS: A retrospective analysis of data on the use of HeLP-Diabetes in routine clinical practice in 4 inner London clinical commissioning groups was undertaken. Data were collected from patients who registered for the program as part of routine health services.. Data on gender, age, ethnicity, and educational attainment were collected at registration, and data on webpage visits (user identification number, date, time, and page visited) were collected automatically by software on the server side of the website. RESULTS: The characteristics of people who registered for the program were found to reflect those of the target population. The mean age was 58.4 years (SD=28.0), over 50.0% were from black and minority ethnic backgrounds, and nearly a third (29.8%) had no qualifications beyond school leaving age. There was no association between demographic characteristics and use of the program, apart from weak evidence of less use by the mixed ethnicity group. There was no evidence of the differential use of the program by any demographic group, apart from weak evidence for people with degrees and school leavers being more likely to use the "Living and working with diabetes" (P=.03) and "Treating diabetes" (P=.04) sections of the website. CONCLUSIONS: This study is one of the first to provide evidence that a digital health intervention can be integrated into routine health services without widening health inequalities. The relative success of the intervention may be attributed to integration into routine health care, and careful design with extensive user input and consideration of literacy levels. Developers of digital health interventions need to acknowledge barriers to access and use, and collect data on the demographic profile of users, to address inequalities.