Experiences in aligning WHO SMART guidelines to classification and terminology standards.

OBJECTIVES: Digital adaptation kits (DAKs) distill WHO guidelines for digital use by representing them as workflows, data dictionaries and decision support tables. This paper aims to highlight key lessons learnt in coding data elements of the antenatal care (ANC) and family planning DAKs to standardised classifications and terminologies (CATs). METHODS: We encoded data elements within the ANC and family planning DAKs to standardised CATs from the WHO CATs and other freely available CATs. RESULTS: The coding process demonstrated approaches to refine the data dictionaries and enhance alignment between data elements and CATs. DISCUSSION: Applying CATs to WHO clinical and public health guidelines can ensure that recommendations are operationalised in a digital system with appropriate consistency and clarity. This requires a multidisciplinary team and careful review to achieve conceptual equivalence between data elements and standardised terminologies. CONCLUSION: The systematic translation of guidelines into digital systems provides an opportunity for leveraging CATs; however, this approach needs further exploration into its implementation in country contexts and transition into machine-readable components.

iCHECK-DH: Guidelines and Checklist for the Reporting on Digital Health Implementations.

BACKGROUND: Implementation of digital health technologies has grown rapidly, but many remain limited to pilot studies due to challenges, such as a lack of evidence or barriers to implementation. Overcoming these challenges requires learning from previous implementations and systematically documenting implementation processes to better understand the real-world impact of a technology and identify effective strategies for future implementation. OBJECTIVE: A group of global experts, facilitated by the Geneva Digital Health Hub, developed the Guidelines and Checklist for the Reporting on Digital Health Implementations (iCHECK-DH, pronounced "I checked") to improve the completeness of reporting on digital health implementations. METHODS: A guideline development group was convened to define key considerations and criteria for reporting on digital health implementations. To ensure the practicality and effectiveness of the checklist, it was pilot-tested by applying it to several real-world digital health implementations, and adjustments were made based on the feedback received. The guiding principle for the development of iCHECK-DH was to identify the minimum set of information needed to comprehensively define a digital health implementation, to support the identification of key factors for success and failure, and to enable others to replicate it in different settings. RESULTS: The result was a 20-item checklist with detailed explanations and examples in this paper. The authors anticipate that widespread adoption will standardize the quality of reporting and, indirectly, improve implementation standards and best practices. CONCLUSIONS: Guidelines for reporting on digital health implementations are important to ensure the accuracy, completeness, and consistency of reported information. This allows for meaningful comparison and evaluation of results, transparency, and accountability and informs stakeholder decision-making. i-CHECK-DH facilitates standardization of the way information is collected and reported, improving systematic documentation and knowledge transfer that can lead to the development of more effective digital health interventions and better health outcomes.

Correction: iCHECK-DH: Guidelines and Checklist for the Reporting on Digital Health Implementations.

[This corrects the article DOI: 10.2196/46694.].

Accuracy of a smartphone application for blood pressure estimation in Bangladesh, South Africa, and Tanzania.

Undetected and unmonitored hypertension carries substantial mortality and morbidity, especially during pregnancy. We assessed the accuracy of OptiBPTM, a smartphone application for estimating blood pressure (BP), across diverse settings. The study was conducted in community settings: Gaibandha, Bangladesh and Ifakara, Tanzania for general populations, and Kalafong Provincial Tertiary Hospital, South Africa for pregnant populations. Based on guidance from the International Organization for Standardization (ISO) 81,060-2:2018 for non-invasive BP devices and global consensus statement, we compared BP measurements taken by two independent trained nurses on a standard auscultatory cuff to the BP measurements taken by a research version of OptiBPTM called CamBP. For ISO criterion 1, the mean error was 0.5 ± 5.8 mm Hg for the systolic blood pressure (SBP) and 0.1 ± 3.9 mmHg for the diastolic blood pressure (DBP) in South Africa; 0.8 ± 7.0 mmHg for the SBP and -0.4 ± 4.0 mmHg for the DBP in Tanzania; 3.3 ± 7.4 mmHg for the SBP and -0.4 ± 4.3 mmHg for the DBP in Bangladesh. For ISO criterion 2, the average standard deviation of the mean error per subject was 4.9 mmHg for the SBP and 3.4 mmHg for the DBP in South Africa; 6.3 mmHg for the SBP and 3.6 mmHg for the DBP in Tanzania; 6.4 mmHg for the SBP and 3.8 mmHg for the DBP in Bangladesh. OptiBPTM demonstrated accuracy against ISO standards in study populations, including pregnant populations, except in Bangladesh for SBP (criterion 2). Further research is needed to improve performance across different populations and integration within health systems.

Digitalization of routine health information systems: Bangladesh, Indonesia, Pakistan.

Objective: To describe a systematic process of transforming paper registers into a digital system optimized to enhance service provision and fulfil reporting requirements. Methods: We designed a formative study around primary health workers providing reproductive, maternal, newborn and child health services in three countries in Bangladesh, Indonesia and Pakistan. The study ran from November 2014 to June 2018. We developed a prototype digital application after conducting a needs assessment of health workers' responsibilities, workflows, routine data requirements and service delivery needs. Methods included desk reviews, focus group discussions, in-depth interviews; data mapping of paper registers; observations of health workers; co-design workshops with health workers; and usability testing. Finally, we conducted an observational feasibility assessment to monitor uptake of the application. Findings: Researchers reviewed a total of 17 paper registers across the sites, which we transformed into seven modules within a digital application running on mobile devices. Modules corresponded to the services provided, including household enumeration, antenatal care, family planning, immunization, nutrition and child health. A total of 65 health workers used the modules during the feasibility assessment, and average weekly form submissions ranged from 8 to 234, depending on the health worker and their responsibilities. We also observed variability in the use of modules, requiring consistent monitoring support for health workers. Conclusion: Lessons learnt from this study shaped key global initiatives and resulted in a software global good. The deployment of digital systems requires well-designed applications, change management and strengthening human resources to realize and sustain health system gains.

Transitioning to Digital Systems: The Role of World Health Organization's Digital Adaptation Kits in Operationalizing Recommendations and Interoperability Standards.

INTRODUCTION: The transition from paper to digital systems requires quality assurance of the underlying content and application of data standards for interoperability. The World Health Organization (WHO) developed digital adaptation kits (DAKs) as an operational and software-neutral mechanism to translate WHO guidelines into a standardized format that can be more easily incorporated into digital systems. METHODS: WHO convened health program area and digital leads, reviewed existing approaches for requirements gathering, mapped to established standards, and incorporated research findings to define DAK components. RESULTS: For each health domain area, the DAKs distill WHO guidelines to specify the health interventions, personas, user scenarios, business process workflows, core data elements mapped to terminology codes, decision-support logic, program indicators, and functional and nonfunctional requirements. DISCUSSION: DAKs aim to catalyze quality of care and facilitate data use and interoperability as part of WHO's vision of SMART (Standards-based, Machine-readable, Adaptive, Requirements-based, and Testable) guidelines. Efforts will be needed to strengthen a collaborative approach for the uptake of DAKs within the local digital ecosystem and national health policies.

Integration of new digital antenatal care tools using the WHO SMART guideline approach: Experiences from Rwanda and Zambia.

OBJECTIVES: Digital tools for decision-support and health records can address the protracted process of guideline adoption at local levels and accelerate countries' implementation of new health policies and programmes. World Health Organization (WHO) launched the SMART Guidelines approach to support the uptake of clinical, public health, and data recommendations within digital systems. SMART guidelines are a package of tools that include Digital Adaptation Kits (DAKs), which distill WHO guidelines into a format that facilitates translation into digital systems. SMART Guidelines also include reference software applications known as digital modules. METHODS: This paper details the structured process to inform the adaptation of the WHO antenatal care (ANC) digital module to align with country-specific ANC packages for Zambia and Rwanda using the DAK. Digital landscape assessments were conducted to determine potential integrations between the ANC digital module and existing systems. A multi-stakeholder team consisting of Ministry of Health technical officers representing maternal health, HIV, digital health, and monitoring and evaluation at district and national levels was assembled to review existing guidelines to adapt the DAK. RESULTS: The landscape analysis resulted in considerations for integrating the ANC module into the broader digital ecosystems of both countries. Adaptations to the DAK included adding national services not reflected in the generic DAK and modification of decision support logic and indicators. Over 80% of the generic DAK content was consistent with processes for both countries. The adapted DAK will inform the customization of country-specific ANC digital modules. CONCLUSION: Both countries found that coordination between maternal and digital health leads was critical to ensuring requirements were accurately reflected within the ANC digital module. Additionally, DAKs provided a structured process for gathering requirements, reviewing and addressing gaps within existing systems, and aligning clinical content.

Decision-support tools via mobile devices to improve quality of care in primary healthcare settings.

BACKGROUND: The ubiquity of mobile devices has made it possible for clinical decision-support systems (CDSS) to become available to healthcare providers on handheld devices at the point-of-care, including in low- and middle-income countries. The use of CDSS by providers can potentially improve adherence to treatment protocols and patient outcomes. However, the evidence on the effect of the use of CDSS on mobile devices needs to be synthesized. This review was carried out to support a World Health Organization (WHO) guideline that aimed to inform investments on the use of decision-support tools on digital devices to strengthen primary healthcare. OBJECTIVES: To assess the effects of digital clinical decision-support systems (CDSS) accessible via mobile devices by primary healthcare providers in the context of primary care settings. SEARCH METHODS: We searched CENTRAL, MEDLINE, Embase, Global Index Medicus, POPLINE, and two trial registries from 1 January 2000 to 9 October 2020. We conducted a grey literature search using mHealthevidence.org and issued a call for papers through popular digital health communities of practice. Finally, we conducted citation searches of included studies. SELECTION CRITERIA: Study design: we included randomized trials, including full-text studies, conference abstracts, and unpublished data irrespective of publication status or language of publication.  Types of participants: we included studies of all cadres of healthcare providers, including lay health workers and other individuals (administrative, managerial, and supervisory staff) involved in the delivery of primary healthcare services using clinical decision-support tools; and studies of clients or patients receiving care from primary healthcare providers using digital decision-support tools. Types of interventions: we included studies comparing digital CDSS accessible via mobile devices with non-digital CDSS or no intervention, in the context of primary care. CDSS could include clinical protocols, checklists, and other job-aids which supported risk prioritization of patients. Mobile devices included mobile phones of any type (but not analogue landline telephones), as well as tablets, personal digital assistants, and smartphones. We excluded studies where digital CDSS were used on laptops or integrated with electronic medical records or other types of longitudinal tracking of clients. DATA COLLECTION AND ANALYSIS: A machine learning classifier that gave each record a probability score of being a randomized trial screened all search results. Two review authors screened titles and abstracts of studies with more than 10% probability of being a randomized trial, and one review author screened those with less than 10% probability of being a randomized trial. We followed standard methodological procedures expected by Cochrane and the Effective Practice and Organisation of Care group. We used the GRADE approach to assess the certainty of the evidence for the most important outcomes. MAIN RESULTS: Eight randomized trials across varying healthcare contexts in the USA,. India, China, Guatemala, Ghana, and Kenya, met our inclusion criteria. A range of healthcare providers (facility and community-based, formally trained, and lay workers) used digital CDSS. Care was provided for the management of specific conditions such as cardiovascular disease, gastrointestinal risk assessment, and maternal and child health. The certainty of evidence ranged from very low to moderate, and we often downgraded evidence for risk of bias and imprecision. We are uncertain of the effect of this intervention on providers' adherence to recommended practice due to the very low certainty evidence (2 studies, 185 participants). The effect of the intervention on patients' and clients' health behaviours such as smoking and treatment adherence is mixed, with substantial variation across outcomes for similar types of behaviour (2 studies, 2262 participants). The intervention probably makes little or no difference to smoking rates among people at risk of cardiovascular disease but probably increases other types of desired behaviour among patients, such as adherence to treatment. The effect of the intervention on patients'/clients' health status and well-being  is also mixed (5 studies, 69,767 participants). It probably makes little or no difference to some types of health outcomes, but we are uncertain about other health outcomes, including maternal and neonatal deaths, due to very low-certainty evidence. The intervention may slightly improve patient or client acceptability and satisfaction (1 study, 187 participants). We found no studies that reported the time between the presentation of an illness and appropriate management, provider acceptability or satisfaction, resource use, or unintended consequences. AUTHORS' CONCLUSIONS: We are uncertain about the effectiveness of mobile phone-based decision-support tools on several outcomes, including adherence to recommended practice. None of the studies had a quality of care framework and focused only on specific health areas.   We need well-designed research that takes a systems lens to assess these issues.

Birth and death notification via mobile devices: a mixed methods systematic review.

BACKGROUND: Ministries of health, donors, and other decision-makers are exploring how they can use mobile technologies to acquire accurate and timely statistics on births and deaths. These stakeholders have called for evidence-based guidance on this topic. This review was carried out to support World Health Organization (WHO) recommendations on digital interventions for health system strengthening. OBJECTIVES: Primary objective: To assess the effects of birth notification and death notification via a mobile device, compared to standard practice. Secondary objectives: To describe the range of strategies used to implement birth and death notification via mobile devices and identify factors influencing the implementation of birth and death notification via mobile devices. SEARCH METHODS: We searched CENTRAL, MEDLINE, Embase, the Global Health Library, and POPLINE (August 2, 2019). We searched two trial registries (August 2, 2019). We also searched Epistemonikos for related systematic reviews and potentially eligible primary studies (August 27, 2019). We conducted a grey literature search using mHealthevidence.org (August 15, 2017) and issued a call for papers through popular digital health communities of practice. Finally, we conducted citation searches of included studies in Web of Science and Google Scholar (May 15, 2020). We searched for studies published after 2000 in any language.  SELECTION CRITERIA: For the primary objective, we included individual and cluster-randomised trials; cross-over and stepped-wedge study designs; controlled before-after studies, provided they have at least two intervention sites and two control sites; and interrupted time series studies. For the secondary objectives, we included any study design, either quantitative, qualitative, or descriptive, that aimed to describe current strategies for birth and death notification via mobile devices; or to explore factors that influence the implementation of these strategies, including studies of acceptability or feasibility. For the primary objective, we included studies that compared birth and death notification via mobile devices with standard practice. For the secondary objectives, we included studies of birth and death notification via mobile device as long as we could extract data relevant to our secondary objectives. We included studies of all cadres of healthcare providers, including lay health workers; administrative, managerial, and supervisory staff; focal individuals at the village or community level; children whose births were being notified and their parents/caregivers; and individuals whose deaths were being notified and their relatives/caregivers. DATA COLLECTION AND ANALYSIS: For the primary objective, two authors independently screened all records, extracted data from the included studies and assessed risk of bias. For the analyses of the primary objective, we reported means and proportions, where appropriate. We used the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach to assess the certainty of the evidence and we prepared a 'Summary of Findings' table. For the secondary objectives, two authors screened all records, one author extracted data from the included studies and assessed methodological limitations using the WEIRD tool and a second author checked the data and assessments. We carried out a framework analysis using the Supporting the Use of Research Evidence (SURE) framework to identify themes in the data. We used the GRADE-CERQual (Confidence in the Evidence from Reviews of Qualitative research) approach to assess our confidence in the evidence and we prepared a 'Summary of Qualitative Findings' table. MAIN RESULTS: For the primary objective, we included one study, which used a controlled before-after study design. The study was conducted in Lao People's Democratic Republic and assessed the effect of using mobile devices for birth notification on outcomes related to coverage and timeliness of Hepatitis B vaccination. However, we are uncertain of the effect of this approach on these outcomes because the certainty of this evidence was assessed as very low. The included study did not assess resource use or unintended consequences. For the primary objective, we did not identify any studies using mobile devices for death notification. For the secondary objective, we included 21 studies. All studies were conducted in low- or middle-income settings. They focussed on identification of births and deaths in rural, remote, or marginalised populations who are typically under-represented in civil registration processes or traditionally seen as having poor access to health services. The review identified several factors that could influence the implementation of birth-death notification via mobile device. These factors were tied to the health system, the person responsible for notifying, the community and families; and include: - Geographic barriers that could prevent people's access to birth-death notification and post-notification services - Access to health workers and other notifiers with enough training, supervision, support, and incentives - Monitoring systems that ensure the quality and timeliness of the birth and death data - Legal frameworks that allow births and deaths to be notified by mobile device and by different types of notifiers - Community awareness of the need to register births and deaths - Socio-cultural norms around birth and death - Government commitment - Cost to the system, to health workers and to families - Access to electricity and network connectivity, and compatibility with existing systems - Systems that protect data confidentiality We have low to moderate confidence in these findings. This was mainly because of concerns about methodological limitations and data adequacy. AUTHORS' CONCLUSIONS: We need more, well-designed studies of the effect of birth and death notification via mobile devices and on factors that may influence its implementation.

Building a Digital Tool for the Adoption of the World Health Organization's Antenatal Care Recommendations: Methodological Intersection of Evidence, Clinical Logic, and Digital Technology.

BACKGROUND: One of the key mandates of the World Health Organization (WHO) is to develop guidelines, defined as "a document containing recommendations for clinical practice or public health policy." Guidelines represent the global standard for information sources shaping clinical practice and public health policies. Despite the rigorous development process and the value of guidelines for setting standards, implementing such standards within local contexts and at the point of care is a well-documented challenge. Digital technologies enable agile information management and may facilitate the adaptation of guidelines to diverse settings of health services delivery. OBJECTIVE: The objective of this paper is to detail the systematic and iterative process involved in transforming the WHO Antenatal Care (ANC) guidelines into a digital decision-support and patient-record application for routine use in primary health care settings, known as the WHO digital ANC module. METHODS: The WHO convened a team of clinical and digital health experts to develop the WHO digital ANC module as a tool to assist health care professionals in the implementation of WHO evidence-based recommendations for pregnant women. The WHO digital ANC module's creation included the following steps: defining a minimum viable product (MVP), developing clinical workflows and algorithms, algorithm testing, developing a data dictionary, and the creation of a user interface or application development. The overall process of development took approximately 1 year to reach a stable prototype and to finalize the underlying content requirements of the data dictionary and decision support algorithms. RESULTS: The first output is a reference software reflecting the generic WHO ANC guideline content, known as the WHO digital ANC module. Within it, all actionable ANC recommendations have related data fields and algorithms to confirm whether the associated task was performed. WHO recommendations that are not carried out by the health care worker are saved as pending tasks on a woman's health record, and those that are adequately fulfilled trigger messages with positive reinforcement. The second output consists of the structured documentation of the different components which contributed to the development of the WHO digital ANC module, such as the data dictionary and clinical decision support workflows. CONCLUSIONS: This is a novel approach to facilitate the adoption and adaptation of recommendations through digital systems at the health service delivery level. It is expected that the WHO digital ANC module will support the implementation of evidence-based practices and provide information for monitoring and surveillance; however, further evidence is needed to understand how the WHO digital ANC module impacts the implementation of WHO recommendations. Further, the module's implementation will inform the WHO's ongoing efforts to create a pathway to adaptive and integrated (Smart) Guidelines in Digital Systems to improve health system quality, coverage, and accountability.

WHO Digital Health Guidelines: a milestone for global health.

In 2019, the World Health Organization (WHO) released the first-ever evidence-based guidelines for digital health. The guideline provides nine recommendations on select digital health interventions that involve the use of a mobile phone or device. It also provides information on implementation considerations, quality and certainty of extant evidence, factors related to acceptability and feasibility of the intervention, and gaps in the evidence that can inform future research. Given the pivotal role digital health can play in supporting health systems, seen especially in light of the COVID-19 pandemic, these guidelines can help provide a roadmap for governments and policymakers in introducing and scaling up digital health interventions to support population health outcomes.

Correction: Building a Digital Tool for the Adoption of the World Health Organization's Antenatal Care Recommendations: Methodological Intersection of Evidence, Clinical Logic, and Digital Technology.

[This corrects the article DOI: 10.2196/16355.].

Targeted client communication via mobile devices for improving maternal, neonatal, and child health.

BACKGROUND: The global burden of poor maternal, neonatal, and child health (MNCH) accounts for more than a quarter of healthy years of life lost worldwide. Targeted client communication (TCC) via mobile devices (MD) (TCCMD) may be a useful strategy to improve MNCH. OBJECTIVES: To assess the effects of TCC via MD on health behaviour, service use, health, and well-being for MNCH. SEARCH METHODS: In July/August 2017, we searched five databases including The Cochrane Central Register of Controlled Trials, MEDLINE and Embase. We also searched two trial registries. A search update was carried out in July 2019 and potentially relevant studies are awaiting classification. SELECTION CRITERIA: We included randomised controlled trials that assessed TCC via MD to improve MNCH behaviour, service use, health, and well-being. Eligible comparators were usual care/no intervention, non-digital TCC, and digital non-targeted client communication. DATA COLLECTION AND ANALYSIS: We used standard methodological procedures recommended by Cochrane, although data extraction and risk of bias assessments were carried out by one person only and cross-checked by a second. MAIN RESULTS: We included 27 trials (17,463 participants). Trial populations were: pregnant and postpartum women (11 trials conducted in low-, middle- or high-income countries (LMHIC); pregnant and postpartum women living with HIV (three trials carried out in one lower middle-income country); and parents of children under the age of five years (13 trials conducted in LMHIC). Most interventions (18) were delivered via text messages alone, one was delivered through voice calls only, and the rest were delivered through combinations of different communication channels, such as multimedia messages and voice calls. Pregnant and postpartum women TCCMD versus standard care For behaviours, TCCMD may increase exclusive breastfeeding in settings where rates of exclusive breastfeeding are less common (risk ratio (RR) 1.30, 95% confidence intervals (CI) 1.06 to 1.59; low-certainty evidence), but have little or no effect in settings where almost all women breastfeed (low-certainty evidence). For use of health services, TCCMD may increase antenatal appointment attendance (odds ratio (OR) 1.54, 95% CI 0.80 to 2.96; low-certainty evidence); however, the CI encompasses both benefit and harm. The intervention may increase skilled attendants at birth in settings where a lack of skilled attendants at birth is common (though this differed by urban/rural residence), but may make no difference in settings where almost all women already have a skilled attendant at birth (OR 1.00, 95% CI 0.34 to 2.94; low-certainty evidence). There were uncertain effects on maternal and neonatal mortality and morbidity because the certainty of the evidence was assessed as very low. TCCMD versus non-digital TCC (e.g. pamphlets) TCCMD may have little or no effect on exclusive breastfeeding (RR 0.92, 95% CI 0.79 to 1.07; low-certainty evidence). TCCMD may reduce 'any maternal health problem' (RR 0.19, 95% CI 0.04 to 0.79) and 'any newborn health problem' (RR 0.52, 95% CI 0.25 to 1.06) reported up to 10 days postpartum (low-certainty evidence), though the CI for the latter includes benefit and harm. The effect on health service use is unknown due to a lack of studies. TCCMD versus digital non-targeted communication No studies reported behavioural, health, or well-being outcomes for this comparison. For use of health services, there are uncertain effects for the presence of a skilled attendant at birth due to very low-certainty evidence, and the intervention may make little or no difference to attendance for antenatal influenza vaccination (RR 1.05, 95% CI 0.71 to 1.58), though the CI encompasses both benefit and harm (low-certainty evidence). Pregnant and postpartum women living with HIV TCCMD versus standard care For behaviours, TCCMD may make little or no difference to maternal and infant adherence to antiretroviral (ARV) therapy (low-certainty evidence). For health service use, TCC mobile telephone reminders may increase use of antenatal care slightly (mean difference (MD) 1.5, 95% CI -0.36 to 3.36; low-certainty evidence). The effect on the proportion of births occurring in a health facility is uncertain due to very low-certainty evidence. For health and well-being outcomes, there was an uncertain intervention effect on neonatal death or stillbirth, and infant HIV due to very low-certainty evidence. No studies reported on maternal mortality or morbidity. TCCMD versus non-digital TCC The effect is unknown due to lack of studies reporting this comparison. TCCMD versus digital non-targeted communication TCCMD may increase infant ARV/prevention of mother-to-child transmission treatment adherence (RR 1.26, 95% CI 1.07 to 1.48; low-certainty evidence). The effect on other outcomes is unknown due to lack of studies. Parents of children aged less than five years No studies reported on correct treatment, nutritional, or health outcomes. TCCMD versus standard care Based on 10 trials, TCCMD may modestly increase health service use (vaccinations and HIV care) (RR 1.21, 95% CI 1.08 to 1.34; low-certainty evidence); however, the effect estimates varied widely between studies. TCCMD versus non-digital TCC TCCMD may increase attendance for vaccinations (RR 1.13, 95% CI 1.00 to 1.28; low-certainty evidence), and may make little or no difference to oral hygiene practices (low-certainty evidence). TCCMD versus digital non-targeted communication TCCMD may reduce attendance for vaccinations, but the CI encompasses both benefit and harm (RR 0.63, 95% CI 0.33 to 1.20; low-certainty evidence). No trials in any population reported data on unintended consequences. AUTHORS' CONCLUSIONS: The effect of TCCMD for most outcomes is uncertain. There may be improvements for some outcomes using targeted communication but these findings were of low certainty. High-quality, adequately powered trials and cost-effectiveness analyses are required to reliably ascertain the effects and relative benefits of TCCMD. Future studies should measure potential unintended consequences, such as partner violence or breaches of confidentiality.

Mobile technologies to support healthcare provider to healthcare provider communication and management of care.

BACKGROUND: The widespread use of mobile technologies can potentially expand the use of telemedicine approaches to facilitate communication between healthcare providers, this might increase access to specialist advice and improve patient health outcomes. OBJECTIVES: To assess the effects of mobile technologies versus usual care for supporting communication and consultations between healthcare providers on healthcare providers' performance, acceptability and satisfaction, healthcare use, patient health outcomes, acceptability and satisfaction, costs, and technical difficulties. SEARCH METHODS: We searched CENTRAL, MEDLINE, Embase and three other databases from 1 January 2000 to 22 July 2019. We searched clinical trials registries, checked references of relevant systematic reviews and included studies, and contacted topic experts. SELECTION CRITERIA: Randomised trials comparing mobile technologies to support healthcare provider to healthcare provider communication and consultations compared with usual care. DATA COLLECTION AND ANALYSIS: We followed standard methodological procedures expected by Cochrane and EPOC. We used the GRADE approach to assess the certainty of the evidence. MAIN RESULTS: We included 19 trials (5766 participants when reported), most were conducted in high-income countries. The most frequently used mobile technology was a mobile phone, often accompanied by training if it was used to transfer digital images. Trials recruited participants with different conditions, and interventions varied in delivery, components, and frequency of contact. We judged most trials to have high risk of performance bias, and approximately half had a high risk of detection, attrition, and reporting biases. Two studies reported data on technical problems, reporting few difficulties. Mobile technologies used by primary care providers to consult with hospital specialists We assessed the certainty of evidence for this group of trials as moderate to low. Mobile technologies: - probably make little or no difference to primary care providers following guidelines for people with chronic kidney disease (CKD; 1 trial, 47 general practices, 3004 participants); - probably reduce the time between presentation and management of individuals with skin conditions, people with symptoms requiring an ultrasound, or being referred for an appointment with a specialist after attending primary care (4 trials, 656 participants); - may reduce referrals and clinic visits among people with some skin conditions, and increase the likelihood of receiving retinopathy screening among people with diabetes, or an ultrasound in those referred with symptoms (9 trials, 4810 participants when reported); - probably make little or no difference to patient-reported quality of life and health-related quality of life (2 trials, 622 participants) or to clinician-assessed clinical recovery (2 trials, 769 participants) among individuals with skin conditions; - may make little or no difference to healthcare provider (2 trials, 378 participants) or participant acceptability and satisfaction (4 trials, 972 participants) when primary care providers consult with dermatologists; - may make little or no difference for total or expected costs per participant for adults with some skin conditions or CKD (6 trials, 5423 participants). Mobile technologies used by emergency physicians to consult with hospital specialists about people attending the emergency department We assessed the certainty of evidence for this group of trials as moderate. Mobile technologies: - probably slightly reduce the consultation time between emergency physicians and hospital specialists (median difference -12 minutes, 95% CI -19 to -7; 1 trial, 345 participants); - probably reduce participants' length of stay in the emergency department by a few minutes (median difference -30 minutes, 95% CI -37 to -25; 1 trial, 345 participants). We did not identify trials that reported on providers' adherence, participants' health status and well-being, healthcare provider and participant acceptability and satisfaction, or costs. Mobile technologies used by community health workers or home-care workers to consult with clinic staff We assessed the certainty of evidence for this group of trials as moderate to low. Mobile technologies: - probably make little or no difference in the number of outpatient clinic and community nurse consultations for participants with diabetes or older individuals treated with home enteral nutrition (2 trials, 370 participants) or hospitalisation of older individuals treated with home enteral nutrition (1 trial, 188 participants); - may lead to little or no difference in mortality among people living with HIV (RR 0.82, 95% CI 0.55 to 1.22) or diabetes (RR 0.94, 95% CI 0.28 to 3.12) (2 trials, 1152 participants); - may make little or no difference to participants' disease activity or health-related quality of life in participants with rheumatoid arthritis (1 trial, 85 participants); - probably make little or no difference for participant acceptability and satisfaction for participants with diabetes and participants with rheumatoid arthritis (2 trials, 178 participants). We did not identify any trials that reported on providers' adherence, time between presentation and management, healthcare provider acceptability and satisfaction, or costs. AUTHORS' CONCLUSIONS: Our confidence in the effect estimates is limited. Interventions including a mobile technology component to support healthcare provider to healthcare provider communication and management of care may reduce the time between presentation and management of the health condition when primary care providers or emergency physicians use them to consult with specialists, and may increase the likelihood of receiving a clinical examination among participants with diabetes and those who required an ultrasound. They may decrease the number of people attending primary care who are referred to secondary or tertiary care in some conditions, such as some skin conditions and CKD. There was little evidence of effects on participants' health status and well-being, satisfaction, or costs.

Targeted client communication via mobile devices for improving sexual and reproductive health.

BACKGROUND: The burden of poor sexual and reproductive health (SRH) worldwide is substantial, disproportionately affecting those living in low- and middle-income countries. Targeted client communication (TCC) delivered via mobile devices (MD) (TCCMD) may improve the health behaviours and service use important for sexual and reproductive health. OBJECTIVES: To assess the effects of TCC via MD on adolescents' knowledge, and on adolescents' and adults' sexual and reproductive health behaviour, health service use, and health and well-being. SEARCH METHODS: In July/August 2017, we searched five databases including The Cochrane Central Register of Controlled Trials, MEDLINE and Embase. We also searched two trial registries. A search update was carried out in July 2019 and potentially relevant studies are awaiting classification. SELECTION CRITERIA: We included randomised controlled trials of TCC via MD to improve sexual and reproductive health behaviour, health service use, and health and well-being. Eligible comparators were standard care or no intervention, non-digital TCC, and digital non-targeted communication. DATA COLLECTION AND ANALYSIS: We used standard methodological procedures recommended by Cochrane, although data extraction and risk of bias assessments were carried out by one person only and cross-checked by a second. We have presented results separately for adult and adolescent populations, and for each comparison. MAIN RESULTS: We included 40 trials (27 among adult populations and 13 among adolescent populations) with a total of 26,854 participants. All but one of the trials among adolescent populations were conducted in high-income countries. Trials among adult populations were conducted in a range of high- to low-income countries. Among adolescents, nine interventions were delivered solely through text messages; four interventions tested text messages in combination with another communication channel, such as emails, multimedia messaging, or voice calls; and one intervention used voice calls alone. Among adults, 20 interventions were delivered through text messages; two through a combination of text messages and voice calls; and the rest were delivered through other channels such as voice calls, multimedia messaging, interactive voice response, and instant messaging services. Adolescent populations TCCMD versus standard care TCCMD may increase sexual health knowledge (risk ratio (RR) 1.45, 95% confidence interval (CI) 1.23 to 1.71; low-certainty evidence). TCCMD may modestly increase contraception use (RR 1.19, 95% CI 1.05 to 1.35; low-certainty evidence). The effects on condom use, antiretroviral therapy (ART) adherence, and health service use are uncertain due to very low-certainty evidence. The effects on abortion and STI rates are unknown due to lack of studies. TCCMD versus non-digital TCC (e.g. pamphlets) The effects of TCCMD on behaviour (contraception use, condom use, ART adherence), service use, health and wellbeing (abortion and STI rates) are unknown due to lack of studies for this comparison. TCCMD versus digital non-targeted communication The effects on sexual health knowledge, condom and contraceptive use are uncertain due to very low-certainty evidence. Interventions may increase health service use (attendance for STI/HIV testing, RR 1.61, 95% CI 1.08 to 2.40; low-certainty evidence). The intervention may be beneficial for reducing STI rates (RR 0.61, 95% CI 0.28 to 1.33; low-certainty evidence), but the confidence interval encompasses both benefit and harm. The effects on abortion rates and on ART adherence are unknown due to lack of studies. We are uncertain whether TCCMD results in unintended consequences due to lack of evidence. Adult populations TCCMD versus standard care For health behaviours, TCCMD may modestly increase contraception use at 12 months (RR 1.17, 95% CI 0.92 to 1.48) and may reduce repeat abortion (RR 0.68 95% CI 0.28 to 1.66), though the confidence interval encompasses benefit and harm (low-certainty evidence). The effect on condom use is uncertain. No study measured the impact of this intervention on STI rates. TCCMD may modestly increase ART adherence (RR 1.13, 95% CI 0.97 to 1.32, low-certainty evidence, and standardised mean difference 0.44, 95% CI -0.14 to 1.02, low-certainty evidence). TCCMD may modestly increase health service utilisation (RR 1.17, 95% CI 1.04 to 1.31; low-certainty evidence), but there was substantial heterogeneity (I2 = 85%), with mixed results according to type of service utilisation (i.e. attendance for STI testing; HIV treatment; voluntary male medical circumcision (VMMC); VMMC post-operative visit; post-abortion care). For health and well-being outcomes, there may be little or no effect on CD4 count (mean difference 13.99, 95% CI -8.65 to 36.63; low-certainty evidence) and a slight reduction in virological failure (RR 0.86, 95% CI 0.73 to 1.01; low-certainty evidence). TCCMD versus non-digital TCC No studies reported STI rates, condom use, ART adherence, abortion rates, or contraceptive use as outcomes for this comparison. TCCMD may modestly increase in service attendance overall (RR: 1.12, 95% CI 0.92-1.35, low certainty evidence), however the confidence interval encompasses benefit and harm. TCCMD versus digital non-targeted communication No studies reported STI rates, condom use, ART adherence, abortion rates, or contraceptive use as outcomes for this comparison. TCCMD may increase service utilisation overall (RR: 1.71, 95% CI 0.67-4.38, low certainty evidence), however the confidence interval encompasses benefit and harm and there was considerable heterogeneity (I2 = 72%), with mixed results according to type of service utilisation (STI/HIV testing, and VMMC). Few studies reported on unintended consequences. One study reported that a participant withdrew from the intervention as they felt it compromised their undisclosed HIV status. AUTHORS' CONCLUSIONS: TCCMD may improve some outcomes but the evidence is of low certainty. The effect on most outcomes is uncertain/unknown due to very low certainty evidence or lack of evidence. High quality, adequately powered trials and cost effectiveness analyses are required to reliably ascertain the effects and relative benefits of TCC delivered by mobile devices. Given the sensitivity and stigma associated with sexual and reproductive health future studies should measure unintended consequences, such as partner violence or breaches of confidentiality.

Electronic clinical decision support algorithms incorporating point-of-care diagnostic tests in low-resource settings: a target product profile.

Health workers in low-resource settings often lack the support and tools to follow evidence-based clinical recommendations for diagnosing, treating and managing sick patients. Digital technologies, by combining patient health information and point-of-care diagnostics with evidence-based clinical protocols, can help improve the quality of care and the rational use of resources, and save patient lives. A growing number of electronic clinical decision support algorithms (CDSAs) on mobile devices are being developed and piloted without evidence of safety or impact. Here, we present a target product profile (TPP) for CDSAs aimed at guiding preventive or curative consultations in low-resource settings. This document will help align developer and implementer processes and product specifications with the needs of end users, in terms of quality, safety, performance and operational functionality. To identify the characteristics of CDSAs, a multidisciplinary group of experts (academia, industry and policy makers) with expertise in diagnostic and CDSA development and implementation in low-income and middle-income countries were convened to discuss a draft TPP. The TPP was finalised through a Delphi process to facilitate consensus building. An agreement greater than 75% was reached for all 40 TPP characteristics. In general, experts were in overwhelming agreement that, given that CDSAs provide patient management recommendations, the underlying clinical algorithms should be human-interpretable and evidence-based. Whenever possible, the algorithm's patient management output should take into account pretest disease probabilities and likelihood ratios of clinical and diagnostic predictors. In addition, validation processes should at a minimum show that CDSAs are implementing faithfully the evidence they are based on, and ideally the impact on patient health outcomes. In terms of operational needs, CDSAs should be designed to fit within clinic workflows and function in connectivity-challenged and high-volume settings. Data collected through the tool should conform to local patient privacy regulations and international data standards.

Tracking health commodity inventory and notifying stock levels via mobile devices: a mixed methods systematic review.

BACKGROUND: Health systems need timely and reliable access to essential medicines and health commodities, but problems with access are common in many settings. Mobile technologies offer potential low-cost solutions to the challenge of drug distribution and commodity availability in primary healthcare settings. However, the evidence on the use of mobile devices to address commodity shortages is sparse, and offers no clear way forward. OBJECTIVES: Primary objective To assess the effects of strategies for notifying stock levels and digital tracking of healthcare-related commodities and inventory via mobile devices across the primary healthcare system Secondary objectives To describe what mobile device strategies are currently being used to improve reporting and digital tracking of health commodities To identify factors influencing the implementation of mobile device interventions targeted at reducing stockouts of health commodities SEARCH METHODS: We searched CENTRAL, MEDLINE Ovid, Embase Ovid, Global Index Medicus WHO, POPLINE K4Health, and two trials registries in August 2019. We also searched Epistemonikos for related systematic reviews and potentially eligible primary studies. We conducted a grey literature search using mHealthevidence.org, and issued a call for papers through popular digital health communities of practice. Finally, we conducted citation searches of included studies. We searched for studies published after 2000, in any language. SELECTION CRITERIA: For the primary objective, we included individual and cluster-randomised trials, controlled before-after studies, and interrupted time series studies. For the secondary objectives, we included any study design, which could be quantitative, qualitative, or descriptive, that aimed to describe current strategies for commodity tracking or stock notification via mobile devices; or aimed to explore factors that influenced the implementation of these strategies, including studies of acceptability or feasibility. We included studies of all cadres of healthcare providers, including lay health workers, and others involved in the distribution of health commodities (administrative staff, managerial and supervisory staff, dispensary staff); and all other individuals involved in stock notification, who may be based in a facility or a community setting, and involved with the delivery of primary healthcare services. We included interventions aimed at improving the availability of health commodities using mobile devices in primary healthcare settings. For the primary objective, we included studies that compared health commodity tracking or stock notification via mobile devices with standard practice. For the secondary objectives, we included studies of health commodity tracking and stock notification via mobile device, if we could extract data relevant to our secondary objectives. DATA COLLECTION AND ANALYSIS: For the primary objective, two authors independently screened all records, extracted data from the included studies, and assessed the risk of bias. For the analyses of the primary objectives, we reported means and proportions where appropriate. We used the GRADE approach to assess the certainty of the evidence, and prepared a 'Summary of findings' table. For the secondary objective, two authors independently screened all records, extracted data from the included studies, and applied a thematic synthesis approach to synthesise the data. We assessed methodological limitation using the Ways of Evaluating Important and Relevant Data (WEIRD) tool. We used the GRADE-CERQual approach to assess our confidence in the evidence, and prepared a 'Summary of qualitative findings' table. MAIN RESULTS: Primary objective For the primary objective, we included one controlled before-after study conducted in Malawi. We are uncertain of the effect of cStock plus enhanced management, or cStock plus effective product transport on the availability of commodities, quality and timeliness of stock management, and satisfaction and acceptability, because we assessed the evidence as very low-certainty. The study did not report on resource use or unintended consequences. Secondary objective For the secondary objectives, we included 16 studies, using a range of study designs, which described a total of eleven interventions. All studies were conducted in African (Tanzania, Kenya, Malawi, Ghana, Ethiopia, Cameroon, Zambia, Liberia, Uganda, South Africa, and Rwanda) and Asian (Pakistan and India) countries. Most of the interventions aimed to make data about stock levels and potential stockouts visible to managers, who could then take corrective action to address them. We identified several factors that may influence the implementation of stock notification and tracking via mobile device. These include challenges tied to infrastructural issues, such as poor access to electricity or internet, and broader health systems issues, such as drug shortages at the national level which cannot be mitigated by interventions at the primary healthcare level (low confidence). Several factors were identified as important, including strong partnerships with local authorities, telecommunication companies, technical system providers, and non-governmental organizations (very low confidence); availability of stock-level data at all levels of the health system (low confidence); the role of supportive supervision and responsive management (moderate confidence); familiarity and training of health workers in the use of the digital devices (moderate confidence); availability of technical programming expertise for the initial development and ongoing maintenance of the digital systems (low confidence); incentives, such as phone credit for personal use, to support regular use of the system (low confidence); easy-to-use systems built with user participation (moderate confidence); use of basic or personal mobile phones to support easier adoption (low confidence); consideration for software features, such as two-way communication (low confidence); and data availability in an easy-to-use format, such as an interactive dashboard (moderate confidence). AUTHORS' CONCLUSIONS: We need more, well-designed, controlled studies comparing stock notification and commodity management via mobile devices with paper-based commodity management systems. Further studies are needed to understand the factors that may influence the implementation of such interventions, and how implementation considerations differ by variations in the intervention.

Mobile Technology for Community Health in Ghana: Is Maternal Messaging and Provider Use of Technology Cost-Effective in Improving Maternal and Child Health Outcomes at Scale?

BACKGROUND: Mobile technologies are emerging as tools to enhance health service delivery systems and empower clients to improve maternal, newborn, and child health. Limited evidence exists on the value for money of mobile health (mHealth) programs in low- and middle-income countries. OBJECTIVE: This study aims to forecast the incremental cost-effectiveness of the Mobile Technology for Community Health (MOTECH) initiative at scale across 170 districts in Ghana. METHODS: MOTECH's "Client Data Application" allows frontline health workers to digitize service delivery information and track the care of patients. MOTECH's other main component, the "Mobile Midwife," sends automated educational voice messages to mobile phones of pregnant and postpartum women. We measured program costs and consequences of scaling up MOTECH over a 10-year analytic time horizon. Economic costs were estimated from informant interviews and financial records. Health effects were modeled using the Lives Saved Tool with data from an independent evaluation of changes in key services coverage observed in Gomoa West District. Incremental cost-effectiveness ratios were presented overall and for each year of implementation. Uncertainty analyses assessed the robustness of results to changes in key parameters. RESULTS: MOTECH was scaled in clusters over a 3-year period to reach 78.7% (170/216) of Ghana's districts. Sustaining the program would cost US $17,618 on average annually per district. Over 10 years, MOTECH could potentially save an estimated 59,906 lives at a total cost of US $32 million. The incremental cost per disability-adjusted life year averted ranged from US $174 in the first year to US $6.54 in the tenth year of implementation and US $20.94 (95% CI US $20.34-$21.55) over 10 years. Uncertainty analyses suggested that the incremental cost-effectiveness ratio was most sensitive to changes in health effects, followed by personnel time. Probabilistic sensitivity analyses suggested that MOTECH had a 100% probability of being cost-effective above a willingness-to-pay threshold of US $50. CONCLUSIONS: This is the first study to estimate the value for money of the supply- and demand-side of an mHealth initiative. The adoption of MOTECH to improve MNCH service delivery and uptake represents good value for money in Ghana and should be considered for expansion. Integration with other mHealth solutions, including e-Tracker, may provide opportunities to continue or combine beneficial components of MOTECH to achieve a greater impact on health.