User Perceptions and Use of an Enhanced Electronic Health Record in Rwanda With and Without Clinical Alerts: Cross-sectional Survey.

BACKGROUND: Electronic health records (EHRs) have been implemented in many low-resource settings but lack strong evidence for usability, use, user confidence, scalability, and sustainability. OBJECTIVE: This study aimed to evaluate staff use and perceptions of an EHR widely used for HIV care in >300 health facilities in Rwanda, providing evidence on factors influencing current performance, scalability, and sustainability. METHODS: A randomized, cross-sectional, structured interview survey of health center staff was designed to assess functionality, use, and attitudes toward the EHR and clinical alerts. This study used the associated randomized clinical trial study sample (56/112, 50% sites received an enhanced EHR), pulling 27 (50%) sites from each group. Free-text comments were analyzed thematically using inductive coding. RESULTS: Of the 100 participants, 90 (90% response rate) were interviewed at 54 health centers: 44 (49%) participants were clinical and 46 (51%) were technical. The EHR top uses were to access client data easily or quickly (62/90, 69%), update patient records (56/89, 63%), create new patient records (49/88, 56%), generate various reports (38/85, 45%), and review previous records (43/89, 48%). In addition, >90% (81/90) of respondents agreed that the EHR made it easier to make informed decisions, was worth using, and has improved patient information quality. Regarding availability, (66/88) 75% said they could always or almost always count on the EHR being available, whereas (6/88) 7% said never/almost never. In intervention sites, staff were significantly more likely to update existing records (P=.04), generate summaries before (P<.001) or during visits (P=.01), and agree that "the EHR provides useful alerts, and reminders" (P<.01). CONCLUSIONS: Most users perceived the EHR as well accepted, appropriate, and effective for use in low-resource settings despite infrastructure limitation in 25% (22/88) of the sites. The implementation of EHR enhancements can improve the perceived usefulness and use of key functions. Successful scale-up and use of EHRs in small health facilities could improve clinical documentation, care, reporting, and disease surveillance in low- and middle-income countries.

Development of standard indicators to assess use of electronic health record systems implemented in low-and medium-income countries.

BACKGROUND: Electronic Health Record Systems (EHRs) are being rolled out nationally in many low- and middle-income countries (LMICs) yet assessing actual system usage remains a challenge. We employed a nominal group technique (NGT) process to systematically develop high-quality indicators for evaluating actual usage of EHRs in LMICs. METHODS: An initial set of 14 candidate indicators were developed by the study team adapting the Human Immunodeficiency Virus (HIV) Monitoring, Evaluation, and Reporting indicators format. A multidisciplinary team of 10 experts was convened in a two-day NGT workshop in Kenya to systematically evaluate, rate (using Specific, Measurable, Achievable, Relevant, and Time-Bound (SMART) criteria), prioritize, refine, and identify new indicators. NGT steps included introduction to candidate indicators, silent indicator ranking, round-robin indicator rating, and silent generation of new indicators. 5-point Likert scale was used in rating the candidate indicators against the SMART components. RESULTS: Candidate indicators were rated highly on SMART criteria (4.05/5). NGT participants settled on 15 final indicators, categorized as system use (4); data quality (3), system interoperability (3), and reporting (5). Data entry statistics, systems uptime, and EHRs variable concordance indicators were rated highest. CONCLUSION: This study describes a systematic approach to develop and validate quality indicators for determining EHRs use and provides LMICs with a multidimensional tool for assessing success of EHRs implementations.

Building workforce capacity for effective use of health information systems: Evaluation of a blended eLearning course in Namibia and Tanzania.

BACKGROUND: Electronic health information systems (HIS) are critical components of national health systems, and have been identified as a key element in the development and strengthening of health systems globally. Novel approaches are needed to effectively and efficiently train health care workers on the use of HIS. One such approach is the use of digital eLearning programs, either alone or blended with face-to-face learning activities. METHODS: We developed a novel blended eLearning course based on an in-person HIS training package previously developed by the United States Centers for Disease Control and Prevention. We then conducted a pilot implementation of the eLearning course in Namibia and Tanzania. RESULTS: The blended eLearning pilot program enrolled 131 people, 72 (55%) from Namibia and 59 (45%) from Tanzania. The majority of enrollees were female (n = 88, 67%) and were nurses (n = 66, 50%). Of the 131 people who participated in the in-person orientation, 95 (73%) completed some or all of the eLearning modules. Across all three modules, the mean score on the post-test was significantly greater than on the pre-test (p < 0.001). When comparing results from previous in-person workshops and the blended eLearning course, we found that participants experienced strong learning gains in both, although learning gains were somewhat greater in the in-person course. Blended eLearning course participants reported good to very good satisfaction with the overall content of the course and with the eLearning modules (3.5 and 3.6 out of 5-point Likert scale). We estimate that the total cost per participant is 2.2-3.4 times greater for the in-person course (estimated cost USD $980) than for the blended eLearning course (estimated cost USD $287-$437). CONCLUSION: A blended eLearning course is an effective method with which to train healthcare workers in the basic features of HIS, and the cost is up to 3.4 times less expensive than for an in-person course with similar content.

Effect of a clinical decision support system on early action on immunological treatment failure in patients with HIV in Kenya: a cluster randomised controlled trial.

BACKGROUND: A clinical decision support system (CDSS) is a computer program that applies a set of rules to data stored in electronic health records to offer actionable recommendations. We aimed to establish whether a CDSS that supports detection of immunological treatment failure among patients with HIV taking antiretroviral therapy (ART) would improve appropriate and timely action. METHODS: We did this prospective, cluster randomised controlled trial in adults and children (aged ≥18 months) who were eligible for, and receiving, ART at HIV clinics in Siaya County, western Kenya. Health facilities were randomly assigned (1:1), via block randomisation (block size of two) with a computer-generated random number sequence, to use electronic health records either alone (control) or with CDSS (intervention). Facilities were matched by type and by number of patients enrolled in HIV care. The primary outcome measure was the difference between groups in the proportion of patients who experienced immunological treatment failure and had a documented clinical action. We used generalised linear mixed models with random effects to analyse clustered data. This trial is registered with ClinicalTrials.gov, number NCT01634802. FINDINGS: Between Sept 1, 2012, and Jan 31, 2014, 13 clinics, comprising 41,062 patients, were randomly assigned to the control group (n=6) or the intervention group (n=7). Data collection at each site took 12 months. Among patients eligible for ART, 10,358 (99%) of 10,478 patients were receiving ART at control sites and 10,991 (99%) of 11,028 patients were receiving ART at intervention sites. Of these patients, 1125 (11%) in the control group and 1342 (12%) in the intervention group had immunological treatment failure, of whom 332 (30%) and 727 (54%), respectively, received appropriate action. The likelihood of clinicians taking appropriate action on treatment failure was higher with CDSS alerts than with no decision support system (adjusted odds ratio 3·18, 95% CI 1·02-9·87). INTERPRETATION: CDSS significantly improved the likelihood of appropriate and timely action on immunological treatment failure. We expect our findings will be generalisable to virological monitoring of patients with HIV receiving ART once countries implement the 2015 WHO recommendation to scale up viral load monitoring. FUNDING: US President's Emergency Plan for AIDS Relief (PEPFAR), through the US Centers for Disease Control and Prevention.

Automating indicator data reporting from health facility EMR to a national aggregate data system in Kenya: An Interoperability field-test using OpenMRS and DHIS2.

Introduction: Developing countries are increasingly strengthening national health information systems (HIS) for evidence-based decision-making. However, the inability to report indicator data automatically from electronic medical record systems (EMR) hinders this process. Data are often printed and manually re-entered into aggregate reporting systems. This affects data completeness, accuracy, reporting timeliness, and burdens staff who support routine indicator reporting from patient-level data. Method: After conducting a feasibility test to exchange indicator data from Open Medical Records System (OpenMRS) to District Health Information System version 2 (DHIS2), we conducted a field test at a health facility in Kenya. We configured a field-test DHIS2 instance, similar to the Kenya Ministry of Health (MOH) DHIS2, to receive HIV care and treatment indicator data and the KenyaEMR, a customized version of OpenMRS, to generate and transmit the data from a health facility. After training facility staff how to send data using DHIS2 reporting module, we compared completeness, accuracy and timeliness of automated indicator reporting with facility monthly reports manually entered into MOH DHIS2. Results: All 45 data values in the automated reporting process were 100% complete and accurate while in manual entry process, data completeness ranged from 66.7% to 100% and accuracy ranged from 33.3% to 95.6% for seven months (July 2013-January 2014). Manual tally and entry process required at least one person to perform each of the five reporting activities, generating data from EMR and manual entry required at least one person to perform each of the three reporting activities, while automated reporting process had one activity performed by one person. Manual tally and entry observed in October 2013 took 375 minutes. Average time to generate data and manually enter into DHIS2 was over half an hour (M=32.35 mins, SD=0.29) compared to less than a minute for automated submission (M=0.19 mins, SD=0.15). Discussion and Conclusion: The results indicate that indicator data sent electronically from OpenMRS-based EMR at a health facility to DHIS2 improves data completeness, eliminates transcription errors and delays in reporting, and reduces the reporting burden on human resources. This increases availability of quality indicator data using available resources to facilitate monitoring service delivery and measuring progress towards set goals.

The effect of electronic medical record-based clinical decision support on HIV care in resource-constrained settings: a systematic review.

BACKGROUND: It is estimated that one million people infected with HIV initiate anti-retroviral therapy (ART) in resource-constrained countries annually. This occurs against a background of overburdened health workers with limited skills to handle rapidly changing treatment standards and guidelines hence compromising quality of care. Electronic medical record (EMR)-based clinical decision support systems (CDSS) are considered a solution to improve quality of care. Little evidence, however, exists on the effectiveness of EMR-based CDSS on quality of HIV care and treatment in resource-constrained settings. OBJECTIVE: The aim of this systematic review was to identify original studies on EMR-based CDSS describing process and outcome measures as well as reported barriers to their implementation in resource-constrained settings. We characterized the studies by guideline adherence, data and process, and barriers to CDSS implementation. METHODS: Two reviewers independently assessed original articles from a search of the MEDLINE, EMBASE, CINAHL and Global Health Library databases until January 2012. The included articles were those that evaluated or described the implementation of EMR-based CDSS that were used in HIV care in low-income countries. RESULTS: A total of 12 studies met the inclusion criteria, 10 of which were conducted in sub-Saharan Africa and 2 in the Caribbean. None of the papers described a strong (randomized controlled) evaluation design. Guideline adherence: One study showed that ordering rates for CD4 tests were significantly higher when reminders were used. Data and process: Studies reported reduction in data errors, reduction in missed appointments, reduction in missed CD4 results and reduction in patient waiting time. Two studies showed a significant increase in time spent by clinicians on direct patient care. Barriers to CDSS implementation: Technical infrastructure problems such as unreliable electric power and erratic Internet connectivity, clinicians' limited computer skills and failure by providers to comply with the reminders are key impediments to the implementation and effective use of CDSS. CONCLUSION: The limited number of evaluation studies, the basic and heterogeneous study designs, and varied outcome measures make it difficult to meaningfully conclude on the effectiveness of CDSS on quality of HIV care and treatment in resource-limited settings. High quality evaluation studies are needed. Factors specific to implementation of EMR-based CDSS in resource-limited setting should be addressed before such countries can demonstrate its full benefits. More work needs to be done to overcome the barriers to EMR and CDSS implementation in developing countries such as technical infrastructure and care providers' computer illiteracy. However, simultaneously evaluating and describing CDSS implementation strategies that work can further guide wise investments in their wider rollout.

Protecting HIV information in countries scaling up HIV services: a baseline study.

BACKGROUND: Individual-level data are needed to optimize clinical care and monitor and evaluate HIV services. Confidentiality and security of such data must be safeguarded to avoid stigmatization and discrimination of people living with HIV. We set out to assess the extent that countries scaling up HIV services have developed and implemented guidelines to protect the confidentiality and security of HIV information. METHODS: Questionnaires were sent to UNAIDS field staff in 98 middle- and lower-income countries, some reportedly with guidelines (G-countries) and others intending to develop them (NG-countries). Responses were scored, aggregated and weighted to produce standard scores for six categories: information governance, country policies, data collection, data storage, data transfer and data access. Responses were analyzed using regression analyses for associations with national HIV prevalence, gross national income per capita, OECD income, receiving US PEPFAR funding, and being a G- or NG-country. Differences between G- and NG-countries were investigated using non-parametric methods. RESULTS: Higher information governance scores were observed for G-countries compared with NG-countries; no differences were observed between country policies or data collection categories. However, for data storage, data transfer and data access, G-countries had lower scores compared with NG-countries. No significant associations were observed between country score and HIV prevalence, per capita gross national income, OECD economic category, and whether countries had received PEPFAR funding. CONCLUSIONS: Few countries, including G-countries, had developed comprehensive guidelines on protecting the confidentiality and security of HIV information. Countries must develop their own guidelines, using established frameworks to guide their efforts, and may require assistance in adapting, adopting and implementing them.

Why and how to monitor the cost and evaluate the cost-effectiveness of HIV services in countries.

The number of people in the world living with HIV is increasing as HIV-related mortality has declined but the annual number of people newly infected with HIV has not. The international response to contain the HIV pandemic, meanwhile, has grown. Since 2006, an international commitment to scale up prevention, treatment, care and support services in middle and lower-income countries by 2010 has been part of the Universal Access programme, which itself plays an important part in achieving the Millennium Development Goals by 2015. Apart from providing technical support, donor countries and agencies have substantially increased their funding to enable countries to scale up HIV services. Many countries have been developing their HIV monitoring and evaluation systems to generate the strategic information required to track their response and ensure the best use of the new funds. Financial information is an important aspect of the strategic information required for scaling up existing services as well as assessing the effect of new ones. It involves two components: tracking the money available and spent on HIV at all levels, through budget tracking, national health accounts and national AIDS spending assessments, and estimating the cost and efficiency of HIV services. The cost of service provision should be monitored over time, whereas evaluations of the cost-effectiveness of services are required periodically; both should be part of any country's HIV monitoring and evaluation system. This paper provides country examples of the complementary relationship between monitoring the cost of HIV services and evaluating their cost-effectiveness. It also summarizes global initiatives that enable countries to develop their own HIV monitoring and evaluation systems and to generate relevant, robust and up-to-date strategic information.