Four statistical frameworks for assessing an immune correlate of protection (surrogate endpoint) from a randomized, controlled, vaccine efficacy trial.

A central goal of vaccine research is to characterize and validate immune correlates of protection (CoPs). In addition to helping elucidate immunological mechanisms, a CoP can serve as a valid surrogate endpoint for an infectious disease clinical outcome and thus qualifies as a primary endpoint for vaccine authorization or approval without requiring resource-intensive randomized, controlled phase 3 trials. Yet, it is challenging to persuasively validate a CoP, because a prognostic immune marker can fail as a reliable basis for predicting/inferring the level of vaccine efficacy against a clinical outcome, and because the statistical analysis of phase 3 trials only has limited capacity to disentangle association from cause. Moreover, the multitude of statistical methods garnered for CoP evaluation in phase 3 trials renders the comparison, interpretation, and synthesis of CoP results challenging. Toward promoting broader harmonization and standardization of CoP evaluation, this article summarizes four complementary statistical frameworks for evaluating CoPs in a phase 3 trial, focusing on the frameworks' distinct scientific objectives as measured and communicated by distinct causal vaccine efficacy parameters. Advantages and disadvantages of the frameworks are considered, dependent on phase 3 trial context, and perspectives are offered on how the frameworks can be applied and their results synthesized.

Under-five mortality before and after implementation of the Liberia National Community Health Assistant (NCHA) program: A study protocol.

Between 2018 and 2022 the Liberian Government implemented the National Community Health Assistant (NCHA) program to improve provision of maternal and child health care to underserved rural areas of the country. Whereas the contributions of this and similar community health worker (CHW) based healthcare programs have been associated with improved process measures, the impact of a governmental CHW program at scale on child mortality has not been fully established. We will conduct a cluster sampled, community-based survey with landmark event calendars to retrospectively assess child births and deaths among all children born to women in the Grand Bassa District of Liberia. We will use a mixed effects Cox proportional hazards model, taking advantage of the staggered program implementation in Grand Bassa districts over a period of 4 years to compare rates of under-5 child mortality between the pre- and post-NCHA program implementation periods. This study will be the first to estimate the impact of the Liberian NCHA program on under-5 mortality.

A controlled effects approach to assessing immune correlates of protection.

An immune correlate of risk (CoR) is an immunologic biomarker in vaccine recipients associated with an infectious disease clinical endpoint. An immune correlate of protection (CoP) is a CoR that can be used to reliably predict vaccine efficacy (VE) against the clinical endpoint and hence is accepted as a surrogate endpoint that can be used for accelerated approval or guide use of vaccines. In randomized, placebo-controlled trials, CoR analysis is limited by not assessing a causal vaccine effect. To address this limitation, we construct the controlled risk curve of a biomarker, which provides the causal risk of an endpoint if all participants are assigned vaccine and the biomarker is set to different levels. Furthermore, we propose a causal CoP analysis based on controlled effects, where for the important special case that the biomarker is constant in the placebo arm, we study the controlled vaccine efficacy curve that contrasts the controlled risk curve with placebo arm risk. We provide identification conditions and formulae that account for right censoring of the clinical endpoint and two-phase sampling of the biomarker, and consider G-computation estimation and inference under a semiparametric model such as the Cox model. We add modular approaches to sensitivity analysis that quantify robustness of CoP evidence to unmeasured confounding. We provide an application to two phase 3 trials of a dengue vaccine indicating that controlled risk of dengue strongly varies with 50$\%$ neutralizing antibody titer. Our work introduces controlled effects causal mediation analysis to immune CoP evaluation.

Stochastic Interventional Vaccine Efficacy and Principal Surrogate Analyses of Antibody Markers as Correlates of Protection against Symptomatic COVID-19 in the COVE mRNA-1273 Trial.

The COVE trial randomized participants to receive two doses of mRNA-1273 vaccine or placebo on Days 1 and 29 (D1, D29). Anti-SARS-CoV-2 Spike IgG binding antibodies (bAbs), anti-receptor binding domain IgG bAbs, 50% inhibitory dilution neutralizing antibody (nAb) titers, and 80% inhibitory dilution nAb titers were measured at D29 and D57. We assessed these markers as correlates of protection (CoPs) against COVID-19 using stochastic interventional vaccine efficacy (SVE) analysis and principal surrogate (PS) analysis, frameworks not used in our previous COVE immune correlates analyses. By SVE analysis, hypothetical shifts of the D57 Spike IgG distribution from a geometric mean concentration (GMC) of 2737 binding antibody units (BAU)/mL (estimated vaccine efficacy (VE): 92.9% (95% CI: 91.7%, 93.9%)) to 274 BAU/mL or to 27,368 BAU/mL resulted in an overall estimated VE of 84.2% (79.0%, 88.1%) and 97.6% (97.4%, 97.7%), respectively. By binary marker PS analysis of Low and High subgroups (cut-point: 2094 BAU/mL), the ignorance interval (IGI) and estimated uncertainty interval (EUI) for VE were [85%, 90%] and (78%, 93%) for Low compared to [95%, 96%] and (92%, 97%) for High. By continuous marker PS analysis, the IGI and 95% EUI for VE at the 2.5th percentile (519.4 BAU/mL) vs. at the 97.5th percentile (9262.9 BAU/mL) of D57 Spike IgG concentration were [92.6%, 93.4%] and (89.2%, 95.7%) vs. [94.3%, 94.6%] and (89.7%, 97.0%). Results were similar for other D29 and D57 markers. Thus, the SVE and PS analyses additionally support all four markers at both time points as CoPs.

Pharmacokinetic serum concentrations of VRC01 correlate with prevention of HIV-1 acquisition.

BACKGROUND: The phase 2b proof-of-concept Antibody Mediated Prevention (AMP) trials showed that VRC01, an anti-HIV-1 broadly neutralising antibody (bnAb), prevented acquisition of HIV-1 sensitive to VRC01. To inform future study design and dosing regimen selection of candidate bnAbs, we investigated the association of VRC01 serum concentration with HIV-1 acquisition using AMP trial data. METHODS: The case-control sample included 107 VRC01 recipients who acquired HIV-1 and 82 VRC01 recipients who remained without HIV-1 during the study. We measured VRC01 serum concentrations with a qualified pharmacokinetic (PK) Binding Antibody Multiplex Assay. We employed nonlinear mixed effects PK modelling to estimate daily-grid VRC01 concentrations. Cox regression models were used to assess the association of VRC01 concentration at exposure and baseline body weight, with the hazard of HIV-1 acquisition and prevention efficacy as a function of VRC01 concentration. We also compared fixed dosing vs. body weight-based dosing via simulations. FINDINGS: Estimated VRC01 concentrations in VRC01 recipients without HIV-1 were higher than those in VRC01 recipients who acquired HIV-1. Body weight was inversely associated with HIV-1 acquisition among both placebo and VRC01 recipients but did not modify the prevention efficacy of VRC01. VRC01 concentration was inversely correlated with HIV-1 acquisition, and positively correlated with prevention efficacy of VRC01. Simulation studies suggest that fixed dosing may be comparable to weight-based dosing in overall predicted prevention efficacy. INTERPRETATION: These findings suggest that bnAb serum concentration may be a useful marker for dosing regimen selection, and operationally efficient fixed dosing regimens could be considered for future trials of HIV-1 bnAbs. FUNDING: Was provided by the National Institutes of Health, National Institute of Allergy and Infectious Diseases (NIAID) (UM1 AI068614, to the HIV Vaccine Trials Network [HVTN]; UM1 AI068635, to the HVTN Statistical Data and Management Center [SDMC], Fred Hutchinson Cancer Center [FHCC]; 2R37 054165 to the FHCC; UM1 AI068618, to HVTN Laboratory Center, FHCC; UM1 AI068619, to the HPTN Leadership and Operations Center; UM1 AI068613, to the HIV Prevention Trials Network [HPTN] Laboratory Center; UM1 AI068617, to the HPTN SDMC; and P30 AI027757, to the Center for AIDS Research, Duke University (AI P30 AI064518) and University of Washington (P30 AI027757) Centers for AIDS Research; R37AI054165 from NIAID to the FHCC; and OPP1032144 CA-VIMC Bill & Melinda Gates Foundation.

Comparing antibody assays as correlates of protection against COVID-19 in the COVE mRNA-1273 vaccine efficacy trial.

The best assay or marker to define mRNA-1273 vaccine-induced antibodies as a correlate of protection (CoP) is unclear. In the COVE trial, participants received two doses of the mRNA-1273 COVID-19 vaccine or placebo. We previously assessed IgG binding antibodies to the spike protein (spike IgG) or receptor binding domain (RBD IgG) and pseudovirus neutralizing antibody 50 or 80% inhibitory dilution titer measured on day 29 or day 57, as correlates of risk (CoRs) and CoPs against symptomatic COVID-19 over 4 months after dose. Here, we assessed a new marker, live virus 50% microneutralization titer (LV-MN50), and compared and combined markers in multivariable analyses. LV-MN50 was an inverse CoR, with a hazard ratio of 0.39 (95% confidence interval, 0.19 to 0.83) at day 29 and 0.51 (95% confidence interval, 0.25 to 1.04) at day 57 per 10-fold increase. In multivariable analyses, pseudovirus neutralization titers and anti-spike binding antibodies performed best as CoRs; combining antibody markers did not improve correlates. Pseudovirus neutralization titer was the strongest independent correlate in a multivariable model. Overall, these results supported pseudovirus neutralizing and binding antibody assays as CoRs and CoPs, with the live virus assay as a weaker correlate in this sample set. Day 29 markers performed as well as day 57 markers as CoPs, which could accelerate immunogenicity and immunobridging studies.

Immune correlates analysis of the ENSEMBLE single Ad26.COV2.S dose vaccine efficacy clinical trial.

Measuring immune correlates of disease acquisition and protection in the context of a clinical trial is a prerequisite for improved vaccine design. We analysed binding and neutralizing antibody measurements 4 weeks post vaccination as correlates of risk of moderate to severe-critical COVID-19 through 83 d post vaccination in the phase 3, double-blind placebo-controlled phase of ENSEMBLE, an international randomized efficacy trial of a single dose of Ad26.COV2.S. We also evaluated correlates of protection in the trial cohort. Of the three antibody immune markers we measured, we found most support for 50% inhibitory dilution (ID50) neutralizing antibody titre as a correlate of risk and of protection. The outcome hazard ratio was 0.49 (95% confidence interval 0.29, 0.81; P = 0.006) per 10-fold increase in ID50; vaccine efficacy was 60% (43%, 72%) at non-quantifiable ID50 (<2.7 IU50 ml-1) and increased to 89% (78%, 96%) at ID50 = 96.3 IU50 ml-1. Comparison of the vaccine efficacy by ID50 titre curves for ENSEMBLE-US, the COVE trial of the mRNA-1273 vaccine and the COV002-UK trial of the AZD1222 vaccine supported the ID50 titre as a correlate of protection across trials and vaccine types.

A controlled effects approach to assessing immune correlates of protection.

An immune correlate of risk (CoR) is an immunologic biomarker in vaccine recipients associated with an infectious disease clinical endpoint. An immune correlate of protection (CoP) is a CoR that can be used to reliably predict vaccine efficacy (VE) against the clinical endpoint and hence is accepted as a surrogate endpoint that can be used for accelerated approval or guide use of vaccines. In randomized, placebo-controlled trials, CoR analysis is limited by not assessing a causal vaccine effect. To address this limitation, we construct the controlled risk curve of a biomarker, which provides the causal risk of an endpoint if all participants are assigned vaccine and the biomarker is set to different levels. Furthermore, we propose a causal CoP analysis based on controlled effects, where for the important special case that the biomarker is constant in the placebo arm, we study the controlled vaccine efficacy curve that contrasts the controlled risk curve with placebo arm risk. We provide identification conditions and formulae that account for right censoring of the clinical endpoint and two-phase sampling of the biomarker, and consider G-computation estimation and inference under a semiparametric model such as the Cox model. We add modular approaches to sensitivity analysis that quantify robustness of CoP evidence to unmeasured confounding. We provide an application to two phase 3 trials of a dengue vaccine indicating that controlled risk of dengue strongly varies with 50$\%$ neutralizing antibody titer. Our work introduces controlled effects causal mediation analysis to immune CoP evaluation.

Analysis of stepped wedge cluster randomized trials in the presence of a time-varying treatment effect.

Stepped wedge cluster randomized controlled trials are typically analyzed using models that assume the full effect of the treatment is achieved instantaneously. We provide an analytical framework for scenarios in which the treatment effect varies as a function of exposure time (time since the start of treatment) and define the "effect curve" as the magnitude of the treatment effect on the linear predictor scale as a function of exposure time. The "time-averaged treatment effect" (TATE) and "long-term treatment effect" (LTE) are summaries of this curve. We analytically derive the expectation of the estimator δ ^ $$ \hat{\delta} $$ resulting from a model that assumes an immediate treatment effect and show that it can be expressed as a weighted sum of the time-specific treatment effects corresponding to the observed exposure times. Surprisingly, although the weights sum to one, some of the weights can be negative. This implies that δ ^ $$ \hat{\delta} $$ may be severely misleading and can even converge to a value of the opposite sign of the true TATE or LTE. We describe several models, some of which make assumptions about the shape of the effect curve, that can be used to simultaneously estimate the entire effect curve, the TATE, and the LTE. We evaluate these models in a simulation study to examine the operating characteristics of the resulting estimators and apply them to two real datasets.

Adherence to Perioperative Antibiotic Prophylaxis Recommendations and Its Impact on Postoperative Surgical Site Infections.

Introduction Surgical site infections (SSIs) are common and carry a significant risk of morbidity and mortality and lead to increased healthcare costs. Perioperative antibiotic prophylaxis decreases the risk of SSIs. There are several guidelines on the use of perioperative antibiotic prophylaxis. The American College of Surgeons (ACS) recommends weight-based antibiotic administration within 60 minutes prior to (two hours for vancomycin/fluoroquinolones) incision and redosing by drug half-life. There are limited data regarding adherence to existing recommendations. Furthermore, there are scarce data on the relationship between adherence to recommendations and the risk of postoperative SSI. Objectives In this study, we aimed to assess the adherence to ACS guidelines for perioperative antimicrobial prophylaxis in the Seattle Children's Hospital (SCH) National Surgical Quality Improvement Program (NSQIP) pediatric cohort and to determine whether adherence to ACS guidelines is associated with a decreased risk of SSI. the secondary objective was to identify risk factors associated with SSI in our patient population. Materials and methods We conducted a secondary analysis of an institutional NSQIP pediatric data cohort between Jan 1, 2012, and Dec 31, 2017. We calculated summary statistics to assess adherence to ACS recommendations and fit a logistic regression model to identify factors associated with the risk of SSI. Patients who did not receive antibiotic prophylaxis were excluded. Results  A total of 6,072 surgeries among 5,532 patients met the inclusion criteria. Adherence was achieved for weight-based dosing in 35% of surgeries, administration prior to the incision in 91%, administration within 60 minutes (two hours for vancomycin/fluoroquinolones) in 86%, correct redosing in 97%, and to all recommendations in 29%. There were no significant associations between any adherence metrics and SSI, although confidence intervals were wide for some metrics. Factors associated with SSI when adherence was met included urgent case status, wound class 2 or 4, the American Society of Anesthesiologists (ASA) class 2-5, and surgery duration. Conclusion There was varying adherence to ACS recommendations on antibiotic prophylaxis in our cohort. More evidence is needed to better understand the effects of adherence to any or all components of the recommendations on SSI. We identified a group of pediatric patients at risk of SSI and a need for further research and targeted interventions.

Immune Correlates Analysis of a Single Ad26.COV2.S Dose in the ENSEMBLE COVID-19 Vaccine Efficacy Clinical Trial.

Anti-spike IgG binding antibody, anti-receptor binding domain IgG antibody, and pseudovirus neutralizing antibody measurements four weeks post-vaccination were assessed as correlates of risk of moderate to severe-critical COVID-19 outcomes through 83 days post-vaccination and as correlates of protection following a single dose of Ad26.COV2.S COVID-19 vaccine in the placebo-controlled phase of ENSEMBLE, an international, randomized efficacy trial. Each marker had evidence as a correlate of risk and of protection, with strongest evidence for 50% inhibitory dilution (ID50) neutralizing antibody titer. The outcome hazard ratio was 0.49 (95% confidence interval 0.29, 0.81; p=0.006) per 10-fold increase in ID50; vaccine efficacy was 60% (43, 72%) at nonquantifiable ID50 (< 2.7 IU50/ml) and rose to 89% (78, 96%) at ID50 = 96.3 IU50/ml. Comparison of the vaccine efficacy by ID50 titer curves for ENSEMBLE-US, the COVE trial of the mRNA-1273 vaccine, and the COV002-UK trial of the AZD1222 vaccine supported consistency of the ID50 titer correlate of protection across trials and vaccine types.

Random effect misspecification in stepped wedge designs.

Stepped wedge cluster randomized trials are often analysed using linear mixed effects models that may include random effects for cluster, time and/or treatment. We investigate the impact of misspecification of the random effects structure of the model. Specifically, we considered two cases of misspecification of the random effects in a cross-sectional stepped wedge cluster randomized trials model - fit a linear mixed effects model with random time effects but the true model includes random treatment effects (case 1) or fit a linear mixed effects model with random treatment effect but the true model includes random time effects (case 2) - and derived the variance of the estimated treatment effect under misspecification. We defined two measures of the effect of misspecification: validity and efficiency. Validity is the ratio of the model-based variance of the treatment effect from the mis-specified model divided by the true variance of the treatment effect from the mis-specified model (based on a sandwich estimate of the variance). Efficiency is the ratio of the model-based variance of the treatment effect from the correctly specified model divided by the true variance of the treatment effect from the mis-specified model. We found that validity is less than 1.0 (anti-conservative) in almost all situations investigated with the exception of case 1 with two sequences, when validity could be greater than 1.0. Efficiency is less than 1 in all cases and depends on the intracluster correlation coefficient, the relative magnitude of the variance of the misclassified variance component, and the number of sequences. In general, there is no universal recommendation as to the most robust approach except for the case of a classic stepped wedge cluster randomized trial with only 2 sequences, where fitting a random time model is less likely to lead to anti-conservative inference compared with fitting a random intervention model.

Model misspecification in stepped wedge trials: Random effects for time or treatment.

Mixed models are commonly used to analyze stepped wedge trials (SWTs) to account for clustering and repeated measures on clusters. One critical issue researchers face is whether to include a random time effect or a random treatment effect. When the wrong model is chosen, inference on the treatment effect may be invalid. We explore asymptotic and finite-sample convergence of variance component estimates when the model is misspecified and how misspecification affects the estimated variance of the treatment effect. For asymptotic results, we rely on analytical solutions rather than simulation studies, which allow us to succinctly describe the convergence of misspecified estimates, even though there are multiple roots for each misspecified model. We found that both direction and magnitude of the bias associated with model-based standard errors depends on the study design and magnitude of the true variance components. We identify some scenarios in which choosing the wrong random effect has a large impact on model-based inference. However, many trends depend on trial design and assumptions about the true correlation structure, so we provide tools for researchers to investigate specific scenarios of interest. We use data from an SWT on disinvesting from weekend services in hospital wards to demonstrate how these results can be applied as a sensitivity analysis, which quantifies the impact of misspecification under a variety of settings and directly compares the potential consequences of different modeling choices. Our results will provide guidance for prespecified model choices and supplement sensitivity analyses to inform confidence in the validity of results.

Measuring Knowledge of Community Health Workers at the Last Mile in Liberia: Feasibility and Results of Clinical Vignette Assessments.

INTRODUCTION: Community health workers (CHWs) can provide lifesaving treatment for children in remote areas, but high-quality care is essential for effective delivery. Measuring the quality of community-based care in remote areas is logistically challenging. Clinical vignettes have been validated in facility settings as a proxy for competency. We assessed feasibility and effectiveness of clinical vignettes to measure CHW knowledge of integrated community case management (iCCM) in Liberia's national CHW program. METHODS: We developed 3 vignettes to measure knowledge of iCCM illnesses (malaria, diarrhea, and pneumonia) in 4 main areas: assessment, diagnosis, treatment, and caregiver instructions. Trained nurse supervisors administered the vignettes to CHWs in 3 counties in rural Liberia as part of routine program supervision between January and May 2019, collected data on CHW knowledge using a standardized checklist tool, and provided feedback and coaching to CHWs in real time after vignette administration. Proportions of vignettes correctly managed, including illness classification, treatment, and referral where necessary, were calculated. We assessed feasibility, defined as the ability of clinical supervisors to administer the vignettes integrated into their routine activities once per year for each CHW, and effectiveness, defined as the ability of the vignettes to measure the primary outcomes of CHW knowledge of diagnosis and treatment including referrals. RESULTS: We were able to integrate this assessment into routine supervision, facilitate real-time coaching, and collect data on iCCM knowledge among 155 CHWs through delivery of 465 vignettes. Diagnosis including severity was correct in 65%-82% of vignettes. CHWs correctly identified danger signs in 44%-50% of vignettes, correctly proposed referral to the facility in 63% of vignettes including danger signs, and chose correct lifesaving treatment in 23%-65% of vignettes. Both diagnosis and lifesaving treatment rates were highest for malaria and lowest for severe pneumonia. CONCLUSION: Administration of vignettes to assess knowledge of correct iCCM case management was feasible and effective in producing results in this setting. Proportions of correct diagnosis and lifesaving treatment varied, with high proportions for uncomplicated disease, but lower for more severe cases, with accurate recognition of danger signs posing a challenge. Future work includes validation of vignettes for use with CHWs through direct observation, strengthening supportive supervision, and program interventions to address identified knowledge gaps.

Design and implementation of a mobile health electronic data capture platform that functions in fully-disconnected settings: a pilot study in rural Liberia.

BACKGROUND: Mobile phones and personal digital assistants have been used for data collection in developing world settings for over three decades, and have become increasingly common. However, the use of electronic data capture (EDC) through mobile phones is limited in many areas by inconsistent network connectivity and poor access to electricity, which thwart data transmission and device usage. This is the case in rural Liberia, where many health workers live and work in areas without any access to cellular connectivity or reliable power. Many existing EDC mobile software tools are built for occasionally-disconnected settings, allowing a user to collect data while out of range of a cell tower and transmit data to a central server when he/she regains a network connection. However, few tools exist that can be used indefinitely in fully-disconnected settings, where a user will never have access to the internet or a cell network. This led us to create and implement an EDC software tool that allows for completely offline data transfer and application updating. RESULTS: We designed, pilot-tested, and scaled an open-source fork of Open Data Kit Collect (an Android application that can be used to create EDC systems) that allows for offline Bluetooth-based bidirectional data transfer, enabling a system in which permanently-offline users can collect data and receive application updates. We implemented this platform among a cohort of 317 community health workers and 28 supervisors in a remote area of rural Liberia with incomplete cellular connectivity and low access to power sources. CONCLUSIONS: Running a fully-offline EDC program that completely bypasses the cellular network was found to be feasible; the system is still running, over 4 years after the initial pilot program. The users of this program can theoretically collect data offline for months or years, assuming they receive hardware support when needed. Fully-offline EDC has applications in settings where cellular network coverage is poor, as well as in disaster relief settings in which portions of the communications infrastructure may be temporarily nonfunctional.

Satellite images and machine learning can identify remote communities to facilitate access to health services.

OBJECTIVE: Community health systems operating in remote areas require accurate information about where people live to efficiently provide services across large regions. We sought to determine whether a machine learning analyses of satellite imagery can be used to map remote communities to facilitate service delivery and planning. MATERIALS AND METHODS: We developed a method for mapping communities using a deep learning approach that excels at detecting objects within images. We trained an algorithm to detect individual buildings, then examined building clusters to identify groupings suggestive of communities. The approach was validated in southeastern Liberia, by comparing algorithmically generated results with community location data collected manually by enumerators and community health workers. RESULTS: The deep learning approach achieved 86.47% positive predictive value and 79.49% sensitivity with respect to individual building detection. The approach identified 75.67% (n = 451) of communities registered through the community enumeration process, and identified an additional 167 potential communities not previously registered. Several instances of false positives and false negatives were identified. DISCUSSION: Analysis of satellite images is a promising solution for mapping remote communities rapidly, and with relatively low costs. Further research is needed to determine whether the communities identified algorithmically, but not registered in the manual enumeration process, are currently inhabited. CONCLUSIONS: To our knowledge, this study represents the first effort to apply image recognition algorithms to rural healthcare delivery. Results suggest that these methods have the potential to enhance community health worker scale-up efforts in underserved remote communities.

A Community Health Worker Intervention to Increase Childhood Disease Treatment Coverage in Rural Liberia: A Controlled Before-and-After Evaluation.

OBJECTIVES: To assess a community health worker (CHW) program's impact on childhood illness treatment in rural Liberia. METHODS: We deployed CHWs in half of Rivercess County in August 2015 with the other half constituting a comparison group until July 2016. All CHWs were provided cash incentives, supply chain support, and monthly clinical supervision. We conducted stratified cluster-sample population-based surveys at baseline (March-April 2015) and follow-up (April-June 2016) and performed a difference-in-differences analysis, adjusted by inverse probability of treatment weighting, to assess changes in treatment of fever, diarrhea, and acute respiratory infection by a qualified provider. RESULTS: We estimated a childhood treatment difference-in-differences of 56.4 percentage points (95% confidence interval [CI] = 36.4, 76.3). At follow-up, CHWs provided 57.6% (95% CI = 42.8, 71.2) of treatment in the intervention group. The difference-in-differences diarrhea oral rehydration therapy was 22.4 percentage points (95% CI = -0.7, 45.5). CONCLUSIONS: Implementation of a CHW program in Rivercess County, Liberia, was associated with large, statistically significant improvements treatment by a qualified provider; however, improvements in correct diarrhea treatment were lower than improvements in coverage. Findings from this study offer support for expansion of Liberia's new National Community Health Assistant Program.

Validation Relaxation: A Quality Assurance Strategy for Electronic Data Collection.

BACKGROUND: The use of mobile devices for data collection in developing world settings is becoming increasingly common and may offer advantages in data collection quality and efficiency relative to paper-based methods. However, mobile data collection systems can hamper many standard quality assurance techniques due to the lack of a hardcopy backup of data. Consequently, mobile health data collection platforms have the potential to generate datasets that appear valid, but are susceptible to unidentified database design flaws, areas of miscomprehension by enumerators, and data recording errors. OBJECTIVE: We describe the design and evaluation of a strategy for estimating data error rates and assessing enumerator performance during electronic data collection, which we term "validation relaxation." Validation relaxation involves the intentional omission of data validation features for select questions to allow for data recording errors to be committed, detected, and monitored. METHODS: We analyzed data collected during a cluster sample population survey in rural Liberia using an electronic data collection system (Open Data Kit). We first developed a classification scheme for types of detectable errors and validation alterations required to detect them. We then implemented the following validation relaxation techniques to enable data error conduct and detection: intentional redundancy, removal of "required" constraint, and illogical response combinations. This allowed for up to 11 identifiable errors to be made per survey. The error rate was defined as the total number of errors committed divided by the number of potential errors. We summarized crude error rates and estimated changes in error rates over time for both individuals and the entire program using logistic regression. RESULTS: The aggregate error rate was 1.60% (125/7817). Error rates did not differ significantly between enumerators (P=.51), but decreased for the cohort with increasing days of application use, from 2.3% at survey start (95% CI 1.8%-2.8%) to 0.6% at day 45 (95% CI 0.3%-0.9%; OR=0.969; P<.001). The highest error rate (84/618, 13.6%) occurred for an intentional redundancy question for a birthdate field, which was repeated in separate sections of the survey. We found low error rates (0.0% to 3.1%) for all other possible errors. CONCLUSIONS: A strategy of removing validation rules on electronic data capture platforms can be used to create a set of detectable data errors, which can subsequently be used to assess group and individual enumerator error rates, their trends over time, and categories of data collection that require further training or additional quality control measures. This strategy may be particularly useful for identifying individual enumerators or systematic data errors that are responsive to enumerator training and is best applied to questions for which errors cannot be prevented through training or software design alone. Validation relaxation should be considered as a component of a holistic data quality assurance strategy.

Implementation research on community health workers' provision of maternal and child health services in rural Liberia.

OBJECTIVE: To assess changes in the use of essential maternal and child health services in Konobo, Liberia, after implementation of an enhanced community health worker (CHW) programme. METHODS: The Liberian Ministry of Health partnered with Last Mile Health, a nongovernmental organization, to implement a pilot CHW programme with enhanced recruitment, training, supervision and compensation. To assess changes in maternal and child health-care use, we conducted repeated cross-sectional cluster surveys before (2012) and after (2015) programme implementation. FINDINGS: Between 2012 and 2015, 54 CHWs, seven peer supervisors and three clinical supervisors were trained to serve a population of 12 127 people in 44 communities. The regression-adjusted percentage of children receiving care from formal care providers increased by 60.1 (95% confidence interval, CI: 51.6 to 68.7) percentage points for diarrhoea, by 30.6 (95% CI: 20.5 to 40.7) for fever and by 51.2 (95% CI: 37.9 to 64.5) for acute respiratory infection. Facility-based delivery increased by 28.2 points (95% CI: 20.3 to 36.1). Facility-based delivery and formal sector care for acute respiratory infection and diarrhoea increased more in agricultural than gold-mining communities. Receipt of one-or-more antenatal care sessions at a health facility and postnatal care within 24 hours of delivery did not change significantly. CONCLUSION: We identified significant increases in uptake of child and maternal health-care services from formal providers during the pilot CHW programme in remote rural Liberia. Clinic-based services, such as postnatal care, and services in specific settings, such as mining areas, require additional interventions to achieve optimal outcomes.

Charting health system reconstruction in post-war Liberia: a comparison of rural vs. remote healthcare utilization.

BACKGROUND: Despite a growing global emphasis on universal healthcare, access to basic primary care for remote populations in post-conflict countries remains a challenge. To better understand health sector recovery in post-conflict Liberia, this paper seeks to evaluate changes in utilization of health services among rural populations across a 5-year time span. METHODS: We assessed trends in healthcare utilization among the national rural population using the Liberian Demographic and Health Survey (DHS) from 2007 and 2013. We compared these results to results obtained from a two-staged cluster survey in 2012 in the district of Konobo, Liberia, to assess for differential health utilization in an isolated, remote region. Our primary outcomes of interest were maternal and child health service care seeking and utilization. RESULTS: Most child and maternal health indicators improved in the DHS rural sub-sample from 2007 to 2013. However, this progress was not reflected in the remote Konobo population. A lower proportion of women received 4+ antenatal care visits (AOR 0.28, P < 0.001) or any postnatal care (AOR 0.25, P <0.001) in Konobo as compared to the 2013 DHS. Similarly, a lower proportion of children received professional care for common childhood illnesses, including acute respiratory infection (9 % vs. 52 %, P < 0.001) or diarrhea (11 % vs. 46 %, P < 0.001). CONCLUSIONS: Our data suggest that, despite the demonstrable success of post-war rehabilitation in rural regions, particularly remote populations in Liberia remain at disproportionate risk for limited access to basic health services. As a renewed effort is placed on health systems reconstruction in the wake of the Ebola-epidemic, a specific focus on solutions to reach isolated populations will be necessary in order to ensure extension of coverage to remote regions such as Konobo.