Exploring the accuracy of self-reported maternal and newborn care in select studies from low and middle-income country settings: do respondent and facility characteristics affect measurement?

BACKGROUND: Accurate data on the receipt of essential maternal and newborn health interventions is necessary to interpret and address gaps in effective coverage. Validation results of commonly used content and quality of care indicators routinely implemented in international survey programs vary across settings. We assessed how respondent and facility characteristics influenced the accuracy of women's recall of interventions received in the antenatal and postnatal periods. METHODS: We synthesized reporting accuracy using data from a known sample of validation studies conducted in Sub-Saharan Africa and Southeast Asia, which assessed the validity of women's self-report of received antenatal care (ANC) (N = 3 studies, 3,169 participants) and postnatal care (PNC) (N = 5 studies, 2,462 participants) compared to direct observation. For each study, indicator sensitivity and specificity are presented with 95% confidence intervals. Univariate fixed effects and bivariate random effects models were used to examine whether respondent characteristics (e.g., age group, parity, education level), facility quality, or intervention coverage level influenced the accuracy of women's recall of whether interventions were received. RESULTS: Intervention coverage was associated with reporting accuracy across studies for the majority (9 of 12) of PNC indicators. Increasing intervention coverage was associated with poorer specificity for 8 indicators and improved sensitivity for 6 indicators. Reporting accuracy for ANC or PNC indicators did not consistently differ by any other respondent or facility characteristic. CONCLUSIONS: High intervention coverage may contribute to higher false positive reporting (poorer specificity) among women who receive facility-based maternal and newborn care while low intervention coverage may contribute to false negative reporting (lower sensitivity). While replication in other country and facility settings is warranted, results suggest that monitoring efforts should consider the context of care when interpreting national estimates of intervention coverage.

New approaches for developing biomarkers of hormonal contraceptive use.

To identify biomarkers of hormonal contraceptive (HC) use in urine and saliva, we conducted a pilot study with 30 women initiating levonorgestrel (LNG) containing combined oral contraceptives (COCs) or depot medroxyprogesterone acetate (DMPA) (15/group). Based on established COC pharmacokinetics, we collected serum and urine samples before COC ingestion and during Days one and three of use, or before DMPA injection and on Days 21 and 60 post-injection. We used liquid chromatography-tandem mass spectrometry (LC-MS/MS) to measure serum/urine LNG and MPA. LNG was undetectable at baseline (specificity 100%); post ingestion, most urine samples had detectable LNG levels (sensitivity: 80% 6 h post Dose one, 93% 6 h post Dose three). We used a DetectX LNG immunoassay kit and showed 100% sensitivity measuring urine LNG. Urine MPA levels were undetectable in 14/15 women at baseline (specificity 91%); post-injection all urine samples had detectable MPA levels (sensitivity: 100% days 21 and 60). Results suggest urine sampling can be used to identify a biomarker of LNG and MPA use. Based on evidence from other steroidal hormonal studies showing changes affecting the transcriptome profile of saliva at 24 h, we used the same (COC, DMPA) timepoints to collect saliva. We performed transcriptome analysis and detected several differentially expressed genes in DMPA users' saliva on Days 21 and 60 compared to baseline; none among COC users. We plan further research of differential gene expression in saliva as a HC biomarker of DMPA use, and will explore longer periods of COC use and saliva collection times, and application of microRNA sequencing to support using saliva as a COC biomarker.

The Validity of Women's Reports of Family Planning Service Quality in Cambodia and Kenya.

Population-based indicators of the coverage of key elements of high-quality family planning services are tracked via household surveys with female respondents, yet little work has been done to establish their validity. We take advantage of existing data sets from Cambodia and Kenya to compare women's responses at exit interviews following a health facility visit against the observations of a trained third-party observer during the visit. The results, which treat the observations as the reference standard, show that indicators that measure contraceptive methods received are accurately reported while indicators of whether the woman received her preferred method and whether information was "discussed" or "explained" during counseling are less reliably reported. Studies designed explicitly to assess the validity of family planning questions in household surveys, especially questions in large survey programs critical for monitoring demographic trends and programmatic coverage, are needed.

The links between women's employment and children at home: Variations in low- and middle-income countries by world region.

As fertility declines in low- and middle-income countries, the time women devote to childbearing and rearing may also be reduced. This shift has been described as one of the positive consequences of the demographic transition, as it opens opportunities for women to pursue educational and employment opportunities that were previously constrained by the demands of bearing and raising children. We estimate the numbers of children residing at home (with their mother) for women in 58 countries in Asia, Latin America, the Middle East and North Africa, and sub-Saharan Africa. We then examine the association between women's employment and having children at home. Finally, we assess trends over recent decades in the relationship between employment and childbearing, and differences in this relationship by mother's occupation. We find a negative association between women's employment and having children at home; this association varies substantially by world region, age of child, and mother's occupation.

Validation studies for population-based intervention coverage indicators: design, analysis, and interpretation.

BACKGROUND: Population-based intervention coverage indicators are widely used to track country and program progress in improving health and to evaluate health programs. Indicator validation studies that compare survey responses to a "gold standard" measure are useful to understand whether the indicator provides accurate information. The Improving Coverage Measurement (ICM) Core Group has developed and implemented a standard approach to validating coverage indicators measured in household surveys, described in this paper. METHODS: The general design of these studies includes measurement of true health status and intervention receipt (gold standard), followed by interviews with the individuals observed, and a comparison of the observations (gold standard) to the responses to survey questions. The gold standard should use a data source external to the respondent to document need for and receipt of an intervention. Most frequently, this is accomplished through direct observation of clinical care, and/or use of a study-trained clinician to obtain a gold standard diagnosis. Follow-up interviews with respondents should employ standard survey questions, where they exist, as well as alternative or additional questions that can be compared against the standard household survey questions. RESULTS: Indicator validation studies should report on participation at every stage, and provide data on reasons for non-participation. Metrics of individual validity (sensitivity, specificity, area under the receiver operating characteristic curve) and population-level validity (inflation factor) should be reported, as well as the percent of survey responses that are "don't know" or missing. Associations between interviewer and participant characteristics and measures of validity should be assessed and reported. CONCLUSIONS: These methods allow respondent-reported coverage measures to be validated against more objective measures of need for and receipt of an intervention, and should be considered together with cognitive interviewing, discriminative validity, or reliability testing to inform decisions about which indicators to include in household surveys. Public health researchers should assess the evidence for validity of existing and proposed household survey coverage indicators and consider validation studies to fill evidence gaps.

Women's recall of maternal and newborn interventions received in the postnatal period: a validity study in Kenya and Swaziland.

BACKGROUND: Despite the concentration of maternal and infant deaths in the early postnatal period, information on the content and quality of postnatal care interventions is not routinely collected in most low and middle-income countries. At present, data on the coverage of postnatal care interventions mostly rely on women's reports collected in household surveys, such as the Demographic and Health Surveys (DHS) and Multiple Indicator Cluster Surveys (MICS), which collect limited information. We assessed the validity of a set of postnatal care indicators that reflect a range of recommended interventions for both mother and newborn and have potential to be included in household surveys for monitoring of population-level coverage. METHODS: We compared women's reports in exit interviews on the content of postnatal care received in health facilities located in Kenya and Swaziland against a gold standard of direct observation by a trained third party. We calculated sensitivity, specificity and the area under the receiver operating curve (AUC) to assess individual-level reporting accuracy and the inflation factor (IF) to assess population-level accuracy. We also examined whether women's reporting accuracy varied significantly by her sociodemographic characteristics. RESULTS: 18 indicators in Kenya and 19 in Swaziland had sufficient sample size for analysis. Of these, 12 indicators in Kenya and five in Swaziland met criteria for acceptable individual and population-level reporting accuracy. Two indicators met acceptability criteria in both Kenya and Swaziland: whether the provider performed a breast exam or an abdominal exam. There was no significant association between women's characteristics and reporting accuracy, across indicators. CONCLUSION: Women are able to accurately report on multiple aspects of care received during a postnatal visit. Findings inform the recommendation of indicators for tracking progress of critical postnatal care interventions for mothers and newborns. Improved measurement of the coverage of maternal and newborn postnatal care is warranted to monitor progress in maternal and newborn care globally.

Trends in the age at reproductive transitions in the developing world: The role of education.

Girls' school participation has expanded considerably in the developing world over the last few decades, a phenomenon expected to have substantial consequences for reproductive behaviour. Using Demographic and Health Survey data from 43 countries, this paper examines trends and differentials in the mean ages at three critical life-cycle events for young women: first sexual intercourse, first marriage, and first birth. We measure the extent to which trends in the timing of these events are driven either by the changing educational composition of populations or by changes in behaviour within education groups. Mean ages have risen over time in all regions for all three events, except age at first sex in Latin America and the Caribbean. Results from a decomposition exercise indicate that increases in educational attainment, rather than trends within education groups, are primarily responsible for the overall trends. Possible explanations for these findings are discussed.

Can surveys of women accurately track indicators of maternal and newborn care? A validity and reliability study in Kenya.

BACKGROUND: Tracking progress on maternal and newborn survival requires accurate information on the coverage of essential interventions. Despite widespread use, most indicators measuring maternal and newborn intervention coverage have not been validated. This study assessed the ability of women delivering in two Kenyan hospitals to recall critical elements of care received during the intrapartum and immediate postnatal period at two time points: hospital discharge and 13-15 months following delivery. METHODS: Women's reports of received care were compared against observations by trained third party observers. Indicators selected for validation were either currently in use or have the potential to be included in population-based surveys. We used a mixed-methods approach to validate women's reporting ability. We calculated individual-reporting accuracy using the area under the receiver operating curve (AUC), population-level accuracy using the inflation factor (IF), and compared the accuracy of women's reporting at baseline and follow-up. We also assessed the consistency of women's reporting over time. We used in-depth interviews with a sub-set of women (n = 20) to assess their understanding of key survey terms. RESULTS: Of 606 women who participated at baseline and agreed to follow-up, 515 were re-interviewed. Thirty-eight indicators had sufficient sample size for validation analysis; ten met criteria for high or moderate reporting accuracy (0.60

Measuring progress in maternal and newborn health care in Mexico: validating indicators of health system contact and quality of care.

BACKGROUND: The majority of births in Mexico take place in a health facility and are attended by a skilled birth attendant, yet maternal mortality has not declined to anticipated levels. Coverage estimates of skilled attendance and other maternal and newborn interventions often rely on women's self-report through a population-based survey, the accuracy of which is not well established. METHODS: We used a facility-based design to validate women's report of skilled birth attendance, as well as other key elements of maternal, newborn intrapartum, and immediate postnatal care. Women's reports of labor and delivery care were collected by exit interview prior to hospital discharge and were compared against direct observation by a trained third party in a Mexican public hospital (n = 597). For each indicator, validity was assessed at the individual level using the area under the receiver operating curve (AUC) and at the population level using the inflation factor (IF). RESULTS: Five of 47 indicators met both validation criteria (AUC > 0.60 and 0.75 < IF < 1.25): urine sample screen, injection or IV medication received during labor, before the birth of the baby (i.e., uterotonic for either induction or augmentation of labor), episiotomy, excessive bleeding, and receipt of blood products. An additional 9 indicators met criteria for the AUC and 18 met criteria for the IF. A skilled attendant indicator had high sensitivity (90.1 %: 95 % CI: 87.1-92.5 %), low specificity (14.0 %: 95 % CI: 5.8-26.7 %) and was suitable for population-level estimation only. CONCLUSION: Women are able to give valid reports on some aspects of the content of care, although questions regarding the indication for interventions are less likely to be known. Questions that include technical terms or refer to specific time periods tended to have lower response levels. A key aspect of efforts to improve maternal and newborn health requires valid measurement of women's access to maternal and newborn health interventions and the quality of such services. Additional work on improving measurement of population coverage indicators is warranted.

Assessing the validity of indicators of the quality of maternal and newborn health care in Kenya.

BACKGROUND: The measurement of progress in maternal and newborn health often relies on data provided by women in surveys on the quality of care they received. The majority of these indicators, however, including the widely tracked "skilled attendance at birth" indicator, have not been validated. We assess the validity of a large set of maternal and newborn health indicators that are included or have the potential to be included in population-based surveys. METHODS: We compare women's reports of care received during labor and delivery in two Kenyan hospitals prior to discharge against a reference standard of direct observations by a trained third party (n = 662). We assessed individual-level reporting accuracy by quantifying the area under the receiver operating curve (AUC) and estimated population-level accuracy using the inflation factor (IF) for each indicator with sufficient numbers for analysis. FINDINGS: Four of 41 indicators performed well on both validation criteria (AUC>0.70 and 0.75

Myths and Misinformation: An Analysis of Text Messages Sent to a Sexual and Reproductive Health Q&A Service in Nigeria.

The almost 50 million young people aged 10-24 in Nigeria face many challenges to their sexual and reproductive health (SRH). MyQuestion is a platform that allows young people to ask SRH questions via text message. Trained counselors provide responses using a database of answers to frequently asked questions or customized replies. We analyze the content of more than 300,000 text messages received by the service since 2007 to address three questions: which health topics are most frequently submitted to the MyQuestion service; what kinds of questions are asked about these topics; and what language is used to convey the questions? We find a substantial unmet need for basic SRH information, with users' questions communicated in ways that convey considerable confusion, misinformation, and urgency. The analysis can be used to improve similar Q&A services and to improve the provision of SRH services for young people more generally.

Estimating the current mean age of mothers at the birth of their first child from household surveys.

BACKGROUND: Estimates of the period mean age at first birth are readily available for countries with accurate vital statistics (i.e., in much of the developed world). In contrast, in most developing countries vital statistics are lacking or incomplete and estimates of the period mean age at first birth are therefore often unavailable. The Demographic and Health Surveys (DHS) program provides a large set of demographic and health statistics for many developing countries, but not the mean age at childbearing or the mean age at first birth. METHODS: We propose two different methods for the estimation of the period mean age at first birth from information collected in DHS surveys. The first method is the same as the one used in populations with accurate vital statistics and is based on a weighted average of single year of age first birth rates. The second is the singulate mean age at first birth. RESULTS: A comparison of the two estimates obtained from the latest surveys in 62 countries shows excellent agreement in countries in which there is no evidence of a rise in childlessness. But, as expected on theoretical grounds, there is less agreement in populations that have experienced an increase in the proportion childless. CONCLUSIONS: Based on these results, we recommend the first method. The measure is relatively straightforward to calculate and, since it refers to recent births, is presumably more accurately reported than indicators based on events that occurred in the more distant past. This measure makes it possible for the first time to assess recent trends in the onset of childbearing in developing countries with multiple DHS surveys and to compare recent period estimates of the mean age at first birth among countries.

A multi-country study of the "intrapartum stillbirth and early neonatal death indicator" in hospitals in low-resource settings.

OBJECTIVE: To determine the feasibility of introducing a simple indicator of quality of obstetric and neonatal care and to determine the proportion of potentially avoidable perinatal deaths in hospitals in low-income countries. METHODS: Between September 1, 2011, and February 29, 2012, data were collected from consecutive women who were admitted to the labor ward of 1 of 6 hospitals in 4 low-income countries. Fetal heart tones on admission were monitored, and demographic and birth data were recorded. RESULTS: Data were obtained for 3555 women and 3593 neonates (including twins). The doptone was used on 97% of women admitted. The overall perinatal mortality rate was 34 deaths per 1000 deliveries. Of the perinatal deaths, 40%-45% occurred in the hospital and were potentially preventable by better hospital care. CONCLUSION: The results demonstrated that it is possible to accurately determine fetal viability on admission via a doptone. Implementation of doptone use, coupled with a concise data record, might form the basis of a low-cost and sustainable program to monitor and evaluate efforts to improve quality of care and ultimately might help to reduce the in-hospital component of perinatal mortality in low-income countries.

Measuring coverage in MNCH: new findings, new strategies, and recommendations for action.

Considerable progress has been made in reducing maternal, newborn, and child mortality worldwide, but many more deaths could be prevented if effective interventions were available to all who could benefit from them. Timely, high-quality measurements of intervention coverage--the proportion of a population in need of a health intervention that actually receives it--are essential to support sound decisions about progress and investments in women's and children's health. The PLOS Medicine "Measuring Coverage in MNCH" Collection of research studies and reviews presents systematic assessments of the validity of health intervention coverage measurement based on household surveys, the primary method for estimating population-level intervention coverage in low- and middle-income countries. In this overview of the Collection, we discuss how and why some of the indicators now being used to track intervention coverage may not provide fully reliable coverage measurements, and how a better understanding of the systematic and random error inherent in these coverage indicators can help in their interpretation and use. We draw together strategies proposed across the Collection for improving coverage measurement, and recommend continued support for high-quality household surveys at national and sub-national levels, supplemented by surveys with lighter tools that can be implemented every 1-2 years and by complementary health-facility-based assessments of service quality. Finally, we stress the importance of learning more about coverage measurement to strengthen the foundation for assessing and improving the progress of maternal, newborn, and child health programs.

Measuring coverage in MNCH: validating women's self-report of emergency cesarean sections in Ghana and the Dominican Republic.

BACKGROUND: Cesarean section is the only surgery for which we have nearly global population-based data. However, few surveys provide additional data related to cesarean sections. Given weaknesses in many health information systems, health planners in developing countries will likely rely on nationally representative surveys for the foreseeable future. The objective is to validate self-reported data on the emergency status of cesarean sections among women delivering in teaching hospitals in the capitals of two contrasting countries: Accra, Ghana and Santo Domingo, Dominican Republic (DR). METHODS AND FINDINGS: This study compares hospital-based data, considered the reference standard, against women's self-report for two definitions of emergency cesarean section based on the timing of the decision to operate and the timing of the cesarean section relative to onset of labor. Hospital data were abstracted from individual medical records, and hospital discharge interviews were conducted with women who had undergone cesarean section in two hospitals. The study assessed sensitivity, specificity, and positive predictive value of responses to questions regarding emergency versus non-emergency cesarean section and estimated the percent of emergency cesarean sections that would be obtained from a survey, given the observed prevalence, sensitivity, and specificity from this study. Hospital data were matched with exit interviews for 659 women delivered via cesarean section for Ghana and 1,531 for the Dominican Republic. In Ghana and the Dominican Republic, sensitivity and specificity for emergency cesarean section defined by decision time were 79% and 82%, and 50% and 80%, respectively. The validity of emergency cesarean defined by operation time showed less favorable results than decision time in Ghana and slightly more favorable results in the Dominican Republic. CONCLUSIONS: Questions used in this study to identify emergency cesarean section are promising but insufficient to promote for inclusion in international survey questionnaires. Additional studies which confirm the accuracy of key facility-based indicators in advance of data collection and which use a longer recall period are warranted.

New findings for maternal mortality age patterns: aggregated results for 38 countries.

BACKGROUND: With recent results showing a global decline in overall maternal mortality during the last two decades and with the target date for achieving the Millennium Development Goals only four years away, the question of how to continue or even accelerate the decline has become more pressing. By knowing where the risk is highest as well as where the numbers of deaths are greatest, it may be possible to re-direct resources and fine-tune strategies for greater effectiveness in efforts to reduce maternal mortality. METHODS: We aggregate data from 38 Demographic and Health Surveys that included a maternal mortality module and were conducted in 2000 or later to produce maternal mortality ratios, rates, and numbers of deaths by five year age groups, separately by residence, region, and overall mortality level. FINDINGS: The age pattern of maternal mortality is broadly similar across regions, type of place of residence, and overall level of maternal mortality. A "J" shaped curve, with markedly higher risk after age 30, is evident in all groups. We find that the excess risk among adolescents is of a much lower magnitude than is generally assumed. The oldest age groups appear to be especially resistant to change. We also find evidence of extremely elevated risk among older mothers in countries with high levels of HIV prevalence. CONCLUSIONS: The largest number of deaths occurs in the age groups from 20-34, largely because those are the ages at which women are most likely to give birth so efforts directed at this group would most effectively reduce the number of deaths. Yet equity considerations suggest that efforts also be directed toward those most at risk, i.e., older women and adolescents. Because women are at risk each time they become pregnant, fulfilling the substantial unmet need for contraception is a cross-cutting strategy that can address both effectiveness and equity concerns.