Paradata analyses to inform population-based survey capture of pregnancy outcomes: EN-INDEPTH study.

BACKGROUND: Paradata are (timestamped) records tracking the process of (electronic) data collection. We analysed paradata from a large household survey of questions capturing pregnancy outcomes to assess performance (timing and correction processes). We examined how paradata can be used to inform and improve questionnaire design and survey implementation in nationally representative household surveys, the major source for maternal and newborn health data worldwide. METHODS: The EN-INDEPTH cross-sectional population-based survey of women of reproductive age in five Health and Demographic Surveillance System sites (in Bangladesh, Guinea-Bissau, Ethiopia, Ghana, and Uganda) randomly compared two modules to capture pregnancy outcomes: full pregnancy history (FPH) and the standard DHS-7 full birth history (FBH+). We used paradata related to answers recorded on tablets using the Survey Solutions platform. We evaluated the difference in paradata entries between the two reproductive modules and assessed which question characteristics (type, nature, structure) affect answer correction rates, using regression analyses. We also proposed and tested a new classification of answer correction types. RESULTS: We analysed 3.6 million timestamped entries from 65,768 interviews. 83.7% of all interviews had at least one corrected answer to a question. Of 3.3 million analysed questions, 7.5% had at least one correction. Among corrected questions, the median number of corrections was one, regardless of question characteristics. We classified answer corrections into eight types (no correction, impulsive, flat (simple), zigzag, flat zigzag, missing after correction, missing after flat (zigzag) correction, missing/incomplete). 84.6% of all corrections were judged not to be problematic with a flat (simple) mistake correction. Question characteristics were important predictors of probability to make answer corrections, even after adjusting for respondent's characteristics and location, with interviewer clustering accounted as a fixed effect. Answer correction patterns and types were similar between FPH and FBH+, as well as the overall response duration. Avoiding corrections has the potential to reduce interview duration and reproductive module completion by 0.4 min. CONCLUSIONS: The use of questionnaire paradata has the potential to improve measurement and the resultant quality of electronic data. Identifying sections or specific questions with multiple corrections sheds light on typically hidden challenges in the survey's content, process, and administration, allowing for earlier real-time intervention (e.g.,, questionnaire content revision or additional staff training). Given the size and complexity of paradata, additional time, data management, and programming skills are required to realise its potential.

Electronic data collection in a multi-site population-based survey: EN-INDEPTH study.

BACKGROUND: Electronic data collection is increasingly used for household surveys, but factors influencing design and implementation have not been widely studied. The Every Newborn-INDEPTH (EN-INDEPTH) study was a multi-site survey using electronic data collection in five INDEPTH health and demographic surveillance system sites. METHODS: We described experiences and learning involved in the design and implementation of the EN-INDEPTH survey, and undertook six focus group discussions with field and research team to explore their experiences. Thematic analyses were conducted in NVivo12 using an iterative process guided by a priori themes. RESULTS: Five steps of the process of selecting, adapting and implementing electronic data collection in the EN-INDEPTH study are described. Firstly, we reviewed possible electronic data collection platforms, and selected the World Bank's Survey Solutions® as the most suited for the EN-INDEPTH study. Secondly, the survey questionnaire was coded and translated into local languages, and further context-specific adaptations were made. Thirdly, data collectors were selected and trained using standardised manual. Training varied between 4.5 and 10 days. Fourthly, instruments were piloted in the field and the questionnaires finalised. During data collection, data collectors appreciated the built-in skip patterns and error messages. Internet connection unreliability was a challenge, especially for data synchronisation. For the fifth and final step, data management and analyses, it was considered that data quality was higher and less time was spent on data cleaning. The possibility to use paradata to analyse survey timing and corrections was valued. Synchronisation and data transfer should be given special consideration. CONCLUSION: We synthesised experiences using electronic data collection in a multi-site household survey, including perceived advantages and challenges. Our recommendations for others considering electronic data collection include ensuring adaptations of tools to local context, piloting/refining the questionnaire in one site first, buying power banks to mitigate against power interruption and paying attention to issues such as GPS tracking and synchronisation, particularly in settings with poor internet connectivity.

Neonatal and child mortality data in retrospective population-based surveys compared with prospective demographic surveillance: EN-INDEPTH study.

BACKGROUND: Global mortality estimates remain heavily dependent on household surveys in low- and middle-income countries, where most under-five deaths occur. Few studies have assessed the accuracy of mortality data or determinants of capturing births in surveys. METHODS: The Every Newborn-INDEPTH study (EN-INDEPTH) included a large, multi-country survey of women aged 15-49 interviewed about livebirths and their survival status in five Health and Demographic Surveillance Systems (HDSSs). The HDSSs undertake regular household visits to register births and deaths for a given population. We analysed EN-INDEPTH survey data to assess background factors associated with not recalling a complete date-of-birth. We calculated Kaplan-Meier survival estimates for both survey and HDSS data and describe age-at-death distributions during the past 5 years for children born to the same women. We assessed the proportion of HDSS-births that could be matched on month-of-birth to survey-births and used regression models to identify factors associated with matching. RESULTS: 69,176 women interviewed in the survey reported 109,817 births and 3064 deaths in children under 5 years in the 5 years prior to the survey. In the HDSS data, the same women had 83,768 registered births and 2335 under-five deaths in the same period. A complete date-of-birth was not reported for 1-7% of survey-births. Birthdates were less likely to be complete for dead children and children born to women of higher parity or with little/no education. Distributions of reported age-at-death indicated heaping at full weeks (neonatal period) and at 12 months. Heaping was more pronounced in the survey data. Survey estimates of under-five mortality rates were similar to HDSS estimates of under-five mortality in two of five sites, higher in the survey in two sites (15%, 41%) and lower (24%) in one site. The proportion of HDSS-births matched to survey-births ranged from 51 to 89% across HDSSs and births of children who had died were less likely to be matched. CONCLUSIONS: Mortality estimates in the survey and HDSS were not markedly different for most sites. However, neither source is a "gold standard" and both sources miss some events. Research is required to improve capture and accuracy to better track newborn and child survival targets.