Improving methods to measure comparable mortality cause (IMMCMC) gold standard verbal autopsy dataset.

OBJECTIVES: Gold standard cause of death data is critically important to improve verbal autopsy (VA) methods in diagnosing cause of death where civil and vital registration systems are inadequate or poor. As part of a three-country research study-Improving Methods to Measure Comparable Mortality by Cause (IMMCMC) study-data were collected on clinicopathological criteria-based gold standard cause of death from hospital record reviews with matched VAs. The purpose of this data note is to make accessible a de-identified format of these gold standard VAs for interested researchers to improve the diagnostic accuracy of VA methods. DATA DESCRIPTION: The study was conducted between 2011 and 2014 in the Philippines, Bangladesh, and Papua New Guinea. Gold standard diagnoses of underlying causes of death for deaths occurring in hospital were matched to VAs conducted using a standardized VA questionnaire developed by the Population Health Metrics Consortium. 3512 deaths were collected in total, comprised of 2491 adults (12 years and older), 320 children (28 days to 12 years), and 702 neonates (0-27 days).

Robustness of the Tariff method for diagnosing verbal autopsies: impact of additional site data on the relationship between symptom and cause.

BACKGROUND: Verbal autopsy (VA) is increasingly being considered as a cost-effective method to improve cause of death information in countries with low quality vital registration. VA algorithms that use empirical data have an advantage over expert derived algorithms in that they use responses to the VA instrument as a reference instead of physician opinion. It is unclear how stable these data driven algorithms, such as the Tariff 2.0 method, are to cultural and epidemiological variations in populations where they might be employed. METHODS: VAs were conducted in three sites as part of the Improving Methods to Measure Comparable Mortality by Cause (IMMCMC) study: Bohol, Philippines; Chandpur and Comila Districts, Bangladesh; and Central and Eastern Highlands Provinces, Papua New Guinea. Similar diagnostic criteria and cause lists as the Population Health Metrics Research Consortium (PHMRC) study were used to identify gold standard (GS) deaths. We assessed changes in Tariffs by examining the proportion of Tariffs that changed significantly after the addition of the IMMCMC dataset to the PHMRC dataset. RESULTS: The IMMCMC study added 3512 deaths to the GS VA database (2491 adults, 320 children, and 701 neonates). Chance-corrected cause specific mortality fractions for Tariff improved with the addition of the IMMCMC dataset for adults (+ 5.0%), children (+ 5.8%), and neonates (+ 1.5%). 97.2% of Tariffs did not change significantly after the addition of the IMMCMC dataset. CONCLUSIONS: Tariffs generally remained consistent after adding the IMMCMC dataset. Population level performance of the Tariff method for diagnosing VAs improved marginally for all age groups in the combined dataset. These findings suggest that cause-symptom relationships of Tariff 2.0 might well be robust across different population settings in developing countries. Increasing the total number of GS deaths improves the validity of Tariff and provides a foundation for the validation of other empirical algorithms.

Improving coverage of civil registration and vital statistics, Bangladesh.

PROBLEM: Bangladesh has no national system for registering deaths and determining their causes. As a result, policy-makers lack reliable and complete data to inform public health decisions. APPROACH: In 2016, the government of Bangladesh introduced a pilot project to strengthen the civil registration and vital statistics system and generate cause of death data in Kaliganj Upazila. Community-based health workers were trained to notify births and deaths to the civil registrar, and to conduct verbal autopsy interviews with family members of a deceased person. International experts in cause-of-death certification and coding trained master trainers on how to complete the international medical certificate of cause of death. These trainers then trained physicians and coders. LOCAL SETTING: Kaliganj Upazila has an estimated population of 304 600, and 5600 births and 1550 deaths annually. Health assistants and family welfare assistants make regular visits to households to track certain health outcomes. RELEVANT CHANGES: Following the start of the project in 2016, the number of births registered within 45 days rose from 873 to 4630 in 2018. The number of deaths registered within 45 days increased from 458 to 1404. During this period, health assistants conducted 7837 verbal autopsy interviews. Between January 2017 and December 2018, 105 master trainers and more than 7000 physicians were trained to complete the international medical certificate of cause of death and they completed more than 12 000 certificates. LESSONS LEARNT: Training community-based health workers, physicians and coders were successful approaches to improve death registration completeness and availability of cause-of-death data.

On the estimation of population cause-specific mortality fractions from in-hospital deaths.

BACKGROUND: Almost all countries without complete vital registration systems have data on deaths collected by hospitals. However, these data have not been widely used to estimate cause of death (COD) patterns in populations because only a non-representative fraction of people in these countries die in health facilities. Methods that can exploit hospital mortality statistics to reliably estimate community COD patterns are required to strengthen the evidence base for disease and injury control programs. We propose a method that weights hospital-certified causes by the probability of death to estimate population cause-specific mortality fractions (CSMFs). METHODS: We used an established verbal autopsy instrument (VAI) to collect data from hospital catchment areas in Chandpur and Comilla Districts, Bangladesh, and Bohol province, the Philippines, between 2011 and 2014, along with demographic covariates for each death. Hospital medical certificates of cause of death (death certificates) were collected and mapped to the corresponding cause categories of the VAI. Tariff 2.0 was used to assign a COD for community deaths. Logistic regression models were created for broad causes in each country to calculate the probability of in-hospital death, given a set of covariate values. The reweighted CSMFs for deaths in the hospital catchment population, represented by each hospital death, were calculated from the corresponding regression models. RESULTS: We collected data on 4228 adult deaths in the Philippines and 3725 deaths in Bangladesh. Short time to hospital and education were consistently associated with in-hospital death in the Philippines and absence of a disability was consistently associated with in-hospital death in Bangladesh. Non-communicable diseases (excluding stroke) and stroke were the leading causes of death in both the Philippines (33.9%, 19.1%) and Bangladesh (46.1%, 21.1%) according to the reweighted method. The reweighted method generally estimated CSMFs that fell between those derived from hospitals and those diagnosed by Tariff 2.0. CONCLUSIONS: Statistical methods can be used to derive estimates of cause-specific probability of death in-hospital for Bangladesh and the Philippines to generate population CSMFs. In regions where hospital death certification is of reasonable quality and routine verbal autopsy is not applied, these estimates could be applied to generate cost-effective and robust CSMFs for the population.

Assessing the quality of medical death certification: a case study of concordance between national statistics and results from a medical record review in a regional hospital in the Philippines.

BACKGROUND: Medical certificates of cause of death (MCCOD) issued by hospital physicians are a key input to vital registration systems. Deaths certified by hospital physicians have been implicitly considered to be of high quality, but recent evidence suggests otherwise. We conducted a medical record review (MRR) of hospital MCCOD in the Philippines and compared the cause of death concordance with certificates coded by the Philippines Statistics Authority (PSA). METHODS: MCCOD for adult deaths in Bohol Regional Hospital (BRH) in 2007-2008 and 2011 were collected and reviewed by a team of study physicians. Corresponding MCCOD coded by the PSA were linked by a hospital identifier. The study physicians wrote a new MCCOD using the patient medical record, noted the quality of the medical record to produce a cause of death, and indicated whether it was necessary to change the underlying cause of death (UCOD). Chance-corrected concordance, cause-specific mortality fraction (CSMF) accuracy, and chance-corrected CSMF were used to examine the concordance between the MRR and PSA. RESULTS: A total of 1052 adult deaths were linked between the MRR and PSA. Median chance-corrected concordance was 0.73, CSMF accuracy was 0.85, and chance-corrected CSMF accuracy was 0.58. 74.8% of medical records were deemed to be of high enough quality to assign a cause of death, yet study physicians indicated that it was necessary to change the UCOD in 41% of deaths, 82% of which required addition of a new UCOD. CONCLUSIONS: Medical records were generally of sufficient quality to assign a cause of death and concordance between the PSA and MRR was reasonably high, suggesting that routine mortality statistics data are reasonably accurate for describing population level causes of death in Bohol. While overall agreement between the PSA and MRR in major cause groups was sufficient for public health purposes, improvements in death certification practices are recommended to help physicians differentiate between treatable (immediate) COD and COD that are important for public health surveillance.

Comparing tariff and medical assistant assigned causes of death from verbal autopsy interviews in Matlab, Bangladesh: implications for a health and demographic surveillance system.

BACKGROUND: Deaths in developing countries often occur outside health facilities, making it extremely difficult to gather reliable cause of death (COD) information. Automated COD assignment using a verbal autopsy instrument (VAI) has been proposed as a reliable and cost-effective alternative to traditional physician-certified verbal autopsy, but its performance is still being evaluated. The purpose of this study was to compare the similarity of diagnosis by Medical Assistants (MA) in the Matlab Health and Demographic Surveillance System (HDSS) with the SmartVA Analyze 1.2 (Tariff 2.0) diagnosis. METHODS: This study took place between January 2011 and April 2014 in Matlab, Bangladesh. MA with 3 years of medical training assigned COD to Matlab residents by reviewing the information collected using the Population Health Metrics Research Consortium (PHMRC) long-form VAI. Smart VA Analyze 1.2 automatically assigned COD using the same questionnaire. COD agreement and cause-specific mortality fractions (CSMFs) were compared for MA and Tariff. RESULTS: Of the 4969 verbal autopsy cases reviewed, 4328 were adults, 296 were children, and 345 were neonates. Cohen's kappa was 0.38 (0.36, 0.40) for adults, 0.43 (0.38, 0.49) for children, and 0.27 (0.22, 0.33) for neonates. For adults, the top two COD for MA were stroke (29.6%) and ischemic heart diseases (IHD) (14.2%) and for Tariff these were stroke (32.0%) and IHD (14.0%). For children, the top two COD for MA were drowning (33.5%) and pneumonia (13.2%) and for Tariff these were also drowning (36.8%) and pneumonia (12.4%). For neonates, the top two COD for MA were birth asphyxia (41.2%) and meningitis/sepsis (22.3%) and for Tariff these were birth asphyxia (37.0%) and preterm delivery (30.9%). CONCLUSION: The CSMFs for Tariff and MA showed very close agreement across all age categories but some differences were observed for neonate preterm delivery and meningitis/sepsis. Given the known advantages of automated methods over physician certified verbal autopsy, the SmartVA software, incorporating the shortened VAI questionnaire and Tariff 2.0, could serve as a cost-effective alternative to Matlab MA to routinely collect and analyze verbal autopsy data in a HDSS to generate essential population level COD data for planning.

Performance of InSilicoVA for assigning causes of death to verbal autopsies: multisite validation study using clinical diagnostic gold standards.

BACKGROUND: Recently, a new algorithm for automatic computer certification of verbal autopsy data named InSilicoVA was published. The authors presented their algorithm as a statistical method and assessed its performance using a single set of model predictors and one age group. METHODS: We perform a standard procedure for analyzing the predictive accuracy of verbal autopsy classification methods using the same data and the publicly available implementation of the algorithm released by the authors. We extend the original analysis to include children and neonates, instead of only adults, and test accuracy using different sets of predictors, including the set used in the original paper and a set that matches the released software. RESULTS: The population-level performance (i.e., predictive accuracy) of the algorithm varied from 2.1 to 37.6% when trained on data preprocessed similarly as in the original study. When trained on data that matched the software default format, the performance ranged from -11.5 to 17.5%. When using the default training data provided, the performance ranged from -59.4 to -38.5%. Overall, the InSilicoVA predictive accuracy was found to be 11.6-8.2 percentage points lower than that of an alternative algorithm. Additionally, the sensitivity for InSilicoVA was consistently lower than that for an alternative diagnostic algorithm (Tariff 2.0), although the specificity was comparable. CONCLUSIONS: The default format and training data provided by the software lead to results that are at best suboptimal, with poor cause-of-death predictive performance. This method is likely to generate erroneous cause of death predictions and, even if properly configured, is not as accurate as alternative automated diagnostic methods.

Collecting verbal autopsies: improving and streamlining data collection processes using electronic tablets.

BACKGROUND: There is increasing interest in using verbal autopsy to produce nationally representative population-level estimates of causes of death. However, the burden of processing a large quantity of surveys collected with paper and pencil has been a barrier to scaling up verbal autopsy surveillance. Direct electronic data capture has been used in other large-scale surveys and can be used in verbal autopsy as well, to reduce time and cost of going from collected data to actionable information. METHODS: We collected verbal autopsy interviews using paper and pencil and using electronic tablets at two sites, and measured the cost and time required to process the surveys for analysis. From these cost and time data, we extrapolated costs associated with conducting large-scale surveillance with verbal autopsy. RESULTS: We found that the median time between data collection and data entry for surveys collected on paper and pencil was approximately 3 months. For surveys collected on electronic tablets, this was less than 2 days. For small-scale surveys, we found that the upfront costs of purchasing electronic tablets was the primary cost and resulted in a higher total cost. For large-scale surveys, the costs associated with data entry exceeded the cost of the tablets, so electronic data capture provides both a quicker and cheaper method of data collection. CONCLUSIONS: As countries increase verbal autopsy surveillance, it is important to consider the best way to design sustainable systems for data collection. Electronic data capture has the potential to greatly reduce the time and costs associated with data collection. For long-term, large-scale surveillance required by national vital statistical systems, electronic data capture reduces costs and allows data to be available sooner.

The quality of medical death certification of cause of death in hospitals in rural Bangladesh: impact of introducing the International Form of Medical Certificate of Cause of Death.

BACKGROUND: Accurate and timely data on cause of death are critically important for guiding health programs and policies. Deaths certified by doctors are implicitly considered to be reliable and accurate, yet the quality of information provided in the international Medical Certificate of Cause of Death (MCCD) usually varies according to the personnel involved in certification, the diagnostic capacity of the hospital, and the category of hospitals. There are no published studies that have analysed how certifying doctors in Bangladesh adhere to international rules when completing the MCCD or have assessed the quality of clinical record keeping. METHODS: The study took place between January 2011 and April 2014 in the Chandpur and Comilla districts of Bangladesh. We introduced the international MCCD to all study hospitals. Trained project physicians assigned an underlying cause of death, assessed the quality of the death certificate, and reported the degree of certainty of the medical records provided for a given cause. We examined the frequency of common errors in completing the MCCD, the leading causes of in-hospital deaths, and the degree of certainty in the cause of death data. RESULTS: The study included 4914 death certificates. 72.9% of medical records were of too poor quality to assign a cause of death, with little difference by age, hospital, and cause of death. 95.6% of death certificates did not indicate the time interval between onset and death, 31.6% required a change in sequence, 13.9% required to include a new diagnosis, 50.7% used abbreviations, 41.5% used multiple causes per line, and 33.2% used an ill-defined condition as the underlying cause of death. 99.1% of death certificates had at least one error. The leading cause of death among adults was stroke (15.8%), among children was pneumonia (31.7%), and among neonates was birth asphyxia (52.8%). CONCLUSION: Physicians in Bangladeshi hospitals had difficulties in completing the MCCD correctly. Physicians routinely made errors in death certification practices and medical record quality was poor. There is an urgent need to improve death certification practices and the quality of hospital data in Bangladesh if these data are to be useful for policy.

Implementing the PHMRC shortened questionnaire: Survey duration of open and closed questions in three sites.

BACKGROUND: More countries are using verbal autopsy as a part of routine mortality surveillance. The length of time required to complete a verbal autopsy interview is a key logistical consideration for planning large-scale surveillance. METHODS: We use the PHMRC shortened questionnaire to conduct verbal autopsy interviews at three sites and collect data on the length of time required to complete the interview. This instrument uses a novel checklist of keywords to capture relevant information from the open response. The open response section is timed separately from the section consisting of closed questions. RESULTS: We found the median time to complete the entire interview was approximately 25 minutes and did not vary substantially by age-specific module. The median time for the open response section was approximately 4 minutes and 60% of interviewees mentioned at least one keyword within the open response section. CONCLUSIONS: The length of time required to complete the interview was short enough for large-scale routine use. The open-response section did not add a substantial amount of time and provided useful information which can be used to increase the accuracy of the predictions of the cause of death. The novel checklist approach further reduces the burden of transcribing and translating a large amount of free text. This makes the PHMRC instrument ideal for national mortality surveillance.