Robustness of the Tariff method for diagnosing verbal autopsies: impact of additional site data on the relationship between symptom and cause.

BACKGROUND: Verbal autopsy (VA) is increasingly being considered as a cost-effective method to improve cause of death information in countries with low quality vital registration. VA algorithms that use empirical data have an advantage over expert derived algorithms in that they use responses to the VA instrument as a reference instead of physician opinion. It is unclear how stable these data driven algorithms, such as the Tariff 2.0 method, are to cultural and epidemiological variations in populations where they might be employed. METHODS: VAs were conducted in three sites as part of the Improving Methods to Measure Comparable Mortality by Cause (IMMCMC) study: Bohol, Philippines; Chandpur and Comila Districts, Bangladesh; and Central and Eastern Highlands Provinces, Papua New Guinea. Similar diagnostic criteria and cause lists as the Population Health Metrics Research Consortium (PHMRC) study were used to identify gold standard (GS) deaths. We assessed changes in Tariffs by examining the proportion of Tariffs that changed significantly after the addition of the IMMCMC dataset to the PHMRC dataset. RESULTS: The IMMCMC study added 3512 deaths to the GS VA database (2491 adults, 320 children, and 701 neonates). Chance-corrected cause specific mortality fractions for Tariff improved with the addition of the IMMCMC dataset for adults (+ 5.0%), children (+ 5.8%), and neonates (+ 1.5%). 97.2% of Tariffs did not change significantly after the addition of the IMMCMC dataset. CONCLUSIONS: Tariffs generally remained consistent after adding the IMMCMC dataset. Population level performance of the Tariff method for diagnosing VAs improved marginally for all age groups in the combined dataset. These findings suggest that cause-symptom relationships of Tariff 2.0 might well be robust across different population settings in developing countries. Increasing the total number of GS deaths improves the validity of Tariff and provides a foundation for the validation of other empirical algorithms.

Performance of InSilicoVA for assigning causes of death to verbal autopsies: multisite validation study using clinical diagnostic gold standards.

BACKGROUND: Recently, a new algorithm for automatic computer certification of verbal autopsy data named InSilicoVA was published. The authors presented their algorithm as a statistical method and assessed its performance using a single set of model predictors and one age group. METHODS: We perform a standard procedure for analyzing the predictive accuracy of verbal autopsy classification methods using the same data and the publicly available implementation of the algorithm released by the authors. We extend the original analysis to include children and neonates, instead of only adults, and test accuracy using different sets of predictors, including the set used in the original paper and a set that matches the released software. RESULTS: The population-level performance (i.e., predictive accuracy) of the algorithm varied from 2.1 to 37.6% when trained on data preprocessed similarly as in the original study. When trained on data that matched the software default format, the performance ranged from -11.5 to 17.5%. When using the default training data provided, the performance ranged from -59.4 to -38.5%. Overall, the InSilicoVA predictive accuracy was found to be 11.6-8.2 percentage points lower than that of an alternative algorithm. Additionally, the sensitivity for InSilicoVA was consistently lower than that for an alternative diagnostic algorithm (Tariff 2.0), although the specificity was comparable. CONCLUSIONS: The default format and training data provided by the software lead to results that are at best suboptimal, with poor cause-of-death predictive performance. This method is likely to generate erroneous cause of death predictions and, even if properly configured, is not as accurate as alternative automated diagnostic methods.

Collecting verbal autopsies: improving and streamlining data collection processes using electronic tablets.

BACKGROUND: There is increasing interest in using verbal autopsy to produce nationally representative population-level estimates of causes of death. However, the burden of processing a large quantity of surveys collected with paper and pencil has been a barrier to scaling up verbal autopsy surveillance. Direct electronic data capture has been used in other large-scale surveys and can be used in verbal autopsy as well, to reduce time and cost of going from collected data to actionable information. METHODS: We collected verbal autopsy interviews using paper and pencil and using electronic tablets at two sites, and measured the cost and time required to process the surveys for analysis. From these cost and time data, we extrapolated costs associated with conducting large-scale surveillance with verbal autopsy. RESULTS: We found that the median time between data collection and data entry for surveys collected on paper and pencil was approximately 3 months. For surveys collected on electronic tablets, this was less than 2 days. For small-scale surveys, we found that the upfront costs of purchasing electronic tablets was the primary cost and resulted in a higher total cost. For large-scale surveys, the costs associated with data entry exceeded the cost of the tablets, so electronic data capture provides both a quicker and cheaper method of data collection. CONCLUSIONS: As countries increase verbal autopsy surveillance, it is important to consider the best way to design sustainable systems for data collection. Electronic data capture has the potential to greatly reduce the time and costs associated with data collection. For long-term, large-scale surveillance required by national vital statistical systems, electronic data capture reduces costs and allows data to be available sooner.

Improving methods to measure comparable mortality cause (IMMCMC) gold standard verbal autopsy dataset.

OBJECTIVES: Gold standard cause of death data is critically important to improve verbal autopsy (VA) methods in diagnosing cause of death where civil and vital registration systems are inadequate or poor. As part of a three-country research study-Improving Methods to Measure Comparable Mortality by Cause (IMMCMC) study-data were collected on clinicopathological criteria-based gold standard cause of death from hospital record reviews with matched VAs. The purpose of this data note is to make accessible a de-identified format of these gold standard VAs for interested researchers to improve the diagnostic accuracy of VA methods. DATA DESCRIPTION: The study was conducted between 2011 and 2014 in the Philippines, Bangladesh, and Papua New Guinea. Gold standard diagnoses of underlying causes of death for deaths occurring in hospital were matched to VAs conducted using a standardized VA questionnaire developed by the Population Health Metrics Consortium. 3512 deaths were collected in total, comprised of 2491 adults (12 years and older), 320 children (28 days to 12 years), and 702 neonates (0-27 days).

The epidemiological transition in Papua New Guinea: new evidence from verbal autopsy studies.

BACKGROUND: Recent economic growth in Papua New Guinea (PNG) would suggest that the country may be experiencing an epidemiological transition, characterized by a reduction in infectious diseases and a growing burden from non-communicable diseases (NCDs). However, data on cause-specific mortality in PNG are very sparse, and the extent of the transition within the country is poorly understood. METHODS: Mortality surveillance was established in four small populations across PNG: West Hiri in Central Province, Asaro Valley in Eastern Highlands Province, Hides in Hela Province and Karkar Island in Madang Province. Verbal autopsies (VAs) were conducted on all deaths identified, and causes of death were assigned by SmartVA and classified into five broad disease categories: endemic NCDs; emerging NCDs; endemic infections; emerging infections; and injuries. Results from previous PNG VA studies, using different VA methods and spanning the years 1970 to 2001, are also presented here. RESULTS: A total of 868 deaths among adolescents and adults were identified and assigned a cause of death. NCDs made up the majority of all deaths (40.4%), with the endemic NCD of chronic respiratory disease responsible for the largest proportion of deaths (10.5%), followed by the emerging NCD of diabetes (6.2%). Emerging infectious diseases outnumbered endemic infectious diseases (11.9% versus 9.5%). The distribution of causes of death differed across the four sites, with emerging NCDs and emerging infections highest at the site that is most socioeconomically developed, West Hiri. Comparing the 1970-2001 VA series with the present study suggests a large decrease in endemic infections. CONCLUSIONS: Our results indicate immediate priorities for health service planning and for strengthening of vital registration systems, to more usefully serve the needs of health priority setting.

Implementing the PHMRC shortened questionnaire: Survey duration of open and closed questions in three sites.

BACKGROUND: More countries are using verbal autopsy as a part of routine mortality surveillance. The length of time required to complete a verbal autopsy interview is a key logistical consideration for planning large-scale surveillance. METHODS: We use the PHMRC shortened questionnaire to conduct verbal autopsy interviews at three sites and collect data on the length of time required to complete the interview. This instrument uses a novel checklist of keywords to capture relevant information from the open response. The open response section is timed separately from the section consisting of closed questions. RESULTS: We found the median time to complete the entire interview was approximately 25 minutes and did not vary substantially by age-specific module. The median time for the open response section was approximately 4 minutes and 60% of interviewees mentioned at least one keyword within the open response section. CONCLUSIONS: The length of time required to complete the interview was short enough for large-scale routine use. The open-response section did not add a substantial amount of time and provided useful information which can be used to increase the accuracy of the predictions of the cause of death. The novel checklist approach further reduces the burden of transcribing and translating a large amount of free text. This makes the PHMRC instrument ideal for national mortality surveillance.