The burden of bacterial antimicrobial resistance in Croatia in 2019: a country-level systematic analysis.

AIM: To deliver the most wide-ranging set of antimicrobial resistance (AMR) burden estimates for Croatia to date. METHODS: A complex modeling approach with five broad modeling components was used to estimate the disease burden for 12 main infectious syndromes and one residual group, 23 pathogenic bacteria, and 88 bug-drug combinations. This was represented by two relevant counterfactual scenarios: deaths/disability-adjusted life years (DALYs) that are attributable to AMR considering a situation where drug-resistant infections are substituted with sensitive ones, and deaths/DALYs associated with AMR considering a scenario where people with drug-resistant infections would instead present without any infection. The 95% uncertainty intervals (UI) were based on 1000 posterior draws in each modeling step, reported at the 2.5% and 97.5% of the draws' distribution, while out-of-sample predictive validation was pursued for all the models. RESULTS: The total burden associated with AMR in Croatia was 2546 (95% UI 1558-3803) deaths and 46958 (28,033-71,628) DALYs, while the attributable burden was 614 (365-943) deaths and 11321 (6,544-17,809) DALYs. The highest number of deaths was established for bloodstream infections, followed by peritoneal and intra-abdominal infections and infections of the urinary tract. Five leading pathogenic bacterial agents were responsible for 1808 deaths associated with resistance: Escherichia coli, Staphylococcus aureus, Acinetobacter baumannii, Klebsiella pneumoniae, and Pseudomonas aeruginosa (ordered by the number of deaths). Trimethoprim/sulfamethoxazole-resistant E coli and methicillin-resistant S. aureus were dominant pathogen-drug combinations in regard to mortality associated with and attributable to AMR, respectively. CONCLUSION: We showed that AMR represented a substantial public health concern in Croatia, which reflects global trends; hence, our detailed country-level findings may fast-track the implementation of multipronged strategies tailored in accordance with leading pathogens and pathogen-drug combinations.

Estimating distributions of health state severity for the global burden of disease study.

BACKGROUND: Many major causes of disability in the Global Burden of Disease (GBD) study present with a range of severity, and for most causes finding population distributions of severity can be difficult due to issues of sparse data, inconsistent measurement, and need to account for comorbidities. We developed an indirect approach to obtain severity distributions empirically from survey data. METHODS: Individual-level data were used from three large population surveys from the US and Australia that included self-reported prevalence of major diseases and injuries as well as generic health status assessments using the 12-Item Short Form Health Survey (SF-12). We developed a mapping function from SF-12 scores to GBD disability weights. Mapped scores for each individual respondent were regressed against the reported diseases and injuries using a mixed-effects model with a logit-transformed response variable. The regression outputs were used to predict comorbidity-corrected health-state weights for the group of individuals with each condition. The distribution of these comorbidity-corrected weights were used to estimate the fraction of individuals with each condition falling into different GBD severity categories, including asymptomatic (implying disability weight of zero). RESULTS: After correcting for comorbid conditions, all causes analyzed had some proportion of the population in the asymptomatic category. For less severe conditions, such as alopecia areata, we estimated that 44.1 % [95 % CI: 38.7 %-49.4 %] were asymptomatic while 28.3 % [26.8 %-29.6 %] of anxiety disorders had asymptomatic cases. For 152 conditions, full distributions of severity were estimated. For anxiety disorders for example, we estimated the mean population proportions in the mild, moderate, and severe states to be 40.9 %, 18.5 %, and 12.3 % respectively. Thirty-seven of the analyzed conditions were used in the GBD 2013 estimates and are reported here. CONCLUSION: There is large heterogeneity in the disabling severity of conditions among individuals. The GBD 2013 approach allows explicit accounting for this heterogeneity in GBD estimates. Existing survey data that have collected health status together with information on the presence of a series of comorbid conditions can be used to fill critical gaps in the information on condition severity while correcting for effects of comorbidity. Our ability to make these estimates may be limited by lack of geographic variation in the data and by the current methodology for disability weights, which implies that severity must be binned rather than expressed in as a full distribution. Future country-specific data collection efforts will be needed to advance this research.

Cigarette smoking prevalence in US counties: 1996-2012.

BACKGROUND: Cigarette smoking is a leading risk factor for morbidity and premature mortality in the United States, yet information about smoking prevalence and trends is not routinely available below the state level, impeding local-level action. METHODS: We used data on 4.7 million adults age 18 and older from the Behavioral Risk Factor Surveillance System (BRFSS) from 1996 to 2012. We derived cigarette smoking status from self-reported data in the BRFSS and applied validated small area estimation methods to generate estimates of current total cigarette smoking prevalence and current daily cigarette smoking prevalence for 3,127 counties and county equivalents annually from 1996 to 2012. We applied a novel method to correct for bias resulting from the exclusion of the wireless-only population in the BRFSS prior to 2011. RESULTS: Total cigarette smoking prevalence varies dramatically between counties, even within states, ranging from 9.9% to 41.5% for males and from 5.8% to 40.8% for females in 2012. Counties in the South, particularly in Kentucky, Tennessee, and West Virginia, as well as those with large Native American populations, have the highest rates of total cigarette smoking, while counties in Utah and other Western states have the lowest. Overall, total cigarette smoking prevalence declined between 1996 and 2012 with a median decline across counties of 0.9% per year for males and 0.6% per year for females, and rates of decline for males and females in some counties exceeded 3% per year. Statistically significant declines were concentrated in a relatively small number of counties, however, and more counties saw statistically significant declines in male cigarette smoking prevalence (39.8% of counties) than in female cigarette smoking prevalence (16.2%). Rates of decline varied by income level: counties in the top quintile in terms of income experienced noticeably faster declines than those in the bottom quintile. CONCLUSIONS: County-level estimates of cigarette smoking prevalence provide a unique opportunity to assess where prevalence remains high and where progress has been slow. These estimates provide the data needed to better develop and implement strategies at a local and at a state level to further reduce the burden imposed by cigarette smoking.

A composite metric for assessing data on mortality and causes of death: the vital statistics performance index.

BACKGROUND: Timely and reliable data on causes of death are fundamental for informed decision-making in the health sector as well as public health research. An in-depth understanding of the quality of data from vital statistics (VS) is therefore indispensable for health policymakers and researchers. We propose a summary index to objectively measure the performance of VS systems in generating reliable mortality data and apply it to the comprehensive cause of death database assembled for the Global Burden of Disease (GBD) 2013 Study. METHODS: We created a Vital Statistics Performance Index, a composite of six dimensions of VS strength, each assessed by a separate empirical indicator. The six dimensions include: quality of cause of death reporting, quality of age and sex reporting, internal consistency, completeness of death reporting, level of cause-specific detail, and data availability/timeliness. A simulation procedure was developed to combine indicators into a single index. This index was computed for all country-years of VS in the GBD 2013 cause of death database, yielding annual estimates of overall VS system performance for 148 countries or territories. RESULTS: The six dimensions impacted the accuracy of data to varying extents. VS performance declines more steeply with declining simulated completeness than for any other indicator. The amount of detail in the cause list reported has a concave relationship with overall data accuracy, but is an important driver of observed VS performance. Indicators of cause of death data quality and age/sex reporting have more linear relationships with simulated VS performance, but poor cause of death reporting influences observed VS performance more strongly. VS performance is steadily improving at an average rate of 2.10% per year among the 148 countries that have available data, but only 19.0% of global deaths post-2000 occurred in countries with well-performing VS systems. CONCLUSIONS: Objective and comparable information about the performance of VS systems and the utility of the data that they report will help to focus efforts to strengthen VS systems. Countries and the global health community alike need better intelligence about the accuracy of VS that are widely and often uncritically used in population health research and monitoring.

A comparison of missing data procedures for addressing selection bias in HIV sentinel surveillance data.

BACKGROUND: Selection bias is common in clinic-based HIV surveillance. Clinics located in HIV hotspots are often the first to be chosen and monitored, while clinics in less prevalent areas are added to the surveillance system later on. Consequently, the estimated HIV prevalence based on clinic data is substantially distorted, with markedly higher HIV prevalence in the earlier periods and trends that reveal much more dramatic declines than actually occur. METHODS: Using simulations, we compare and contrast the performance of the various approaches and models for handling selection bias in clinic-based HIV surveillance. In particular, we compare the application of complete-case analysis and multiple imputation (MI). Several models are considered for each of the approaches. We demonstrate the application of the methods through sentinel surveillance data collected between 2002 and 2008 from India. RESULTS: Simulations suggested that selection bias, if not handled properly, can lead to biased estimates of HIV prevalence trends and inaccurate evaluation of program impact. Complete-case analysis and MI differed considerably in their ability to handle selection bias. In scenarios where HIV prevalence remained constant over time (i.e. ß = 0), the estimated ß^1 derived from MI tended to be biased downward. Depending on the imputation model used, the estimated bias ranged from -1.883 to -0.048 in logit prevalence. Furthermore, as the level of selection bias intensified, the extent of bias also increased. In contrast, the estimates yielded by complete-case analysis were relatively unbiased and stable across the various scenarios. The estimated bias ranged from -0.002 to 0.002 in logit prevalence. CONCLUSIONS: Given that selection bias is common in clinic-based HIV surveillance, when analyzing data from such sources appropriate adjustment methods need to be applied. The results in this paper suggest that indiscriminant application of imputation models can lead to biased results.

Left behind: widening disparities for males and females in US county life expectancy, 1985-2010.

BACKGROUND: The United States spends more than any other country on health care. The poor relative performance of the US compared to other high-income countries has attracted attention and raised questions about the performance of the US health system. An important dimension to poor national performance is the large disparities in life expectancy. METHODS: We applied a mixed effects Poisson statistical model and Gaussian Process Regression to estimate age-specific mortality rates for US counties from 1985 to 2010. We generated uncertainty distributions for life expectancy at each age using standard simulation methods. RESULTS: Female life expectancy in the United States increased from 78.0 years in 1985 to 80.9 years in 2010, while male life expectancy increased from 71.0 years in 1985 to 76.3 years in 2010. The gap between female and male life expectancy in the United States was 7.0 years in 1985, narrowing to 4.6 years in 2010. For males at the county level, the highest life expectancy steadily increased from 75.5 in 1985 to 81.7 in 2010, while the lowest life expectancy remained under 65. For females at the county level, the highest life expectancy increased from 81.1 to 85.0, and the lowest life expectancy remained around 73. For male life expectancy at the county level, there have been three phases in the evolution of inequality: a period of rising inequality from 1985 to 1993, a period of stable inequality from 1993 to 2002, and rising inequality from 2002 to 2010. For females, in contrast, inequality has steadily increased during the 25-year period. Compared to only 154 counties where male life expectancy remained stagnant or declined, 1,405 out of 3,143 counties (45%) have seen no significant change or a significant decline in female life expectancy from 1985 to 2010. In all time periods, the lowest county-level life expectancies are seen in the South, the Mississippi basin, West Virginia, Kentucky, and selected counties with large Native American populations. CONCLUSIONS: The reduction in the number of counties where female life expectancy at birth is declining in the most recent period is welcome news. However, the widening disparities between counties and the slow rate of increase compared to other countries should be viewed as a call for action. An increased focus on factors affecting health outcomes, morbidity, and mortality such as socioeconomic factors, difficulty of access to and poor quality of health care, and behavioral, environmental, and metabolic risk factors is urgently required.

Prevalence of physical activity and obesity in US counties, 2001-2011: a road map for action.

BACKGROUND: Obesity and physical inactivity are associated with several chronic conditions, increased medical care costs, and premature death. METHODS: We used the Behavioral Risk Factor Surveillance System (BRFSS), a state-based random-digit telephone survey that covers the majority of United States counties, and the National Health and Nutrition Examination Survey (NHANES), a nationally representative sample of the US civilian noninstitutionalized population. About 3.7 million adults aged 20 years or older participated in the BRFSS from 2000 to 2011, and 30,000 adults aged 20 or older participated in NHANES from 1999 to 2010. We calculated body mass index (BMI) from self-reported weight and height in the BRFSS and adjusted for self-reporting bias using NHANES. We calculated self-reported physical activity-both any physical activity and physical activity meeting recommended levels-from self-reported data in the BRFSS. We used validated small area estimation methods to generate estimates of obesity and physical activity prevalence for each county annually for 2001 to 2011. RESULTS: Our results showed an increase in the prevalence of sufficient physical activity from 2001 to 2009. Levels were generally higher in men than in women, but increases were greater in women than men. Counties in Kentucky, Florida, Georgia, and California reported the largest gains. This increase in level of activity was matched by an increase in obesity in almost all counties during the same time period. There was a low correlation between level of physical activity and obesity in US counties. From 2001 to 2009, controlling for changes in poverty, unemployment, number of doctors per 100,000 population, percent rural, and baseline levels of obesity, for every 1 percentage point increase in physical activity prevalence, obesity prevalence was 0.11 percentage points lower. CONCLUSIONS: Our study showed that increased physical activity alone has a small impact on obesity prevalence at the county level in the US. Indeed, the rise in physical activity levels will have a positive independent impact on the health of Americans as it will reduce the burden of cardiovascular diseases and diabetes. Other changes such as reduction in caloric intake are likely needed to curb the obesity epidemic and its burden.

Modeling causes of death: an integrated approach using CODEm.

BACKGROUND: Data on causes of death by age and sex are a critical input into health decision-making. Priority setting in public health should be informed not only by the current magnitude of health problems but by trends in them. However, cause of death data are often not available or are subject to substantial problems of comparability. We propose five general principles for cause of death model development, validation, and reporting. METHODS: We detail a specific implementation of these principles that is embodied in an analytical tool - the Cause of Death Ensemble model (CODEm) - which explores a large variety of possible models to estimate trends in causes of death. Possible models are identified using a covariate selection algorithm that yields many plausible combinations of covariates, which are then run through four model classes. The model classes include mixed effects linear models and spatial-temporal Gaussian Process Regression models for cause fractions and death rates. All models for each cause of death are then assessed using out-of-sample predictive validity and combined into an ensemble with optimal out-of-sample predictive performance. RESULTS: Ensemble models for cause of death estimation outperform any single component model in tests of root mean square error, frequency of predicting correct temporal trends, and achieving 95% coverage of the prediction interval. We present detailed results for CODEm applied to maternal mortality and summary results for several other causes of death, including cardiovascular disease and several cancers. CONCLUSIONS: CODEm produces better estimates of cause of death trends than previous methods and is less susceptible to bias in model specification. We demonstrate the utility of CODEm for the estimation of several major causes of death.

Developing a comprehensive time series of GDP per capita for 210 countries from 1950 to 2015.

BACKGROUND: Income has been extensively studied and utilized as a determinant of health. There are several sources of income expressed as gross domestic product (GDP) per capita, but there are no time series that are complete for the years between 1950 and 2015 for the 210 countries for which data exist. It is in the interest of population health research to establish a global time series that is complete from 1950 to 2015. METHODS: We collected GDP per capita estimates expressed in either constant US dollar terms or international dollar terms (corrected for purchasing power parity) from seven sources. We applied several stages of models, including ordinary least-squares regressions and mixed effects models, to complete each of the seven source series from 1950 to 2015. The three US dollar and four international dollar series were each averaged to produce two new GDP per capita series. RESULTS AND DISCUSSION: Nine complete series from 1950 to 2015 for 210 countries are available for use. These series can serve various analytical purposes and can illustrate myriad economic trends and features. The derivation of the two new series allows for researchers to avoid any series-specific biases that may exist. The modeling approach used is flexible and will allow for yearly updating as new estimates are produced by the source series. CONCLUSION: GDP per capita is a necessary tool in population health research, and our development and implementation of a new method has allowed for the most comprehensive known time series to date.

Estimating health care delivery system value for each US state and testing key associations.

OBJECTIVE: To estimate health care systems' value in treating major illnesses for each US state and identify system characteristics associated with value. DATA SOURCES: Annual condition-specific death and incidence estimates for each US state from the Global Burden Disease 2019 Study and annual health care spending per person for each state from the National Health Expenditure Accounts. STUDY DESIGN: Using non-linear meta-stochastic frontier analysis, mortality incidence ratios for 136 major treatable illnesses were regressed separately on per capita health care spending and key covariates such as age, obesity, smoking, and educational attainment. State- and year-specific inefficiency estimates were extracted for each health condition and combined to create a single estimate of health care delivery system value for each US state for each year, 1991-2014. The association between changes in health care value and changes in 23 key health care system characteristics and state policies was measured. DATA COLLECTION/EXTRACTION METHODS: Not applicable. PRINCIPAL FINDINGS: US state with relatively high spending per person or relatively poor health-outcomes were shown to have low health care delivery system value. New Jersey, Maryland, Florida, Arizona, and New York attained the highest value scores in 2014 (81 [95% uncertainty interval 72-88], 80 [72-87], 80 [71-86], 77 [69-84], and 77 [66-85], respectively), after controlling for health care spending, age, obesity, smoking, physical activity, race, and educational attainment. Greater market concentration of hospitals and of insurers were associated with worse health care value (p-value ranging from <0.01 to 0.02). Higher hospital geographic density and use were also associated with worse health care value (p-value ranging from 0.03 to 0.05). Enrollment in Medicare Advantage HMOs was associated with better value, as was more generous Medicaid income eligibility (p-value 0.04 and 0.01). CONCLUSIONS: Substantial variation in the value of health care exists across states. Key health system characteristics such as market concentration and provider density were associated with value.

Exposing misclassified HIV/AIDS deaths in South Africa.

OBJECTIVE: To quantify the deaths from human immunodeficiency virus (HIV) infection or acquired immunodeficiency syndrome (AIDS) that are misattributed to other causes in South Africa's death registration data and to adjust for this bias. METHODS: Deaths in the World Health Organization's mortality database were distributed among 48 mutually exclusive causes. For each cause, age- and sex-specific global death rates were compared with the average rate among people aged 65-69, 70-74 and 75-79 years to generate "relative" global death rates. Relative rates were also computed for South Africa alone. Differences between global and South African relative death rates were used to identify the causes to which deaths from HIV/AIDS were misattributed in South Africa and quantify the HIV/AIDS deaths misattributed to each. These deaths were then reattributed to HIV/AIDS. FINDINGS: In South Africa, deaths from HIV/AIDS are often misclassified as being caused by 14 other conditions. Whereas in 1996-2006 deaths attributed to HIV/AIDS accounted for 2.0-2.5% of all registered deaths in South Africa, our analysis shows that the true cause-specific mortality fraction rose from 19% (uncertainty range: 7-28%) to 48% (uncertainty range: 38-50%) over that period. More than 90% of HIV/AIDS deaths were found to have been misattributed to other causes during 1996-2006. CONCLUSION: Adjusting for cause of death misclassification, a simple procedure that can be carried out in any country, can improve death registration data and provide empirical estimates of HIV/AIDS deaths that may be useful in assessing estimates from demographic models.

Falling behind: life expectancy in US counties from 2000 to 2007 in an international context.

BACKGROUND: The United States health care debate has focused on the nation's uniquely high rates of lack of insurance and poor health outcomes relative to other high-income countries. Large disparities in health outcomes are well-documented in the US, but the most recent assessment of county disparities in mortality is from 1999. It is critical to tracking progress of health reform legislation to have an up-to-date assessment of disparities in life expectancy across counties. US disparities can be seen more clearly in the context of how progress in each county compares to international trends. METHODS: We use newly released mortality data by age, sex, and county for the US from 2000 to 2007 to compute life tables separately for each sex, for all races combined, for whites, and for blacks. We propose, validate, and apply novel methods to estimate recent life tables for small areas to generate up-to-date estimates. Life expectancy rates and changes in life expectancy for counties are compared to the life expectancies across nations in 2000 and 2007. We calculate the number of calendar years behind each county is in 2000 and 2007 compared to an international life expectancy time series. RESULTS: Across US counties, life expectancy in 2007 ranged from 65.9 to 81.1 years for men and 73.5 to 86.0 years for women. When compared against a time series of life expectancy in the 10 nations with the lowest mortality, US counties range from being 15 calendar years ahead to over 50 calendar years behind for men and 16 calendar years ahead to over 50 calendar years behind for women. County life expectancy for black men ranges from 59.4 to 77.2 years, with counties ranging from seven to over 50 calendar years behind the international frontier; for black women, the range is 69.6 to 82.6 years, with counties ranging from eight to over 50 calendar years behind. Between 2000 and 2007, 80% (men) and 91% (women) of American counties fell in standing against this international life expectancy standard. CONCLUSIONS: The US has extremely large geographic and racial disparities, with some communities having life expectancies already well behind those of the best-performing nations. At the same time, relative performance for most communities continues to drop. Efforts to address these issues will need to tackle the leading preventable causes of death.

Performance of the Tariff Method: validation of a simple additive algorithm for analysis of verbal autopsies.

BACKGROUND: Verbal autopsies provide valuable information for studying mortality patterns in populations that lack reliable vital registration data. Methods for transforming verbal autopsy results into meaningful information for health workers and policymakers, however, are often costly or complicated to use. We present a simple additive algorithm, the Tariff Method (termed Tariff), which can be used for assigning individual cause of death and for determining cause-specific mortality fractions (CSMFs) from verbal autopsy data. METHODS: Tariff calculates a score, or "tariff," for each cause, for each sign/symptom, across a pool of validated verbal autopsy data. The tariffs are summed for a given response pattern in a verbal autopsy, and this sum (score) provides the basis for predicting the cause of death in a dataset. We implemented this algorithm and evaluated the method's predictive ability, both in terms of chance-corrected concordance at the individual cause assignment level and in terms of CSMF accuracy at the population level. The analysis was conducted separately for adult, child, and neonatal verbal autopsies across 500 pairs of train-test validation verbal autopsy data. RESULTS: Tariff is capable of outperforming physician-certified verbal autopsy in most cases. In terms of chance-corrected concordance, the method achieves 44.5% in adults, 39% in children, and 23.9% in neonates. CSMF accuracy was 0.745 in adults, 0.709 in children, and 0.679 in neonates. CONCLUSIONS: Verbal autopsies can be an efficient means of obtaining cause of death data, and Tariff provides an intuitive, reliable method for generating individual cause assignment and CSMFs. The method is transparent and flexible and can be readily implemented by users without training in statistics or computer science.

Improving the public health utility of global cardiovascular mortality data: the rise of ischemic heart disease.

BACKGROUND: High-quality, cause-specific mortality data are critical for effective health policy. Yet vague cause of death codes, such as heart failure, are highly prevalent in global mortality data. We propose an empirical method correcting mortality data for the use of heart failure as an underlying cause of death. METHODS: We performed a regression analysis stratified by sex, age, and country development status on all available ICD-10 mortality data, consisting of 142 million deaths across 838 country-years. The analysis yielded predicted fractions with which to redistribute heart failure-attributed deaths to the appropriate underlying causes of death. Age-adjusted death rates and rank causes of death before and after correction were calculated. RESULTS: Heart failure accounts for 3.1% of all deaths in the dataset. Ischemic heart disease has the highest redistribution proportion for ages 15-49 and 50+ in both sexes and country development levels, causing gains in age-adjusted death rates in both developed and developing countries. COPD and hypertensive heart disease also make significant rank gains. Reproductive-aged women in developing country-years yield the most diverse range of heart failure causes. CONCLUSIONS: Ischemic heart disease becomes the No. 1 cause of death in several developed countries, including France and Japan, underscoring the cardiovascular epidemic in high-income countries. Age-adjusted death rate increases for ischemic heart disease in low- and middle-income countries, such as Argentina and South Africa, highlight the rise of the cardiovascular epidemic in regions where public health efforts have historically focused on infectious diseases. This method maximizes the use of available data, providing better evidence on major causes of death to inform policymakers in allocating finite resources.

Direct estimation of cause-specific mortality fractions from verbal autopsies: multisite validation study using clinical diagnostic gold standards.

BACKGROUND: Verbal autopsy (VA) is used to estimate the causes of death in areas with incomplete vital registration systems. The King and Lu method (KL) for direct estimation of cause-specific mortality fractions (CSMFs) from VA studies is an analysis technique that estimates CSMFs in a population without predicting individual-level cause of death as an intermediate step. In previous studies, KL has shown promise as an alternative to physician-certified verbal autopsy (PCVA). However, it has previously been impossible to validate KL with a large dataset of VAs for which the underlying cause of death is known to meet rigorous clinical diagnostic criteria. METHODS: We applied the KL method to adult, child, and neonatal VA datasets from the Population Health Metrics Research Consortium gold standard verbal autopsy validation study, a multisite sample of 12,542 VAs where gold standard cause of death was established using strict clinical diagnostic criteria. To emulate real-world populations with varying CSMFs, we evaluated the KL estimations for 500 different test datasets of varying cause distribution. We assessed the quality of these estimates in terms of CSMF accuracy as well as linear regression and compared this with the results of PCVA. RESULTS: KL performance is similar to PCVA in terms of CSMF accuracy, attaining values of 0.669, 0.698, and 0.795 for adult, child, and neonatal age groups, respectively, when health care experience (HCE) items were included. We found that the length of the cause list has a dramatic effect on KL estimation quality, with CSMF accuracy decreasing substantially as the length of the cause list increases. We found that KL is not reliant on HCE the way PCVA is, and without HCE, KL outperforms PCVA for all age groups. CONCLUSIONS: Like all computer methods for VA analysis, KL is faster and cheaper than PCVA. Since it is a direct estimation technique, though, it does not produce individual-level predictions. KL estimates are of similar quality to PCVA and slightly better in most cases. Compared to other recently developed methods, however, KL would only be the preferred technique when the cause list is short and individual-level predictions are not needed.

Performance of physician-certified verbal autopsies: multisite validation study using clinical diagnostic gold standards.

BACKGROUND: Physician review of a verbal autopsy (VA) and completion of a death certificate remains the most widely used approach for VA analysis. This study provides new evidence about the performance of physician-certified verbal autopsy (PCVA) using defined clinical diagnostic criteria as a gold standard for a multisite sample of 12,542 VAs. The study was also designed to analyze issues related to PCVA, such as the impact of a second physician reader on the cause of death assigned, the variation in performance with and without household recall of health care experience (HCE), and the importance of local information for physicians reading VAs. METHODS: The certification was performed by 24 physicians. The assignment of VA was random and blinded. Each VA was certified by one physician. Half of the VAs were reviewed by a different physician with household recall of health care experience included. The completed death certificate was processed for automated ICD-10 coding of the underlying cause of death. PCVA was compared to gold standard cause of death assignment based on strictly defined clinical diagnostic criteria that are part of the Population Health Metrics Research Consortium (PHMRC) gold standard verbal autopsy study. RESULTS: For individual cause assignment, the overall chance-corrected concordance for PCVA against the gold standard cause of death is less than 50%, with substantial variability by cause and physician. Physicians assign the correct cause around 30% of the time without HCE, and addition of HCE improves performance in adults to 45% and slightly higher in children to 48%. Physicians estimate cause-specific mortality fractions (CSMFs) with considerable error for adults, children, and neonates. Only for neonates for a cause list of six causes with HCE is accuracy above 0.7. In all three age groups, CSMF accuracy improves when household recall of health care experience is available. CONCLUSIONS: Results show that physician coding for cause of death assignment may not be as robust as previously thought. The time and cost required to initially collect the verbal autopsies must be considered in addition to the analysis, as well as the impact of diverting physicians from servicing immediate health needs in a population to review VAs. All of these considerations highlight the importance and urgency of developing better methods to more reliably analyze past and future verbal autopsies to obtain the highest quality mortality data from populations without reliable death certification.

Random forests for verbal autopsy analysis: multisite validation study using clinical diagnostic gold standards.

BACKGROUND: Computer-coded verbal autopsy (CCVA) is a promising alternative to the standard approach of physician-certified verbal autopsy (PCVA), because of its high speed, low cost, and reliability. This study introduces a new CCVA technique and validates its performance using defined clinical diagnostic criteria as a gold standard for a multisite sample of 12,542 verbal autopsies (VAs). METHODS: The Random Forest (RF) Method from machine learning (ML) was adapted to predict cause of death by training random forests to distinguish between each pair of causes, and then combining the results through a novel ranking technique. We assessed quality of the new method at the individual level using chance-corrected concordance and at the population level using cause-specific mortality fraction (CSMF) accuracy as well as linear regression. We also compared the quality of RF to PCVA for all of these metrics. We performed this analysis separately for adult, child, and neonatal VAs. We also assessed the variation in performance with and without household recall of health care experience (HCE). RESULTS: For all metrics, for all settings, RF was as good as or better than PCVA, with the exception of a nonsignificantly lower CSMF accuracy for neonates with HCE information. With HCE, the chance-corrected concordance of RF was 3.4 percentage points higher for adults, 3.2 percentage points higher for children, and 1.6 percentage points higher for neonates. The CSMF accuracy was 0.097 higher for adults, 0.097 higher for children, and 0.007 lower for neonates. Without HCE, the chance-corrected concordance of RF was 8.1 percentage points higher than PCVA for adults, 10.2 percentage points higher for children, and 5.9 percentage points higher for neonates. The CSMF accuracy was higher for RF by 0.102 for adults, 0.131 for children, and 0.025 for neonates. CONCLUSIONS: We found that our RF Method outperformed the PCVA method in terms of chance-corrected concordance and CSMF accuracy for adult and child VA with and without HCE and for neonatal VA without HCE. It is also preferable to PCVA in terms of time and cost. Therefore, we recommend it as the technique of choice for analyzing past and current verbal autopsies.

Robust metrics for assessing the performance of different verbal autopsy cause assignment methods in validation studies.

BACKGROUND: Verbal autopsy (VA) is an important method for obtaining cause of death information in settings without vital registration and medical certification of causes of death. An array of methods, including physician review and computer-automated methods, have been proposed and used. Choosing the best method for VA requires the appropriate metrics for assessing performance. Currently used metrics such as sensitivity, specificity, and cause-specific mortality fraction (CSMF) errors do not provide a robust basis for comparison. METHODS: We use simple simulations of populations with three causes of death to demonstrate that most metrics used in VA validation studies are extremely sensitive to the CSMF composition of the test dataset. Simulations also demonstrate that an inferior method can appear to have better performance than an alternative due strictly to the CSMF composition of the test set. RESULTS: VA methods need to be evaluated across a set of test datasets with widely varying CSMF compositions. We propose two metrics for assessing the performance of a proposed VA method. For assessing how well a method does at individual cause of death assignment, we recommend the average chance-corrected concordance across causes. This metric is insensitive to the CSMF composition of the test sets and corrects for the degree to which a method will get the cause correct due strictly to chance. For the evaluation of CSMF estimation, we propose CSMF accuracy. CSMF accuracy is defined as one minus the sum of all absolute CSMF errors across causes divided by the maximum total error. It is scaled from zero to one and can generalize a method's CSMF estimation capability regardless of the number of causes. Performance of a VA method for CSMF estimation by cause can be assessed by examining the relationship across test datasets between the estimated CSMF and the true CSMF. CONCLUSIONS: With an increasing range of VA methods available, it will be critical to objectively assess their performance in assigning cause of death. Chance-corrected concordance and CSMF accuracy assessed across a large number of test datasets with widely varying CSMF composition provide a robust strategy for this assessment.

Performance of InterVA for assigning causes of death to verbal autopsies: multisite validation study using clinical diagnostic gold standards.

BACKGROUND: InterVA is a widely disseminated tool for cause of death attribution using information from verbal autopsies. Several studies have attempted to validate the concordance and accuracy of the tool, but the main limitation of these studies is that they compare cause of death as ascertained through hospital record review or hospital discharge diagnosis with the results of InterVA. This study provides a unique opportunity to assess the performance of InterVA compared to physician-certified verbal autopsies (PCVA) and alternative automated methods for analysis. METHODS: Using clinical diagnostic gold standards to select 12,542 verbal autopsy cases, we assessed the performance of InterVA on both an individual and population level and compared the results to PCVA, conducting analyses separately for adults, children, and neonates. Following the recommendation of Murray et al., we randomly varied the cause composition over 500 test datasets to understand the performance of the tool in different settings. We also contrasted InterVA with an alternative Bayesian method, Simplified Symptom Pattern (SSP), to understand the strengths and weaknesses of the tool. RESULTS: Across all age groups, InterVA performs worse than PCVA, both on an individual and population level. On an individual level, InterVA achieved a chance-corrected concordance of 24.2% for adults, 24.9% for children, and 6.3% for neonates (excluding free text, considering one cause selection). On a population level, InterVA achieved a cause-specific mortality fraction accuracy of 0.546 for adults, 0.504 for children, and 0.404 for neonates. The comparison to SSP revealed four specific characteristics that lead to superior performance of SSP. Increases in chance-corrected concordance are attained by developing cause-by-cause models (2%), using all items as opposed to only the ones that mapped to InterVA items (7%), assigning probabilities to clusters of symptoms (6%), and using empirical as opposed to expert probabilities (up to 8%). CONCLUSIONS: Given the widespread use of verbal autopsy for understanding the burden of disease and for setting health intervention priorities in areas that lack reliable vital registrations systems, accurate analysis of verbal autopsies is essential. While InterVA is an affordable and available mechanism for assigning causes of death using verbal autopsies, users should be aware of its suboptimal performance relative to other methods.

Simplified Symptom Pattern Method for verbal autopsy analysis: multisite validation study using clinical diagnostic gold standards.

BACKGROUND: Verbal autopsy can be a useful tool for generating cause of death data in data-sparse regions around the world. The Symptom Pattern (SP) Method is one promising approach to analyzing verbal autopsy data, but it has not been tested rigorously with gold standard diagnostic criteria. We propose a simplified version of SP and evaluate its performance using verbal autopsy data with accompanying true cause of death. METHODS: We investigated specific parameters in SP's Bayesian framework that allow for its optimal performance in both assigning individual cause of death and in determining cause-specific mortality fractions. We evaluated these outcomes of the method separately for adult, child, and neonatal verbal autopsies in 500 different population constructs of verbal autopsy data to analyze its ability in various settings. RESULTS: We determined that a modified, simpler version of Symptom Pattern (termed Simplified Symptom Pattern, or SSP) performs better than the previously-developed approach. Across 500 samples of verbal autopsy testing data, SSP achieves a median cause-specific mortality fraction accuracy of 0.710 for adults, 0.739 for children, and 0.751 for neonates. In individual cause of death assignment in the same testing environment, SSP achieves 45.8% chance-corrected concordance for adults, 51.5% for children, and 32.5% for neonates. CONCLUSIONS: The Simplified Symptom Pattern Method for verbal autopsy can yield reliable and reasonably accurate results for both individual cause of death assignment and for determining cause-specific mortality fractions. The method demonstrates that verbal autopsies coupled with SSP can be a useful tool for analyzing mortality patterns and determining individual cause of death from verbal autopsy data.