Assessing the Diagnostic Accuracy of Physicians for Home Death Certification in Shanghai: Application of SmartVA.

Approximately 30% of deaths in Shanghai either occur at home or are not medically attended. The recorded cause of death (COD) in these cases may not be reliable. We applied the Smart Verbal Autopsy (VA) tool to assign the COD for a representative sample of home deaths certified by 16 community health centers (CHCs) from three districts in Shanghai, from December 2017 to June 2018. The results were compared with diagnoses from routine practice to ascertain the added value of using SmartVA. Overall, cause-specific mortality fraction (CSMF) accuracy improved from 0.93 (93%) to 0.96 after the application of SmartVA. A comparison with a "gold standard (GS)" diagnoses obtained from a parallel medical record review investigation found that 86.3% of the initial diagnoses made by the CHCs were assigned the correct COD, increasing to 90.5% after the application of SmartVA. We conclude that routine application of SmartVA is not indicated for general use in CHCs, although the tool did improve diagnostic accuracy for residual causes, such as other or ill-defined cancers and non-communicable diseases.

Are cause of death data for Shanghai fit for purpose? A retrospective study of medical records.

OBJECTIVES: To assess the quality of cause of death reporting in Shanghai for both hospital and home deaths. DESIGN AND SETTING: Medical records review (MRR) to independently establish a reference data set against which to compare original and adjusted diagnoses from a sample of three tertiary hospitals, one secondary level hospital and nine community health centres in Shanghai. PARTICIPANTS: 1757 medical records (61% males, 39% females) of deaths that occurred in these sample sites in 2017 were reviewed using established diagnostic standards. INTERVENTIONS: None. PRIMARY OUTCOME: Original underlying cause of death (UCOD) from medical facilities. SECONDARY OUTCOME: Routine UCOD assigned from the Shanghai Civil Registration and Vital Statistics (CRVS) system and MRR UCODs from MRR. RESULTS: The original UCODs as assigned by doctors in the study facilities were of relatively low quality, reduced to 31% of deaths assigned to garbage codes, reduced to 2.3% following data quality and follow back procedures routinely applied by the Shanghai CRVS system. The original UCOD had lower chance-corrected concordance and cause-specific mortality fraction accuracy of 0.57 (0.44, 0.70) and 0.66, respectively, compared with 0.75 (0.66, 0.85) and 0.96, respectively, after routine data checking procedures had been applied. CONCLUSIONS: Training in correct death certification for clinical doctors, especially tertiary hospital doctors, is essential to improve UCOD quality in Shanghai. A routine quality control system should be established to actively track diagnostic performance and provide feedback to individual doctors or facilities as needed.

Improving medical certification of cause of death: effective strategies and approaches based on experiences from the Data for Health Initiative.

BACKGROUND: Accurate and timely cause of death (COD) data are essential for informed public health policymaking. Medical certification of COD generally provides the majority of COD data in a population and is an essential component of civil registration and vital statistics (CRVS) systems. Accurate completion of the medical certificate of cause of death (MCCOD) should be a relatively straightforward procedure for physicians, but mistakes are common. Here, we present three training strategies implemented in five countries supported by the Bloomberg Philanthropies Data for Health (D4H) Initiative at the University of Melbourne (UoM) and evaluate the impact on the quality of certification. METHODS: The three training strategies evaluated were (1) training of trainers (TOT) in the Philippines, Myanmar, and Sri Lanka; (2) direct training of physicians by the UoM D4H in Papua New Guinea (PNG); and (3) the implementation of an online and basic training strategy in Peru. The evaluation involved an assessment of MCCODs before and after training using an assessment tool developed by the University of Melbourne. RESULTS: The TOT strategy led to reductions in incorrectly completed certificates of between 28% in Sri Lanka and 40% in the Philippines. Following direct training of physicians in PNG, the reduction in incorrectly completed certificates was 30%. In Peru, the reduction in incorrect certificates was 30% after implementation and training on an online system only and 43% after training on both the online system and basic medical certification principles. CONCLUSIONS: The results of this study indicate that a variety of training strategies can produce benefits in the quality of certification, but further improvements are possible. The experiences of D4H suggest several aspects of the strategies that should be further developed to improve outcomes, particularly key stakeholder engagement from early in the intervention and local committees to oversee activities and support an improved culture in hospitals to support better diagnostic skills and practices.

Saving lives through certifying deaths: assessing the impact of two interventions to improve cause of death data in Perú.

BACKGROUND: Mortality statistics derived from cause of death data are an important source of information for population health monitoring, priority setting and planning. In Perú, almost all death certificates are issued by doctors because it is a legal requirement. However, the quality of cause of death data is poor. In August 2016, the Ministry of Health of Perú decided to make two specific interventions to improve cause of death data: to introduce an online death certification system and to train doctors in standard death certification practices. METHODS: The study comprised a random sample of 300 pre-intervention death certificates, 900 death certificates that were part of the online intervention, and 900 death certificates that were part of both the online and training interventions. All the deaths had occurred between January and September 2017. We used the Assessing the quality of death certification tool from the University of Melbourne for the assessment. We examined the frequency of common errors in death certificates, the frequency of any error and the average error score for each category of: age group, sex, doctor's seniority, doctor's speciality, level of health facility and broad cause of death. RESULTS: The average error score declined by 38% due to the online intervention and by a further 26% due to the training intervention. Improved certification practices remained after controlling for potentially confounding factors. Main improvements were reductions in the absence of a time interval (66% of certificates), incorrect sequence of causes (22%), and ill-defined conditions (13%). CONCLUSIONS: This study demonstrates how the two interventions introduced by the Ministry of Health in Perú improved the correctness of death certificates. The study also provides evidence on necessary changes to the training program to address the poor certification practices that have remained after implementation of the online system.

Collecting verbal autopsies: improving and streamlining data collection processes using electronic tablets.

BACKGROUND: There is increasing interest in using verbal autopsy to produce nationally representative population-level estimates of causes of death. However, the burden of processing a large quantity of surveys collected with paper and pencil has been a barrier to scaling up verbal autopsy surveillance. Direct electronic data capture has been used in other large-scale surveys and can be used in verbal autopsy as well, to reduce time and cost of going from collected data to actionable information. METHODS: We collected verbal autopsy interviews using paper and pencil and using electronic tablets at two sites, and measured the cost and time required to process the surveys for analysis. From these cost and time data, we extrapolated costs associated with conducting large-scale surveillance with verbal autopsy. RESULTS: We found that the median time between data collection and data entry for surveys collected on paper and pencil was approximately 3 months. For surveys collected on electronic tablets, this was less than 2 days. For small-scale surveys, we found that the upfront costs of purchasing electronic tablets was the primary cost and resulted in a higher total cost. For large-scale surveys, the costs associated with data entry exceeded the cost of the tablets, so electronic data capture provides both a quicker and cheaper method of data collection. CONCLUSIONS: As countries increase verbal autopsy surveillance, it is important to consider the best way to design sustainable systems for data collection. Electronic data capture has the potential to greatly reduce the time and costs associated with data collection. For long-term, large-scale surveillance required by national vital statistical systems, electronic data capture reduces costs and allows data to be available sooner.

Reporting of ethics in peer-reviewed verbal autopsy studies: a systematic review.

Introduction: Verbal autopsy (VA) is a method that determines the cause of death by interviewing a relative of the deceased about the events occurring before the death, in regions where medical certification of cause of death is incomplete. This paper aims to review the ethical standards reported in peer-reviewed VA studies. Methods: A systematic review of Medline and Ovid was conducted by two independent researchers. Data were extracted and analysed for articles based on three key areas: Institutional Review Board (IRB) clearance and consenting process; data collection and management procedures, including: time between death and interview; training and education of interviewer, confidentiality of data and data security; and declarations of funding and conflict of interest. Results: The review identified 802 articles, of which 288 were included. The review found that 48% all the studies reported having IRB clearance or obtaining consent of participants. The interviewer training and education levels were reported in 62% and 21% of the articles, respectively. Confidentiality of data was reported for 14% of all studies, 18% did not report the type of respondent interviewed and 51% reported time between death and the interview for the VA. Data security was reported in 8% of all studies. Funding was declared in 63% of all studies and conflict of interest in 42%. Reporting of all these variables increased over time. Conclusions: The results of this systematic review show that although there has been an increase in ethical reporting for VA studies, there still remains a large gap in reporting.

The quality of medical death certification of cause of death in hospitals in rural Bangladesh: impact of introducing the International Form of Medical Certificate of Cause of Death.

BACKGROUND: Accurate and timely data on cause of death are critically important for guiding health programs and policies. Deaths certified by doctors are implicitly considered to be reliable and accurate, yet the quality of information provided in the international Medical Certificate of Cause of Death (MCCD) usually varies according to the personnel involved in certification, the diagnostic capacity of the hospital, and the category of hospitals. There are no published studies that have analysed how certifying doctors in Bangladesh adhere to international rules when completing the MCCD or have assessed the quality of clinical record keeping. METHODS: The study took place between January 2011 and April 2014 in the Chandpur and Comilla districts of Bangladesh. We introduced the international MCCD to all study hospitals. Trained project physicians assigned an underlying cause of death, assessed the quality of the death certificate, and reported the degree of certainty of the medical records provided for a given cause. We examined the frequency of common errors in completing the MCCD, the leading causes of in-hospital deaths, and the degree of certainty in the cause of death data. RESULTS: The study included 4914 death certificates. 72.9% of medical records were of too poor quality to assign a cause of death, with little difference by age, hospital, and cause of death. 95.6% of death certificates did not indicate the time interval between onset and death, 31.6% required a change in sequence, 13.9% required to include a new diagnosis, 50.7% used abbreviations, 41.5% used multiple causes per line, and 33.2% used an ill-defined condition as the underlying cause of death. 99.1% of death certificates had at least one error. The leading cause of death among adults was stroke (15.8%), among children was pneumonia (31.7%), and among neonates was birth asphyxia (52.8%). CONCLUSION: Physicians in Bangladeshi hospitals had difficulties in completing the MCCD correctly. Physicians routinely made errors in death certification practices and medical record quality was poor. There is an urgent need to improve death certification practices and the quality of hospital data in Bangladesh if these data are to be useful for policy.

Implementing the PHMRC shortened questionnaire: Survey duration of open and closed questions in three sites.

BACKGROUND: More countries are using verbal autopsy as a part of routine mortality surveillance. The length of time required to complete a verbal autopsy interview is a key logistical consideration for planning large-scale surveillance. METHODS: We use the PHMRC shortened questionnaire to conduct verbal autopsy interviews at three sites and collect data on the length of time required to complete the interview. This instrument uses a novel checklist of keywords to capture relevant information from the open response. The open response section is timed separately from the section consisting of closed questions. RESULTS: We found the median time to complete the entire interview was approximately 25 minutes and did not vary substantially by age-specific module. The median time for the open response section was approximately 4 minutes and 60% of interviewees mentioned at least one keyword within the open response section. CONCLUSIONS: The length of time required to complete the interview was short enough for large-scale routine use. The open-response section did not add a substantial amount of time and provided useful information which can be used to increase the accuracy of the predictions of the cause of death. The novel checklist approach further reduces the burden of transcribing and translating a large amount of free text. This makes the PHMRC instrument ideal for national mortality surveillance.

New challenges for verbal autopsy: Considering the ethical and social implications of verbal autopsy methods in routine health information systems.

Verbal autopsy (VA) methods are designed to collect cause-of-death information from populations where many deaths occur outside of health facilities and where death certification is weak or absent. A VA consists of an interview with a relative or carer of a recently deceased individual in order to gather information on the signs and symptoms the decedent presented with prior to death. These details are then used to determine and assign a likely cause-of-death. At a population level this information can be invaluable to help guide prioritisation and direct health policy and services. To date VAs have largely been restricted to research contexts but many countries are now venturing to incorporate VA methods into routine civil registration and vital statistics (CRVS) systems. Given the sensitive nature of death, however, there are a number of ethical, legal and social issues that should be considered when scaling-up VAs, particularly in the cross-cultural and socio-economically disadvantaged environments in which they are typically applied. Considering each step of the VA process this paper provides a narrative review of the social context of VA methods. Harnessing the experiences of applying and rolling out VAs as part of routine CRVS systems in a number of low and middle income countries, we identify potential issues that countries and implementing institutions need to consider when incorporating VAs into CRVS systems and point to areas that could benefit from further research and deliberation.

Strengthening health information systems for disability-related rehabilitation in LMICs.

The purpose of this study was to describe the state of rehabilitation health information systems (HIS) in different settings, and identify key processes and actions which contribute to the development of HIS which can effectively support low- and middle-income countries (LMICs) allocate resources to health-related rehabilitation to people with disabilities. Nine case studies were conducted across different disability and developmental settings using documentary review and semi-structured key informant interviews (N = 41). Results were analysed against the six building blocks of a HIS, based on the Health Metrics Network Framework and Standards for Country Health Information Systems and existing HIS capacity. Key barriers or enablers to good disability data collection and use, were documented for each HIS component. Research results suggest there is no gold standard HIS for rehabilitation. There was broad consensus however, that effective health related disability planning requires reliable data on disability prevalence, functional status, access to rehabilitation services and functional outcomes of rehabilitation. For low-resource settings, and where routine HIS are already challenged, planning to include disability and rehabilitation foci starting with a minimum dataset on functioning, and progressively improving the system for increased utility and harmonization, is likely to be most effective and minimize the potential for overburdening fragile systems. The recommendations from this study are based on the successes and challenges of countries with established information systems, and will assist LMICs to prioritize strategic measures to strengthen the collection and use of data for rehabilitation, and progressively realize the rights of people with disabilities.

Use of Smartphone for Verbal Autopsy.

Traditionally, verbal autopsies (VA) are collected on paper-based questionnaires and reviewed by physicians for cause of death assignment, it is resource intensive and time consuming. The Population Health Metrics Research Consortium VA questionnaires was made available on an Android-based application and cause of death was derived using the Tariff method. Over one year, all adult deaths occurring in 48 villages in 4 counties were identified and a VA interview was conducted using the smartphone VA application. A total of 507 adult deaths were recorded and VA interviews were conducted. Cardiovascular disease was the leading cause of death (35.3%) followed by injury (14.6%) and neoplasms (13.5%). The total cost of the pilot study was USD28 835 (USD0.42 per capita). The interviewers found use of smartphones to conduct interviews to be easier. The study showed that using a smartphone application for VA interviews was feasible for implementation in rural China.

A feasibility study on using smartphones to conduct short-version verbal autopsies in rural China.

BACKGROUND: Currently there are two main sources of mortality data with cause of death assignments in China. Both sources-the Ministry of Health-Vital Registration system and the Chinese Disease Surveillance Point system-present their own challenges. A new approach to cause of death assignment is a smartphone-based shortened version of a verbal autopsy survey. This study evaluates the feasibility and acceptability of this new method conducted by township health care providers (THP) and village doctors (VD) in rural China, where a large proportion of deaths occur in homes and cause of death data are inaccurate or lacking. METHODS: The Population Health Metrics Research Consortium mobile phone-based shortened verbal autopsy questionnaire was made available on an Android system-based application, and cause of death was derived using the Tariff method (Tariff 2.0); we called this set of tools "msVA." msVA was administered to relatives of the deceased by six THPs and six VDs in 24 villages located in six townships of Luquan County, Hebei Province, China. Subsequently, interviews were conducted among 12 interviewers, 12 randomly selected respondents, and five study staff to assess the feasibility and acceptability of using msVA for mortality data collection. RESULTS: Between July 2013 and August 2013, 268 deaths took place in the study villages. Among the 268 deaths, 227 VAs were completed (nine refusals, 31 migrations and one loss of data due to breakdown of the smartphone). The average time for a VA interview was 21.5 ± 3.4 min (20.1 ± 3.5 min for THP and 23.2 ± 4.1 min for VD). Both THPs and VDs could be successful interviewers; the latter needed more training but had more willingness to implement msVA in the future. The interviews revealed that both interviewers and relatives of the deceased found msVA to be feasible, acceptable, and more desirable than traditional methods. The cost of conducting a new VA was $8.87 per death. CONCLUSIONS: Conduction of msVA by VDs in their own villages was feasible and acceptable in rural northern China. Broader implementation of msVA across rural China could potentially improve the coverage and quality of cause of death data, allowing for better national health evaluation and program planning.

Health information systems and disability in the Lao PDR: a qualitative study.

The Convention on the Rights of Persons with Disabilities provides an opportunity to strengthen disability-related health information. This study analysed the health information system in Lao PDR and sought evidence of interventions to improve disability-related health information. The study was based on a literature review and key informant interviews (N = 17) informed by the Health Metrics Network's Framework and Standards and the Performance of Routine Information System Management framework. The Lao health information system is in an embryonic stage with health data often incomplete, inaccurate and poorly used. Indicators related to disability or functioning are not included, and capacity to diagnose the health condition of disability is limited. No studies of health information interventions were found. As a State Party to the CRPD, the Lao PDR has a legal obligation to collect health-related information on people with disabilities. Given the nascent stage of development of the health information system in the Lao PDR and diagnostic capacity, indicators related to basic functioning and access to services should be integrated into household level surveys. As the health information system further develops, small, incremental changes in the type of disability information and rehabilitation and the way it is collected can be implemented. Copyright © 2015 John Wiley & Sons, Ltd.

A shortened verbal autopsy instrument for use in routine mortality surveillance systems.

BACKGROUND: Verbal autopsy (VA) is recognized as the only feasible alternative to comprehensive medical certification of deaths in settings with no or unreliable vital registration systems. However, a barrier to its use by national registration systems has been the amount of time and cost needed for data collection. Therefore, a short VA instrument (VAI) is needed. In this paper we describe a shortened version of the VAI developed for the Population Health Metrics Research Consortium (PHMRC) Gold Standard Verbal Autopsy Validation Study using a systematic approach. METHODS: We used data from the PHMRC validation study. Using the Tariff 2.0 method, we first established a rank order of individual questions in the PHMRC VAI according to their importance in predicting causes of death. Second, we reduced the size of the instrument by dropping questions in reverse order of their importance. We assessed the predictive performance of the instrument as questions were removed at the individual level by calculating chance-corrected concordance and at the population level with cause-specific mortality fraction (CSMF) accuracy. Finally, the optimum size of the shortened instrument was determined using a first derivative analysis of the decline in performance as the size of the VA instrument decreased for adults, children, and neonates. RESULTS: The full PHMRC VAI had 183, 127, and 149 questions for adult, child, and neonatal deaths, respectively. The shortened instrument developed had 109, 69, and 67 questions, respectively, representing a decrease in the total number of questions of 40-55%. The shortened instrument, with text, showed non-significant declines in CSMF accuracy from the full instrument with text of 0.4%, 0.0%, and 0.6% for the adult, child, and neonatal modules, respectively. CONCLUSIONS: We developed a shortened VAI using a systematic approach, and assessed its performance when administered using hand-held electronic tablets and analyzed using Tariff 2.0. The length of a VA questionnaire was shortened by almost 50% without a significant drop in performance. The shortened VAI developed reduces the burden of time and resources required for data collection and analysis of cause of death data in civil registration systems.

Systematic review of statistics on causes of deaths in hospitals: strengthening the evidence for policy-makers.

OBJECTIVE: To systematically review the reliability of hospital data on cause of death and encourage periodic reviews of these data using a standard method. METHODS: We searched Google Scholar, Pubmed and Biblioteca Virtual de la Salud for articles in English, Spanish and Portuguese that reported validation studies of data on cause of death. We analysed the results of 199 studies that had used medical record reviews to validate the cause of death reported on death certificates or by the vital registration system. FINDINGS: The screened studies had been published between 1983 and 2013 and their results had been reported in English (n = 124), Portuguese (n = 25) or Spanish (n = 50). Only 29 of the studies met our inclusion criteria. Of these, 13 had examined cause of death patterns at the population level - with a view to correcting cause-specific mortality fractions - while the other 16 had been undertaken to identify discrepancies in the diagnosis for specific diseases before and after medical record review. Most of the selected studies reported substantial misdiagnosis of causes of death in hospitals. There was wide variation in study methodologies. Many studies did not describe the methods used in sufficient detail to be able to assess the reproducibility or comparability of their results. CONCLUSION: The assumption that causes of death are being accurately reported in hospitals is unfounded. To improve the reliability and usefulness of reported causes of death, national governments should do periodic medical record reviews to validate the quality of their hospital cause of death data, using a standard.