Reliability and validity of an innovative high performing healthcare system assessment tool.

BACKGROUND: Universal Health coverage (UHC) is the mantra of the twenty-first century yet knowing when it has been achieved or how to best influence its progression remains elusive. An innovative framework for High Performing Healthcare (HPHC) attempts to address these issues. It focuses on measuring four constructs of Accountable, Affordable, Accessible, and Reliable (AAAR) healthcare that contribute to better health outcomes and impact. The HPHC tool collects information on the perceived functionality of health system processes and provides real-time data analysis on the AAAR constructs, and on processes for health system resilience, responsiveness, and quality, that include roles of community, private sector, as well as both demand, and supply factors affecting health system performance. The tool attempts to capture the multidimensionality of UHC measurement and evidence that links health system strengthening activities to outcomes. This paper provides evidence on the reliability and validity of the tool. METHODS: Internet survey with non-probability sampling was used for testing reliability and validity of the HPHC tool. The volunteers were recruited using international networks and listservs. Two hundred and thirteen people from public, private, civil society and international organizations volunteered from 35 low-and-middle-income countries. Analyses involved testing reliability and validity and validation from other international sources of information as well as applicability in different setting and contexts. RESULTS: The HPHC tool's AAAR constructs, and their sub-domains showed high internal consistency (Cronbach alpha >.80) and construct validity. The tool scores normal distribution displayed variations among respondents. In addition, the tool demonstrated its precision and relevance in different contexts/countries. The triangulation of HPHC findings with other international data sources further confirmed the tool's validity. CONCLUSIONS: Besides being reliable and valid, the HPHC tool adds value to the state of health system measurement by focusing on linkages between AAAR processes and health outcomes. It ensures that health system stakeholders take responsibility and are accountable for better system performance, and the community is empowered to participate in decision-making process. The HPHC tool collects and analyzes data in real time with minimum costs, supports monitoring, and promotes adaptive management, policy, and program development for better health outcomes.

Evaluation of the Performance of Routine Information System Management (PRISM) framework: evidence from Uganda.

BACKGROUND: Sound policy, resource allocation and day-to-day management decisions in the health sector require timely information from routine health information systems (RHIS). In most low- and middle-income countries, the RHIS is viewed as being inadequate in providing quality data and continuous information that can be used to help improve health system performance. In addition, there is limited evidence on the effectiveness of RHIS strengthening interventions in improving data quality and use. The purpose of this study is to evaluate the usefulness of the newly developed Performance of Routine Information System Management (PRISM) framework, which consists of a conceptual framework and associated data collection and analysis tools to assess, design, strengthen and evaluate RHIS. The specific objectives of the study are: a) to assess the reliability and validity of the PRISM instruments and b) to assess the validity of the PRISM conceptual framework. METHODS: Facility- and worker-level data were collected from 110 health care facilities in twelve districts in Uganda in 2004 and 2007 using records reviews, structured interviews and self-administered questionnaires. The analysis procedures include Cronbach's alpha to assess internal consistency of selected instruments, test-retest analysis to assess the reliability and sensitivity of the instruments, and bivariate and multivariate statistical techniques to assess validity of the PRISM instruments and conceptual framework. RESULTS: Cronbach's alpha analysis suggests high reliability (0.7 or greater) for the indices measuring a promotion of a culture of information, RHIS tasks self-efficacy and motivation. The study results also suggest that a promotion of a culture of information influences RHIS tasks self-efficacy, RHIS tasks competence and motivation, and that self-efficacy and the presence of RHIS staff have a direct influence on the use of RHIS information, a key aspect of RHIS performance. CONCLUSIONS: The study results provide some empirical support for the reliability and validity of the PRISM instruments and the validity of the PRISM conceptual framework, suggesting that the PRISM approach can be effectively used by RHIS policy makers and practitioners to assess the RHIS and evaluate RHIS strengthening interventions. However, additional studies with larger sample sizes are needed to further investigate the value of the PRISM instruments in exploring the linkages between RHIS data quality and use, and health systems performance.

Health system modelling research: towards a whole-health-system perspective for identifying good value for money investments in health system strengthening.

Global health research has typically focused on single diseases, and most economic evaluation research to date has analysed technical health interventions to identify 'best buys'. New approaches in the conduct of economic evaluations are needed to help policymakers in choosing what may be good value (ie, greater health, distribution of health, or financial risk protection) for money (ie, per budget expenditure) investments for health system strengthening (HSS) that tend to be programmatic. We posit that these economic evaluations of HSS interventions will require developing new analytic models of health systems which recognise the dynamic connections between the different components of the health system, characterise the type and interlinks of the system's delivery platforms; and acknowledge the multiple constraints both within and outside the health sector which limit the system's capacity to efficiently attain its objectives. We describe priority health system modelling research areas to conduct economic evaluation of HSS interventions and ultimately identify good value for money investments in HSS.

Towards gender equality in health in Afghanistan.

The Afghanistan gender inequality index shows that 70% loss in development is due to the limited participation of women in the workforce, low education and poor women's health outcomes. However, since the fall of the Taliban regime in 2002, gender inequalities in health have improved. This paper will review factors that led to these improvements. The review draws upon information from various sources, including formative and applied research, surveys and existing information systems. The review showed gender differentials in morbidity, mortality and accessing and utilising health services. Health professionals have expressed inadequate medical knowledge and interpersonal skills to address sensitive issues, such as domestic, physical and sexual violence. Discussing sexuality and its impact on health remains taboo both within and outside of the medical profession. Strict cultural norms restrict a woman's autonomy to seek health care, choose a marriage partner and have control over her body, indicating a need to increase awareness about how harmful social practices adversely affect health. The policy review showed that the Ministry of Public Health has made a commitment to reducing gender inequity in health and developed a two-pronged action plan to improve health providers' skills in handling gender-sensitive issues and mass media campaigns to change social norms.

Success in reducing maternal and child mortality in Afghanistan.

After the collapse of the Taliban regime in 2002, Afghanistan adopted a new development path and billions of dollars were invested in rebuilding the country's economy and health systems with the help of donors. These investments have led to substantial improvements in maternal and child health in recent years and ultimately to a decrease in maternal and child mortality. The 2010 Afghanistan Mortality Survey (AMS) provides important new information on the levels and trends in these indicators. The AMS estimated that there are 327 maternal deaths for every 100,000 live births (95% confidence interval = 260-394) and 97 deaths before the age of five years for every 1000 children born. Decreases in these mortality rates are consistent with changes in key determinants of mortality, including an increasing age at marriage, higher contraceptive use, lower fertility, better immunisation coverage, improvements in the percentage of women delivering in health facilities and receiving antenatal and postnatal care, involvement of community health workers and increasing access to the Basic Package of Health Services. Despite the impressive gains in these areas, many challenges remain. Further improvements in health services in Afghanistan will require sustained efforts on the part of both the Government of Afghanistan and international donors.

PRISM framework: a paradigm shift for designing, strengthening and evaluating routine health information systems.

The utility and effectiveness of routine health information systems (RHIS) in improving health system performance in developing countries has been questioned. This paper argues that the health system needs internal mechanisms to develop performance targets, track progress, and create and manage knowledge for continuous improvement. Based on documented RHIS weaknesses, we have developed the Performance of Routine Information System Management (PRISM) framework, an innovative approach to design, strengthen and evaluate RHIS. The PRISM framework offers a paradigm shift by putting emphasis on RHIS performance and incorporating the organizational, technical and behavioural determinants of performance. By describing causal pathways of these determinants, the PRISM framework encourages and guides the development of interventions for strengthening or reforming RHIS. Furthermore, it conceptualizes and proposes a methodology for measuring the impact of RHIS on health system performance. Ultimately, the PRISM framework, in spite of its challenges and competing paradigms, proposes a new agenda for building and sustaining information systems, for the promotion of an information culture, and for encouraging accountability in health systems.