Improving Outcome Accountability of Block Grants: Lessons Learned From the Preventive Health and Health Services Block Grant Evaluation.

The flexibility federal block grants provide recipients poses challenges for evaluation. These challenges include aggregating data on wide-ranging activities grant recipients implement and the outcomes they achieve. In 2014, we began designing an evaluation to address the challenges of assessing outcomes and to improve outcome accountability for the Preventive Health and Health Services Block Grant. Through the use of evaluability assessment methodology, review of existing data and the literature, and key informant interviews, we developed a measurement framework to assess outcomes resulting from recipients' ability to use grant funds to meet their locally prioritized needs. We argue our evaluation approach demonstrates that block grants, and other similarly flexible programs, can be evaluated through appropriately designed measures. Our efforts challenge the idea that flexibility presents an insurmountable barrier to evaluation and outcome accountability for federal block grants.

Defining and assessing quality improvement outcomes: a framework for public health.

We describe an evidence-based framework to define and assess the impact of quality improvement (QI) in public health. Developed to address programmatic and research-identified needs for articulating the value of public health QI in aggregate, this framework proposes a standardized set of measures to monitor and improve the efficiency and effectiveness of public health programs and operations. We reviewed the scientific literature and analyzed QI initiatives implemented through the Centers for Disease Control and Prevention's National Public Health Improvement Initiative to inform the selection of 5 efficiency and 8 effectiveness measures. This framework provides a model for identifying the types of improvement outcomes targeted by public health QI efforts and a means to understand QI's impact on the practice of public health.

Advances in public health accreditation readiness and quality improvement: evaluation findings from the National Public Health Improvement Initiative.

INTRODUCTION: Continuous quality improvement is a central tenet of the Public Health Accreditation Board's (PHAB) national voluntary public health accreditation program. Similarly, the Centers for Disease Control and Prevention launched the National Public Health Improvement Initiative (NPHII) in 2010 with the goal of advancing accreditation readiness, performance management, and quality improvement (QI). OBJECTIVE: Evaluate the extent to which NPHII awardees have achieved program goals. DESIGN: NPHII awardees responded to an annual assessment and program monitoring data requests. Analysis included simple descriptive statistics. SETTING: Seventy-four state, tribal, local, and territorial public health agencies receiving NPHII funds. PARTICIPANTS: NPHII performance improvement managers or principal investigators. MAIN OUTCOME MEASURE(S): Development of accreditation prerequisites, completion of an organizational self-assessment against the PHAB Standards and Measures, Version 1.0, establishment of a performance management system, and implementation of QI initiatives to increase efficiency and effectiveness. RESULTS: Of the 73 responding NPHII awardees, 42.5% had a current health assessment, 26% had a current health improvement plan, and 48% had a current strategic plan in place at the end of the second program year. Approximately 26% of awardees had completed an organizational PHAB self-assessment, 72% had established at least 1 of the 4 components of a performance management system, and 90% had conducted QI activities focused on increasing efficiencies and/or effectiveness. CONCLUSIONS: NPHII appears to be supporting awardees' initial achievement of program outcomes. As NPHII enters its third year, there will be additional opportunities to advance the work of NPHII, compile and disseminate results, and inform a vision of high-quality public health necessary to improve the health of the population.

Conceptualizing and measuring community preparedness within public health preparedness and response: complexities and lessons learned.

Since 2001, the Centers for Disease Control and Prevention's Public Health Emergency Preparedness cooperative agreement has supported state, territorial, and local public health departments in preparing for and responding to public health emergencies. This conceptual article describes complexities identified and lessons learned in developing community preparedness performance measures for the Centers for Disease Control and Prevention's public health preparedness program. Challenges arose in (a) defining community; (b) measuring meaningful community engagement; and (c) determining a strategy for collecting, aggregating, and analyzing data from diverse state, territorial, and local health departments. This article contributes to prior work describing conceptual challenges in developing standardized measures of performance at the federal level and suggests ways to potentially mitigate general performance measurement challenges as well as measurement complexities specific to community preparedness. It may be informative for those state, territorial, and local health departments currently implementing (or contemplating implementing) community preparedness activities and for individuals more generally engaged in performance measurement.

Building performance-based accountability with limited empirical evidence: performance measurement for public health preparedness.

Efforts to respond to performance-based accountability mandates for public health emergency preparedness have been hindered by a weak evidence base linking preparedness activities with response outcomes. We describe an approach to measure development that was successfully implemented in the Centers for Disease Control and Prevention Public Health Emergency Preparedness Cooperative Agreement. The approach leverages insights from process mapping and experts to guide measure selection, and provides mechanisms for reducing performance-irrelevant variation in measurement data. Also, issues are identified that need to be addressed to advance the science of measurement in public health emergency preparedness.