Two-State Collaborative Study of a Multifaceted Intervention to Decrease Ventilator-Associated Events.

OBJECTIVES: Ventilator-associated events are associated with increased mortality, prolonged mechanical ventilation, and longer ICU stay. Given strong national interest in improving ventilated patient care, the National Institute of Health and Agency for Healthcare Research and Quality funded a two-state collaborative to reduce ventilator-associated events. We describe the collaborative's impact on ventilator-associated event rates in 56 ICUs. DESIGN: Longitudinal quasi-experimental study. SETTING: Fifty-six ICUs at 38 hospitals in Maryland and Pennsylvania from October 2012 to March 2015. INTERVENTIONS: We organized a multifaceted intervention to improve adherence with evidence-based practices, unit teamwork, and safety culture. Evidence-based interventions promoted by the collaborative included head-of-bed elevation, use of subglottic secretion drainage endotracheal tubes, oral care, chlorhexidine mouth care, and daily spontaneous awakening and breathing trials. Each unit established a multidisciplinary quality improvement team. We coached teams to establish comprehensive unit-based safety programs through monthly teleconferences. Data were collected on rounds using a common tool and entered into a Web-based portal. MEASUREMENTS AND RESULTS: ICUs reported 69,417 ventilated patient-days of intervention compliance observations and 1,022 unit-months of ventilator-associated event data. Compliance with all evidence-based interventions improved over the course of the collaborative. The quarterly mean ventilator-associated event rate significantly decreased from 7.34 to 4.58 cases per 1,000 ventilator-days after 24 months of implementation (p = 0.007). During the same time period, infection-related ventilator-associated complication and possible and probable ventilator-associated pneumonia rates decreased from 3.15 to 1.56 and 1.41 to 0.31 cases per 1,000 ventilator-days (p = 0.018, p = 0.012), respectively. CONCLUSIONS: A multifaceted intervention was associated with improved compliance with evidence-based interventions and decreases in ventilator-associated event, infection-related ventilator-associated complication, and probable ventilator-associated pneumonia. Our study is the largest to date affirming that best practices can prevent ventilator-associated events.

Clinician advice to quit smoking among seniors.

OBJECTIVE: Little smoking research in the past 20years includes persons 50 and older; herein we describe patterns of clinician cessation advice to US seniors, including variation by Medicare beneficiary characteristics. METHOD: In 2012-4, we analyzed 2010 Consumer Assessment of Healthcare Providers and Systems (CAHPS) survey data from Medicare beneficiaries over age 64 (n=346,674). We estimated smoking rates and the proportion of smokers whose clinicians encouraged cessation. RESULTS: 12% of male and 8% of female respondents aged 65 and older smoke. The rate decreases with age (14% of 65-69, 3% of 85+) and education (12-15% with no high school degree, 5-6% with BA+). Rates are highest among American Indian/Alaskan Native (16%), multiracial (14%), and African-American (13%) seniors, and in the Southeast (14%). Only 51% of smokers say they receive cessation advice "always" or "usually" at doctor visits, with advice more often given to the young, those in low-smoking regions, Asians, and women. For all results cited p<0.05. CONCLUSIONS: Smoking cessation advice to seniors is variable. Providers may focus on groups or areas in which smoking is less common or when they are most comfortable giving advice. More consistent interventions are needed, including cessation advice from clinicians.

Specialties differ in which aspects of doctor communication predict overall physician ratings.

BACKGROUND: Effective doctor communication is critical to positive doctor-patient relationships and predicts better health outcomes. Doctor communication is the strongest predictor of patient ratings of doctors, but the most important aspects of communication may vary by specialty. OBJECTIVE: To determine the importance of five aspects of doctor communication to overall physician ratings by specialty. DESIGN: For each of 28 specialties, we calculated partial correlations of five communication items with a 0-10 overall physician rating, controlling for patient demographics. PATIENTS: Consumer Assessment of Healthcare Providers and Systems Clinician and Group (CG-CAHPS®) 12-month Survey data collected 2005-2009 from 58,251 adults at a 534-physician medical group. MAIN MEASURES: CG-CAHPS includes a 0 ("Worst physician possible") to 10 ("Best physician possible") overall physician rating. Five doctor communication items assess how often the physician: explains things; listens carefully; gives easy-to-understand instructions; shows respect; and spends enough time. KEY RESULTS: Physician showing respect was the most important aspect of communication for 23/28 specialties, with a mean partial correlation (0.27, ranging from 0.07 to 0.44 across specialties) that accounted for more than four times as much variance in the overall physician rating as any other communication item. Three of five communication items varied significantly across specialties in their associations with the overall rating (p < 0.05). CONCLUSIONS: All patients valued respectful treatment; the importance of other aspects of communication varied significantly by specialty. Quality improvement efforts by all specialties should emphasize physicians showing respect to patients, and each specialty should also target other aspects of communication that matter most to their patients. The results have implications for improving provider quality improvement and incentive programs and the reporting of CAHPS data to patients. Specialists make important contributions to coordinated patient care, and thus customized approaches to measurement, reporting, and quality improvement efforts are important.

Experiences of care among Medicare beneficiaries with ESRD: Medicare Consumer Assessment of Healthcare Providers and Systems (CAHPS) survey results.

BACKGROUND: Patients with end-stage renal disease (ESRD) have special health needs; little is known about their care experiences. STUDY DESIGN: Secondary analysis of 2009-2010 Medicare Consumer Assessment of Healthcare Providers and Systems (CAHPS) data, using representative random samples of Medicare beneficiaries. Description of Medicare beneficiaries with ESRD and investigation of differences in patient experiences by sociodemographic characteristics and coverage type. SETTING & PARTICIPANTS: Data were collected from 823,564 Medicare beneficiaries (3,794 with ESRD) as part of the Medicare CAHPS survey, administered by mail with telephone follow-up of nonrespondents. PREDICTOR: ESRD status, age, education, self-reported general and mental health status, race/ethnicity, sex, Medicare coverage type, state of residence, and other demographic measures. OUTCOMES: 6 composite measures of patient experience in 4 care domains (access to care, physician communication, customer service, and access to prescription drugs and drug information) and 4 ratings (overall care, personal physician, specialist physician, and prescription drug plan). RESULTS: Patients with ESRD reported better care experiences than non-ESRD beneficiaries for 7 of 10 measures (P < 0.05) after adjustment for patient characteristics, geography, and coverage type, although to only a small extent (adjusted mean difference, <3 points [scale, 0-100]). Black patients with ESRD and less educated patients were more likely than other patients with ESRD to report poor experiences. LIMITATIONS: Inability to distinguish patient experiences of care for different treatment modalities. CONCLUSIONS: On average, beneficiaries with ESRD report patient experiences that are at least as positive as non-ESRD beneficiaries. However, black and less educated patients with ESRD reported worse experiences than other ESRD patients. Stratified reporting of patient experience by race/ethnicity or education in patients with ESRD can be used to monitor this disparity. Physician choice and confidence and trust in physicians may be particularly important for patients with ESRD.

Measuring success for health care quality improvement interventions.

BACKGROUND: The lack of a standard measure of quality improvement (QI) success and the use of subjective or self-reported measures of QI success has constrained efforts to formally evaluate QI programs and to understand how the various contextual factors impact QI success. OBJECTIVES: The objective of this study was to assess how best to measure "QI success" by comparing self-reported and externally rated measures of QI success. RESEARCH DESIGN: We performed a retrospective evaluation that analyzed data on different measures of QI success for organizations after their staff completed the QI training. SUBJECTS: The sample included 30 organizations whose staff had received QI training during 2006-2008, and who had used this training to carry out at least some subsequent QI initiative in their organizations. MEASURES: We developed 2 measures of self-reported QI success based on survey responses and 4 externally rated measures of QI success based on outcome data provided by the participating organizations in addition to qualitative data generated from the interviews. RESULTS: We found some variation in the mean scores of the different QI success measures and only moderate to small correlations between the self-report and externally rated QI measures. CONCLUSIONS: This study confirms that there are important differences between self-reported and externally rated measures of QI success and provides researchers with a methodology and criteria to externally rate measures of QI success.

Advancing the science of patient safety.

Despite a decade's worth of effort, patient safety has improved slowly, in part because of the limited evidence base for the development and widespread dissemination of successful patient safety practices. The Agency for Healthcare Research and Quality sponsored an international group of experts in patient safety and evaluation methods to develop criteria to improve the design, evaluation, and reporting of practice research in patient safety. This article reports the findings and recommendations of this group, which include greater use of theory and logic models, more detailed descriptions of interventions and their implementation, enhanced explanation of desired and unintended outcomes, and better description and measurement of context and of how context influences interventions. Using these criteria and measuring and reporting contexts will improve the science of patient safety.

Using health information technology-related performance measures and tools to improve chronic care.

BACKGROUND: The American Medical Association led a collaborative initiative to explore opportunities for improving the quality of outpatient chronic care through the use of nationally endorsed clinical performance measures and tools. The measures and tools focused on adult diabetes, major depressive disorder, chronic stable coronary artery disease, heart failure, hypertension, and asthma. METHODS: The RAND Corporation conducted an independent, formative assessment of the initiative's four pilot activities using the Context-Input-Process-Product evaluation model. RESULTS: Pilots 1 and 2 demonstrated the feasibility and value of implementing performance measures and tools in practices with electronic health information systems, while highlighting the difficulty of using them in practices with paper-based systems and in community-based models, where multiple stakeholders are expected to share patient data. Pilot 3 illustrated the usefulness of validating performance measures before their use for internal quality improvement or external reporting. Pilot 4 documented the challenges involved in exporting clinical performance data from a physician practice to external entities for multiple potential uses. DISCUSSION: Improving the quality of chronic care through clinical performance measurement, data aggregation, and reporting will require expanded use of clinical performance measures for both internal quality improvement and pay-for-performance; integrating electronic health records (EHRs) or electronic-based registries into more physician offices; more accurate measurement and documentation of diagnoses and care procedures; EHR products that make it easier to capture certain types of information; and simplified, standardized processes for performance data extraction and exporting.

Shortcomings in Medicare bonus payments for physicians in underserved areas.

This study examines trends in Medicare spending for basic payments and bonus payments for physician services provided to beneficiaries residing in nonmetropolitan counties. For our analysis, we used Medicare Part B physician/supplier claims data for 1992, 1994, 1996, and 1998. Payments under the congressionally mandated bonus payment program acccounted for less than 1 percent of expenditures for physician services in nonmetropolitan, underserved counties. Physician payments increased from 1992 to 1998, while bonus payments increased through 1996 but then declined by 13 percent by 1998. The share of bonus payments to primary care physicians declined throughout the decade, but the share for primary care services increased.

Effect of CAHPS performance information on health plan choices by Iowa Medicaid beneficiaries.

This evaluation tested if Consumer Assessment of Health Plans Study (CAHPS) information on plan performance affected health plan choices by new beneficiaries in Iowa Medicaid. New cases entering Medicaid in selected counties during February through May 2000 were assigned randomly to experimental or control groups. The control group received standard Medicaid enrollment materials, and the experimental group received these materials plus a CAHPS report. We found that CAHPS information did not affect health plan choices by Iowa Medicaid beneficiaries, similar to previously reported findings for New Jersey Medicaid. However, it did affect plan choice in an earlier laboratory experiment. The value of this information may be limited to a subset of receptive consumers who actively study information received, even then only when (1) ratings of available plans differ greatly, (2) ratings differ from prior beliefs about plan quality, and (3) reports are easy to understand.

Effects of CAHPS health plan performance information on plan choices by New Jersey Medicaid beneficiaries.

OBJECTIVE: To assess the effects of CAHPS health plan performance information on plan choices and decision processes by New Jersey Medicaid beneficiaries. DATA SOURCES/STUDY SETTING: The study sample was a statewide sample of all new Medicaid cases that chose Medicaid health plans during April 1998. The study used state data on health maintenance organization (HMO) enrollments and survey data for a subset of these cases. STUDY DESIGN: An experimental design was used, with new Medicaid cases randomly assigned to experimental or control groups. The experimental group received a CAHPS report along with the standard enrollment materials, and the control group did not. DATA COLLECTION: The HMO enrollment data were obtained from the state in June 1998, and evaluation survey data were collected from July to October 1998. PRINCIPAL FINDINGS: No effects of CAHPS information on HMO choices were found for the total sample. Further examination revealed that only about half the Medicaid cases said they received and read the plan report and there was an HMO with dominant Medicaid market share but low CAHPS performance scores. The subset of cases who read the report and did not choose this dominant HMO chose HMOs with higher CAHPS scores, on average, than did those in an equivalent control group. CONCLUSIONS: Health plan performance information can influence plan choices by Medicaid beneficiaries, but will do so only if they actually read it. These findings suggest a need for enhancing dissemination of the information as well as further education to encourage informed choices.

CAHPS in practice: the Iowa demonstration.

This article evaluates the usefulness of the Consumer Assessment of Health Plan Study (CAHPS) surveys and reports in a demonstration and evaluation with three health care purchasers in Iowa. The CAHPS survey detected significant differences between health plans in several domains of health care. Both paper- and Web-based consumer reports were developed and distributed to consumers and stakeholders. There was some agreement on the value of these reports, but areas for improvement were noted.

Measurement of implementation components and contextual factors in a two-state healthcare quality initiative to reduce ventilator-associated pneumonia.

OBJECTIVE: To develop and field test an implementation assessment tool for assessing progress of hospital units in implementing improvements for the prevention of ventilator-associated pneumonia (VAP) in a two-state collaborative, including data on actions implemented by participating teams and contextual factors that may influence their efforts. Using the data collected, learn how implementation actions can be improved and analyze effects of implementation progress on outcome measures. DESIGN: We developed the tool as an interview protocol that included quantitative and qualitative items addressing actions on the Comprehensive Unit-based Safety Program (CUSP) and clinical interventions for use in guiding data collection via telephone interviews. SETTING: We conducted interviews with leaders of improvement teams from units participating in the two-state VAP prevention initiative. METHODS: We collected data from 43 hospital units as they implemented actions for the VAP initiative and performed descriptive analyzes of the data with comparisons across the 2 states. RESULTS: Early in the VAP prevention initiative, most units had made only moderate progress overall in using many of the CUSP actions known to support their improvement processes. For contextual factors, a relatively small number of barriers were found to have important negative effects on implementation progress (in particular, barriers related to workload and time issues). We modified coaching provided to the unit teams to reinforce training in weak spots that the interviews identified. CONCLUSION: These assessments provided important new knowledge regarding the implementation science of quality improvement projects, including feedback during implementation, and give a better understanding of which factors most affect implementation.

A retrospective evaluation of the Perfecting Patient Care University training program for health care organizations.

This study evaluated how the Perfecting Patient Care (PPC) University, a quality improvement (QI) training program for health care leaders and clinicians, affected the ability of organizations to improve the health care they provide. This training program teaches improvement methods based on Lean concepts and principles of the Toyota Production System and is offered in several formats. A retrospective evaluation was performed that gathered data on training, other process factors, and outcomes after staff completed the PPC training. A majority of respondents reported gaining QI competencies and cultural achievements from the training. Organizations had high average scores for the success measures of "outcomes improved" and "sustainable monitoring" but lower scores for diffusion of QI efforts. Total training dosage was significantly associated with the measures of QI success. This evaluation provides evidence that organizations gained the PPC competencies and cultural achievements and that training dosage is a driver of QI success.

Care experiences of managed care Medicare enrollees near the end of life.

OBJECTIVES: To compare reports about care experiences of individuals who died within 1 year of survey with reports of those who did not. DESIGN: Medicare Advantage (MA) Consumer Assessment of Healthcare Providers and Systems (CAHPS) surveys asked about care experiences. Survey completion dates were linked to Social Security Administration death records to identify enrollees dying within 1 year of survey completion. Propensity-score weighting combined with regression-based case-mix adjustment was used to compare these individuals' experiences with experiences of those who were alive 1 year later. SETTING: Nationally representative sample of MA enrollees. PARTICIPANTS: Four hundred two thousand five hundred ninety-three MA enrollees responding to 2008 and 2009 CAHPS Surveys. MEASUREMENTS: Outcomes were five care ratings (plan, prescription drug coverage, doctor, specialists, care) and five composite measures of care (getting needed care, getting care quickly, doctor communication, getting drugs, getting drug information). Analyses were adjusted for age, sex, race and ethnicity, education, Medicaid status, geographic region, and several health status measures. RESULTS: Twelve thousand one hundred two enrollees (3%) died within 1 year of survey completion (near-end-of-life group). Those enrollees reported slightly better experiences than other enrollees with respect to getting care quickly (+2%, P < .001) and gave slightly higher ratings for their plans (+1%, P = .02) and prescription drug coverage (+1%, P < .001). There were no measures of participant experience for which the near-end-of-life group reported worse experiences than other enrollees. CONCLUSION: Contrary to analyses based on retrospective reports from surviving relatives after an individual's death, MA enrollees' reports about care within 1 year of death were as good as or better than reports of other MA enrollees. Future research might investigate whether results are similar in other Medicare populations.

CMS Innovation Center Health Care Innovation Awards: Evaluation Plan.

The Center for Medicare and Medicaid Innovation within the Centers for Medicare & Medicaid Services (CMS) has funded 108 Health Care Innovation Awards, funded through the Affordable Care Act, for applicants who proposed compelling new models of service delivery or payment improvements that promise to deliver better health, better health care, and lower costs through improved quality of care for Medicare, Medicaid, and Children's Health Insurance Program enrollees. CMS is also interested in learning how new models would affect subpopulations of beneficiaries (e.g., those eligible for Medicare and Medicaid and complex patients) who have unique characteristics or health care needs that could be related to poor outcomes. In addition, the initiative seeks to identify new models of workforce development and deployment, as well as models that can be rapidly deployed and have the promise of sustainability. This article describes a strategy for evaluating the results. The goal for the evaluation design process is to create standardized approaches for answering key questions that can be customized to similar groups of awardees and that allow for rapid and comparable assessment across awardees. The evaluation plan envisions that data collection and analysis will be carried out on three levels: at the level of the individual awardee, at the level of the awardee grouping, and as a summary evaluation that includes all awardees. Key dimensions for the evaluation framework include implementation effectiveness, program effectiveness, workforce issues, impact on priority populations, and context. The ultimate goal is to identify strategies that can be employed widely to lower cost while improving care.

Multiple patient safety events within a single hospitalization: a national profile in US hospitals.

The objective was to examine co-occurrence of iatrogenic events in US hospitals. Using Agency for Healthcare Research and Quality patient safety indicators (PSIs), the authors defined multiple patient safety events (MPSEs) as the occurrence of multiple PSIs during a single hospitalization. The National Inpatient Sample was analyzed to estimate the national prevalence of MPSEs, risk factors for MPSEs, and the average length of stay and average hospital charges associated with MPSEs. MPSEs occurred in approximately 1 in every 1000 hospitalizations, affecting more than 30 000 patients in 2004. Significant risk factors for MPSEs include age, black race, Medicare coverage, and treatment at urban teaching hospitals. Compared with all admissions, the average length of stay for MPSE admissions was 4 times longer, and the average charge for MPSE admissions was 8 times greater. Despite the low prevalence, MPSEs affect large numbers of hospital patients in America. MPSEs have distinct characteristics and are far more resource intensive than hospital admissions generally.

How event reporting by US hospitals has changed from 2005 to 2009.

CONTEXT: Information is needed on the performance of hospitals' adverse-event reporting systems and the effects of national patient-safety initiatives, including the Patient Safety and Quality Improvement Act (PSQIA) of 2005. Results are presented of a 2009 survey of a sample of non-federal US hospitals and changes between 2005 and 2009 are examined. METHODS: The Adverse Event Reporting System survey was fielded in 2005 and 2009 using a mixed-mode design with stratified random samples of non-federal US hospitals; risk managers were respondents. Response rates were 81% in 2005 and 79% in 2009. RESULTS: Virtually all hospitals reported they had centralised adverse-event-reporting systems. However, scores on four performance indexes suggested that hospitals have not effectively implemented key components of reporting systems. Average index scores improved somewhat between 2005 and 2009 for supportive environment (0.7 increase; p<0.05) and types of staff reporting (0.08 increase; p<0.001). Average scores did not change for timely distribution of event reports or discussion with key departments and committees. Some within-hospital inconsistencies in responses between 2005 and 2009 were found. These self-reported responses may be optimistic assessments of hospital performance. CONCLUSIONS: The 2009 survey confirmed improvement needs identified by the 2005 survey for hospitals' event reporting processes, while finding signs of progress. Optimising the use of surveys to assess the effects of national patient-safety initiatives such as PSQIA will require decreasing within-hospital variations in reporting rates.

Understanding variations in Medicare Consumer Assessment of Health Care Providers and Systems scores: California as an example.

OBJECTIVE: To understand reasons why California has lower Consumer Assessment of Healthcare Providers and Systems (CAHPS) scores than the rest of the country, including differing patterns of CAHPS scores between Medicare Advantage (MA) and fee-for-service, effects of additional demographic characteristics of beneficiaries, and variation across MA plans within California. STUDY DESIGN/DATA COLLECTION: Using 2008 CAHPS survey data for fee-for-service Medicare beneficiaries and MA members, we compared mean case mix adjusted Medicare CAHPS scores for California and the remainder of the nation. PRINCIPAL FINDINGS: California fee-for-service Medicare had lower scores than non-California fee-for-service on 11 of 14 CAHPS measures; California MA had lower scores only for physician services measures and higher scores for other measures. Adding race/ethnicity and urbanity to risk adjustment improved California standing for all measures in both MA and fee-for-service. Within the MA plans, one large plan accounted for the positive performance in California MA; other California plans performed below national averages. CONCLUSIONS: This study shows that the mix of fee-for-service and MA enrollees, demographic characteristics of populations, and plan-specific factors can all play a role in observed regional variations. Anticipating value-based payments, further study of successful MA plans could generate lessons for enhancing patient experience for the Medicare population.