Quality improvement lessons learned from National Implementation of the "Patient Safety Events in Community Care: Reporting, Investigation, and Improvement Guidebook".

OBJECTIVE: To evaluate nationwide implementation of a Guidebook designed to standardize safety practices across VA-delivered and VA-purchased care (i.e., Community Care) and identify lessons learned and strategies to improve them. DATA SOURCES AND STUDY SETTING: Qualitative data collected from key informants at 18 geographically diverse VA facilities across 17 Veterans Integrated Services Networks (VISNs). STUDY DESIGN: We conducted semi-structured interviews from 2019 to 2022 with VISN Patient Safety Officers (PSOs) and VA facility patient safety and quality managers (PSMs and QMs) and VA Facility Community Care (CC) staff to assess lessons learned by examining organizational contextual factors affecting Guidebook implementation based on the Consolidated Framework for Implementation Research (CFIR). DATA COLLECTION/EXTRACTION METHODS: Interviews were conducted virtually with 45 facility staff and 10 VISN PSOs. Using directed content analysis, we identified CFIR factors affecting implementation. These factors were mapped to the Expert Recommendations for Implementing Change (ERIC) strategy compilation to identify lessons learned that could be useful to our operational partners in improving implementation processes. We met frequently with our partners to discuss findings and plan next steps. PRINCIPAL FINDINGS: Six CFIR constructs were identified as both facilitators and barriers to Guidebook implementation: (1) planning for implementation; (2) engaging key knowledge holders; (3) available resources; (4) networks and communications; (5) culture; and (6) external policies. The two CFIR constructs that were only barriers included: (1) cosmopolitanism and (2) executing implementation. CONCLUSIONS: Our findings suggest several important lessons: (1) engage all collaborators involved in implementation; (2) ensure end-users have opportunities to provide feedback; (3) describe collaborators' purpose and roles/responsibilities clearly at the start; (4) communicate information widely and repeatedly; and (5) identify how multiple high priorities can be synergistic. This evaluation will help our partners and key VA leadership to determine next steps and future strategies for improving Guidebook implementation through collaboration with VA staff.

Standardizing Patient Safety Event Reporting between Care Delivered or Purchased by the Veterans Health Administration (VHA).

BACKGROUND: Increasing community care (CC) use by veterans has introduced new challenges in providing integrated care across the Veterans Health Administration (VHA) and CC. VHA's well-recognized patient safety program has been particularly challenging for CC staff to adopt and implement. To standardize VHA safety practices across both settings, VHA implemented the Patient Safety Guidebook in 2018. The authors compared national- and facility-level trends in VHA and CC safety event reporting post-Guidebook implementation. METHODS: In this retrospective study using patient safety event data from VHA's event reporting system (2020-2022), the research team examined trends in patient safety events, adverse events, close calls (near misses), and recovery rates (ratio of close calls to adverse events plus close calls) in VHA and CC using linear regression models to determine whether the average changes in VHA and CC safety events at the national and facility levels per quarter were significant. RESULTS: A total of 499,332 safety events were reported in VHA and CC. Although VHA patient safety event trends were not significant (p > 0.05), there was a significant negative trend for adverse events (p = 0.02) and positive trends for close calls (p = 0.003) and recovery rates (p = 0.004). In CC there were significant negative trends for patient safety events and adverse events (p = 0.02) and a significant positive trend for recovery rates (p = 0.03). There was less variation in VHA than in CC facilities with significant decreases (for example, interquartile ranges in VHA and CC were 0.03 vs. 0.05, respectively). CONCLUSION: Fluctuations in different safety events over time were likely due to the disruption of care caused by COVID-19 as well as organizational factors. Notably, the increases in recovery rates reflect less staff focus on harmful events and more attention to close calls (preventable events). Although safety practice adoption from VHA to CC was feasible, additional implementation strategies are needed to sustain standardized safety reporting across settings.

Facilitators and Barriers to Verifying Penicillin Allergies in a Veteran Nursing Home Population.

BACKGROUND: Unconfirmed penicillin allergies are common and may contribute to adverse outcomes, especially in frail older patients. Evidence-based clinical pathways for evaluating penicillin allergies have been effectively and safely applied in selected settings, but not in nursing home populations. OBJECTIVE: To identify potential facilitators and barriers to implementing a strategy to verify penicillin allergies in Veterans Health Administration nursing homes, known as Community Living Centers (CLCs). METHODS: We conducted semistructured interviews with staff, patients, and family members at 1 CLC to assess their understanding of penicillin allergies and receptiveness to verifying the allergy. We also asked staff about the proposed allergy assessment strategy, including willingness to delabel by history and feasibility of performing oral challenges or skin testing on their unit. RESULTS: From 24 interviews (11 front-line staff, 4 leadership, 3 patients, 6 family members), we identified several facilitators or barriers. Staff recognized the importance of allergy verification and were willing to support and assist in implementing verification strategies. The CLC residents were willing to have their allergy status verified. However, some family members expressed reluctance to verifying their relative's allergy status owing to safety concerns. Front-line staff also expressed concern over having the necessary resources, including time and expertise, to implement the strategy. Staff suggested involving clinical pharmacists and educating staff, patients, and family members as ways to overcome these barriers. CONCLUSIONS: Concerns about safety and staff resources are important potential barriers to implementing verification strategies. Involvement of pharmacists and education of both staff and patients and family members will be important components of any successful intervention.

Development and Validation of Quality Measures for Testosterone Prescribing.

Context: Accurate measures to assess appropriateness of testosterone prescribing are needed to improve prescribing practices. Objective: This work aimed to develop and validate quality measures around the initiation and monitoring of testosterone prescribing. Methods: This retrospective cohort study comprised a national cohort of male patients receiving care in the Veterans Health Administration who initiated testosterone during January or February 2020. Using laboratory data and diagnostic codes, we developed 9 initiation and 7 monitoring measures. These were based on the current Endocrine Society guidelines supplemented by expert opinion and prior work. We chose measures that could be operationalized using national VA electronic health record (EHR) data. We assessed criterion validity for these 16 measures by manual review of 142 charts. Main outcome measures included positive and negative predictive values (PPVs, NPVs), overall accuracy (OA), and Matthews Correlation Coefficients (MCCs). Results: We found high PPVs (>78%), NPVs (>98%), OA (≥94%), and MCCs (>0.85) for the 10 measures based on laboratory data (5 initiation and 5 monitoring). For the 6 measures relying on diagnostic codes, we similarly found high NPVs (100%) and OAs (≥98%). However, PPVs for measures of acute conditions occurring before testosterone initiation (ie, acute myocardial infarction or stroke) or new conditions occurring after initiation (ie, prostate or breast cancer) PPVs were much lower (0% to 50%) due to few or no cases. Conclusion: We developed several valid EHR-based quality measures for assessing testosterone-prescribing practices. Deployment of these measures in health care systems can facilitate identification of quality gaps in testosterone-prescribing and improve care of men with hypogonadism.

Beyond Medication Adherence: The Role of Patients' Beliefs and Life Context in Blood Pressure Control.

Objective: Despite numerous interventions to address adherence to antihypertensive medications, continued high rates of uncontrolled blood pressure (BP) suggest a need to better understand patient factors beyond adherence associated with BP control. We examined how patients' BP-related beliefs, and aspects of life context affect BP control, beyond medication adherence. Methods: We conducted a cross-sectional telephone survey of primary care patients with hypertension between 2010 and 2011 (N=103; 93 had complete data on all variables and were included in the regression analyses). We assessed patient sociodemographics (including race/ethnicity), medication adherence, BP-related beliefs, aspects of life context, and used clinical BP assessments. Results: Regression models including sociodemographics, medication adherence, and either beliefs or context consistently predicted BP control. Adding context after beliefs added no predictive value while adding beliefs after context significantly predicted BP control. Practical Implications: Results suggest that when clinicians must choose a dimension on which to intervene, focusing on beliefs would be the most fruitful approach to effecting change in BP control.

The Patient Safety Indicator Perioperative Pulmonary Embolism or Deep Vein Thrombosis: Is there associated surveillance bias in the Veterans Health Administration?

BACKGROUND: Studies disagree whether surveillance bias is associated with perioperative venous thromboembolism (VTE) performance measures. A prior VA study used a chart-based outcome; no studies have used the fully specified administrative data-based AHRQ Patient Safety Indicator, PSI-12, as their primary outcome. If surveillance bias were present, we hypothesized that inpatient surveillance rates would be associated with higher PSI-12 rates, but with lower post-discharge VTE rates. METHODS: Using VA data, we examined Pearson correlations between hospital-level VTE imaging rates and risk-adjusted PSI-12 rates and post-discharge VTE rates. To determine the robustness of findings, we conducted several sensitivity analyses. RESULTS: Hospital imaging rates were positively correlated with both PSI-12 (r = 0.24, p = 0.01) and post-discharge VTE rates (r = 0.16, p = 0.09). Sensitivity analyses yielded similar findings. CONCLUSIONS: Like the prior VA study, we found no evidence of PSI-12-related surveillance bias. Given the use of PSI-12 in nationwide measurement, these findings warrant replication using similar methods in the non-VA setting.

Tailoring an educational program on the AHRQ Patient Safety Indicators to meet stakeholder needs: lessons learned in the VA.

BACKGROUND: Given that patient safety measures are increasingly used for public reporting and pay-for performance, it is important for stakeholders to understand how to use these measures for improvement. The Agency for Healthcare Research and Quality (AHRQ) Patient Safety Indicators (PSIs) are one particularly visible set of measures that are now used primarily for public reporting and pay-for-performance among both private sector and Veterans Health Administration (VA) hospitals. This trend generates a strong need for stakeholders to understand how to interpret and use the PSIs for quality improvement (QI). The goal of this study was to develop an educational program and tailor it to stakeholders' needs. In this paper, we share what we learned from this program development process. METHODS: Our study population included key VA stakeholders involved in reviewing performance reports and prioritizing and initiating quality/safety initiatives. A pre-program formative evaluation through telephone interviews and web-based surveys assessed stakeholders' educational needs/interests. Findings from the formative evaluation led to development and implementation of a cyberseminar-based program, which we tailored to stakeholders' needs/interests. A post-program survey evaluated program participants' perceptions about the PSI educational program. RESULTS: Interview data confirmed that the concepts we had developed for the interviews could be used for the survey. Survey results informed us on what program delivery mode and content topics were of high interest. Six cyberseminars were developed-three of which focused on two content areas that were noted of greatest interest: learning how to use PSIs for monitoring trends and understanding how to interpret PSIs. We also used snapshots of VA PSI reports so that participants could directly apply learnings. Although initial interest in the program was high, actual attendance was low. However, post-program survey results indicated that perceptions about the program were positive. CONCLUSIONS: Conducting a formative evaluation was a highly important process in program development. The useful information that we collected through the interviews and surveys allowed us to tailor the program to stakeholders' needs and interests. Our experiences, particularly with the formative evaluation process, yielded valuable lessons that can guide others when developing and implementing similar educational programs.

Does Surveillance Bias Influence the Validity of Measures of Inpatient Complications? A Systematic Review.

Surveillance bias may threaten the accuracy of inpatient complication measures. A systematic literature review was conducted to examine whether surveillance bias influences the validity of selected Patient Safety Indicator- and health care associated infection-related measures. Ten venous thromboembolism (VTE) articles were identified: 7 trauma related, 3 postoperative, and 1 central line-associated bloodstream infection (CLABSI) article. Nine VTE articles found positive associations between deep vein thrombosis imaging and VTE diagnoses. Because imaging also may be symptom driven, most studies performed additional analyses to corroborate findings. Six trauma-related and 2 postoperative VTE studies concluded that surveillance bias was present, the latter based on circumstantial evidence. The non-VTE study found a significant positive correlation between surveillance aggressiveness and intensive care unit CLABSI rates. Even considering VTE, relatively little is known about the impact of surveillance bias on inpatient complication measures. Given the implications of misclassifying hospitals on quality, this issue requires further investigation using more direct measurement methods.

Results of a Regional Effort to Improve Warfarin Management.

BACKGROUND: Improved anticoagulation control with warfarin reduces adverse events and represents a target for quality improvement. No previous study has described an effort to improve anticoagulation control across a health system. OBJECTIVE: To describe the results of an effort to improve anticoagulation control in the New England region of the Veterans Health Administration (VA). METHODS: Our intervention encompassed 8 VA sites managing warfarin for more than 5000 patients in New England (Veterans Integrated Service Network 1 [VISN 1]). We provided sites with a system to measure processes of care, along with targeted audit and feedback. We focused on processes of care associated with site-level anticoagulation control, including prompt follow-up after out-of-range international normalized ratio (INR) values, minimizing loss to follow-up, and use of guideline-concordant INR target ranges. We used a difference-in-differences (DID) model to examine changes in anticoagulation control, measured as percentage time in therapeutic range (TTR), as well as process measures and compared VISN 1 sites with 116 VA sites located outside VISN 1. RESULTS: VISN 1 sites improved on TTR, our main indicator of quality, from 66.4% to 69.2%, whereas sites outside VISN 1 improved from 65.9% to 66.4% (DID 2.3%, P < 0.001). Improvement in TTR correlated strongly with the extent of improvement on process-of-care measures, which varied widely across VISN 1 sites. CONCLUSIONS: A regional quality improvement initiative, using performance measurement with audit and feedback, improved TTR by 2.3% more than control sites, which is a clinically important difference. Improving relevant processes of care can improve outcomes for patients receiving warfarin.

A Systematic Review of Patient Safety Measures in Adult Primary Care.

Safety measure development has focused on inpatient care despite outpatient visits far outnumbering inpatient admissions. Some measures are clearly identified as outpatient safety measures when published, yet outcomes from quality improvement studies also may be useful measures. The authors conducted a systematic review of the literature to identify published articles detailing safety measures applicable to adult primary care. A total of 21 articles were identified, providing specifications for 182 safety measures. Each measure was classified into one of 6 outpatient safety dimensions: medication management, sentinel events, care coordination, procedures and treatment, laboratory testing and monitoring, and facility structures/resources. Compared to the multitude of available inpatient safety measures, the number of existing adult primary care measures is low. The measures identified by this systematic review may yield further insight into the breadth of safety events causing harm in primary care, while also identifying areas of patient safety in primary care that may be understudied.

Does adding clinical data to administrative data improve agreement among hospital quality measures?

BACKGROUND: Hospital performance measures based on patient mortality and readmission have indicated modest rates of agreement. We examined if combining clinical data on laboratory tests and vital signs with administrative data leads to improved agreement with each other, and with other measures of hospital performance in the nation's largest integrated health care system. METHODS: We used patient-level administrative and clinical data, and hospital-level data on quality indicators, for 2007-2010 from the Veterans Health Administration (VA). For patients admitted for acute myocardial infarction (AMI), heart failure (HF) and pneumonia we examined changes in hospital performance on 30-d mortality and 30-d readmission rates as a result of adding clinical data to administrative data. We evaluated whether this enhancement yielded improved measures of hospital quality, based on concordance with other hospital quality indicators. RESULTS: For 30-d mortality, data enhancement improved model performance, and significantly changed hospital performance profiles; for 30-d readmission, the impact was modest. Concordance between enhanced measures of both outcomes, and with other hospital quality measures - including Joint Commission process measures, VA Surgical Quality Improvement Program (VASQIP) mortality and morbidity, and case volume - remained poor. CONCLUSIONS: Adding laboratory tests and vital signs to measure hospital performance on mortality and readmission did not improve the poor rates of agreement across hospital quality indicators in the VA. INTERPRETATION: Efforts to improve risk adjustment models should continue; however, evidence of validation should precede their use as reliable measures of quality.

Using Harm-Based Weights for the AHRQ Patient Safety for Selected Indicators Composite (PSI-90): Does It Affect Assessment of Hospital Performance and Financial Penalties in Veterans Health Administration Hospitals?

OBJECTIVE: To assess whether hospital profiles for public reporting and pay-for-performance, measured by the Agency for Healthcare Research and Quality (AHRQ) Patient Safety for Selected Indicators (PSI-90) composite measure, were affected by using the recently developed harm-based weights. DATA SOURCES/STUDY SETTING: Retrospective analysis of 2012-2014 data from the Veterans Health Administration (VA). STUDY DESIGN: The AHRQ PSI software (v5.0) was applied to obtain the original volume-based PSI-90 scores for 132 acute-care hospitals. We constructed a modified PSI-90 using the harm-based weights developed by AHRQ. We compared hospital profiles for public reporting and pay-for-performance between these two PSI-90s and assessed patterns in these changes. PRINCIPAL FINDINGS: The volume-based and the harm-based PSI-90s were strongly correlated (r = 0.67, p < .0001). The use of the harm-based PSI-90 had a relatively small impact on public reporting (i.e., 5 percent hospitals changed categorization), but it had a much larger impact on pay-for-performance (e.g., 15 percent of hospitals would have faced different financial penalties under the Medicare Hospital-Acquired Condition Reduction Program). Because of changes in weights of specific PSIs, hospital profile changes occurred systematically. CONCLUSIONS: Use of the harm-based weights in PSI-90 has the potential to significantly change payments under pay-for-performance programs. Policy makers should carefully develop transition plans for guiding hospitals through changes in any quality metrics used for pay-for-performance.

Do Acute Myocardial Infarction and Heart Failure Readmissions Flagged as Potentially Preventable by the 3M Potentially Preventable Readmissions Software Have More Process-of-Care Problems?

BACKGROUND: The 3M Potentially Preventable Readmissions (3M-PPR) software matches clinically related index admission and readmission diagnoses that may signify in-hospital or postdischarge quality problems. To assess whether the PPR algorithm identifies preventable readmissions, we compared processes of care between PPR software-flagged and nonflagged cases. METHODS AND RESULTS: Using 2006 to 2010 national VA administrative data, we identified acute myocardial infarction and heart failure discharges associated with 30-day all-cause readmissions, then flagged cases (PPR-Yes/PPR-No) using the 3M-PPR software. To assess care quality, we abstracted medical records of 100 readmissions per condition using tools containing explicit processes organized into admission work-up, in-hospital evaluation/treatment, discharge readiness, postdischarge period. We derived quality scores, scaled to a maximum of 25 per section (maximum total score=100) and compared cases on total and section-specific mean scores. For acute myocardial infarction, 77 of 100 cases were flagged as PPR-Yes. Section quality scores were highest for in-hospital evaluation/treatment (20.5±2.8) and lowest for postdischarge care (6.8±9.1). Total and section-related mean scores did not differ by PPR status; respective PPR-Yes versus PPR-No total scores were 61.6±11.1 and 60.4±9.4; P=0.98. For heart failure, 86 of 100 cases were flagged as PPR-Yes. Section scores were highest for discharge readiness (18.8±2.4) and lowest for postdischarge care (7.3±8.1). Like acute myocardial infarction, total and section-related mean scores did not differ by PPR status; PPR-Yes versus PPR-No total scores were 61.2±10.8 and 63.4±7.0, respectively; P=0.47. CONCLUSIONS: Among VA acute myocardial infarction and heart failure readmissions, the 3M-PPR software does not distinguish differences in case-level quality of care. Whether 3M-PPR software better identifies preventable readmissions by using other methods to capture poorly documented processes or performing different comparisons requires further study.

Hospital Characteristics and the Agency for Healthcare Research and Quality Inpatient Quality Indicators: A Systematic Review.

BACKGROUND: The Agency for Healthcare Research and Quality Inpatient Quality Indicators (IQIs) include inpatient mortality for selected procedures and medical conditions. They have assumed an increasingly prominent role in hospital comparisons. Healthcare delivery and policy-related decisions need to be driven by reliable research that shows associations between hospital characteristics and quality of inpatient care delivered. OBJECTIVES: To systematically review the literature on associations between hospital characteristics and IQIs. METHODS: We systematically searched PubMed and gray literature (2000-2012) for studies relevant to 14 hospital characteristics and 17 IQIs. We extracted data for study characteristics, IQIs analyzed, and hospital characteristics (e.g., teaching status, bed size, patient volume, rural vs. urban location, and nurse staffing). RESULTS: We included 16 studies, which showed few significant associations. Four hospital characteristics (higher hospital volume, higher nurse staffing, urban vs. rural status, and higher hospital financial resources) had statistically significant associations with lower mortality and selected IQIs in approximately half of the studies. For example, there were no associations between nurse staffing and four IQIs; however, approximately 50% of studies showed a statistically significant relationship between nurse staffing and lower mortality for six IQIs. For two hospital characteristics-higher bed size and disproportionate share percentage-all statistically significant associations had higher mortality. Five hospital characteristics (teaching status, system affiliation, ownership, minority-serving hospitals, and electronic health record status) had some studies with significantly positive and some with significantly negative associations, and many studies with no association. CONCLUSIONS: We found few associations between hospital characteristics and mortality IQIs. Differences in study methodology, coding across hospitals, and hospital case-mix adjustment may partly explain these results. Ongoing research will evaluate potential mechanisms for the identified associations.

Measuring readmissions after surgery: do different methods tell the same story?

BACKGROUND: Readmission is widely used as a quality metric to assess hospital performance. However, different methods to calculate readmissions may produce various results, leading to differences in classification with respect to hospital performance. This study compared 2 commonly used approaches to measure surgical readmissions: the 30-day all-cause hospital-wide readmissions (HWRs) and the potentially preventable readmissions (PPRs). METHODS: We examined the correlation between hospitals' risk-adjusted HWR and PPR rates and whether there was agreement in categorizing hospital performance between these measures among 111 hospitals with inpatient surgical programs in the Veterans Health Administration. RESULTS: We found that hospitals' HWR and PPR rates were highly correlated (r = .85, P < .0001). The overall agreement between these 2 methods in categorizing hospital performance was 82% for all surgeries, 82% for colectomy, 84% for coronary bypass, and 87% for hip/knee replacement, respectively. CONCLUSIONS: Despite differences in methodologies, the HWR and the PPR measures provided relatively consistent perceptions of hospitals' performance on surgical readmissions.

Does Use of a Hospital-wide Readmission Measure Versus Condition-specific Readmission Measures Make a Difference for Hospital Profiling and Payment Penalties?

BACKGROUND: The Centers for Medicare and Medicaid Services (CMS) use public reporting and payment penalties as incentives for hospitals to reduce readmission rates. In contrast to the current condition-specific readmission measures, CMS recently developed an all-condition, 30-day all-cause hospital-wide readmission measure (HWR) to provide a more comprehensive view of hospital performance. OBJECTIVES: We examined whether assessment of hospital performance and payment penalties depends on the readmission measure used. RESEARCH DESIGN: We used inpatient data to examine readmissions for patients discharged from VA acute-care hospitals from Fiscal Years 2007-2010. We calculated risk-standardized 30-day readmission rates for 3 condition-specific measures (heart failure, acute myocardial infarction, and pneumonia) and the HWR measure, and examined agreement between the HWR measure and each of the condition-specific measures on hospital performance. We also assessed the effect of using different readmission measures on hospitals' payment penalties. RESULTS: We found poor agreement between the condition-specific measures and the HWR measure on those hospitals identified as low or high performers (eg, among those hospitals classified as poor performers by the heart failure readmission measure, only 28.6% were similarly classified by the HWR measure). We also found differences in whether a hospital would experience payment penalties. The HWR measure penalized only 60% of those hospitals that would have received penalties based on at least 1 of the condition-specific measures. CONCLUSIONS: The condition-specific measures and the HWR measure provide a different picture of hospital performance. Future research is needed to determine which measure aligns best with CMS's overall goals to reduce hospital readmissions and improve quality.

Partnering With VA Stakeholders to Develop a Comprehensive Patient Safety Data Display: Lessons Learned From the Field.

Health care systems are increasingly burdened by the large numbers of safety measures currently being reported. Within the Veterans Administration (VA), most safety reporting occurs within organizational silos, with little involvement by the frontline users of these measures. To provide a more integrated picture of patient safety, the study team partnered with multiple VA stakeholders and engaged potential frontline users at 2 hospitals to develop a Guiding Patient Safety (GPS) tool. The GPS is currently in its fourth generation; once approval is obtained from senior leadership, implementation will begin. Stakeholders were enthusiastic about the GPS's user-friendly format, comprehensive content, and potential utility for improving safety. These findings suggest that stakeholder engagement is a critical first step in the development of tools that will more likely be used by frontline users. Policy makers and researchers may consider adopting this innovative partnered-research model in developing future national initiatives to deliver meaningful programs to frontline users.

Do pneumonia readmissions flagged as potentially preventable by the 3M PPR software have more process of care problems? A cross-sectional observational study.

BACKGROUND: In the USA, administrative data-based readmission rates such as the Centers for Medicare and Medicaid Services' all-cause readmission measures are used for public reporting and hospital payment penalties. To improve this measure and identify better quality improvement targets, 3M developed the Potentially Preventable Readmissions (PPRs) measure. It matches clinically related index admission and readmission diagnoses that may indicate readmissions resulting from admission- or post-discharge-related quality problems. OBJECTIVE: To examine whether PPR software-flagged pneumonia readmissions are associated with poorer quality of care. METHODS: Using a retrospective observational study design and Veterans Health Administration (VA) data, we identified pneumonia discharges associated with 30-day readmissions, and then flagged cases as PPR-yes or PPR-no using the PPR software. To assess quality of care, we abstracted electronic medical records of 100 random readmissions using a tool containing explicit care processes organised into admission work-up, in-hospital evaluation/treatment, discharge readiness and post-discharge period. We derived quality scores, scaled to a maximum of 25 per section (maximum total score=100) and compared cases by total and section-specific mean scores using t tests and effect size (ES) to characterise the clinical significance of findings. RESULTS: Our abstraction sample was selected from 11,278 pneumonia readmissions (readmission rate=16.5%) during 1 October 2005-30 September 2010; 77% were flagged as PPR-yes. Contrary to expectations, total and section mean quality scores were slightly higher, although non-significantly, among PPR-yes (N=77) versus PPR-no (N=23) cases (respective total scores, 71.2±8.7 vs 65.8±11.5, p=0.14); differences demonstrated ES >0.30 overall and for admission work-up and post-discharge period sections. CONCLUSIONS: Among VA pneumonia readmissions, PPR categorisation did not produce the expected quality of care findings. Either PPR-yes cases are not more preventable, or preventability assessment requires other data collection methods to capture poorly documented processes (eg, direct observation).