Program-specific transplant rate ratios: Association with allocation priority at listing and posttransplant outcomes.

The Scientific Registry of Transplant Recipients (SRTR) is considering more prominent reporting of program-specific adjusted transplant rate ratios (TRRs). To enable more useful reporting of TRRs, SRTR updated the transplant rate models to adjust explicitly for components of allocation priority. We evaluated potential associations between TRRs and components of allocation priority that could indicate programs' ability to manipulate TRRs by denying or delaying access to low-priority candidates. Despite a strong association with unadjusted TRRs, we found no candidate-level association between the components of allocation priority and adjusted TRRs. We found a strong program-level association between median laboratory Model for End-stage Liver Disease (MELD) score at listing and program-specific adjusted TRRs (r = .37; P < .001). The program-level association was likely confounded by regional differences in donor supply/demand and listing practices. In kidney transplantation, higher program-specific adjusted TRRs were weakly associated with better adjusted posttransplant outcomes (r = -.14; P = .035) and lower adjusted waitlist mortality rate ratios (r = -.15; P = .022), but these associations were absent in liver, lung, and heart transplantation. Program-specific adjusted TRRs were unlikely to be improved by listing candidates with high allocation priority and can provide useful information for transplant candidates and programs.

Potential Implications of Recent and Proposed Changes in the Regulatory Oversight of Solid Organ Transplantation in the United States.

Every 6 months, the Scientific Registry of Transplant Recipients (SRTR) publishes evaluations of every solid organ transplant program in the United States, including evaluations of 1-year patient and graft survival. The Centers for Medicare & Medicaid Services (CMS) and the Organ Procurement and Transplantation Network (OPTN) Membership and Professional Standards Committee (MPSC) use SRTR's 1-year evaluations for regulatory review of transplant programs. Concern has been growing that the regulatory scrutiny of transplant programs with lower-than-expected outcomes is harmful, causing programs to undertake fewer high-risk transplants and leading to unnecessary organ discards. As a result, CMS raised its threshold for a "Condition-Level Deficiency" designation of observed relative to expected 1-year graft or patient survival from 1.50 to 1.85. Exceeding this threshold in the current SRTR outcomes report and in one of the four previous reports leads to scrutiny that may result in loss of Medicare funding. For its part, OPTN is reviewing a proposal from the MPSC to also change its performance criteria thresholds for program review, to review programs with "substantive clinical differences." We review the details and implications of these changes in transplant program oversight.

Effects of High-Risk Kidneys on Scientific Registry of Transplant Recipients Program Quality Reports.

There is a perception that transplanting high-risk kidneys causes programs to be identified as underperforming, thereby increasing the frequency of discards and diminishing access to transplant. Thus, the Organ Procurement and Transplantation Network (OPTN) has considered excluding transplants using kidneys from donors with high Kidney Donor Profile Index (KDPI) scores (≥0.85) when assessing program performance. We examined whether accepting high-risk kidneys (KDPI ≥0.85) for transplant yields worse outcome evaluations. Despite a clear relationship between KDPI and graft failure and mortality, there was no relationship between a program's use of high-KDPI kidneys and poor performance evaluations after risk adjustment. Excluding high-KDPI donor transplants from the June 2015 evaluations did not alter the programs identified as underperforming, because in every case underperforming programs also had worse-than-expected outcomes among lower-risk donor transplants. Finally, we found that hypothetically accepting and transplanting additional kidneys with KDPI similar to that of kidneys currently discarded would not adversely affect program evaluations. Based on the study findings, there is no evidence that programs that accept higher-KDPI kidneys are at greater risk for low performance evaluations, and risk aversion may limit access to transplant for candidates while providing no measurable benefit to program evaluations.

External validation of the estimated posttransplant survival score for allocation of deceased donor kidneys in the United States.

The US kidney allocation system adopted in 2013 will allocate the best 20% of deceased donor kidneys (based on the kidney donor risk index [KDRI]) to the 20% of waitlisted patients with the highest estimated posttransplant survival (EPTS). The EPTS has not been externally validated, raising concerns as to its suitability to discriminate between kidney transplant candidates. We examined EPTS using data from the Australia and New Zealand Dialysis and Transplant (ANZDATA) Registry. We included 4983 adult kidney-only deceased donor transplants over 2000-2011. We constructed three Cox models for patient survival: (i) EPTS alone; (ii) EPTS plus donor age, hypertension and HLA-DR mismatch; and (iii) EPTS plus log(KDRI). All models demonstrated moderately good discrimination, with Harrell's C statistics of 0.67, 0.68 and 0.69, respectively. These results are virtually identical to the internal validation that demonstrated a c-statistic of 0.69. These results provide external validation of the EPTS as a moderately good tool for discriminating posttransplant survival of adult kidney-only transplant recipients.

A scientific registry of transplant recipients bayesian method for identifying underperforming transplant programs.

In response to recommendations from a recent consensus conference and from the Committee of Presidents of Statistical Societies, the Scientific Registry of Transplant Recipients explored the use of Bayesian hierarchical, mixed-effects models in assessing transplant program performance in the United States. Identification of underperforming centers based on 1-year patient and graft survival using a Bayesian approach was compared with current observed-to-expected methods. Fewer small-volume programs (<10 transplants per 2.5-year period) were identified as underperforming with the Bayesian method than with the current method, and more mid-volume programs (10-249 transplants per 2.5-year period) were identified. Simulation studies identified optimal Bayesian-based flagging thresholds that maximize true positives while holding false positive flagging rates to approximately 5% regardless of program volume. Compared against previous program surveillance actions from the Organ Procurement and Transplantation Network Membership and Professional Standards Committee, the Bayesian method would have reduced the number of false positive program identifications by 50% for kidney, 35% for liver, 43% for heart and 57% for lung programs, while preserving true positives for, respectively, 96%, 71%, 58% and 83% of programs identified by the current method. We conclude that Bayesian methods to identify underperformance improve identification of programs that need review while minimizing false flags.

Bayesian methods for assessing transplant program performance.

Based on recommendations from a recent consensus conference and a report commissioned by the Centers for Medicare & Medicaid Services to the Committee of Presidents of Statistical Societies, the Scientific Registry of Transplant Recipients (SRTR) plans to adopt Bayesian methods for assessing transplant program performance. Current methods for calculating program-specific reports (PSRs) often generate implausible point estimates of program performance, wide confidence intervals and underpowered conventional statistical tests. Although technically correct, these methods produce statistical summaries that are prone to misinterpretation. The Bayesian approach assumes that performance of most programs is about average and few programs perform much better or much worse than average; thus, strong evidence is required to conclude that performance is extremely good or poor. In Bayesian statistics, inference is performed via a posterior probability distribution, which reflects both the available data and prior beliefs about what model parameter values are most likely. In the PSRs, the posterior distribution of a program-specific hazard ratio will show whether a program is likely to be performing better or worse than average. Bayesian-derived PSRs will be available for preview by programs on the private SRTR website in mid-2014 and will likely replace current methods for public reporting in early 2015.

New quality monitoring tools provided by the Scientific Registry of Transplant Recipients: CUSUM.

The Scientific Registry of Transplant Recipients (SRTR) has been providing data on transplant program performance through semi-annual release of program-specific reports (PSRs). A consensus conference held in February 2012 recommended that SRTR also supply transplant programs with tools such as the cumulative sum (CUSUM) technique to facilitate quality assessment and performance improvement. SRTR developed the process, methodologies, programming code and web capabilities necessary to bring the CUSUM charts to the community, and began releasing them to all liver, kidney, heart and lung programs in July 2013. Observed-minus-expected CUSUM charts provide a general picture of a program's performance (all-cause graft failure and mortality within the first-year posttransplant) over a 3-year period; one-sided charts can determine when performance appears to be sufficiently worrisome to warrant action by the program. CUSUM charts are intended for internal quality improvement by allowing programs to better track performance in near-real time and day to day, and will not be used to indicate whether a program will be flagged for review. The CUSUM technique is better suited for real-time quality monitoring than the current PSRs in allowing monthly outcomes monitoring and presenting data recorded as recently as 2 months before the release of the CUSUM charts.

The equitable allocation of deceased donor lungs for transplant in children in the United States.

On June 5, 2013, a US Federal Court ordered a temporary restraining order to allow two children within the court's jurisdiction to be registered on the adolescent lung transplant waiting list. On June 10, 2013, the Organ Procurement and Transplantation Network's Executive Committee altered lung allocation policy to offer candidates aged younger than 12 years greater access to adult lungs at the discretion of the national Lung Review Board. The Scientific Registry of Transplant Recipients reviewed trends over time in deceased donor lung transplant waitlist mortality and transplant rates, comparing children and adults. Mortality rates of candidates active on the waiting list have been higher for children aged 0-5 years, but have not differed for children aged 6-11 years compared with adolescents aged 12-17 years or adults aged 18 years or older. Transplant rates among active waitlist candidates have been comparable across all age groups. Thus, there is little evidence that the allocation system led to differences in waitlist mortality or transplant rates for children compared with adults. However, these comparisons are difficult to interpret given that current policies likely led to unaccounted differences in the severity of illness at the time of listing.

Predicting end-stage renal disease after liver transplant.

Few equations have been developed to predict end-stage renal disease (ESRD) after deceased donor liver transplant. This retrospective observational cohort study analyzed all adult deceased donor liver transplant recipients in the Scientific Registry of Transplant Recipients (SRTR) database, 1995-2010. The prediction equation for ESRD was developed using candidate predictor variables available in SRTR after implementation of the allocation policy based on the model for end-stage liver disease. ESRD was defined as initiation of maintenance dialysis therapy, kidney transplant or registration on the kidney transplant waiting list. We used Cox proportional hazard models to develop separate equations for assessing risk of ESRD by 6 months posttransplant and between 6 months and 5 years posttransplant. Variables in the 6-month equation included recipient age, history of diabetes, history of dialysis before liver transplant, history of malignancy, body mass index, serum creatinine and liver donor risk index. Variables in the 6-month to 5-year equation included recipient race, history of diabetes, hepatitis C status, serum albumin, serum bilirubin and serum creatinine. The prediction equations have good calibration and discrimination (C statistics 0.74-0.78). We have produced risk prediction equations that can be used to aid in understanding the risk of ESRD after liver transplant.

Optimizing the program-specific reporting of pancreas transplant outcomes.

The Scientific Registry of Transplant Recipients is charged with providing program-specific reports for organ transplant programs in the United States. Monitoring graft survival for pancreas transplant programs has been problematic as there are three different pancreas transplant procedures that may have different outcomes, and analyzing them separately reduces events and statistical power. We combined two consecutive 2.5-year cohorts of transplant recipients to develop Cox proportional hazards models predicting outcomes, and tested these models in the second 2.5-year cohort. We used separate models for 1- and 3-year graft and patient survival for each transplant type: simultaneous pancreas-kidney (SPK), pancreas after kidney (PAK) and pancreas transplant alone (PTA). We first built a predictive model for each pancreas transplant type, and then pooled the transplant types within centers to compare total observed events with total predicted events. Models for 1-year pancreas graft and patient survival yielded C statistics of 0.65 (95% confidence interval, 0.63-0.68) and 0.66 (0.61-0.72), respectively, comparable to C statistics for 1-year patient and graft survival for other organ transplants. Model calibration (Hosmer-Lemeshow method) was also acceptable. We conclude that pooling the results of SPK, PAK and PTA can produce potentially useful models for reporting program-specific pancreas transplant outcomes.

Report of a consensus conference on transplant program quality and surveillance.

Public reports of organ transplant program outcomes by the US Scientific Registry of Transplant Recipients have been both groundbreaking and controversial. The reports are used by regulatory agencies, private insurance providers, transplant centers and patients. Failure to adequately adjust outcomes for risk may cause programs to avoid performing transplants involving suitable but high-risk candidates and donors. At a consensus conference of stakeholders held February 13-15, 2012, the participants recommended that program-specific reports be better designed to address the needs of all users. Additional comorbidity variables should be collected, but innovation should also be protected by excluding patients who are in approved protocols from statistical models that identify underperforming centers. The potential benefits of hierarchical and mixed-effects statistical methods should be studied. Transplant centers should be provided with tools to facilitate quality assessment and performance improvement. Additional statistical methods to assess outcomes at small-volume transplant programs should be developed. More data on waiting list risk and outcomes should be provided. Monitoring and reporting of short-term living donor outcomes should be enhanced. Overall, there was broad consensus that substantial improvement in reporting outcomes of transplant programs in the United States could and should be made in a cost-effective manner.