Identification of outliers and positive deviants for healthcare improvement: looking for high performers in hypoglycemia safety in patients with diabetes.

ABSTRACT

BACKGROUND: The study objectives were to determine (1) how statistical outliers exhibiting low rates of diabetes overtreatment performed on a reciprocal measure - rates of diabetes undertreatment; and (2) the impact of different criteria on high performing outlier status. METHODS: The design was serial cross-sectional, using yearly Veterans Health Administration (VHA) administrative data (2009-2013). Our primary outcome measure was facility rate of HbA1c overtreatment of diabetes in patients at risk for hypoglycemia. Outlier status was assessed by using two approaches calculating a facility outlier value within year, comparator group, and A1c threshold while incorporating at risk population sizes; and examining standardized model residuals across year and A1c threshold. Facilities with outlier values in the lowest decile for all years of data using more than one threshold and comparator or with time-averaged model residuals in the lowest decile for all A1c thresholds were considered high performing outliers. RESULTS: Using outlier values, three of the 27 high performers from 2009 were also identified in 2010-2013 and considered outliers. There was only modest overlap between facilities identified as top performers based on three thresholds A1c < 6%, A1c < 6.5%, and A1c < 7%. There was little effect of facility complexity or regional Veterans Integrated Service Networks (VISNs) on outlier identification. Consistent high performing facilities for overtreatment had higher rates of undertreatment (A1c > 9%) than VA average in the population of patients at high risk for hypoglycemia. CONCLUSIONS: Statistical identification of positive deviants for diabetes overtreatment was dependent upon the specific measures and approaches used. Moreover, because two facilities may arrive at the same results via very different pathways, it is important to consider that a "best" practice may actually reflect a separate "worst" practice.