Potential effectiveness of quality assurance screening using large but imperfect databases.

To examine the effect of imprecise classification of patient risk (severity of illness) on an otherwise highly accurate quality assurance screening technique, data on clinical outcomes were generated for a simulated hospital system consisting of 108 facilities treating approximately 565,000 patients a year. In these simulations, marked differences in facility size, casemix distribution, and quality of care were combined with random variations in outcome. Pooled data for all 108 facilities were used to create algorithms that combined 468 discrete patient risk classifications into either ten or three groups with broad, overlapping ranges of patient-specific risks of unfavorable clinical results. When derived algorithms were applied to independently generated facility-specific data, the ability to identify hospital systems with and without quality of care problems was maintained with ten, but not with three, risk groups. However, even three moderately heterogeneous risk groups were sufficient to preserve a high degree of sensitivity and specificity in screening for potential quality of care problems within individual facilities. Thus, outcome-based quality assurance screening can be highly accurate in actual health care situations in which only imprecise estimations of patient-specific risk can be achieved.

Potential effectiveness of comparative casemix-corrected adverse outcome rates as quality monitors in a simulated hospital system.

To facilitate outcome-based medical quality assurance, a screening technique was developed which corrected facility-specific data for casemix using mean outcome rates from pooled data to establish norms for "uniform-risk groups" of patients. A stimulated health care system (108 facilities treating over 500,000 annually) was created to evaluate this technique's ability to distinguish between systems whose adverse outcomes were determined solely by casemix and random variations and those with true differences in quality of care. Specificity and sensitivity of quality of care decisions for individual facilities also were assessed. The screening technique achieved excellent differentiation between "homogeneous" systems and those with facility-specific variations in quality of care. No more than 3% of facilities without quality of care problems were ever inaccurately labeled, unless systematic or random errors in patient risk classification were introduced. Sensitivity in detecting substandard facilities was 35% when true deviation from standard was 2.5%, and rose to virtually 100% when deviation was 25% or greater. Thus, simulation can serve as an efficient method of testing the potential performance of casemix-corrected quality assurance screening under a wide variety of circumstances.