Rethinking fundamental assumptions: SUPPORT's implications for future reform. Study to Understand Prognoses and Preferences and Risks of Treatment.

BACKGROUND: The intervention in SUPPORT, the Study to Understand Prognoses and Preferences for Outcomes and Risks of Treatments, was ineffective in changing communication, decision-making, and treatment patterns despite evidence that counseling and information were delivered as planned. The previous paper in this volume shows that modest alterations in the intervention design probably did not explain the lack of substantial effects. OBJECTIVE: To explore the possibility that improved individual, patient-level decision-making is not the most effective strategy for improving end-of-life care and that improving routine practices may be more effective. DESIGN: This paper reflects our efforts to synthesize findings from SUPPORT and other sources in order to explore our conceptual models, their consistency with the data, and their leverage for change. RESULTS: Many of the assumptions underlying the model of improved decision-making are problematic. Furthermore, the results of SUPPORT suggest that implementing an effective intervention based on a normative model of shared decision-making can be quite difficult. Practice patterns and social expectations may be strong influences in shaping patients' courses of care. Innovations in system function, such as quality improvement or changing the financing incentives, may offer more powerful avenues for reform. CONCLUSIONS: SUPPORT's intervention may have failed to have an impact because strong psychological and social forces underlie present practices. System-level innovation and quality improvement in routine care may offer more powerful opportunities for improvement.

Ineffectiveness of the SUPPORT intervention: review of explanations.

BACKGROUND: The aim of the Study to Understand Prognoses and Preferences for Outcomes and Risks of Treatments -- SUPPORT -- was to improve the care of seriously ill patients by improving decision-making for patients with life-threatening illnesses. Several theories have been proposed to explain why the SUPPORT intervention was unsuccessful at improving outcomes. OBJECTIVE: To review and discuss explanations offered by others regarding why the SUPPORT intervention failed to have a discernible impact on its prespecified outcome measures. DESIGN: A descriptive review of published articles and book chapters, with synthesis of data-based and conceptual insights. METHODS: The Medline, Bioethicsline, and Ethx databases were searched for citations to SUPPORT articles between 1994 and the end of 1998. This search was supplemented by other published materials that had come to the authors' attention. RESULTS: The critiques and explanations regarding the reasons the SUPPORT intervention did not improve outcomes were catalogued and organized into 11 major categories, the first 10 of which are explored in the present study: (1) the inception cohort was biased against an effect of the intervention, (2) the intervention was not implemented as designed, (3) the intervention failed because nurses were too readily ignored, (4) the intervention was too polite, (5) the intervention presented information ineffectively, (6) the intervention did not focus on primary care physicians, (7) the intervention falsely dichotomized do not resuscitate (DNR) decisions, (8) the intervention needed more years on site or an earlier start with each patient, (9) the intervention required more appropriate outcome measures, (10) the intervention was irrelevant because usual care is not seriously flawed, (11) the conceptual model behind SUPPORT was fundamentally flawed in aiming to improve individual, patient-level decision-making as the way to improve seriously ill, hospitalized patients' experiences. CONCLUSIONS: Although some of the critiques were found to raise important concerns, we conclude in each case that the explanation offered is inadequate to explain the failure of the intervention. We urge further reflection on the fundamental assumptions that informed the design of that intervention and refer the reader to a more comprehensive treatment of that issue in the companion paper in this volume.

Hemodynamic assessment in managing the critically ill: is physician confidence warranted?

Prior to right-heart catheterization of 846 patients, 198 study physicians estimated values of pulmonary capillary wedge pressure (WP), cardiac index (Cl), and systemic vascular resistance index (VRI). The physicians also expressed their confidence in these estimates. Actual values of WP, Cl, and VRI as determined by catheterization enabled the authors to evaluate the quality of the physicians' judgments. The discrimination of the judgments was modest; areas under the ROC curves for WP, Cl, and VRI were 0.724, 0.681, and 0.656, respectively. Calculated using clinically relevant cutoff values, sensitivities were 64%, 50%, and 64%, and specificities were 71%, 75%, and 63%, respectively. Calibration of the estimates of WP, Cl, and VRI was also modest; physicians tended to overestimate low values and underestimate high values. Physicians were generally confident of their estimates, but there was no relation between confidence and accuracy. Experienced physicians were no more accurate than less experienced ones, although they were significantly more confident. The authors conclude that physicians should not use their levels of confidence in their subjective estimates of cardiac function in deciding whether to base therapy on these estimates.

Hindsight bias: an impediment to accurate probability estimation in clinicopathologic conferences.

Although clinicopathologic conferences (CPCs) have been valued for teaching differential diagnosis, their instructional value may be compromised by hindsight bias. This bias occurs when those who know the actual diagnosis overestimate the likelihood that they would have been able to predict the correct diagnosis had they been asked to do so beforehand. Evidence for the presence of the hindsight bias was sought among 160 physicians and trainees attending four CPCs. Before the correct diagnosis was announced, half of the conference audience estimated the probability that each of five possible diagnoses was correct (foresight subjects). After the correct diagnosis was announced the remaining (hindsight) subjects estimated the probability they would have assigned to each of the five possible diagnoses had they been making the initial differential diagnosis. Only 30% of the foresight subjects ranked the correct diagnosis as first, versus 50% of the hindsight subjects (p less than 0.02). Although less experienced physicians consistently demonstrated the hindsight bias, more experienced physicians succumbed only on easier cases.

The covariance decomposition of the probability score and its use in evaluating prognostic estimates. SUPPORT Investigators.

The probability score (PS) or Brier score has been used in a large number of studies in which physician judgment performance was assessed. However, the covariance decomposition of the PS has not previously been used to evaluate medical judgment. The authors introduce the technique and demonstrate it by analyzing prognostic estimates of three groups: physicians, their patients, and the patients' decision-making surrogates. The major components of the covariance decomposition--bias, slope, and scatter--are displayed in covariance graphs for each of the three groups. The decomposition reveals that whereas the physicians have the best overall estimation performance, their bias and their scatter are not always superior to those of the other two groups. This is primarily due to two factors. First, the physicians' prognostic estimates are pessimistic. Second, the patients place the large majority of their estimates in the most optimistic category, thereby achieving low scatter. The authors suggest that the calculational simplicity of this decomposition, its informativeness, and the intuitive nature of its components make it a useful tool with which to analyze medical judgment.

A national survey of psychologists who offer neuropsychological services.

We conducted a national survey of psychologists who offer neuropsychological services to determine levels of training, current practices, and views on professional issues. All subjects were listed in the National Register of Health Service Providers in Psychology and/or the American Psychological Association Directory as having some affiliation with neuropsychology. Results suggest marked diversity within the field, with the modal practitioner being minimally involved in neuropsychological activities. We also compare clinicians with varying involvement and experience in neuropsychology and analyze the concentration of practitioners by geographic regions. We compare our results to those obtained in prior surveys and discuss implications of the findings.

Neuropsychologists' training, experience, and judgment accuracy.

It is often assumed that judgment accuracy improves as clinical training and experience increase, but the few studies on this topic within neuropsychology have yielded negative findings. In an extension of prior research, we obtained information on background training and experience from a nationally representative sample of clinical neuropsychologists and had each practitioner appraise one from among a series of 10 cases. Except for a possible tendency among more experienced practitioners to overdiagnose abnormality, no systematic relations were obtained between training, experience, and accuracy across a series of neuropsychologic judgments. Comparable results were obtained when analysis was limited to the top versus bottom 20% of the sample. This and other studies raise doubt that clinical neuropsychologists train and practice under conditions conducive to experiential learning. The potential benefits of further research on experience and judgment accuracy are discussed.

Effects of task difficulty on subsequent preference for visual complexity.

Ss rated on a 1 to 13 Likert scale their preference for four environmental scenes which differed in complexity. Ss were then given 30 sec. to solve each of 10 3-, 5-, or 7-letter single-solution anagrams. Finally the Ss rated four new slides which were of the same complexity level as the slides seen during the first session. Change scores were calculated by subtracting the preference ratings at each complexity level during the first session from the rating at the corresponding complexity level during the second session. The Anagram Complexity X Slide Complexity interaction was significant: the complexity of the slides showing the most positive change scores was inversely related to the complexity of the anagrams attempted. The results were discussed in terms of optimal level of stimulation theories.

Systematic errors in medical decision making: judgment limitations.

Much of medical practice involves the exercise of such basic cognitive tasks as estimating probabilities and synthesizing information. Scientists studying cognitive processes have identified impediments to accurate performance on these tasks. Together the impediments foster "cognitive bias." Five factors that can detract from accurate probability estimation and three that impair accurate information synthesis are discussed. Examples of all eight factors are illustrated by reference to published articles. The authors suggest ways to minimize the negative influences of these factors.