Understanding processes of change: how some patients reveal more than others-and some groups of therapists less-about what matters in psychotherapy.

OBJECTIVE: We identify difficulties researchers encounter in psychotherapy process-outcome investigations, and we describe several limitations of the popular "variance accounted for" approach to understanding the effects of psychotherapy. METHODS & RESULTS: Using data simulations, we show how the expected correlation between an excellent measure of therapy quality and outcome would be surprisingly small (approximately .25) under conditions likely to be common in psychotherapy research. Even when we modeled conditions designed to increase the likelihood that strong process-outcome relationships would be observed, we found that the expected correlations were still only in the modest range (.38-.51). CONCLUSIONS: We discuss the implications of our analysis for the interpretation of process-outcome findings as well as for design considerations in future investigations.

Mediation analysis: a retrospective snapshot of practice and more recent directions.

R. Baron and D. A. Kenny's (1986) paper introducing mediation analysis has been cited over 9,000 times, but concerns have been expressed about how this method is used. The authors review past and recent methodological literature and make recommendations for how to address 3 main issues: association, temporal order, and the no omitted variables assumption. The authors briefly visit the topics of reliability and the confirmatory-exploratory distinction. In addition, to provide a sense of the extent to which the earlier literature had been absorbed into practice, the authors examined a sample of 50 articles from 2002 citing R. Baron and D. A. Kenny and containing at least 1 mediation analysis via ordinary least squares regression. A substantial proportion of these articles included problematic reporting; as of 2002, there appeared to be room for improvement in conducting such mediation analyses. Future literature reviews will demonstrate the extent to which the situation has improved.

Sequence of improvement in depressive symptoms across cognitive therapy and pharmacotherapy.

BACKGROUND: The authors examined the patterns of improvement in cognitive and vegetative symptoms of major depression in individuals treated with cognitive therapy (CT) or pharmacotherapy (PT). METHOD: Outpatients diagnosed with major depressive disorder (n=180) were randomized to receive either CT or PT. Cognitive and vegetative symptoms of major depression were measured by the Beck Depression Inventory-II at baseline and regularly throughout 16 weeks of treatment. RESULTS: Multivariate hierarchical linear modeling demonstrated the same patterns of change over time for cognitive and vegetative symptoms within CT and within PT. LIMITATIONS: Self-report measures may not be sufficiently specific to capture subtle differences in improvements between vegetative and cognitive symptoms. CONCLUSIONS: These results are consistent with Beck's [Beck, A.T., 1984, November. Cognition and theory [Letter to the editor]. Arch. Gen. Psychiatry 41, 1112-1114.] hypothesis that CT and PT have a similar site of action, which when targeted, results in changes in both cognitive and vegetative features.

Mediation Analysis with Survival Outcomes: Accelerated Failure Time vs. Proportional Hazards Models.

OBJECTIVE: Survival time is an important type of outcome variable in treatment research. Currently, limited guidance is available regarding performing mediation analyses with survival outcomes, which generally do not have normally distributed errors, and contain unobserved (censored) events. We present considerations for choosing an approach, using a comparison of semi-parametric proportional hazards (PH) and fully parametric accelerated failure time (AFT) approaches for illustration. METHOD: We compare PH and AFT models and procedures in their integration into mediation models and review their ability to produce coefficients that estimate causal effects. Using simulation studies modeling Weibull-distributed survival times, we compare statistical properties of mediation analyses incorporating PH and AFT approaches (employing SAS procedures PHREG and LIFEREG, respectively) under varied data conditions, some including censoring. A simulated data set illustrates the findings. RESULTS: AFT models integrate more easily than PH models into mediation models. Furthermore, mediation analyses incorporating LIFEREG produce coefficients that can estimate causal effects, and demonstrate superior statistical properties. Censoring introduces bias in the coefficient estimate representing the treatment effect on outcome-underestimation in LIFEREG, and overestimation in PHREG. With LIFEREG, this bias can be addressed using an alternative estimate obtained from combining other coefficients, whereas this is not possible with PHREG. CONCLUSIONS: When Weibull assumptions are not violated, there are compelling advantages to using LIFEREG over PHREG for mediation analyses involving survival-time outcomes. Irrespective of the procedures used, the interpretation of coefficients, effects of censoring on coefficient estimates, and statistical properties should be taken into account when reporting results.

The Personalized Advantage Index: translating research on prediction into individualized treatment recommendations. A demonstration.

BACKGROUND: Advances in personalized medicine require the identification of variables that predict differential response to treatments as well as the development and refinement of methods to transform predictive information into actionable recommendations. OBJECTIVE: To illustrate and test a new method for integrating predictive information to aid in treatment selection, using data from a randomized treatment comparison. METHOD: Data from a trial of antidepressant medications (N = 104) versus cognitive behavioral therapy (N = 50) for Major Depressive Disorder were used to produce predictions of post-treatment scores on the Hamilton Rating Scale for Depression (HRSD) in each of the two treatments for each of the 154 patients. The patient's own data were not used in the models that yielded these predictions. Five pre-randomization variables that predicted differential response (marital status, employment status, life events, comorbid personality disorder, and prior medication trials) were included in regression models, permitting the calculation of each patient's Personalized Advantage Index (PAI), in HRSD units. RESULTS: For 60% of the sample a clinically meaningful advantage (PAI≥3) was predicted for one of the treatments, relative to the other. When these patients were divided into those randomly assigned to their "Optimal" treatment versus those assigned to their "Non-optimal" treatment, outcomes in the former group were superior (d = 0.58, 95% CI .17-1.01). CONCLUSIONS: This approach to treatment selection, implemented in the context of two equally effective treatments, yielded effects that, if obtained prospectively, would rival those routinely observed in comparisons of active versus control treatments.

Bias resulting from the use of 'assay sensitivity' as an inclusion criterion for meta-analysis.

Assay sensitivity has been proposed as a criterion for including psychiatric clinical outcome studies in meta-analyses. The authors assess the performance of assay sensitivity as a method for determining study appropriateness for meta-analysis by calculating expected standard drug vs placebo effect sizes for various combinations of high quality and flawed studies. In the absence of flawed studies, expected effect sizes are close to unbiased only when sample sizes are very large. In the presence of flawed studies, expected effect sizes tend to be substantially biased except under simultaneous conditions of high power, a large proportion of flawed studies, and a population standard vs placebo effect size of flawed studies considerably lower than that of high quality studies. The authors conclude that this method is not robust and can lead to serious bias. Unless it can be shown that specific conditions hold, assay sensitivity should not be used to make quality judgments of studies.

Reducing exposure of clinical research subjects to placebo treatments.

The ethics of pill placebo and placebo psychotherapy conditions in clinical research are controversial. Even when not life threatening, mental disorders dramatically diminish the quality of life. Pill-placebo conditions in drug treatment research have been justified on the grounds that a placebo versus standard drug comparison is necessary to test the quality of the study, viz., the assay sensitivity method. The assay sensitivity method of judging study quality, however, results in misclassification of the quality of some studies, leading to bias in effect size estimation in the context of meta-analyses. This bias is of particular concern in relation to studies comparing psychotherapies to psychotropic drugs, which are conducted outside of the Food and Drug Administration (FDA) context. In cases in which control conditions may be justified on grounds other than as essential elements of an assay sensitivity test, statistical methods to reduce the number of study participants exposed to placebo should be strongly considered. Of the methods available, group sequential methods are the most widely used. Group sequential methods involve successive looks at accumulating data, with rules for terminating a trial (or an arm of a trial) early if results are strong enough.