Envisioning an improved research strategy for detecting moderators/mediators in intervention studies: A commentary on McClure et al.

The goal of moderator/mediator research in treatment evaluation is to provide guidance to clinicians to choose the best treatment for each patient with a disorder (moderators), and to advise on its optimal protocol or implementation (mediators): personalized/precision medicine. McClure et al. report a systematic review of studies addressing moderators/mediators of the treatment effect of digital interventions for eating disorders, finding no robust moderators or mediators. They attribute this failure to methodological problems, an assessment with which I concur. The focus of this discussion is to clarify which methodological approaches are not likely to be successful, and to envision a research strategy encompassing both hypothesis-generating (exploratory) and hypothesis-testing approaches likely to produce better results not only for eating disorders, but also for all medical treatments.

Messages for Clinicians: Moderators and Mediators of Treatment Outcome in Randomized Clinical Trials.

Many problems in randomized clinical trial design, execution, analysis, presentation and interpretation stem in part from an inadequate understanding of the roles of moderators and mediators of treatment outcome. As a result, 1) the results of clinical research are slow to have an impact on clinical decision making and thus to benefit patients; 2) it is difficult for clinicians or patients to apply randomized clinical trial results comparing two treatments (treatment versus control); 3) when such trials are conducted at various sites, the results often do not replicate; 4) when the results influence clinical decision making, the results clinicians obtain do not match what researchers report; and 5) the treatment effects comparing treatment and control conditions, particularly for psychiatric treatments, often seem trivial. In this review article, the author reviews and integrates the methodological literature concerning dealing with covariates in trials to emphasize their impact on clinical decision making. The goal of trials should ultimately be to establish who should get the treatment condition rather than the control condition (moderators) and to determine how to obtain the best outcomes with whatever is the preferred treatment (mediators). The author makes recommendations to clinicians as to which trials might best be ignored and which carefully considered, and urges clinical researchers to focus on studies best designed to reduce the burden of mental illness on patients.

Methodological comments: McFarlane et al.

Does the McFarlane et al study provide a model for practical trials? Unfortunately, not. The methodological problems are here reviewed and used as a springboard to set out methodological criteria that might define such a model.

A mediator effect size in randomized clinical trials.

To understand the process by which a treatment (T) achieves an effect on outcome (O) and thus to improve the effect of T on O, it is vital to detect mediators, to compare the impact of different mediators, and to develop hypotheses about the causal factors (all mediators) linking T and O. An index is needed to facilitate interpretation of the potential clinical importance of a mediator (M) of choice of T on treatment O in randomized clinical trials (RCTs). Ideally such a mediator effect size should (1) be invariant under any rescaling of M and O consistent with the model used, and (2) reflect the difference between the overall observed effect of T on O and what the maximal effect of T on O could be were the association between T and M broken. A mediator effect size is derived first for the traditional linear model, and then more generally for any categorical (ordered or non-ordered) potential mediator. Issues such as the problem of multiple treatments, outcomes and mediators, and of causal inferences, and the correspondence between this approach and earlier ones, are discussed. Illustrations are given of the application of the approach.

The reliability of clinical diagnoses: state of the art.

Reliability of clinical diagnosis is essential for good clinical decision making as well as productive clinical research. The current review emphasizes the distinction between a disorder and a diagnosis and between validity and reliability of diagnoses, and the relationships that exist between them. What is crucial is that reliable diagnoses are essential to establishing valid diagnoses. The present review discusses the theoretical background underlying the evaluation of diagnoses, possible designs of reliability studies, estimation of the reliability coefficient, the standards for assessment of reliability, and strategies for improving reliability without compromising validity.

Moderators of remission with interpersonal counselling or drug treatment in primary care patients with depression: randomised controlled trial.

BACKGROUND: Despite depressive disorders being very common there has been little research to guide primary care physicians on the choice of treatment for patients with mild to moderate depression. AIMS: To evaluate the efficacy of interpersonal counselling compared with selective serotonin reuptake inhibitors (SSRIs), in primary care attenders with major depression and to identify moderators of treatment outcome. METHOD: A randomised controlled trial in nine centres (DEPICS, Australian New Zealand Clinical Trials Registry number: ACTRN12608000479303). The primary outcome was remission of the depressive episode (defined as a Hamilton Rating Scale for Depression score ≤7 at 2 months). Daily functioning was assessed using the Work and Social Adjustment Scale. Logistic regression models were used to identify moderators of treatment outcome. RESULTS: The percentage of patients who achieved remission at 2 months was significantly higher in the interpersonal counselling group compared with the SSRI group (58.7% v. 45.1%, P = 0.021). Five moderators of treatment outcome were found: depression severity, functional impairment, anxiety comorbidity, previous depressive episodes and smoking habit. CONCLUSIONS: We identified some patient characteristics predicting a differential outcome with pharmacological and psychological interventions. Should our results be confirmed in future studies, these characteristics will help clinicians to define criteria for first-line treatment of depression targeted to patients' characteristics.

Toward sound objective evaluation of clinical measures.

The quality of all clinical decision-making, as well as power and precision in clinical research results, depends fundamentally on the quality of the measures used. Yet evaluations of the quality of clinical measures likely to be used either in clinic or research applications are difficult to execute or to critique because the criteria for judging such studies are so ill-defined. Here a set of guidelines is proposed, modeled on CONSORT guidelines for randomized clinical trials, first defining the terms often inconsistently used in the research literature and then identifying certain errors that seem to recur in evaluation studies.

Discovering, comparing, and combining moderators of treatment on outcome after randomized clinical trials: a parametric approach.

No one treatment is likely to affect all patients with a disorder in the same way. A treatment highly effective for some may be ineffective or even harmful for others. Statistically significant or not, the effect sizes of many treatments tend to be small. Consequently, emphasis in clinical research is gradually shifting (1) to increased focus on effect sizes and (2) to discovery and documentation of moderators of treatment effect on outcome in randomized clinical trials, that is, personalized medicine, in which individual differences between patients are explicitly acknowledged. How to test a null hypothesis of moderation of treatment outcome is reasonably well known. The focus here is on how, under parametric assumptions, to define the strength of moderation, that is, a moderator effect size, either for scientific purposes or for assessment of clinical significance, in order to compare moderators and choose among them and to develop a composite moderator, which might more strongly moderate the effect of a treatment on outcome than any single moderator that might ultimately provide guidance for clinicians as to whom to prescribe what treatment.

Determining gene moderation of environmental risk factors for a mental disorder: a "perfect storm" of methodological problems.

For most of the twentieth century, the focus was on "nature" versus "nurture", i.e. genetic versus environmental effects on disorders. Now it is increasingly recognized that a disorder may reflect genes and environments "working together". A gene may moderate an environmental risk factor, it may be mediated by an environmental risk factor. The environmental risk factor may be proxy to the gene, and the two may be independent risk factors. Which of these situations pertain influences both subsequent research and clinical and policy decision-making. However, recent meta-analyses attempting to confirm the Caspi et al. (Science, 301, 386-389, 2003) hypothesis indicate that the methodological issues relating to establishing specifically a moderating effect of a gene on an environmental factor are not well understood. The discussion here concerns the definition of "moderator", how it is distinct from other ways in which gene and environment can "work together", the methods needed to establish such a moderator, and the public health significance of such efforts.