Biases in research: risk factors for non-replicability in psychotherapy and pharmacotherapy research.

Replicability of findings is an essential prerequisite of research. For both basic and clinical research, however, low replicability of findings has recently been reported. Replicability may be affected by research biases not sufficiently controlled for by the existing research standards. Several biases such as researcher allegiance or selective reporting are well-known for affecting results. For psychotherapy and pharmacotherapy research, specific additional biases may affect outcome (e.g. therapist allegiance, therapist effects or impairments in treatment implementation). For meta-analyses further specific biases are relevant. In psychotherapy and pharmacotherapy research these biases have not yet been systematically discussed in the context of replicability. Using a list of 13 biases as a starting point, we discuss each bias's impact on replicability. We illustrate each bias by selective findings of recent research, showing that (1) several biases are not yet sufficiently controlled for by the presently applied research standards, (2) these biases have a pernicious effect on replicability of findings. For the sake of research credibility, it is critical to avoid these biases in future research. To control for biases and to improve replicability, we propose to systematically implement several measures in psychotherapy and pharmacotherapy research, such as adversarial collaboration (inviting academic rivals to collaborate), reviewing study design prior to knowing the results, triple-blind data analysis (including subjects, investigators and data managers/statisticians), data analysis by other research teams (crowdsourcing), and, last not least, updating reporting standards such as CONSORT or the Template for Intervention Description and Replication (TIDieR).

Changes in Object Relations over the Course of Psychodynamic Psychotherapy.

This study explores whether object relations (OR) functioning improves over the course of psychodynamic psychotherapy, and whether this improvement is related to symptom decrease as well as therapist technique. The sample consisted of 75 outpatients engaged in short-term psychodynamic psychotherapy at a university-based psychological service clinic. OR functioning was assessed pre- and post-treatment by independent raters using the Social Cognition and Object Relations Scale from in-session patient relational narratives. The Comparative Psychotherapy Process Scale was used to assess therapist activity and psychotherapy techniques early in treatment. Independent clinical ratings of global OR and psychotherapy techniques were conducted, and rater agreement was found to be in the excellent range. Regarding the results, global OR (overall quality and level of interpersonal functioning) significantly improved with large effect size after psychodynamic therapy. Change in global OR functioning was significantly and positively related to the incidence of psychodynamic techniques in early sessions, as were number of psychotherapy sessions attended. Patient self-reported reliable change in symptomatology and reliable change in global OR were significantly related as well. Multilevel model analyses confirmed pairwise correlations accounting for therapist effects on a variety of process-outcome measures, number of sessions attended, initial levels of psychiatric symptoms, employment of therapeutic techniques as well overall OR functioning at outcome. Limitations of the present study, future research directions and implications for clinical practice are also discussed. Copyright © 2016 John Wiley & Sons, Ltd. KEY PRACTITIONER MESSAGE: Psychodynamic psychotherapy seems to be effective in improving object relations functioning. Consider use of psychodynamic techniques early in treatment with patients expressing more pathological object representations. Improvements in object relations functioning during psychodynamic psychotherapy are also related to adaptive changes in patient self-reported symptomatology. Therapist effects were also present for the study. As such therapists should be mindful to assess patient change and their use of technique at several points in treatment and flexibly adjust their approach as necessary.

Is psychotherapy effective? A re-analysis of treatments for depression.

AimsThe aim of this study was to reanalyse the data from Cuijpers et al.'s (2018) meta-analysis, to examine Eysenck's claim that psychotherapy is not effective. Cuijpers et al., after correcting for bias, concluded that the effect of psychotherapy for depression was small (standardised mean difference, SMD, between 0.20 and 0.30), providing evidence that psychotherapy is not as effective as generally accepted. METHODS: The data for this study were the effect sizes included in Cuijpers et al. (2018). We removed outliers from the data set of effects, corrected for publication bias and segregated psychotherapy from other interventions. In our study, we considered wait-list (WL) controls as the most appropriate estimate of the natural history of depression without intervention. RESULTS: The SMD for all interventions and for psychotherapy compared to WL controls was approximately 0.70, a value consistent with past estimates of the effectiveness of psychotherapy. Psychotherapy was also more effective than care-as-usual (SMD = 0.31) and other control groups (SMD = 0.43). CONCLUSIONS: The re-analysis reveals that psychotherapy for adult patients diagnosed with depression is effective.

Reliability and validity of DSM-IV axis V.

OBJECTIVE: The authors investigated the reliability and convergent and discriminant validity of the DSM-IV Global Assessment of Functioning Scale and two experimental DSM-IV axis V global rating scales, the Global Assessment of Relational Functioning Scale and the Social and Occupational Functioning Assessment Scale. METHOD: Forty-four patients admitted to a university-based outpatient community clinic were rated by trained clinicians on the three DSM-IV axis V scales. Patients also completed self-report measures of DSM-IV symptoms as well as measures of relational, social, and occupational functioning. RESULTS: The Global Assessment of Functioning Scale, Global Assessment of Relational Functioning Scale, and Social and Occupational Functioning Assessment Scale all exhibited very high levels of interrater reliability. Factor analysis revealed that the Global Assessment of Relational Functioning Scale and the Social and Occupational Functioning Assessment Scale are each more related to the Global Assessment of Functioning Scale individually than they are to each other. The Global Assessment of Functioning Scale was significantly related to concurrent patient responses on the SCL-90-R global severity index. The Social and Occupational Functioning Assessment Scale was significantly related to concurrent patient responses on the SCL-90-R global severity index and to a greater degree with both the Social Adjustment Scale global score and the Inventory of Interpersonal Problems total score. Although the Global Assessment of Relational Functioning Scale was not significantly related to any of the three self-report measures, it was related to the presence of clinician-rated axis II pathology. CONCLUSIONS: The three axis V scales can be scored reliably. The Global Assessment of Relational Functioning Scale and the Social and Occupational Functioning Assessment Scale evaluate different constructs. These findings support the validity of the Global Assessment of Functioning Scale as a scale of global psychopathology; the Social and Occupational Functioning Assessment Scale as a measure of problems in social, occupational, and interpersonal functioning; and the Global Assessment of Relational Functioning Scale as an index of personality pathology. The authors discuss further refinement and use of the three axis V measures in treatment research.

A review of individual psychotherapy outcomes for adult survivors of childhood sexual abuse.

This review examines the current literature on individual psychotherapy outcomes with adult survivors of childhood sexual abuse. As compared to the literature on treatment with victims of sexual trauma in adulthood, fewer researchers have explored the field of psychotherapy outcome with this population, particularly with regard to individual treatments. For this reason, a review of psychotherapy outcome for adult survivors of child sexual abuse is needed. This review first presents issues salient to the study of treatment with this population, such as prevalence, short-term impact, and long-term sequelae of childhood sexual trauma. Next, the eight studies that have been published on individual psychotherapy for this population are presented and evaluated according to both efficacy and effectiveness criteria for a methodologically sound study. Last, directions for future research with this population include continued integration of efficacy and effectiveness methodology, use of multi-method/multi-rater assessment data, as well as further investigation of interpersonal variables such as the therapeutic alliance.

Rorschach and MMPI-2 indices of early psychotherapy termination.

This study was an investigation of the differences between 97 patients who had prematurely terminated psychotherapy (M = 1 session) and 81 who had participated in individual psychotherapy for at least 6 months and 24 sessions (M = 18 months/72 sessions) on selected Minnesota Multiphasic Personality Inventory--2 (MMPI-2) and Rorschach variables. None of the between-group comparisons using the MMPI-2 proved to be significant. However, a multivariate analysis of variance of 9 Rorschach variables in 3 conceptual categories--(a) interpersonal relatedness, (b) psychological resources versus resource demand, and (c) level of psychopathology--proved to be significant at p = .008. The Rorschach scores from the interpersonal-relational category proved to be the most robust in differentiating the 2 groups. The theoretical implications of interpersonal variables are discussed in relation to the termination and continuation of patients in psychotherapy.

Psychometric characteristics of the cluster B personality disorders under DSM-III-R and DSM-IV.

A stated goal for the latest revision of the DSM was improving the performance of the Axis II system. To evaluate the degree to which this goal was achieved, we performed a psychometric analysis of the Cluster B personality disorders (PD) as they are defined under the DSM-III-R and its successor the DSM-IV. Ninety-four patients with a primary PD diagnosis were rated for the presence of DSM-III-R and DSM-IV Cluster B PD criteria. From this symptom level data, the convergence, divergence, and internal consistency of the Cluster B criteria sets were determined. Also, kappa values were computed between the DSM-III-R and DSM-IV versions of these disorders as an index of coverage or agreement across the two systems. The results indicated that, in general, the DSM-IV Cluster B PDs represent an improvement over their DSM-III-R predecessors. However, some psychometric limitations, particularly regarding the convergence of the criteria sets, continue to be present.

The clinical utility of the MMPI-2 Antisocial Practices Content Scale.

This study explored the ability of the MMPI-2 Antisocial Practices Content Scale (ASP) to correctly classify DSM-IV Antisocial Personality Disorder (APD). ASP scores and scores on the MMPI-2 Psychopathic Deviate (Pd) scale were compared in an APD group (n = 10), a Borderline Personality Disorder group (n = 16), a Narcissistic Personality Disorder group (n = 9), an Other Personality Disorder group (representing personality disorders from Clusters A and C; n = 14), and a nonclinical population (NC; n = 67). The ASP exhibited an ability to differentiate APD from other personality disorders and was significantly correlated to DSM-IV diagnostic criteria for APD while the Pd was not. Diagnostic efficiency statistics (sensitivity, specificity, positive predictive power, negative predictive power, overall correct classification rate, and kappa) were calculated under four different conditions in a clinically relevant manner (Kessel & Zimmerman, 1993. Psychological Assessment, 53, 395-399). The results of this study illustrate the usefulness of the ASP as a tool for the assessment of antisocial attitudes, beliefs, and behaviors in individuals suspected of having APD. This information may be valuable to clinicians who are planning treatment or assessing treatment outcome for clients with APD.

The Rorschach: facts, fictions, and future.

A large body of empirical evidence supports the reliability, validity, and utility of the Rorschach. This same evidence reveals that the recent criticisms of the Rorschach are largely without merit. This article systematically addresses several significant Rorschach components: interrater and temporal consistency reliability, normative data and diversity, methodological issues, specific applications in the evaluation of thought disorder and suicide, meta-analyses, incremental validity, clinician judgment, patterns of use, and clinical utility. Strengths and weaknesses of the test are addressed, and research recommendations are made. This information should give the reader both an appreciation for the substantial, but often overlooked, research basis for the Rorschach and an appreciation of the challenges that lie ahead.

An empirical study of seriously disturbed suicidal patients.

Aspects of unconscious processes in a group of seriously disturbed psychiatric patients are examined in an effort to predict near-lethal suicide attempts and explore psychoanalytic formulations of suicide. The Rorschach Inkblot Test, the most widely used projective measure in suicide research (Bongar 1991), was chosen for its potential to shed light on specific unconscious processes. Psychic states commonly associated with suicide were measured by psychoanalytic Rorschach analog scales and then subjected to a progression of statistical analyses in order to predict future occurrence and lethality of suicide attempts. On the basis of a priori hypotheses, the authors developed a suicide index comprising four psychoanalytic Rorschach signs that predicted, with considerable accuracy, which patients would later make near-lethal suicide attempts. The best predictors were unconscious processes indicative of penetrating affective overstimulation, disturbance in the capacity to maintain adequate ego boundaries, and depressive affective states characterized by a morbid preoccupation with death and inner decay. These findings provide empirical support for several well-known formulations of the unconscious motivations for suicide.

Exploring the inner world of self-mutilating borderline patients: a Rorschach investigation.

Psychiatric patients who engage in self-destructive behavior by cutting, burning, or abrading their skin are currently one of the most difficult-to-treat groups in both inpatient and outpatient settings. The complexities of treating these patients, the risk factors associated with this symptom, and the rise in the prevalence of self-mutilation in America's adolescents and young adults provided the impetus for the current study. This article explores aspects of aggression, dependency, object relations, defensive structure, and psychic boundary integrity that may contribute to the genesis and maintenance of self-mutilation. Rorschach protocols from 90 borderline personality-disordered inpatients (48 self-mutilators and 42 non-self-mutilators) were scored using five psychoanalytic content scales. Results indicate that self-mutilating patients exhibit greater incidence of primary process aggression, severe boundary disturbance, pathological object representations, defensive idealization, devaluation, and splitting than did a matched group of non-self-mutilating borderline patients. Clinical theory and technical recommendations are considered in light of the current empirical findings.

Early memories: an exploration of theoretically derived queries and their clinical utility.

The authors take Mayman's (1968) procedure for gathering early memories as their point of departure by developing three theoretically and clinically relevant queries based on object relations theory. Two empirical studies were conducted to assess the projective potential and concurrent validity of these queries, while case vignettes illuminate their construct validity and clinical utility. The results of both studies and the case vignettes lend support to Mayman's theoretical conception of early memories as a unique and powerful projective tool.

Assessing transitional phenomena with the transitional object memory probe.

Winnicott's concept of transitional relatedness has captured the interest of psychoanalysts because it provides an understanding of the dialectical process occurring between inner and outer reality, and by extension, between analyst and analysand. Clinical observations related to transitional phenomena have led the authors to develop a projective early memory probe that assesses transitional phenomena. The transitional object early memory probe was tested both for its empirical validity and for its clinical utility in psychodynamic psychotherapy. Construct validity was assessed by comparing memory scores to the Rorschach Transitional Object Scale, as well as to therapist ratings of patient behaviors. Results demonstrated moderate correlations between early memory scores and Rorschach scale scores. Equally important was the finding that early memory scores were significantly correlated with therapist ratings of key behavioral patterns in therapy. A case vignette highlights the clinical application of the transitional object probe in assessing the capacity for transitional relatedness. In this case, the data gleaned from the patient's memories provided the therapist with a sharper focus on their role in the patient's growing capacity for more vital and creative contact with reality.

The effects of social cognition and object representation on psychotherapy continuation.

This study investigates the extent to which the number of psychotherapy sessions attended is predicted by the Rorschach Mutuality of Autonomy Scale (MOA; Urist, 1977), Holt primary (A1) and secondary (A2) process aggression variables (Holt, 1977), and ratings of Thematic Apperception Test (TAT) narratives using Westen's (1995) eight Social Cognition and Object Relations Scale (SCORS) variables (complexity of representation of people, affective quality of representations, emotional investment in relationships, emotional investment in values and moral standards, understanding of social causality, experience and management of aggressive impulses, self-esteem, identity and coherence of self). Seventy-six patients with a DSM-IV Axis II diagnosis participated in this study. Two separate stepwise regression analyses (one for Rorschach variables, N = 76, and one for the SCORS ([TAT]) variables, n = 63) indicated that the Rorschach MOA PATH score (sum of scale points 5, 6, and 7; positive), as well as two individual SCORS variables (in order of relative magnitude, affective quality of representations, negative, and emotional investment in relationships, positive), were predictive of the number of psychotherapy sessions attended by patients. The conceptual nature and clinical utility of these variables are discussed in relation to the termination and continuation of psychodynamic psychotherapy.

Raw scores vs percentage conversions in factorial solutions for projective test variables.

Factor analytic solutions for raw scores and corresponding percentage conversion scores derived from Rorschach and Hand Test variables were directly compared. Number of factors, total and individual communalities, and test overlap were sufficiently different to question either approach as preferable for summarizing interrelationships among projective tests and test variables. Most of the substantial loadings tended to be either artifactual, relatively uninformative, or appeared on one factor analysis but not the other.

Relationships between Hand Test variables and maladjustment in school children.

Twenty-two Hand Test variables significantly differentiated a group of children referred to school psychologists for social and emotional maladjustment from a control group matched on age and sex. Eleven variables emerged from a subsequent stepwise discriminant analysis, resulting in a 80.85% hit rate. Results are interpreted as providing statistical support for the use of the Hand Test as a screening and/or ancillary projective technique in assessing school children.

A survey of graduate students' experiences, interests, and attitudes about learning the Rorschach.

A questionnaire about the experiences, interests, and attitudes of graduate students learning the Rorschach was sent to 235 student affiliates of the Society for Personality Assessment. The questionnaire asked for information concerning their Rorschach training in (a) class instruction, (b) adjunctive instruction (i.e., extraclass supervision and experiences at practicum sites), (c) research, and (d) personal experiences and views concerning this training. Seventy-one percent (166) of the questionnaires were returned. Results indicate that almost half the respondents gave from zero to four Rorschachs in their first course, that only half the students took a second course on the Rorschach, and that the Exner Comprehensive System was learned by a majority of the students. Students felt somewhat unsure of their skills concerning the formulation of Rorschach findings into concepts that could be communicated in a written report. Students reported that they lacked instruction concerning the research findings and application of the Rorschach. Students also did not feel grounded well enough in theory. Specific recommendations based on the information gathered from this survey are offered to improve teaching practices in the Rorschach as well as in personality-assessment training.

Discriminative validity of the MacAndrew Alcoholism Scale with Cluster B personality disorders.

This study was designed to assess the ability of the Minnesota Multiphasic Personality Inventory (MMPI-2) MacAndrew Alcoholism Scale (MAC-R) to differentiate between outpatients with personality disorders with Substance-Related Disorders (SRDs) and without SRDs. MMPI-2 validity, clinical, and MAC-R scale scores were compared in an SRD Cluster B group (comprised of Narcissistic, Antisocial, Borderline, and Histrionic; n = 15), a non-SRD Cluster B group (n = 33), and a non-SRD group with personality disorders from Clusters A and C (n = 18). Results revealed that the substance-abusing Cluster B group scored significantly higher on the MAC-R ( p <.0001) as well as the Psychopathic Deviate scale ( p <.01). Dimensional analyses illustrated that MAC-R scores were related to the presence of an SRD diagnosis (rpb =.70, p <.0001) and diagnostic criteria for Antisocial Personality Disorder (r =.60, p <.0001). Stepwise regression revealed that (in order of magnitude) the presence of a substance-abuse diagnosis followed by diagnostic criteria for Antisocial and Histrionic Personality Disorders were most related to MAC-R scores (R =.78, R(2) =.60). This indicates that the MAC-R may be more related to the presence of an SRD than has been suggested, and when used in outpatient settings as MacAndrew (1965) intended, the MAC-R may be useful as a screening device for assessing SRD among outpatients with Axis II psychopathology.

Rorschach aggression variables: a study of reliability and validity.

This study investigated the extent to which 6 Rorschach variables of aggression (A1, A2, AG, MOR, AgC, AgPast) are related to one another, to the Diagnostic and Statistical Manual of Mental Disorders (4th ed. [DSM-IV]; American Psychiatric Association, 1994) Cluster B personality disorder criteria, and to self-report measures of anger, aggression, and antisocial behavior. Seventy-eight patients were found to meet DSM-IV criteria for an Axis II disorder, Cluster A personality disorder (paranoid, schizoid, schizotypal) = 9, Cluster B (antisocial personality disorder [ANPD] = 16, borderline personality disorder [BPD] = 23, histrionic personality disorder = 5, narcissistic personality disorder = 12) = 56, and Cluster C personality disorder (avoidant, dependent, obsessive-compulsive) = 13. The results of this study indicated that (a) these 6 Rorschach aggression variables can be scored reliably; (b) 2 factors, revealed by factor analysis, accounted for 77% of the total variance; (c) selected variables were found to be empirically related to DSM-IV diagnostic criteria for ANPD and BPD; and (d) selected variables were found to be empirically related to a self-report measure of anger and antisocial practices. The conceptual nature and clinical utility of these Rorschach aggression variables as well as implications for future research are discussed.