Legal Admissibility of the Rorschach and R-PAS: A Review of Research, Practice, and Case Law.

The special issue editors selected us to form an "adversarial collaboration" because our publications and teaching encompass both supportive and critical attitudes toward the Rorschach and its recently developed system for use, the Rorschach Performance Assessment System (R-PAS). We reviewed the research literature and case law to determine if the Rorschach and specifically R-PAS meet legal standards for admissibility in court. We included evidence on norms, reliability, validity, utility, general acceptance, forensic evaluator use, and response style assessment, as well as United States and selected European case law addressing challenges to mental examination motions, admissibility, and weight. Compared to other psychological tests, the Rorschach is not challenged at unusually high rates. Although the recently introduced R-PAS is not widely referenced in case law, evidence suggests that information from it is likely to be ruled admissible when used by a competent evaluator and selected variables yield scores that are sufficiently reliable and valid to evaluate psychological processes that inform functional psycholegal capacities. We identify effective and ethical but also inappropriate uses (e.g., psychological profiling) of R-PAS in criminal, civil, juvenile, and family court. We recommend specific research to clarify important aspects of R-PAS and advance its utility in forensic mental health assessment.

Appropriate validation standards for licensure examinations: Comment on Callahan et al. (2020).

Callahan et al. (2020) asserted that the Examination for Professional Practice in Psychology (Part 2-Skills) has not undergone appropriate validation. Although they recognized that content validity is the foundation of licensure examinations, they suggested additional validational strategies that are not recommended for licensure examination development. This response clarifies the appropriate validation standards for the examination. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Psychologists' Perspectives on Therapy Termination and the Use of Therapy Engagement/Retention Strategies.

Practicing psychologists (n = 269) were surveyed regarding their perspectives on client reasons for termination at different points in therapy and their use of strategies to engage and retain clients in therapy. Psychologists estimated that one-third of their caseload unilaterally terminated (M = 13% before the third therapy session; M = 20% after the third session). They viewed lack of readiness for change/insufficient motivation as the most important barrier to early treatment engagement, and symptom improvement as the most important reason for clients' unilateral decisions to end therapy after the third session. Most psychologists reported occasional use of the majority of engagement and retention strategies. Although some strategies were used by most psychologists (e.g., building the early working alliance), fewer than 25% of psychologists reported the frequent use of time-limited treatment, appointment reminders or case management procedures. As the implementation of these strategies in clinical practice has the potential to greatly influence client retention rates, future research should examine psychologists' perspectives on and barriers to using these strategies. Copyright © 2016 John Wiley & Sons, Ltd. KEY PRACTITIONER MESSAGE: Therapists tend to underestimate the number of clients who make unilateral decisions to end treatment in their own practices. Therapists are unlikely to take steps to engage and retain clients in treatment unless they believe that unilateral termination is a significant problem. Clients who unilaterally end treatment are often experiencing problems with the process of therapy (e.g., dissatisfaction, lack of fit, feeling as though therapy is going nowhere), whereas therapists often attribute failed therapy to clients. It is important to be aware of this tendency and look for other explanations. It is worthwhile to actively solicit clients' barriers in an effort to mitigate them. The empirical literature provides ample evidence that it is helpful for therapists to deliberately employ strategies to engage and retain clients in therapy. All therapists would benefit from considering which strategies fit with their practices. Although almost all therapists emphasize building the early working alliance, and this is essential to good outcome, other evidence-based methods of engaging clients in therapy are largely underutilized, such as systematically monitoring client progress and barriers, placing time limits on treatment, using appointment reminders, and case management.

A Competency Framework for the Practice of Psychology: Procedures and Implications.

OBJECTIVE: Several competency models for training and practice in professional psychology have been proposed in the United States and Canada. Typically, the procedures used in developing and finalizing these models have involved both expert working groups and opportunities for input from interested parties. What has been missing, however, are empirical data to determine the degree to which the model reflects the views of members of the profession as a whole. METHOD: Using survey data from 466 licensed or registered psychologists (approximately half of whom completed one of two versions of the survey), we examined the degree to which psychologists, both those engaged primarily in practice and those involved in doctoral training, agreed with the competency framework developed by the Association of State and Provincial Psychology Boards' Practice Analysis Task Force (Rodolfa et al., 2013). RESULTS: When distinct time points in training and licensure or registration were considered (i.e., entry-level supervised practice in practicum settings, advanced-level supervised practice during internship, entry level independent practice, and advanced practice), there was limited agreement by survey respondents with the competency framework's proposal about when specific competencies should be attained. In contrast, greater agreement was evident by respondents with the competency framework when the reference point was focused on entry to independent practice (i.e., the competencies necessary for licensure or registration). CONCLUSION: We discuss the implications of these findings for the development of competency models, as well as for the implementation of competency requirements in both licensure or registration and training contexts.

Evaluating the measurement of mental health service accessibility, acceptability, and availability in the Canadian Community Health Survey.

Given the underutilization of mental health services by those with mental health problems, growing attention has focused on barriers to utilizing services. Several researchers have used the Canadian Community Health Survey (CCHS) cycle 1.2 dataset, including measures of barriers because of accessibility, acceptability, and availability, to explore the gap between mental health service need and use. Because the psychometric properties of these barrier measures have not been evaluated, the reliability and validity of the 3 measures were examined in the present study. Confirmatory factor analyses were conducted using data from CCHS participants who had indicated unmet need regarding information on mental illness and its treatments; availability of services, medication, and psychotherapy or counseling (n = 353); as well as the full sample of participants reporting any unmet need in the past year (n = 1,784). The hypothesized 3-factor model (i.e., accessibility, acceptability, and availability) failed to converge with both samples. Exploratory factor analysis was conducted using data from the full sample (n = 1,784), and a possible 2-factor solution was obtained. Reliability analyses on this 2-factor model, as well as the 3-factor model included in the CCHS, demonstrated that internal consistency values failed to attain acceptable levels of reliability (i.e., α < .70). Based on these analyses, these barrier measures are neither reliable nor valid. These measures should not be used to examine barriers to service within the CCHS 1.2 dataset, and caution should be exercised in interpreting the findings of studies that used these measures.

Evidence-Based Practice: Separating Science From Pseudoscience.

Evidence-based practice (EBP) requires that clinicians be guided by the best available evidence. In this article, we address the impact of science and pseudoscience on psychotherapy in psychiatric practice. We describe the key principles of evidence-based intervention. We describe pseudoscience and provide illustrative examples of popular intervention practices that have not been abandoned, despite evidence that they are not efficacious and may be harmful. We distinguish efficacy from effectiveness, and describe modular approaches to treatment. Reasons for the persistence of practices that are not evidence based are examined at both the individual and the professional system level. Finally, we offer suggestions for the promotion of EBP through clinical practice guidelines, modelling of scientific decision making, and training in core skills.

What clinicians want: findings from a psychotherapy practice research network survey.

Practice research networks may be one way of advancing knowledge translation and exchange (KTE) in psychotherapy. In this study, we document this process by first asking clinicians what they want from psychotherapy research. Eighty-two psychotherapists in 10 focus groups identified and discussed psychotherapy research topics relevant to their practices. An analysis of these discussions led to the development of 41 survey items. In an online survey, 1,019 participants, mostly practicing clinicians, rated the importance to their clinical work of these 41 psychotherapy research topics. Ratings were reduced using a principal components analysis in which 9 psychotherapy research themes emerged, accounting for 60.66% of the variance. Two postsurvey focus groups of clinicians (N = 22) aided in interpreting the findings. The ranking of research themes from most to least important were-Therapeutic Relationship/Mechanisms of Change, Therapist Factors, Training and Professional Development, Client Factors, Barriers and Stigma, Technology and Adjunctive Interventions, Progress Monitoring, Matching Clients to Therapist or Therapy, and Treatment Manuals. Few differences were noted in rankings based on participant age or primary therapeutic orientation. Postsurvey focus group participants were not surprised by the top-rated items, as they were considered most proximal and relevant to therapists and their work with clients during therapy sessions. Lower ranked items may be perceived as externally imposed agendas on the therapist and therapy. We discuss practice research networks as a means of creating new collaborations consistent with KTE goals. Findings of this study can help to direct practitioner-researcher collaborations.

The Process of Seeking Psychotherapy and Its Impact on Therapy Expectations and Experiences.

UNLABELLED: Seeking psychotherapy can be conceptualized as having three stages: deciding that therapy might help, deciding to seek therapy and contacting a therapist. The present study examined the duration and difficulty of clients' decisions to seek psychotherapy and whether these experiences were predictive of expected difficulty and commitment to the therapy process. One-hundred and fifty-five adults seeking individual psychological services from a university training clinic were assessed before intake; 107 of these participants also completed measures between the third and fourth therapy sessions and at post-therapy. Deciding that therapy might help was reported to be the most difficult step and took the longest, with each subsequent step becoming easier and briefer. At each step, the more difficult the decision, the more time participants took to make it. Higher distress was associated with more difficulty in deciding that therapy might help and deciding to seek therapy. Duration and difficulty of decisions to seek therapy were positively correlated with expectations of difficulty in therapy as measured prior to treatment and following the third session but were not associated with participants' commitment to therapy. The implications of these results for clinicians and mental health services are discussed. KEY PRACTITIONER MESSAGE: The most difficult and time-consuming step for those who seek mental health services is recognizing that their distressing experiences are connected to mental health; clinicians may aid this challenge by providing information on the nature of mental disorders and common symptoms of emotional distress on websites or through other means (e.g., physicians' waiting rooms and advertising campaigns). The next most difficult and time-consuming step for those who seek mental health services is deciding that psychotherapy may help; by providing easily accessible information (e.g., on websites) about what psychotherapy entails, including clinicians' expectations of clients and the benefits/challenges of therapy, potential clients may be able to progress through this step more rapidly and with less difficulty. Clients' expectations of the value of psychotherapy and their commitment to engage in therapy do not appear to be affected by how long it took, or how difficult it was, to obtain psychotherapy. Factors such as forming a strong therapeutic alliance and providing support and guidance during the initial sessions of therapy may be more important in helping potential clients commit to therapy than what they experienced in their efforts to receive psychotherapy.

Comparing approaches for categorizing measure reliability in the assessment of anxiety in older adults.

A systematic review of the anxiety measures most commonly used with older adults found that most are developed for a younger population and lacked sufficient evidence of their psychometric properties when used with an older population. As researchers and clinicians need a fast way to judge whether a measure is appropriate when faced with daily assessment tasks, recent efforts have been made in developing guidelines to operationalize the criteria necessary to designate a measure as evidence-based. The goal of this study is to: (a) apply the reliability criteria for two existing evidence based assessment categorization systems to the most commonly used anxiety measures with older adults and (b) compare the results of these categorization systems to the results obtained in a previous reliability generalization study that gave the mean reliability of each measure. We discuss the strengths and limitations of both approaches and suggest how researchers and clinicians can identify psychometrically sound measures without having to conduct more labour-intensive meta-analysis studies.

Assessment of anxiety in older adults: a systematic review of commonly used measures.

OBJECTIVES: The authors set out to systematically review the research literature in order to identify the anxiety measures most commonly used in the assessment of older adults. Once identified, the literature was reviewed to determine the extent to which these instruments had age-relevant norms and psychometric data supporting their use with older adults. METHOD: Literature searches were conducted in PsycINFO and PubMed to identify research articles in which anxiety measures were completed by older adults. After screening for suitability, a total of 213 articles were reviewed to determine the most commonly used anxiety measures with older adults to examine the psychometric properties of these instruments and to evaluate whether the instruments are appropriate for use with older adults. RESULTS: A total of 91 different anxiety measures were used in the 213 included articles. Twelve anxiety measures were most commonly used in the literature and of those three were specifically developed for older adults. CONCLUSIONS: Of the most commonly used measures, the majority lacked sufficient evidence to warrant their use with older adults. Based on psychometric evidence, three measures (Beck Anxiety Inventory, Penn State Worry Questionnaire, and Geriatric Mental Status Examination) showed psychometric properties sufficient to justify the use of these instruments when assessing anxiety in older adults. In addition, two measures developed specifically for older adults (Worry Scale and Geriatric Anxiety Inventory) were also found to be appropriate for use with older adults.

Age-related patterns in mental health-related complementary and alternative medicine utilization in Canada.

BACKGROUND: The aim of this study was to examine whether age-related differences in rates of use of complementary and alternative medicine (CAM) specifically for mental health problems parallel well-known age-related differences in use of conventional mental health services and medications. METHODS: A sample of middle-aged (45-64 years; n = 10,762), younger-old (65-74; n = 4,113) and older-old adults (75 years and older; n = 3,623) was drawn from the 2001-2002 Canadian Community Health Survey (CCHS), Cycle 1.2, Mental Health and Wellbeing. Age-related utilization rates of conventional and complementary mental health services and medications/products were calculated. Logistic regression analyses were used to examine the strength of association between age group and utilization of services and medications or products in the context of other important sociodemographic and clinical characteristics. RESULTS: When considered in the context of other sociodemographic and clinical characteristics, older age was positively associated with mental health-related utilization of alternative health products. Older age was not significantly associated with mental health-related consultations with CAM providers. CONCLUSIONS: Overall, age-related patterns in mental health-related use of CAM did not directly correspond to age-related patterns in conventional mental health care utilization, suggesting different sets of predictors involved in seeking each type of care.

Reasons for terminating psychotherapy: a general population study.

Clients' (N=693) reasons for ending psychotherapy and their associations with demographics, mental disorder caseness, and type of mental health care service provider were examined. The most frequently reported reason for termination was feeling better, however, a substantial minority of individuals reported terminating because of treatment dissatisfaction or wanting to solve problems independently. Lower income was associated with lower odds of termination because of feeling better and higher odds of termination because of a perception that therapy was not helping. Meeting criteria for an anxiety disorder, a mood disorder, or substance dependence decreased the odds of termination because of feeling better. These findings provide important information on the challenges to the successful completion of psychotherapy.

Client and therapist views of contextual factors related to termination from psychotherapy: a comparison between unilateral and mutual terminators.

Contextual variables potentially influencing premature termination were examined. Clients (n=83) and therapists (n=35) provided parallel data on early working alliance, psychotherapy termination decision (unilateral vs. mutual), clients' reasons for termination, and barriers to treatment participation. When clients unilaterally ended therapy, therapists were only partially aware of either the extent of clients' perceived improvements or their dissatisfaction. When termination was mutually determined, there were no differences between client and therapist ratings of termination reasons. Although working alliance and barriers to treatment participation were rated as lower in the context of unilateral termination by clients and therapists, all clients rated the early alliance and barriers to treatment more highly than did therapists. Results have implications for understanding premature termination and suggest future research examining the utility of therapist feedback regarding contextual variables in terms of retaining clients in therapy.

Evidence-based assessment.

Evidence-based assessment (EBA) emphasizes the use of research and theory to inform the selection of assessment targets, the methods and measures used in the assessment, and the assessment process itself. Our review focuses on efforts to develop and promote EBA within clinical psychology. We begin by highlighting some weaknesses in current assessment practices and then present recent efforts to develop EBA guidelines for commonly encountered clinical conditions. Next, we address the need to attend to several critical factors in developing such guidelines, including defining psychometric adequacy, ensuring appropriate attention is paid to the influence of comorbidity and diversity, and disseminating accurate and up-to-date information on EBAs. Examples are provided of how data on incremental validity and clinical utility can inform EBA. Given the central role that assessment should play in evidence-based practice, there is a pressing need for clinically relevant research that can inform EBAs.

Interpreting the magnitude of the placebo effect: mountain or Molehill?

The ambiguity involved in interpreting numbers and words is central to Hróbjartsson and Gøtzsche's (this issue) claim of "powerful spin" in the Wampold, Minami, Tierney, Baskin, and Bhati (2005) re-analysis of their meta-analytic findings on the placebo effect in medicine. Meta-analytic results reported by the two sets of authors are nearly identical, yet their conclusions differ dramatically. In our comment, we discuss the findings of the respective authors and consider options for representing and interpreting the magnitude of meta-analytic effect size estimates. We conclude that although the meta-analyses described indicate that placebo effects do exist and cannot be dismissed as unimportant, given contextual information, it is consistent with existing research to describe the obtained mean effect size for placebos in medicine as small in magnitude.

Utilization of mental health care services among older adults with depression.

Despite the availability of effective treatments for late life depression, data indicate that only a small minority of adults over the age of 65 years with depression access any kind of care for emotional or mental health problems. Using data from the Canadian Community Health Survey (Cycle 1.1), we compared patterns of mental health service utilization among middle-aged (45-64 years), younger old (65-74 years), and older old (75 years and older) adults with and without depression and identified predictors associated with accessing different services (n=59,302). Compared to middle-aged adults with depression, individuals aged 65 and older with depression were less likely to report any mental health consultation in the past year and especially unlikely to report consulting with professionals other than a family physician. Age remained a significant predictor of mental health service utilization even after accounting for other relevant variables such as gender, marital status, years of education, depression caseness, and number of chronic medical conditions. Although the prevalence of depression is lower in older age groups, the present study provides compelling evidence that mental health services are particularly underutilized by depressed older adults.

Introduction to the special section on developing guidelines for the evidence-based assessment (EBA) of adult disorders.

The goal of this special section is to encourage greater awareness of evidence-based assessment (EBA) in the development of a scientifically supported clinical psychology. In this introductory article, the authors describe the elements that authors in this special section were asked to consider in their focused reviews (including the scope of available psychometric evidence, advancements in psychopathology research, and evidence of attention to factors such as gender, age, and ethnicity in measure validation). The authors then present central issues evident in the articles that deal with anxiety, depression, personality disorders, and couple distress and in the accompanying commentaries. The authors conclude by presenting key themes emerging from the articles in this special section, including gaps in psychometric information, limited information about the utility of assessment, the discrepancy between recommended EBAs and current training and practice, and the need for further data on the process of clinical assessment.

Evidence-based assessment of child and adolescent disorders: issues and challenges.

The main purpose of this article and this special section is to encourage greater attention to evidence-based assessment (EBA) in the development of a scientifically supported clinical child and adolescent psychology. This increased attention is especially important in light of (a) the omission of assessment considerations in recent efforts to promote evidence-based treatments for children and (b) ongoing changes in the nature of clinical child assessment. We discuss several key considerations in the development of guidelines for EBA, including the purposes of assessment, the role of disorder or problem specificity, the scope of assessment, assessment process parameters, possible "cross-cutting" assessment issues, psychometric considerations, and issues related to the clinician's integration of assessment data. We conclude the article with suggestions for how current, summary information on EBA can be developed, maintained, and disseminated.

Introduction to the special section on incremental validity and utility in clinical assessment.

This special section focuses on the incremental validity and utility of clinical assessment data. The lack of replicated incremental validity research limits the ability of psychologists to establish their assessment practices on a solid empirical footing. The articles in this section deal with conceptual, methodological, and content issues in the development and use of clinical measures. The authors addressed several aspects of incremental validity research, including (a) the evaluation of the magnitude of validity increments, (b) the use of incremental validity data for test development and validation, (c) the costs of assessment, (d) the use of multi-informant and multimethod assessments, and (e) the treatment utility of assessment. Obstacles limiting the use of incremental validity research to inform and guide clinical practice are also emphasized.

The incremental validity of psychological testing and assessment: conceptual, methodological, and statistical issues.

There has been insufficient effort in most areas of applied psychology to evaluate incremental validity. To further this kind of validity research, the authors examined applicable research designs, including those to assess the incremental validity of test instruments, of test-informed clinical inferences, and of newly developed measures. The authors also considered key statistical and measurement issues that can influence incremental validity findings, including the entry order of predictor variables, how to interpret the size of a validity increment, and possible artifactual effects in the criteria selected for incremental validity research. The authors concluded by suggesting steps for building a cumulative research base concerning incremental validity and by describing challenges associated with applying nomothetic research findings to individual clinical cases.