An Investigation of Interrater Reliability for the Rorschach Performance Assessment System (R-PAS) in a Nonpatient U.S. Sample.

The Rorschach Performance Assessment System (R-PAS) aims to provide an evidence-based approach to administration, coding, and interpretation of the Rorschach Inkblot Method (RIM). R-PAS analyzes individualized communications given by respondents to each card to code a wide pool of possible variables. Due to the large number of possible codes that can be assigned to these responses, it is important to consider the concordance rates among different assessors. This study investigated interrater reliability for R-PAS protocols. Data were analyzed from a nonpatient convenience sample of 50 participants who were recruited through networking, local marketing, and advertising efforts from January 2013 through October 2014. Blind recoding was used and discrepancies between the initial and blind coders' ratings were analyzed for each variable with SPSS yielding percent agreement and intraclass correlation values. Data for Location, Space, Contents, Synthesis, Vague, Pairs, Form Quality, Populars, Determinants, and Cognitive and Thematic codes are presented. Rates of agreement for 1,168 responses were higher for more simplistic coding (e.g., Location), whereas agreement was lower for more complex codes (e.g., Cognitive and Thematic codes). Overall, concordance rates achieved good to excellent agreement. Results suggest R-PAS is an effective method with high interrater reliability supporting its empirical basis.

Addressing Issues in the Development and Use of the Composite International Reference Values as Rorschach Norms for Adults.

This article describes 3 studies evaluating normative reference data for the Rorschach Comprehensive System (CS; Exner, 2003, 2007), with a particular focus on the viability of the Composite International Reference Values (CIRVs) that were compiled from 21 adult studies by Meyer, Erdberg, and Shaffer (2007). Study 1 documented how the CIRV norms are virtually identical when organized into 3 groups differentiated by the quality of their data collection effort, including an optimal group of 4 samples that relied on multiple experienced examiners and provided ongoing quality control over administration and coding. Analyses also showed that relative to the group of more optimal samples, the group of less optimal samples did not produce more variability in summary scores within or across samples or lower interrater reliability for coding. Study 2 used the existing CS reference norms to generate T scores for the CIRV means and documented how the CS norms make other samples of healthy nonpatients look psychologically impaired in multiple domains. Study 3 documented with examples from 4 different countries how 2 sets of within-country local norms produced notably different results on some variables, which compromises the ability of local norms to be used instead of the CIRVs. Taken together, the 3 studies provide support for the use of CIRVs in clinical practice as norms that are generalizable across samples, settings, languages, and cultures and that account for the natural variability that is present when clinicians and researchers contend with the ambiguity contained in the standard CS reference materials concerning the proper ways to administer and code. We conclude by urging CS users to rely on the CIRVs when making clinical inferences and to adopt alternative methods of ensuring they are following cohesively standardized administration and coding guidelines.

Rorschach intercoder reliability for protocol-level comprehensive system variables in an international sample.

This study examines the intercoder reliability of Rorschach Comprehensive System (CS; Exner, 2001) protocol-level variables. A large international sample was combined to obtain intercoder agreement for 489 Rorschach protocols coded using the CS. Intercoder agreement was calculated using an Iota coefficient, a statistical coefficient similar to kappa that is corrected for chance. Iota values for the variables analyzed ranged from .31 to 1.00, with 2 in the poor range of agreement, 4 in the fair range, 25 in the good range, and 116 in the excellent range of agreement. Discrepancies between variables are discussed.

Rorschach Comprehensive System data for 100 nonpatient children from the United States in two age groups.

Building on our previously published study (Hamel, Shaffer, & Erdberg, 2000), which provided data on 100 nonpatient children aged 6 to 12 from the United States, we here provide reference data for two more homogeneous age subgroups: 6 to 9 (N = 50) and 10 to 12 (N = 50). Inclusion criteria are described, and expanded interrater reliability statistics at the response level are presented along with scores for the Rorschach Comprehensive System (CS; Exner, 2001) at each age grouping. In addition to the children being administered the Rorschach, their parents were given the Conners' Parent Rating Scale-93 (CPRS-93; Conners, 1989), and these results are presented as well.

Rorschach Comprehensive System data for a sample of 283 adult nonpatients from the United States.

This article presents an update on adult nonpatients living in central California previously described by Shaffer, Erdberg, and Haroian (1999). In this study an additional 160 nonpatients were administered the Rorschach for a total sample size of 283. Graduate students enrolled in a 2-year Rorschach research seminar administered the Rorschach (Exner, 1995), WAIS-R (Weschler, 1981), and MMPI-2 (Butcher, Dahlstrom, Graham, Tellegen, & Kaemmer, 1989), and were provided with extensive supervision and ongoing quality control. All protocols were recoded by the first two authors and a psychologist with extensive Rorschach experience. Exclusion criteria are described. Interrater reliability statistics at the response level are presented along with scores for Rorschach Comprehensive System (CS), WAIS-R, and MMPI-2 variables.

Rorschach Comprehensive System data for a sample of 52 older adult nonpatients from the United States.

This study is composed of 52 older adults, ages 60 to 80, all from the United States. In addition to being administered the Rorschach (Rorschach, 1942), all participants also were administered the Wechsler Adult Intelligence Scale-Revised (WAIS-R; Weschler, 1981), the Minnesota Multiphasic Personality Inventory-2 (MMPI-2; Butcher, Dahlstrom, Graham, Tellegen, & Kaemmer, 1989), the Neuro Cognitive Status Examination (NCSE; Kiernan, Mueller, Langston, & Van Dyke, 1987), and the Trail Making Test Part B (Reitan, 1958; Reitan & Wolfson, 1993). Examiners included the principal author as well as three other trained examiners. Inclusion criteria are described. Interrater reliability statistics at the response level are presented along with scores for the Rorschach Comprehensive System (CS; Exner, 1995).

Rorschach Comprehensive System data for a sample of 42 nonpatient Mexican American children from the United States.

This study is composed of 42 Mexican American children, ages 8 to 10 years, all from the United States. In addition to the children being administered the Rorschach, they also were administered the Children's Hispanic Background Scale (CHBS) and the Children's Personality Questionnaire (CPQ). The principal author was the sole examiner. Inclusion criteria are described. Interrater reliability statistics at the response level are presented along with scores for the Rorschach Comprehensive System (CS; Exner, 1995).

Rorschach Comprehensive System data for a sample of 37 nonpatient/nondelinquent adolescents from the United States.

This study is composed of 37 American adolescents, ages 15 to 17 years of age, all from the United States. The principal author was the sole examiner. Inclusion criteria are described. Interrater reliability statistics at the response level are presented along with scores for the Rorschach Comprehensive System (CS; Exner, 2003). The results revealed differences from the published norms on variables that reflect poor form quality, fewer popular responses, more attention to detail, more poor human relationship responses, and a more simplistic, affective-free approach to the environment, among others.

Introduction to the JPA special supplement on International Reference Samples for the Rorschach Comprehensive System.

This Introduction provides an overview to the JPA Special Supplement on International Reference Samples for the Rorschach Comprehensive System (CS; Exner, 2001). It contains a history of this project and a table including all of the lead authors, their country and the type and size of their sample. Suggestions for conducting this type of research are offered, including information on normative vs. nonpatient samples, inclusion/exclusion criteria, the use of collateral instruments, and concerns relative to representativeness. Reliability and coding concerns are addressed, and information on the Popular response in Japan are reviewed. Finally, trends within and across the data are highlighted.

Toward international normative reference data for the comprehensive system.

We build on the work of all the authors contributing to this Special Supplement by summarizing findings across their samples of data, and we also draw on samples published elsewhere. Using 21 samples of adult data from 17 countries we create a composite set of internationally-based reference means and standard deviations from which we compute T-scores for each sample. Figures illustrate how the scores in each sample are distributed and how the samples compare across variables in eight Rorschach Comprehensive System (CS; Exner, 2003) clusters. The adult samples from around the world are generally quite similar, and thus we encourage clinicians to integrate the composite international reference values into their clinical interpretation of protocols. However, the 31 child and adolescent samples from 5 countries produce unstable and often quite extreme values on many scores. Until the factors contributing to the variability among these samples are more fully understood, we discourage clinicians from using many CS scores to make nomothetic, score-based inferences about psychopathology in children and adolescents.