Legal Admissibility of the Rorschach and R-PAS: A Review of Research, Practice, and Case Law.

The special issue editors selected us to form an "adversarial collaboration" because our publications and teaching encompass both supportive and critical attitudes toward the Rorschach and its recently developed system for use, the Rorschach Performance Assessment System (R-PAS). We reviewed the research literature and case law to determine if the Rorschach and specifically R-PAS meet legal standards for admissibility in court. We included evidence on norms, reliability, validity, utility, general acceptance, forensic evaluator use, and response style assessment, as well as United States and selected European case law addressing challenges to mental examination motions, admissibility, and weight. Compared to other psychological tests, the Rorschach is not challenged at unusually high rates. Although the recently introduced R-PAS is not widely referenced in case law, evidence suggests that information from it is likely to be ruled admissible when used by a competent evaluator and selected variables yield scores that are sufficiently reliable and valid to evaluate psychological processes that inform functional psycholegal capacities. We identify effective and ethical but also inappropriate uses (e.g., psychological profiling) of R-PAS in criminal, civil, juvenile, and family court. We recommend specific research to clarify important aspects of R-PAS and advance its utility in forensic mental health assessment.

Complexity and Cognitive Engagement in the Rorschach Task: An fMRI Study.

Recently, an eye-tracking study found that Complexity and other R-PAS variables located in the Engagement and Cognitive Processing domain correlated with a proxy marker for cognitive effort and engagement. The goal of the current study was to test the robustness and validity of those eye-tracking findings by inspecting fMRI data. We hypothesized that the greater the level of engagement and cognitive effort put in place by a Rorschach test-taker, the greater the engagement of his/her cortical areas reflecting ongoing top-down attentional processes should be. We re-analyzed archival fMRI data from 26 healthy participants exposed to the Rorschach inkblots with the instruction to think of what they might be. The association of various Engagement and Cognitive Processing R-PAS scores to increased BOLD signals in the Dorsal Attention Network of the brain was examined. As expected, Complexity showed the strongest effect size across all R-PAS variables under investigation (d = 0.43), followed by Synthesis (d = 0.32) and Human Movement (d = 0.21). Noteworthy, the correlation between the effect sizes found in the current fMRI study and those found in the previously published eye-tracking study consists of an impressive r = .80.

An Inventory of Problems-29 (IOP-29) study investigating feigned schizophrenia and random responding in a British community sample.

Compared to other Western countries, malingering research is still relatively scarce in the United Kingdom, partly because only a few brief and easy-to-use symptom validity tests (SVTs) have been validated for use with British test-takers. This online study examined the validity of the Inventory of Problems-29 (IOP-29) in detecting feigned schizophrenia and random responding in 151 British volunteers. Each participant took three IOP-29 test administrations: (a) responding honestly; (b) pretending to suffer from schizophrenia; and (c) responding at random. Additionally, they also responded to a schizotypy measure (O-LIFE) under standard instruction. The IOP-29's feigning scale (FDS) showed excellent validity in discriminating honest responding from feigned schizophrenia (AUC = .99), and its classification accuracy was not significantly affected by the presence of schizotypal traits. Additionally, a recently introduced IOP-29 scale aimed at detecting random responding (RRS) demonstrated very promising results.

An Inventory of Problems-29 Sensitivity Study Investigating Feigning of Four Different Symptom Presentations Via Malingering Experimental Paradigm.

The Inventory of Problems-29 (IOP-29) is a newly developed, self-administered test designed to assist practitioners evaluating the credibility of various symptom presentations. In a recent simulation study comparing real patients against experimental feigners, its classification accuracy compared favorably with that of the Structured Inventory of Malingered Symptomatology and generalized better to different types of symptom reports (i.e., mood- or trauma-related vs. psychosis-related). Extending on these findings, this sensitivity study tested whether the IOP-29 would be similarly accurate in detecting feigning of conditions related to depression, mild traumatic brain injury, posttraumatic stress disorder (PTSD), and schizophrenia. Additionally, it also evaluated the susceptibility of the IOP-29 to uncooperative or random-like responding. Examination of 1,200 IOP-29 records from 400 nonclinical Italian volunteers who took the Italian version of the IOP-29 3 times, in 3 different conditions (i.e., standard instructions, feigned disorder, and random-like responding), confirmed that the IOP-29 might be similarly valid across a wide variety of mental health and cognitive complaints. Indeed, using the standard a priori cutoff of the IOP-29 (i.e., ≥ .50), sensitivity estimates ranged from .86 (for feigned PTSD) to .95 (for feigned depression) and did not significantly differ from one symptom presentation to another. These findings were obtained despite the fact that participants were coached to respond in a believable manner. Interestingly, the random-like responding generated extremely variable IOP-29 scores.

An Inventory of Problems-29 Study on Random Responding Using Experimental Feigners, Honest Controls, and Computer-Generated Data.

Self-reports could be affected by 2 primary sources of distortion: content-related (CRD) and content-unrelated (CUD) distortions. CRD and CUD, however, might covary, and similar detection strategies have been used to capture both. Thus, we hypothesized that a scale developed to detect random responding-arguably, one of the most evident examples of CUD-would likely be sensitive to both CUD and, albeit to a lesser extent, CRD. Study 1 (N = 1,901) empirically tested this hypothesis by developing a random responding scale (RRS) for the recently introduced Inventory of Problems-29 (Viglione, Giromini, & Landis, 2017), and by testing it with both experimental feigners and honest controls. Results supported our hypothesis and offered some insight on how to pull apart CRD- from CUD-related variance. Study 2 (N = 700) then evaluated whether our RRS would perform similarly well with data from human participants instructed to respond at random versus computer-generated random data. Interestingly, the sensitivity of our RRS dropped dramatically when considering the data from human participants. Together with the results of additional analyses inspecting the patterns of responses provided by our human random responders, these findings thus posed a major question: Is humans' random responding really random?

The Effect of CS Administration or an R-Optimized Alternative on Potential Projective Material in Rorschach Responses From Six Studies and a Meta-Analysis of Their Findings.

The Rorschach Performance Assessment System (R-PAS; Meyer, Viglione, Mihura, Erard, & Erdberg, 2011) introduced R-optimized administration to reduce variability in the number of Responses (R). We provide new data from six studies of participants randomly assigned to receive a version of this method or Comprehensive System (CS; Exner, 2003) administration. We examine how administration methods affect 3 types of codes most likely to contain potential projective material and the frequency of these codes for the 1st, 2nd, 3rd, 4th, or last response to a card (R in Card). In a meta-analytic summary, we found 37% of responses have 1 type of code, 19% have 2 types, and 3% have all 3 types, with stable proportions across responses within cards. Importantly, administration method had no impact on potential projective variable means. Differential skew across samples made variability harder to interpret. Initial results suggesting differences in 3 of the 18 specific Type by R in Card pairs did not follow a coherent pattern and disappeared when using raw counts from all participants. Overall, data do not support concerns that R-optimized administration might alter potential projective processes, or make potentially "signature" last responses to the card any different in R-PAS than the CS.

Criterion Validity of the Rorschach Developmental Index With Children.

The Rorschach Developmental Index (DI) is a composite measure of psychological development. The aim of this study was to examine the criterion validity of the DI by investigating its relationship with age, nonverbal intelligence, and the impact of institutionalization in a sample of Brazilian children. The sample included 231 children, ages 7 to 11, who were either living with their families or in an institutionalized setting. The results indicated that the DI is strongly associated with nonverbal intelligence. Multiple regression analysis revealed that 71% of the variability in the DI was associated with nonverbal intelligence, whereas age explained only 2%. Moreover, the DI scores and nonverbal intelligence scores were significantly lower for children living in institutionalized settings. The DI appears to be effective in assessing various levels of psychological development, especially when expressed in cognitive ability. The DI was developed in the United States, and this study suggests that the DI can be adapted to diverse cultures, regions, or languages.

The Effect of CS Administration or an R-Optimized Alternative on R-PAS Variables: A Meta-Analysis of Findings From Six Studies.

Controlling the number of Rorschach responses (R) as a method to reduce variability in the length of records has stimulated controversy among researchers for many years. Recently, the Rorschach Performance Assessment System (R-PAS; Meyer, Viglione, Mihura, Erard, & Erdberg, 2011 ) introduced an R-Optimized method to reduce variability in R. Using 4 published and 2 previously unpublished studies (N = 713), we examine the extent to which 51 Comprehensive System-based scores on the R-PAS profile pages are affected as a result of receiving Comprehensive System (CS; Exner, 2003 ) administration versus a version of R-Optimized administration. As hypothesized, R-the intended target of R-Optimized administration-showed reliable weighted average differences across each method of administration. As expected, its mean modestly increased and its standard deviation notably decreased. Also as hypothesized, the next largest effects were decreases in the variability (SD) of 2 variables directly related to R, R8910% and Complexity. No other reliable differences were observed. Therefore, because R-Optimized administration does not notably modify the existing CS-based normative values for other profiled R-PAS variables, the data do not support concerns that R-Optimized administration notably modifies the Rorschach task or that existing CS research data would not generalize to R-PAS. However, because R-Optimized administration reduces variability in R, it allows a single set of norms to apply readily to more people.

Development and Validation of the False Disorder Score: The Focal Scale of the Inventory of Problems.

This article introduces the Inventory of Problems (IOP)-a new, computerized, 181-item tool designed to discriminate bona fide from feigned mental illness and cognitive impairment-and presents the development and validation of its focal, feigning scale, the False Disorder Score (IOP-FDS). The initial sample included (a) 211 patients and 64 offenders who took the IOP under standard conditions, and (b) 210 community volunteers and 64 offenders who feigned mental illness. We split this sample into three subsamples. The first (n = 301) was used to select the variables to generate the IOP-FDS; the second (n = 148) scaled the IOP-FDS into a probability score; and the third (n = 100) tested its validity with an independent data set. In this third subsample, the IOP-FDS had sensitivity = .90, specificity = .80, and a greater area under the curve (AUC = .95) than the IOP-29 (.91). For 40 participants, the Personality Assessment Inventory (PAI) was available, too. Within this subgroup, the IOP-FDS outperformed the selected PAI validity scales (AUC = .99 vs. AUC ≤ .85).

Introducing the concept of neurobiological foundation of Rorschach responses using the example of Oral Dependent Language.

We introduce the concept of "neurobiological foundation" of Rorschach interpretations as an extension of the concept of behavioral representation as a foundation for interpretation of R-PAS variables. Here, we propose that if there is a parallelism between the mental, verbal and perceptual behaviors occurring within the microcosm of the Rorschach task and those occurring in the external environment [behavioral foundation], then the same brain regions engaged by the test-taker when producing of a given code, should be engaged also when reproducing, in the external environment, the same psychological processes underlying that specific Rorschach code [neurobiological foundation]. To investigate this concept, we used archival, fMRI data and tested whether producing Oral Dependency Language (ODL) responses would associate with increased activation in brain regions associated with dependency-related, psychological processes. Results from a sample of 21 non-clinical volunteers partially confirmed our hypothesis, providing some support to the neurobiological foundation of the ODL code.

The Development of the Inventory of Problems-29: A Brief Self-Administered Measure for Discriminating Bona Fide From Feigned Psychiatric and Cognitive Complaints.

This article describes the development of the Inventory of Problems-29 (IOP-29), a new, short, paper-and-pencil, self-administered measure of feigned mental and cognitive disorders. Four clinical comparison simulation studies were conducted. Study 1 (n = 451) selected the items and produced an index of potential feigning. Study 2 (n = 331) scaled this index to produce a probability score, and examined its psychometric properties. Study 3 tested the generalizability of Study 2's findings with 2 additional samples (ns = 128 and 90). Results supported the utility of the IOP-29 for discriminating bona fide from feigned psychiatric and cognitive complaints. Validity was demonstrated in feigning mild traumatic brain injury, psychosis, posttraumatic stress disorder, and depression. Within the independent samples of Studies 2 and 3, the brief IOP-29 performed similarly to the MMPI-2 and Personality Assessment Inventory, and perhaps better than the Test of Memory Malingering. Classifications within these samples with base rates of .5 produced sensitivity, specificity, positive predictive power, and negative predictive power statistics of about .80. Further research is needed testing the IOP-29 in ecologically valid field studies.

A Survey of Challenges Experienced by New Learners Coding the Rorschach.

Learning to code the imagery, communication, and behavior associated with Rorschach responding is challenging. Although there is some survey research on graduate students' impressions of their Rorschach training, research has not identified which coding decisions students find to be the most problematic and time-consuming. We surveyed students to identify what they struggled with most when learning coding and to quantify how difficult it is to learn how to code. Participants (n = 191) from the United States, Brazil, Denmark, Israel, and Italy rated 57 aspects of coding using a 4-point scale that encompassed both the time required to code and the subjective difficulty of doing so. Mean ratings for coding in general indicated that students considered the overall task challenging. Ratings also revealed that students struggled most with Cognitive Special Scores, Determinants, and extrapolating from the tables to code Form Quality for objects that were not specifically listed. The findings offer suggestions about how to improve the guidelines for some of the more difficult variables and where it is most necessary to focus teaching time. Taking these steps might help new students in learning the Rorschach.

The Effects of Using the International Versus Comprehensive System Rorschach Norms For Children, Adolescents, and Adults.

Currently, there is some debate about whether to use Comprehensive System norms (CS; Exner, 2003 ) or the Composite International Reference Values (CIRV; Meyer, Erdberg, & Shaffer, 2007 ) when interpreting Rorschach Inkblot Method (RIM; Rorschach, 1921 ) protocols administered with the CS method. The goal of this study is to assist clinicians in making this decision by providing information about the effects of choosing one option or the other. Accordingly, this research evaluates the effects of using the CS versus CIRV norms with children, adolescents, and adults. First, we identified 43 variables for which the CS and the CIRV for children and adolescents differ from each other by at least a Cohen's d value of .50. Next, we evaluated whether these divergent variables are the same as those previously identified as divergent for the adult population. Results showed that for both children and adolescents, as well as for adults, relying on CS norms versus CIRV would result in interpretations that are more pathological in terms of (a) perception and thinking, (b) psychological resources and cognitive and emotional abilities, and (c) representations of human relationships. A discussion on the clinical effects of using one versus the other set of norms follows.

Introducing a Bayesian Approach to Determining Degree of Fit With Existing Rorschach Norms.

This article offers a new methodological approach to investigate the degree of fit between an independent sample and 2 existing sets of norms. Specifically, with a new adaptation of a Bayesian method, we developed a user-friendly procedure to compare the mean values of a given sample to those of 2 different sets of Rorschach norms. To illustrate our technique, we used a small, U.S. community sample of 80 adults and tested whether it resembled more closely the standard Comprehensive System norms (CS 600; Exner, 2003), or a recently introduced, internationally based set of Rorschach norms (Meyer, Erdberg, & Shaffer, 2007 ). Strengths and limitations of this new statistical technique are discussed.

Cross-Cultural Validation of the Rorschach Developmental Index.

The Developmental Index (DI) has recently been introduced as a composite Rorschach measure of psychological development and maturation, which can be used both with the Comprehensive System (Exner, 2003), and with the recently developed Rorschach Performance Assessment System (Meyer, Viglione, Mihura, Erard, & Erdberg, 2011). As the DI is new, and its validity has not yet been investigated with independent non-U.S. samples, we tested the correlation between DI and age using 3 relatively large samples, 2 of which were from outside the United States (total N = 902). Other Rorschach variables presumably associated with maturation, such as complexity and productivity, were also investigated. As expected, the DI significantly correlated with age, with small variations across the 3 samples. Importantly, the correlation between DI and age remained statistically significant also after controlling for productivity (i.e., the number of responses) and complexity.

Developing an Alternative Rorschach Administration Method to Optimize the Number of Responses and Enhance Clinical Inferences.

UNLABELLED: Variability in the number of Rorschach responses (R) has stimulated controversy among clinicians and researchers for many years, and recent research reveals that R is much more variable than previously thought. Because R is correlated with other scores, its excessive variability may reduce the reliability, validity and clinical utility of these other scores. We present two experimental studies and additional results from other clinical datasets with the aim of developing a new administration procedure to diminish variability in R by reducing the number of very short and long records. In the first experiment, protocols were obtained using standard Comprehensive System administration or an alternative where we encouraged a second response if only one was given to a card and allowed only four responses on each card. This alternative method reduced the proportion of short records but produced an undesirable number of long records. To minimize the proportion of long records, in a second experiment, we added an instruction to give two or maybe three responses per card when introducing the test. Comparisons to CS administrations revealed that this procedure reduced variability in R by limiting the proportion of both short and long records. This reduced range was largely retained in an outpatient sample of older respondents with schizophrenia and a mixed clinical sample. Thus, we recommend this method of optimizing the range of R, which has since been included with very minor changes in the Rorschach Performance Assessment System. KEY PRACTITIONER MESSAGE: Alternative administration method successfully eliminated overly short and excessively long records. Utility is potentially increased by greatly reducing both short records that often lack reliability and validity, as well as long records that consume an excessive amount of examiner administration and scoring time. Psychometric properties and the ability to apply parametric statistics are likely increased across variables given that the distribution of R is more normal. Re-administration due to inadequate R is almost never needed. Results are consistent with the conclusion that this alternative procedure reduces examiner variability by offering simple, but explicit instructions for encouraging sufficient productivity. Overall variability of R produced using the refined alternative procedure was significantly less than that produced using the traditional CS method, although more in line with Exner's (2003) normative expectations. Suggests that when using the alternative method, R becomes less of a confound for all other scores that are moderately to highly correlated with R. Also demonstrated that the reduced variability of R and the reduced number of less useful short and long records are generalized to clinical samples. Additional research (Reese, Viglione, & Giromini, 2014) provides support for these conclusions with child clinical samples.

A comparison between comprehensive system and an early version of the rorschach performance assessment system administration with outpatient children and adolescents.

For many years, the effects of variability in the length of Rorschach records has been debated, and a new administration procedure aimed at reducing the proportion of short and long records has recently been introduced. Using an outpatient sample of children and adolescents, this study explored the impact of an early version of the Rorschach Performance Assessment System (R-PAS) administration, on the central tendencies of Rorschach variables. Specifically, the mean values of 51 variables in 142 Comprehensive System versus 99 R-PAS collected records were compared with each other. Results found comparable mean values across CS and R-PAS administration methods for the variables that guide interpretation with children and adolescents. Both methods produced a comparable number of long (> 27 responses) records. The only relevant difference that emerged is that the early R-PAS administration version yielded significantly fewer short (14-16 responses) records and lower variability in the number of responses.

Measuring psychological development with the Rorschach.

The aim of this study was to develop and to provide an initial validation of a Rorschach index measuring developmental progress and growth. The Developmental Index (DI) was created in a 3-step, sequential strategy with adult and child data in which we (a) selected potential DI variables from quantitative research literature and from data available to us, (b) identified an optimal group of DI variables and created the DI equation using an independent nonpatient sample, and (c) provided an initial cross-validation of the DI using an independent clinical sample. Age and age rank categorization groups from normative data associated with the Wechsler intelligence scales (Wechsler, 2003, 2008) and contrasts between adults and children served as criteria for development. These samples include a large amount of data from a diverse international subject pool using the Comprehensive System of the Rorschach. Interim validity checks were undertaken to ensure the analytic strategy was sound. The DI includes 12 variables with individual weights determined by regression analysis. The initial independent cross-validation of the DI with a clinical sample revealed that it discriminated well between children and adults. Thus, initial support for this scale as a measure of psychological development across cultures and geography was established, but future research is needed. Clinical implications and specific research needs are presented.

Mirroring activity in the brain and movement determinant in the Rorschach test.

Human movement (M) responses to the Rorschach are related to cognitive sophistication, creativity, and empathy. Recent studies also link Ms to EEG-mu suppression, an index of mirroring activity in the brain. In this article, we further investigate the link between Ms and mu suppression by testing some clinical interpretative distinctions. Previously collected EEG data recorded during the administration of the Rorschach were reanalyzed. We hypothesized that (a) among several responses investigated, only M would be associated with mu suppression, and (b) Ms with active movement, ordinary form quality, or whole human figures would be most strongly associated with mu suppression. Hypothesis 1 was fully confirmed, thus supporting that the traditional interpretation of M has a neurobiological foundation. Hypothesis 2 was partially confirmed; that is, active Ms were associated with mu suppression more strongly than passive Ms (p < .05), but no other significant differences emerged. Clinical implications are discussed.

Rorschach scores in applied clinical practice: a survey of perceived validity by experienced clinicians.

We surveyed practicing clinicians who regularly used the Rorschach about the perceived clinical validity of specific Rorschach scores from many coding systems. The survey included quantitative feedback on the validity of specific variables as well as qualitative input in several areas, including the validity of specific variables, the potentially unique information that can be obtained from them, coding challenges associated with Comprehensive System (CS) codes, and recommendations for CS developments. Participants were recruited by applying a snowball sampling strategy. Based on responses from 246 experienced clinicians from 26 countries, composite judgments on rated variables were quite reliable (e.g., M α = .95 across 88 CS variables), despite limited agreement among any 2 judges. The aggregated judgments clearly differentiated among scores that were considered more and less clinically valid and the overall results aligned with recently obtained meta-analytic conclusions from the traditional validity literature (Mihura, Meyer, Dumitrascu, & Bombel, 2012). The judges also provided guidance concerning revisions and enhancements that would facilitate Rorschach-based assessment in the future. We discuss the implication of the quantitative and qualitative findings and provide suggestions for future directions based on the results.