Current obstacles in replicating risk assessment findings: a systematic review of commonly used actuarial instruments.

An actuarial risk assessment instrument can be considered valid if independent investigations using novel samples can replicate the findings of the instrument's development study. In order for a study to qualify as a replication, it has to adhere to the methodological protocol of the development study with respect to key design characteristics, as well as ensuring that manual-recommended guidelines of test administration have been followed. A systematic search was conducted to identify predictive validity studies (N = 84) on three commonly used actuarial instruments: the Violence Risk Appraisal Guide (VRAG), the Sex Offender Risk Appraisal Guide (SORAG), and the Static-99. Sample (sex, age, criminal history) and design (follow-up, attrition, recidivism) characteristics, as well as markers of assessment integrity (scoring reliability, item omissions, prorating procedure), were extracted from 84 studies comprising 108 samples. None of the replications matched the development study of the instrument they were attempting to cross-validate with respect to key sample and design characteristics. Furthermore none of the replications strictly followed the manual-recommended guidelines for the instruments' administration. Additional replication studies that follow the methodological protocols outlined in actuarial instruments' development studies are needed before claims of generalizability can be made.

Internet-based self-assessment after the Tsunami: lessons learned.

BACKGROUND: In the aftermath of the Tsunami disaster in 2004, an online psychological self-assessment (ONSET) was developed and made available by the University of Zurich in order to provide an online screening instrument for Tsunami victims to test if they were traumatized and in need of mental health care. The objective of the study was to report the lessons learnt that were made using an Internet-based, self-screening instrument after a large-scale disaster and to discuss its outreach and usefulness. METHODS: Users of the online self-assessment decided after finishing the procedure whether their dataset could be used for quality control and scientific evaluation Their answers were stored anonymously only if they consented (which was the case in 88% of the sample), stratified analyses according to level of exposure were conducted. RESULTS: A total of 2,914 adult users gave their consent for analysis of the screenings. Almost three quarter of the sample filled out the ONSET questionnaire within the first four weeks. Forty-one percent of the users reported direct exposure to the Tsunami disaster. Users who were injured by the Tsunami and users who reported dead or injured family members showed the highest degree of PTSD symptoms. CONCLUSION: ONSET was used by a large number of subjects who thought to be affected by the catastrophe in order to help them decide if they needed to see a mental health professional. Furthermore, men more frequently accessed the instrument than women, indicating that Internet-based testing facilitates reaching out to a different group of people than "ordinary" public mental health strategies.

The prediction of criminal recidivism: the implication of sampling in prognostic models.

BACKGROUND: Instruments based on actuarial forensic risk assessment models are sensitive to the calibration sample, and the inclusion criteria for the subjects of a study population will influence the features of the resulting model. If the same instrument is used in populations that are not part of the calibration sample, the discriminative validity of the instrument is jeopardized; thus the definition of the calibration sample is very important. The aim of this study was to examine how sensitive prognostic models are to the calibration sample. METHOD: Two samples (N = 773) of offenders sentenced to at least 10 months in prison for a violent or sexual offense were used in this study. The "sanction sample" (recruited during August 2000, N = 515) consisted of all violent and sexual offenders actively administrated by the Criminal Justice System of Zurich, Switzerland. The "verdict sample" (recruited over two years, N = 258) included all offenders convicted in the Canton of Zurich during a two-year period. Both samples were unbiased, since all subjects that met the study criteria were included. In the first analysis, differences between the two samples were shown with respect to socio-demographic, criminological, and psychiatric variables using bivariate logistic regressions. In the second analysis, recidivism was estimated separately for both samples, using a logistic regression model as a function of a set of psychiatric, socio-demographic and criminological variables. RESULTS: Bivariate logistic regression showed that different risk factors for recidivism existed for both samples. CONCLUSION: Forensic risk assessment models are very sensitive to the calibration sample. There is strong evidence that, even when index-offenses and the socio-cultural background are the same, risk factors for recidivism differ depending on the stage of the judicial process in which the subjects are (e.g. whether a subject is indicted, on conditional release, on parole, or no longer under the supervision of a parole board). Unfortunately, none of the currently available actuarial risk assessment instruments that have been validated in European countries consider the different stages of the judiciary process.