Machine learning item selection for short scale construction: A proof-of-concept using the SIMS.

ObjectiveThis proof-of-concept paper provides evidence to support machine learning (ML) as a valid alternative to traditional psychometric techniques in the development of short forms of longer parent psychological tests. ML comprises a variety of feature selection techniques that can be efficiently applied to identify the set of items that best replicates the characteristics of the original test. MethodsIn the present study, we integrated a dataset of 329 participants from published and unpublished datasets used in previous research on the Structured Inventory of Malingered Symptomatology (SIMS) to develop a short version of the scale. The SIMS is a multi-axial self-report questionnaire and a highly efficient psychometric measure of symptom validity, which is frequently applied in forensic settings. Results State-of-the-art ML item selection techniques achieved a 72% reduction in length while capturing 92% of the variance of the original SIMS. The new SIMS short form now consists of 21 items. ConclusionsThe results suggest that the proposed ML-based item selection technique represents a promising alternative to standard psychometric correlation-based methods (i.e. item selection, item response theory), especially when selection techniques (e.g. wrapper) are employed that evaluate global, rather than local, item value.

Detecting malingering mental illness in forensics: Known-Group Comparison and Simulation Design with MMPI-2, SIMS and NIM.

BACKGROUND: Criminal defendants may often exaggerate psychiatric symptoms either to appear non-accountable for their actions or to mitigate their imprisonment. Several psychometric tests have been proposed to detect malingering. These instruments are often validated by Simulation Design (SD) protocols, where normal participants are explicitly requested to either simulate a mental disorder or respond honestly. However, the real scenarios (clinical or forensic) are often very challenging because of the presence of genuine patients, so that tests accuracy frequently differs from that one obtained in well-controlled experimental settings. Here we assessed the effectiveness in criminal defendants of three well-known malingering-detecting tests (MMPI-2, SIMS and NIM) by using both Known-Group Comparison (KGC) and Simulation Design (SD) protocols. METHODS: The study involved 151 male inmates. Participants to the KGC protocol were all characterized by a positive psychiatric history. They were considered as genuine patients (KGC_Controls) if they had some psychiatric disorders already before imprisonment and scored above the cutoff of SCL-90-R, a commonly used test for mental illness, and as suspected malingerers (KGC_SM) if they were diagnosed as psychiatric patients only after imprisonment and scored below the SCL-90-R cutoff. Participants to SD protocol had no history of psychiatric disease and scored below the SCL-90-R cutoff. They were randomly assigned to either group: Controls (requested to answer honestly, SD_Controls) and simulated malingerers (requested to feign a psychiatric disease, SD_SM). All participants were then submitted to MMPI-2, NIM and SIMS. RESULTS: Results showed that while MMPI-2, SIMS and NIM were all effective in discriminating malingerers in the SD, SIMS only significantly discriminated between KGC_Controls and KGC_SM in the Known-Group Comparison. Receiver Operating Characteristic (ROC) curves analysis confirmed the better sensitivity of SIMS with respect to the other tests but raised some issues on SIMS specificity. DISCUSSION: Results support the sensitivity of SIMS for the detection of malingering in forensic populations. However, some specificity issues emerged suggesting that further research and a good forensic practice should keep into account multiple measures of malingering, including psychometric data, clinical and social history and current clinical situation. These methodological constraints must be kept in mind during detection of malingering in criminal defendants reporting psychiatric symptoms.