Psychometric Assessment of an Item Bank for Adaptive Testing on Patient-Reported Experience of Care Environment for Severe Mental Illness: Validation Study.

BACKGROUND: The care environment significantly influences the experiences of patients with severe mental illness and the quality of their care. While a welcoming and stimulating environment enhances patient satisfaction and health outcomes, psychiatric facilities often prioritize staff workflow over patient needs. Addressing these challenges is crucial to improving patient experiences and outcomes in mental health care. OBJECTIVE: This study is part of the Patient-Reported Experience Measure for Improving Quality of Care in Mental Health (PREMIUM) project and aims to establish an item bank (PREMIUM-CE) and to develop computerized adaptive tests (CATs) to measure the experience of the care environment of adult patients with schizophrenia, bipolar disorder, or major depressive disorder. METHODS: We performed psychometric analyses including assessments of item response theory (IRT) model assumptions, IRT model fit, differential item functioning (DIF), item bank validity, and CAT simulations. RESULTS: In this multicenter cross-sectional study, 498 patients were recruited from outpatient and inpatient settings. The final PREMIUM-CE 13-item bank was sufficiently unidimensional (root mean square error of approximation=0.082, 95% CI 0.067-0.097; comparative fit index=0.974; Tucker-Lewis index=0.968) and showed an adequate fit to the IRT model (infit mean square statistic ranging between 0.7 and 1.0). DIF analysis revealed no item biases according to gender, health care settings, diagnosis, or mode of study participation. PREMIUM-CE scores correlated strongly with satisfaction measures (r=0.69-0.78; P<.001) and weakly with quality-of-life measures (r=0.11-0.21; P<.001). CAT simulations showed a strong correlation (r=0.98) between CAT scores and those of the full item bank, and around 79.5% (396/498) of the participants obtained a reliable score with the administration of an average of 7 items. CONCLUSIONS: The PREMIUM-CE item bank and its CAT version have shown excellent psychometric properties, making them reliable measures for evaluating the patient experience of the care environment among adults with severe mental illness in both outpatient and inpatient settings. These measures are a valuable addition to the existing landscape of patient experience assessment, capturing what truly matters to patients and enhancing the understanding of their care experiences. TRIAL REGISTRATION: ClinicalTrials.gov NCT02491866; https://clinicaltrials.gov/study/NCT02491866.

Development of the PREMIUM computerized adaptive testing for measuring the access and care coordination for patients with severe mental illness.

Severe mental illness (SMI) patients often have complex health needs, which makes it difficult to access and coordinate their care. This study aimed to develop a computerized adaptive testing (CAT) tool, PREMIUM CAT-ACC, to measure SMI patients' experience with access and care coordination. This multicenter and cross-sectional study included 496 adult in- and out-patients with SMI (i.e., schizophrenia, bipolar disorder, or major depressive disorder). Psychometric analysis of the 13-item bank showed adequate properties, with preliminary evidence of external validity and no substantial differential item functioning for sex, age, care setting, and diagnosis, making it suitable for CAT administration. A post-hoc CAT simulation demonstrated that the tool was efficient and accurate, with an average of seven items, compared to the full item bank administration. Its use by clinicians can contribute to optimizing patient care pathways and transitioning towards more person-centered healthcare.

Calibration and Validation of a PREMIUM-DT Item Bank to Measure the Experience of Drug Therapy for Patients with Severe Mental Illness.

The aim of this study was to (1) calibrate an item bank to measure patients' experience of drug therapy for adult patients with SMIs and (2) develop computerized adaptive testing (CAT) to improve its use in routine practice. This is a cross-sectional, multicentric study involving 541 patients with schizophrenia, bipolar disorder, and major depressive disorder. Analyses based on classical test and item response theories were performed. After 7 highly inter-correlated items and 4 items with low factor loadings were removed, the remaining 26 items were sufficiently unidimensional (RMSEA = 0.069, CFI = 0.969, TLI = 0.963) and showed adequate fit to the generalized partial credit model. There was no differential item functioning by gender, age, care setting, or diagnosis from moderate- to large-magnitude. The mean score was 46.0 ± 16.9 and was significantly higher for patients reporting good medication adherence. The resulting PREMIUM-DT item bank has strong psychometric properties, and CAT facilitates widespread use in clinical settings (an average of 8 items administered, corresponding to a reliability of >0.90). Our results suggest that practical information and information about the side effects of psychotropic treatments and how to cope with them should be targeted as a priority to improve patients' experience of drug therapy.