Extending the PROMIS item bank "ability to participate in social roles and activities": a psychometric evaluation using IRT.

OBJECTIVE: Our objective was to explore whether the extension of the PROMIS item bank Ability to Participate in Social Roles and Activities (APSRA) with new items would result in more effective targeting (i.e., selecting items that are appropriate for each individual's trait level), and more reliable measurements across all latent trait levels. METHODS: A sample of 1,022 Dutch adults completed all 35 items of the original item bank plus 17 new items (in Dutch). The new items presented in this publication have been translated provisionally from Dutch into English for presentation purposes. We evaluated the basic IRT assumptions unidimensionality, local independence, and monotonicity. Furthermore, we examined the item parameters, and assessed differential item functioning (DIF) for sex, education, region, age, and ethnicity. In addition, we compared the test information functions, item parameters, and θ scores, for the original and extended item bank in order to assess whether the measurement range had improved. RESULTS: We found that the extended item bank was compatible with the basic IRT assumptions and showed good reliability. Moreover, the extended item bank improved the measurement in the lower trait range, which is important for reliably assessing functioning in clinical populations (i.e., persons reporting lower levels of participation). CONCLUSION: We extended the PROMIS-APSRA item bank and improved its psychometric quality. Our study contributes to PROMIS measurement innovation, which allows for the addition of new items to existing item banks, without changing the interpretation of the scores and while maintaining the comparability of the scores with other PROMIS instruments.

Evaluating the Construct Validity of the Norwegian Version of the Level of Personality Functioning Scale - Brief Form 2.0 in a Large Clinical Sample.

The Level of Personality Functioning - Brief Form 2.0 (LPFS-BF 2.0) is a 12-item self-report questionnaire developed to gain a quick impression of the severity of personality pathology according to the DSM-5 Alternative Model for Personality Disorders (AMPD). The current study evaluated the construct validity and reliability of the Norwegian version of the LPFS-BF 2.0 in a large clinical sample (N = 1673). Dimensionality was examined using confirmatory factor analysis and bifactor analysis followed by an analysis of distinctiveness of the subscales using the proportional reduction in mean squared error (PRMSE), and the concurrent validity was examined using correlations with self-report questionnaires and clinical interviews assessing PDs according to section II of the DSM-5. Taking the findings of the dimensionality and concurrent validity results together, we found moderate to good support for the use of total scores for the Norwegian version of the LPFS-BF 2.0. We would advise against the use of subscale scores, since the subscales provided only a small amount of reliable unique variance.

A systematic review of measurement uncertainty visualizations in the context of standardized assessments.

This systematic review summarized findings of 29 studies evaluating visual presentation formats appropriate for communicating measurement uncertainty associated with standardized clinical assessment instruments. Studies were identified through systematic searches of multiple databases (Medline, Embase, PsycInfo, ERIC, Scopus, and Web of Science). Strikingly, we found no studies which were conducted using samples of clinicians and included clinical decision-making scenarios. Included studies did however find that providing participants with information about measurement uncertainty may increase awareness of uncertainty and promote more optimal decision making. Formats which visualize the shape of the underlying probability distribution were found to promote more accurate probability estimation and appropriate interpretations of the underlying probability distribution shape. However, participants in the included studies did not seem to benefit from the additional information provided by such plots during decision-making tasks. Further explorations into how presentations of measurement uncertainty impact clinical decision making are needed to examine whether findings of the included studies generalize to clinician populations. This review provides an important overview of pitfalls associated with formats commonly used to communicate measurement uncertainty in clinical assessment instruments, and a potential starting point for further explorations into promising alternatives. Finally, our review offers specific recommendations on how remaining research questions might be addressed.

Visualizing Uncertainty to Promote Clinicians' Understanding of Measurement Error.

Measurement error is an inherent part of any test score. This uncertainty is generally communicated in ways that can be difficult to understand for clinical practitioners. In this empirical study, we evaluate the impact of several communication formats on the interpretation of measurement accuracy and its influence on the decision-making process in clinical practice. We provided 230 clinical practitioners with score reports in five formats: textual, error bar, violin plot, diamond plot, and quantile dot plot. We found that quantile dot plots significantly increased accuracy in the assessment of measurement uncertainty compared with other formats. However, a direct relation between visualization format and decision quality could not be found. Although traditional confidence intervals and error bars were favored by many participants due to their familiarity, responses revealed several misconceptions that make the suitability of these formats for communicating uncertainty questionable. Our results indicate that new visualization formats can successfully reduce errors in interpretation.

The Incremental Utility of Criteria A and B of the DSM-5 Alternative Model for Personality Disorders for Predicting DSM-IV/DSM-5 Section II Personality Disorders.

The DSM-5 Alternative Model for Personality Disorders (AMPD) includes two main criteria: moderate or greater impairment in personality functioning (Criterion A) and the presence of one or more pathological personality traits (Criterion B). The aim of the study was to investigate the incremental utility of Criteria A and B for predicting DSM-5 Section II personality disorders (PD). The sample (N = 317) consisted of three well-defined groups: non-clinical participants (n = 35), psychiatric patients with PD (n = 193), and without PD (n = 83). All were assessed using the Structured Clinical Interview for the DSM-5 Alternative Model for Personality Disorders Module I (SCID-5-AMPD-I): Level of Personality Functioning Scale (LPFS), and the Personality Inventory for DSM-5 (PID-5). Logistic regression analyses showed that the SCID-5-AMPD-I could predict the presence of PDs in general, and the three specific PDs that were investigated (i.e., Antisocial, Borderline, and Avoidant PDs). The PID-5 domains enhanced prediction of the specific PDs, but not the presence of PDs in general, when entered in the second step. Our results support the AMPD model: Criterion A predicted the presence of DSM-5 Section II PDs in general, whereas measures of Criterion B incremented prediction of Antisocial, Borderline, and Avoidant PDs.

Do Flexible Administration Procedures Promote Individualized Clinical Assessments? An Explorative Analysis of How Clinicians Utilize the Funnel Structure of the SCID-5-AMPD Module I: LPFS.

The current study examined clinicians' utilization of the SCID-5-AMPD-I funnel structure. Across 237 interviews, conducted as part of the NorAMP study, we found that clinicians administered on average 2-3 adjacent levels under each subdomain, effectively administering only about 50% of available items. Comparing administration patterns of interviews, no two interviews contained the exact same set of administered items. On average, when comparing individual interviews, only about half of the administered items in each interview were administered in both interviews. Cross-classified mixed effects models were estimated to examine the factors affecting item administration. Results indicated that the interplay between patient preliminary scores and item level had a substantial impact on item administration, suggesting clinicians tend to administer items corresponding to expected patient severity. Overall, our findings suggest clinicians utilize the SCID-5-AMPD-I funnel structure to conduct efficient and individually tailored assessments informed by relevant patient characteristics. Adopting similar non-fixed administration procedures for other interviews could potentially provide similar benefits compared to traditional fixed-form administration procedures. The current study can serve as a template for verifying and evaluating future adoptions of non-fixed administration procedures in other interviews.

Empirical Priors in Polytomous Computerized Adaptive Tests: Risks and Rewards in Clinical Settings.

The use of empirical prior information about participants has been shown to substantially improve the efficiency of computerized adaptive tests (CATs) in educational settings. However, it is unclear how these results translate to clinical settings, where small item banks with highly informative polytomous items often lead to very short CATs. We explored the risks and rewards of using prior information in CAT in two simulation studies, rooted in applied clinical examples. In the first simulation, prior precision and bias in the prior location were manipulated independently. Our results show that a precise personalized prior can meaningfully increase CAT efficiency. However, this reward comes with the potential risk of overconfidence in wrong empirical information (i.e., using a precise severely biased prior), which can lead to unnecessarily long tests, or severely biased estimates. The latter risk can be mitigated by setting a minimum number of items that are to be administered during the CAT, or by setting a less precise prior; be it at the expense of canceling out any efficiency gains. The second simulation, with more realistic bias and precision combinations in the empirical prior, places the prevalence of the potential risks in context. With similar estimation bias, an empirical prior reduced CAT test length, compared to a standard normal prior, in 68% of cases, by a median of 20%; while test length increased in only 3% of cases. The use of prior information in CAT seems to be a feasible and simple method to reduce test burden for patients and clinical practitioners alike.

Symptom burden and health-related quality of life in chronic kidney disease: A global systematic review and meta-analysis.

BACKGROUND: The importance of patient-reported outcome measurement in chronic kidney disease (CKD) populations has been established. However, there remains a lack of research that has synthesised data around CKD-specific symptom and health-related quality of life (HRQOL) burden globally, to inform focused measurement of the most relevant patient-important information in a way that minimises patient burden. The aim of this review was to synthesise symptom prevalence/severity and HRQOL data across the following CKD clinical groups globally: (1) stage 1-5 and not on renal replacement therapy (RRT), (2) receiving dialysis, or (3) in receipt of a kidney transplant. METHODS AND FINDINGS: MEDLINE, PsycINFO, and CINAHL were searched for English-language cross-sectional/longitudinal studies reporting prevalence and/or severity of symptoms and/or HRQOL in CKD, published between January 2000 and September 2021, including adult patients with CKD, and measuring symptom prevalence/severity and/or HRQOL using a patient-reported outcome measure (PROM). Random effects meta-analyses were used to pool data, stratified by CKD group: not on RRT, receiving dialysis, or in receipt of a kidney transplant. Methodological quality of included studies was assessed using the Joanna Briggs Institute Critical Appraisal Checklist for Studies Reporting Prevalence Data, and an exploration of publication bias performed. The search identified 1,529 studies, of which 449, with 199,147 participants from 62 countries, were included in the analysis. Studies used 67 different symptom and HRQOL outcome measures, which provided data on 68 reported symptoms. Random effects meta-analyses highlighted the considerable symptom and HRQOL burden associated with CKD, with fatigue particularly prevalent, both in patients not on RRT (14 studies, 4,139 participants: 70%, 95% CI 60%-79%) and those receiving dialysis (21 studies, 2,943 participants: 70%, 95% CI 64%-76%). A number of symptoms were significantly (p < 0.05 after adjustment for multiple testing) less prevalent and/or less severe within the post-transplantation population, which may suggest attribution to CKD (fatigue, depression, itching, poor mobility, poor sleep, and dry mouth). Quality of life was commonly lower in patients on dialysis (36-Item Short Form Health Survey [SF-36] Mental Component Summary [MCS] 45.7 [95% CI 45.5-45.8]; SF-36 Physical Component Summary [PCS] 35.5 [95% CI 35.3-35.6]; 91 studies, 32,105 participants for MCS and PCS) than in other CKD populations (patients not on RRT: SF-36 MCS 66.6 [95% CI 66.5-66.6], p = 0.002; PCS 66.3 [95% CI 66.2-66.4], p = 0.002; 39 studies, 24,600 participants; transplant: MCS 50.0 [95% CI 49.9-50.1], p = 0.002; PCS 48.0 [95% CI 47.9-48.1], p = 0.002; 39 studies, 9,664 participants). Limitations of the analysis are the relatively few studies contributing to symptom severity estimates and inconsistent use of PROMs (different measures and time points) across the included literature, which hindered interpretation. CONCLUSIONS: The main findings highlight the considerable symptom and HRQOL burden associated with CKD. The synthesis provides a detailed overview of the symptom/HRQOL profile across clinical groups, which may support healthcare professionals when discussing, measuring, and managing the potential treatment burden associated with CKD. PROTOCOL REGISTRATION: PROSPERO CRD42020164737.

More is more: Evidence for the incremental value of the SCID-II/SCID-5-PD specific factors over and above a general personality disorder factor.

Currently, 3 competing conceptualizations of personality dysfunction can be distinguished: the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) categorical model delineating 10 distinct types of personality disorders (PDs); the alternative model for PDs (DSM-5 Section III), which assesses personality functioning and traits separately; and the International Classification of Diseases, 11th Version conceptualization, which provides 1 single code for the presence of a PD (which is based on problems in functioning) as well as codes that specify the level of the disorder (mild/moderate/severe), and prominent trait domains or patterns (5 domains and 1 pattern). The current study aims to assess the incremental value of the DSM-5 PDs over and above a global personality dysfunction factor, using expert ratings obtained with the Structured Clinical Interview for DSM-IV PDs and the Structured Clinical Interview for DSM-5 PDs interview in a large sample of clinical patients (N = 3,851). All estimated bifactor models provided adequate fit to the data. We found a surprisingly low explained common variance for the g-factor (<40%), indicating that ignoring the specific PD factors would lead to a substantial loss of information. The strongest specific PDs in terms of explained common variance were the avoidant, schizotypal, and schizoid PD factors and the conduct disorder criteria set if included. Correlations between our factors and external variables were relatively low, except for the Severity Indices of Personality Problems, which aims to measure personality functioning. Our findings suggest that specific PDs still have an important role to play in the assessment of personality pathology. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Norwegian Clinicians' Experiences of Learnability and Usability of SCID-II, SCID-5-PD and SCID-5-AMPD-I Interviews: A Sequential Multi-Group Qualitative Approach.

The DSM-5 presents two competing diagnostic frameworks for personality disorders: the standard categorical model and the Alternative Model of Personality Disorders (AMPD). The AMPD was initially criticized for being too complex and theory laden for clinical implementation. Though inter-rater reliability studies have contested initial claims of the model's complexity, little attention has been paid to how clinicians experience the usability and learnability of either model. We interviewed twenty Norwegian clinicians about their experiences with either the SCID-II/5-PD (n = 9), SCID-5-AMPD-I (n = 8), or both (n = 3). Separate thematic analyses were conducted for SCID-II/5-PD and SCID-5-AMPD-I groups, and group themes were compared. We identified four themes for each group, relating to required skills, training, challenges and information gained through the interview. We found that training and clinical experience were considered to be important for both interviews. Moreover, the SCID-5-AMPD-I was considered to rely more explicitly on theory specific to the development and content of the AMPD model in general and the LPFS specifically We also identified shared and unique challenges and shortcomings of each interview. We comment on how our findings relate to the debate surrounding the AMPD, and recommend development of clear training guidelines for both interviews.

Inventory of assessment practices in people with profound intellectual and multiple disabilities in three European countries.

BACKGROUND: Knowledge about the quality of assessment methods used in the support of people with profound intellectual and multiple disabilities (PIMD) is scarce. This study aimed to provide an overview of the assessment methods used in practice and to examine whether these instruments were studied for their psychometric properties for people with PIMD. METHOD: Professionals (N = 148) from three European countries completed a survey on assessment practices. We performed a literature search to find information about the psychometric properties of the instruments that were identified in the survey. RESULTS: Of the participants, 78.1% used assessments that were not developed for people with PIMD. Documentation on psychometric properties was found for 8 out of 116 instruments. CONCLUSIONS: Most of the instruments in use were not designed for people with PIMD, and information about their quality is lacking. Guidelines are needed regarding the use and development of assessment methods for people with PIMD.

Support for the higher-order factor structure of the WHODAS 2.0 self-report version in a Dutch outpatient psychiatric setting.

PURPOSE: Previous studies of the World Health Organization Disability Assessment Schedule 2.0 (WHODAS 2.0) interview version suggested a second-order model, with a general disability factor and six factors on a lower level. The goal of this study is to investigate if we can find support for a similar higher-order factor structure of the 36-item self-report version of the WHODAS 2.0 in a Dutch psychiatric outpatient sample. We aim to give special attention to the differences between the non-working group sample and the working group sample. Additionally, we intend to provide preliminary norms for clinical interpretation of the WHODAS 2.0 scores in psychiatric settings. METHODS: Patients seeking specialized ambulatory treatment, primarily for depressive or anxiety symptoms, completed the WHODAS 2.0 as part of the initial interview. The total sample consisted of 770 patients with a mean age of 37.5 years (SD = 13.3) of whom 280 were males and 490 were females. Several factorial compositions (i.e., one unidimensional model and two second-order models) were modeled using confirmatory factor analysis (CFA). Descriptive statistics, model-fit statistics, reliability of the (sub)scales, and preliminary norms for interpreting test scores are reported. RESULTS: For the non-working group, the second-order model with a general disability factor and six factors on a lower level, provided an adequate fit. Whereas, for the working group, the second-order model with a general disability factor and seven factors on a lower level seemed more appropriate. The WHODAS 2.0 36-item self-report form showed adequate levels of reliability. Percentile ranks and normalized T-scores are provided to aid clinical evaluations. CONCLUSION: Our results lend support for a factorial structure of the WHODAS 2.0 36-item self-report version that is comparable to the interview version. While we conjecture that a seven-factor solution might give a better reflection of item content and item variance, further research is needed to assess the clinical relevance of such a model. At this point, we recommend using the second-order structure with six factors that matches past findings of the interview form.

The importance of harmonising diagnostic criteria sets for pathological grief.

Five diagnostic criteria sets for pathological grief are currently used in research. Studies evaluating their performance indicate that it is not justified to generalise findings regarding prevalence rates and predictive validity across studies using different diagnostic criteria of pathological grief. We provide recommendations to move the bereavement field forward.

Psychometric Properties of the Externalizing Spectrum Inventory: Replication and Extension across Clinical and Non-Clinical Samples.

The Externalizing Spectrum Inventory aims at assessing personality features that underlie externalizing disorders such as substance abuse and antisocial behaviors. The objective was to replicate the psychometric properties of the 160-item Externalizing Spectrum Inventory in Dutch clinical and non-clinical samples. First, Cronbach's alpha, test-retest reliability and the factor structure were analyzed on a mixed sample of inpatients (n = 149), undergraduates (n = 227), and community participants (n = 178). The factor structure was evaluated through confirmatory and exploratory factor analyses; for the latter Parallel Analysis was used, based on Minimum Rank Factor Analysis. Next, the criterion validity was analyzed using the Aggression Questionnaire and the NEO-Five Factor Inventory as external measures. The Dutch Externalizing Spectrum Inventory subscales showed sufficient reliability (α=.68-.94; ICC=.68-.91), except in the undergraduate sample (α=.49-.96; ICC=.43-.97). The factor structure of the Externalizing Spectrum Inventory was not confirmed and the exploratory analysis yielded different factor solutions across samples. The criterion validity was supported with regard to trait aggression and partly supported with regard to the Five Factor Model. The results suggest that the ESI-160 and its original factor model can be used for prediction purposes. However, further research of the factor structure is strongly recommended.

A Psychometric Analysis of the Structured Clinical Interview for the DSM-5 Alternative Model for Personality Disorders Module I (SCID-5-AMPD-I): Level of Personality Functioning Scale.

The current study aims to examine the psychometric properties of the Structured Clinical Interview for the DSM-5 Alternative Model for Personality Disorders Module I (SCID-5-AMPD-I) assessing the Level of Personality Functioning Scale (LPFS) in a heterogeneous sample of 282 nonpsychotic patients. Latent variable models were used to investigate the dimensionality of the LPFS. The results indicate that the LPFS, as assessed by the SCID-5-AMPD-I, can be considered as a unidimensional construct that can be measured reliably across a wide range of the latent trait. Threshold parameters for the 12 indicators of the LPFS increased gradually over the latent scale, indicating that the five LPFS levels were ordered as predicted by the model. In general, the increase of threshold parameters was relatively small for the shift from Level 2 to Level 3. A better distinction among the different severity levels might be obtained by fine-tuning the interview guidelines or the Level 2 indicators themselves.

A multi-center psychometric evaluation of the Severity Indices of Personality Problems 118 (SIPP-118): Do we really need all those facets?

PURPOSE: The Severity Indices of Personality Problems 118 (SIPP-118) is a self-report questionnaire that aims to measure core components of (mal)adaptive personality functioning that can change over time. In this study, we aimed to assess the facet strength of the 16 facets across three large clinical samples. METHODS: Data from Norwegian and Dutch psychiatric patients were analyzed in this international multi-center study (N1 = 2814, N2 = 4751, N3 = 2217). Bi-factor modeling was used to assess to what degree the SIPP items tap into an overall general factor. The incremental value (distinctiveness) of the facets was studied using proportional reduction in mean squared error (PRMSE) based statistics. RESULTS: The estimated model showed adequate fit. The explained common variance (ECV) attributable to the general factor equaled 50% for all three samples. All but two facets (stable self-image and frustration tolerance) showed sufficient levels of distinctiveness. The findings were observed to be comparable across the three samples. CONCLUSION: Our findings showed that the general factor was relatively weak, and the facets had a clear incremental value.

Making Fixed-Precision Between-Item Multidimensional Computerized Adaptive Tests Even Shorter by Reducing the Asymmetry Between Selection and Stopping Rules.

Fixed-precision between-item multidimensional computerized adaptive tests (MCATs) are becoming increasingly popular. The current generation of item-selection rules used in these types of MCATs typically optimize a single-valued objective criterion for multivariate precision (e.g., Fisher information volume). In contrast, when all dimensions are of interest, the stopping rule is typically defined in terms of a required fixed marginal precision per dimension. This asymmetry between multivariate precision for selection and marginal precision for stopping, which is not present in unidimensional computerized adaptive tests, has received little attention thus far. In this article, we will discuss this selection-stopping asymmetry and its consequences, and introduce and evaluate three alternative item-selection approaches. These alternatives are computationally inexpensive, easy to communicate and implement, and result in effective fixed-marginal-precision MCATs that are shorter in test length than with the current generation of item-selection approaches.

Evaluation of Diagnostic Thresholds for Criterion A in the Alternative DSM-5 Model for Personality Disorders.

The Level of Personality Functioning Scale (LPFS) of the Alternative DSM-5 Model for Personality Disorders (AMPD) was formulated to assess the presence and severity of personality disorders (PDs). Moderate impairment (Level 2) in personality functioning, as measured by the LPFS, was incorporated into the AMPD as a diagnostic threshold for PD in Criterion A of the general criteria, as well as for the "any two areas present" rule for assigning a specific PD diagnosis. This study represents the first evaluation of the diagnostic decision rules for Criterion A, in a clinical sample (N = 282). The results indicate that an overall diagnostic threshold for PDs should be used with caution because it may not identify all DSM-IV PDs. The "any two areas present" rule proved to be a reasonable alternative, although this finding should be interpreted with caution because the LPFS does not measure the disorder-specific A criteria.

A Psychometric Evaluation of the DSM-IV Criteria for Antisocial Personality Disorder: Dimensionality, Local Reliability, and Differential Item Functioning Across Gender.

This study aims at evaluating the psychometric properties of the antisocial personality disorder (ASPD) criteria in a large sample of patients, most of whom had one or more personality disorders (PD). PD diagnoses were assessed by experienced clinicians using the Structured Clinical Interview for Diagnostic and Statistical Manual of Mental Disorders, 4th edition, Axis II PDs. Analyses were performed within an item response theory framework. Results of the analyses indicated that ASPD is a unidimensional construct that can be measured reliably at the upper range of the latent trait scale. Differential item functioning across gender was restricted to two criteria and had little impact on the latent ASPD trait level. Patients fulfilling both the adult ASPD criteria and the conduct disorder criteria had similar latent trait distributions as patients fulfilling only the adult ASPD criteria. Overall, the ASPD items fit the purpose of a diagnostic instrument well, that is, distinguishing patients with moderate from those with high antisocial personality scores.

Measurement Efficiency for Fixed-Precision Multidimensional Computerized Adaptive Tests: Comparing Health Measurement and Educational Testing Using Example Banks.

It is currently not entirely clear to what degree the research on multidimensional computerized adaptive testing (CAT) conducted in the field of educational testing can be generalized to fields such as health assessment, where CAT design factors differ considerably from those typically used in educational testing. In this study, the impact of a number of important design factors on CAT performance is systematically evaluated, using realistic example item banks for two main scenarios: health assessment (polytomous items, small to medium item bank sizes, high discrimination parameters) and educational testing (dichotomous items, large item banks, small- to medium-sized discrimination parameters). Measurement efficiency is evaluated for both between-item multidimensional CATs and separate unidimensional CATs for each latent dimension. In this study, we focus on fixed-precision (variable-length) CATs because it is both feasible and desirable in health settings, but so far most research regarding CAT has focused on fixed-length testing. This study shows that the benefits associated with fixed-precision multidimensional CAT hold under a wide variety of circumstances.