A systematic review of measurement uncertainty visualizations in the context of standardized assessments.

This systematic review summarized findings of 29 studies evaluating visual presentation formats appropriate for communicating measurement uncertainty associated with standardized clinical assessment instruments. Studies were identified through systematic searches of multiple databases (Medline, Embase, PsycInfo, ERIC, Scopus, and Web of Science). Strikingly, we found no studies which were conducted using samples of clinicians and included clinical decision-making scenarios. Included studies did however find that providing participants with information about measurement uncertainty may increase awareness of uncertainty and promote more optimal decision making. Formats which visualize the shape of the underlying probability distribution were found to promote more accurate probability estimation and appropriate interpretations of the underlying probability distribution shape. However, participants in the included studies did not seem to benefit from the additional information provided by such plots during decision-making tasks. Further explorations into how presentations of measurement uncertainty impact clinical decision making are needed to examine whether findings of the included studies generalize to clinician populations. This review provides an important overview of pitfalls associated with formats commonly used to communicate measurement uncertainty in clinical assessment instruments, and a potential starting point for further explorations into promising alternatives. Finally, our review offers specific recommendations on how remaining research questions might be addressed.

Visualizing Uncertainty to Promote Clinicians' Understanding of Measurement Error.

Measurement error is an inherent part of any test score. This uncertainty is generally communicated in ways that can be difficult to understand for clinical practitioners. In this empirical study, we evaluate the impact of several communication formats on the interpretation of measurement accuracy and its influence on the decision-making process in clinical practice. We provided 230 clinical practitioners with score reports in five formats: textual, error bar, violin plot, diamond plot, and quantile dot plot. We found that quantile dot plots significantly increased accuracy in the assessment of measurement uncertainty compared with other formats. However, a direct relation between visualization format and decision quality could not be found. Although traditional confidence intervals and error bars were favored by many participants due to their familiarity, responses revealed several misconceptions that make the suitability of these formats for communicating uncertainty questionable. Our results indicate that new visualization formats can successfully reduce errors in interpretation.

Empirical Priors in Polytomous Computerized Adaptive Tests: Risks and Rewards in Clinical Settings.

The use of empirical prior information about participants has been shown to substantially improve the efficiency of computerized adaptive tests (CATs) in educational settings. However, it is unclear how these results translate to clinical settings, where small item banks with highly informative polytomous items often lead to very short CATs. We explored the risks and rewards of using prior information in CAT in two simulation studies, rooted in applied clinical examples. In the first simulation, prior precision and bias in the prior location were manipulated independently. Our results show that a precise personalized prior can meaningfully increase CAT efficiency. However, this reward comes with the potential risk of overconfidence in wrong empirical information (i.e., using a precise severely biased prior), which can lead to unnecessarily long tests, or severely biased estimates. The latter risk can be mitigated by setting a minimum number of items that are to be administered during the CAT, or by setting a less precise prior; be it at the expense of canceling out any efficiency gains. The second simulation, with more realistic bias and precision combinations in the empirical prior, places the prevalence of the potential risks in context. With similar estimation bias, an empirical prior reduced CAT test length, compared to a standard normal prior, in 68% of cases, by a median of 20%; while test length increased in only 3% of cases. The use of prior information in CAT seems to be a feasible and simple method to reduce test burden for patients and clinical practitioners alike.

More is more: Evidence for the incremental value of the SCID-II/SCID-5-PD specific factors over and above a general personality disorder factor.

Currently, 3 competing conceptualizations of personality dysfunction can be distinguished: the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) categorical model delineating 10 distinct types of personality disorders (PDs); the alternative model for PDs (DSM-5 Section III), which assesses personality functioning and traits separately; and the International Classification of Diseases, 11th Version conceptualization, which provides 1 single code for the presence of a PD (which is based on problems in functioning) as well as codes that specify the level of the disorder (mild/moderate/severe), and prominent trait domains or patterns (5 domains and 1 pattern). The current study aims to assess the incremental value of the DSM-5 PDs over and above a global personality dysfunction factor, using expert ratings obtained with the Structured Clinical Interview for DSM-IV PDs and the Structured Clinical Interview for DSM-5 PDs interview in a large sample of clinical patients (N = 3,851). All estimated bifactor models provided adequate fit to the data. We found a surprisingly low explained common variance for the g-factor (<40%), indicating that ignoring the specific PD factors would lead to a substantial loss of information. The strongest specific PDs in terms of explained common variance were the avoidant, schizotypal, and schizoid PD factors and the conduct disorder criteria set if included. Correlations between our factors and external variables were relatively low, except for the Severity Indices of Personality Problems, which aims to measure personality functioning. Our findings suggest that specific PDs still have an important role to play in the assessment of personality pathology. (PsycInfo Database Record (c) 2022 APA, all rights reserved).