Increasing Authenticity of Simulation-Based Assessment in Diagnostic Radiology.

INTRODUCTION: Clinical reasoning in diagnostic imaging professions is a complex skill that requires processing of visual information and image manipulation skills. We developed a digital simulation-based test method to increase authenticity of image interpretation skill assessment. METHODS: A digital application, allowing volumetric image viewing and manipulation, was used for three test administrations of the national Dutch Radiology Progress Test for residents. This study describes the development and implementation process in three phases. To assess authenticity of the digital tests, perceived image quality and correspondence to clinical practice were evaluated and compared with previous paper-based tests (PTs). Quantitative and qualitative evaluation results were used to improve subsequent tests. RESULTS: Authenticity of the first digital test was not rated higher than the PTs. Test characteristics and environmental conditions, such as image manipulation options and ambient lighting, were optimized based on participants' comments. After adjustments in the third digital test, participants favored the image quality and clinical correspondence of the digital image questions over paper-based image questions. CONCLUSIONS: Digital simulations can increase authenticity of diagnostic radiology assessments compared with paper-based testing. However, authenticity does not necessarily increase with higher fidelity. It can be challenging to simulate the image interpretation task of clinical practice in a large-scale assessment setting, because of technological limitations. Optimizing image manipulation options, the level of ambient light, time limits, and question types can help improve authenticity of simulation-based radiology assessments.

Identifying error types in visual diagnostic skill assessment.

BACKGROUND: Misinterpretation of medical images is an important source of diagnostic error. Errors can occur in different phases of the diagnostic process. Insight in the error types made by learners is crucial for training and giving effective feedback. Most diagnostic skill tests however penalize diagnostic mistakes without an eye for the diagnostic process and the type of error. A radiology test with stepwise reasoning questions was used to distinguish error types in the visual diagnostic process. We evaluated the additional value of a stepwise question-format, in comparison with only diagnostic questions in radiology tests. METHODS: Medical students in a radiology elective (n=109) took a radiology test including 11-13 cases in stepwise question-format: marking an abnormality, describing the abnormality and giving a diagnosis. Errors were coded by two independent researchers as perception, analysis, diagnosis, or undefined. Erroneous cases were further evaluated for the presence of latent errors or partial knowledge. Inter-rater reliabilities and percentages of cases with latent errors and partial knowledge were calculated. RESULTS: The stepwise question-format procedure applied to 1351 cases completed by 109 medical students revealed 828 errors. Mean inter-rater reliability of error type coding was Cohen's κ=0.79. Six hundred and fifty errors (79%) could be coded as perception, analysis or diagnosis errors. The stepwise question-format revealed latent errors in 9% and partial knowledge in 18% of cases. CONCLUSIONS: A stepwise question-format can reliably distinguish error types in the visual diagnostic process, and reveals latent errors and partial knowledge.

The effect of iterative reconstruction on computed tomography assessment of emphysema, air trapping and airway dimensions.

OBJECTIVES: To determine the influence of iterative reconstruction (IR) on quantitative computed tomography (CT) measurements of emphysema, air trapping, and airway wall and lumen dimensions, compared to filtered back-projection (FBP). METHODS: Inspiratory and expiratory chest CTs of 75 patients (37 male, 38 female; mean age 64.0 ± 5.7 years) were reconstructed using FBP and IR. CT emphysema, CT air trapping and airway dimensions of a segmental bronchus were quantified using several commonly used quantification methods. The two algorithms were compared using the concordance correlation coefficient (p (c)) and Wilcoxon signed rank test. RESULTS: Only the E/I-ratio(MLD) as a measure of CT air trapping and airway dimensions showed no significant differences between the algorithms, whereas all CT emphysema and the other CT air trapping measures were significantly different at IR when compared to FBP (P < 0.001). CONCLUSION: The evaluated IR algorithm significantly influences quantitative CT measures in the assessment of emphysema and air trapping. However, the E/I-ratio(MLD) as a measure of CT air trapping, as well as the airway measurements, is unaffected by this reconstruction method. Quantitative CT of the lungs should be performed with careful attention to the CT protocol, especially when iterative reconstruction is introduced. KEY POINTS : • New techniques in CT allow numerous quantitative measurements of lung function. • Iterative reconstruction influences quantitative CT measurements of emphysema and air trapping. • Expiratory-to-inspiratory ratio of mean lung density and airway measurements are unaffected by iterative reconstruction. • Quantitative lung-CT should be performed with careful attention to the CT protocol.