Automated Test-Item Generation System for Retrieval Practice in Radiology Education.

ABSTRACT

OBJECTIVE: To develop and disseminate an automated item generation (AIG) system for retrieval practice (self-testing) in radiology and to obtain trainee feedback on its educational utility. MATERIALS AND METHODS: An AIG software program (Radmatic) that is capable of generating large numbers of distinct multiple-choice self-testing items from a given "item-model" was created. Instead of writing multiple individual self-testing items, an educator creates an "item-model" for one of two distinct item styles true/false knowledge based items and image-based items. The software program then uses the item model to generate self-testing items upon trainee request. This internet-based system was made available to all radiology residents at our institution in conjunction with our didactic conferences. After obtaining institutional review board approval and informed consent, a written survey was conducted to obtain trainee feedback. RESULTS: Two faculty members with no computer programming experience were able to create item-models using a standard template. Twenty five of 54 (46%) radiology residents at our institution participated in the study. Twelve of these 25 (48%) study participants reported using the self-testing items regularly, which correlated well with the anonymous website usage statistics. The residents' overall impression and satisfaction with the self-testing items was quite positive, with a score of 7.89 ± 1.91 (mean ± SD) out of 10. Lack of time and email overload were the main reasons provided by residents for not using self-testing items. CONCLUSION: AIG enabled self-testing is technically feasible, and is perceived positively by radiology residents as useful to their education.