Development and validation of a simulation-based assessment of operative competence for higher specialist trainees in general surgery.
Surg Endosc
; 38(9): 5086-5095, 2024 Sep.
Article
em En
| MEDLINE
| ID: mdl-39020120
ABSTRACT
BACKGROUND:
Simulation is increasingly being explored as an assessment modality. This study sought to develop and collate validity evidence for a novel simulation-based assessment of operative competence. We describe the approach to assessment design, development, pilot testing, and validity investigation.METHODS:
Eight procedural stations were generated using both virtual reality and bio-hybrid models. Content was identified from a previously conducted Delphi consensus study of trainers. Trainee performance was scored using an equally weighted Objective Structured Assessment of Technical Skills (OSATS) tool and a modified Procedure-Based Assessment (PBA) tool. Validity evidence was analyzed in accordance with Messick's validity framework. Both 'junior' (ST2-ST4) and 'senior' trainees (ST 5-ST8) were included to allow for comparative analysis.RESULTS:
Thirteen trainees were assessed by ten assessors across eight stations. Inter-station reliability was high (α = 0.81), and inter-rater reliability was acceptable (inter-class correlation coefficient 0.77). A significant difference in mean station score was observed between junior and senior trainees (44.82 vs 58.18, p = .004), while overall mean scores were moderately correlated with increasing training year (rs = .74, p = .004, Kendall's tau-b .57, p = 0.009). A pass-fail score generated using borderline regression methodology resulted in all 'senior' trainees passing and 4/6 of junior trainees failing the assessment.CONCLUSION:
This study reports validity evidence for a novel simulation-based assessment, designed to assess the operative competence of higher specialist trainees in general surgery.Palavras-chave
Texto completo:
1
Coleções:
01-internacional
Base de dados:
MEDLINE
Assunto principal:
Cirurgia Geral
/
Competência Clínica
/
Avaliação Educacional
/
Treinamento por Simulação
Limite:
Humans
Idioma:
En
Ano de publicação:
2024
Tipo de documento:
Article