Accuracy of Entrustment-Based Assessment: Implications for Programs and Patients.
J Grad Med Educ
; 16(1): 30-36, 2024 Feb.
Article
em En
| MEDLINE
| ID: mdl-38304606
ABSTRACT
Background Although entrustment-supervision ratings are more intuitive compared to other rating scales, it is not known whether their use accurately assesses the appropriateness of care provided by a resident. Objective To determine the frequency of incorrect entrustment ratings assigned by faculty and whether accuracy of an entrustment-supervision scale differed by resident performance when the scripted resident performance level is known. Methods Faculty participants rated standardized residents in 10 videos using a 4-point entrustment-supervision scale. We calculated the frequency of rating a resident incorrectly. We performed generalizability (G) and decision (D) studies for all 10 cases (768 ratings) and repeated the analysis using only cases with an entrustment score of 2. Results The mean score by 77 raters for all videos was 2.87 (SD=0.86) with a mean of 2.37 (SD=0.72), 3.11 (SD=0.67) and 3.78 (SD=0.43) for the scripted levels of 2, 3, and 4. Faculty ratings differed from the scripted score for 331of 768 (43%) ratings. Most errors were ratings higher than the scripted score (223, 67%). G studies estimated the variance proportions of rater and case to be 4.99% and 54.29%. D studies estimated that 3 raters would need to watch 10 cases. The variance proportion of rater was 8.5% when the analysis was restricted to level 2 entrustment, requiring 15 raters to watch 5 cases. Conclusions Participants underestimated residents' potential need for greater supervision. Overall agreement between raters and scripted scores were low.
Texto completo:
1
Coleções:
01-internacional
Contexto em Saúde:
14_ODS3_health_workforce
Base de dados:
MEDLINE
Assunto principal:
Internato e Residência
Tipo de estudo:
Prognostic_studies
Limite:
Humans
Idioma:
En
Revista:
J Grad Med Educ
Ano de publicação:
2024
Tipo de documento:
Article