Constructing a Validity Map for a Workplace-Based Assessment System: Cross-Walking Messick and Kane.
Acad Med
; 96(7S): S64-S69, 2021 07 01.
Article
em En
| MEDLINE
| ID: mdl-34183604
ABSTRACT
PROBLEM:
Health professions education has shifted to a competency-based paradigm in which many programs rely heavily on workplace-based assessment (WBA) to produce data for summative decisions about learners. However, WBAs are complex and require validity evidence beyond psychometric analysis. Here, the authors describe their use of a rhetorical argumentation process to develop a map of validity evidence for summative decisions in an entrustment-based WBA system.APPROACH:
To organize evidence, the authors cross-walked 2 contemporary validity frameworks, one that emphasizes sources of evidence (Messick) and another that stresses inferences in an argument (Kane). They constructed a validity map using 4steps:
(1) Asking critical questions about the stated interpretation and use, (2) Seeking validity evidence as a response, (3) Categorizing evidence using both Messick's and Kane's frameworks, and (4) Building a visual representation of the collected and organized evidence. The authors used an iterative approach, adding new critical questions and evidence over time.OUTCOMES:
The first map draft produced 25 boxes of evidence that included all 5 sources of evidence detailed by Messick and spread across all 4 inferences described by Kane. The rhetorical question-response process allowed for structured critical appraisal of the WBA system, leading to the identification of evidentiary gaps. NEXTSTEPS:
Future map iterations will integrate evidence quality indicators and allow for deeper dives into the evidence. The authors intend to share their map with graduate medical education stakeholders (e.g., accreditors, institutional leaders, learners, patients) to understand if it adds value for evaluating their WBA programs' validity arguments.
Texto completo:
1
Coleções:
01-internacional
Base de dados:
MEDLINE
Assunto principal:
Competência Clínica
/
Local de Trabalho
/
Educação Baseada em Competências
/
Educação de Pós-Graduação em Medicina
Tipo de estudo:
Prognostic_studies
Limite:
Humans
Idioma:
En
Revista:
Acad Med
Ano de publicação:
2021
Tipo de documento:
Article