The effect on reliability and sensitivity to level of training of combining analytic and holistic rating scales for assessing communication skills in an internal medicine resident OSCE.
Patient Educ Couns
; 100(7): 1382-1386, 2017 Jul.
Article
in En
| MEDLINE
| ID: mdl-28228339
ABSTRACT
OBJECTIVE:
Although previous research has compared checklists to rating scales for assessing communication, the purpose of this study was to compare the effect on reliability and sensitivity to level of training of an analytic, a holistic, and a combined analytic-holistic rating scale in assessing communication skills.METHODS:
The University of Alberta Internal Medicine Residency runs OSCEs for postgraduate year (PGY) 1 and 2 residents and another for PGY-4 residents. Communication stations were scored with an analytic scale (empathy, non-verbal skills, verbal skills, and coherence subscales) and a holistic scale. Authors analyzed reliability of individual and combined scales using generalizability theory and evaluated each scale's sensitivity to level of training.RESULTS:
For analytic, holistic, and combined scales, 12, 12, and 11 stations respectively yielded a Phi of 0.8 for the PGY-1,2 cohort, and 16, 16, and 14 stations yielded a Phi of 0.8 for the PGY-4 cohort. PGY-4 residents scored higher on the combined scale, the analytic rating scale, and the non-verbal and coherence subscales.CONCLUSION:
A combined analytic-holistic rating scale increased score reliability and was sensitive to level of training. PRACTICE IMPLICATIONS Given increased validity evidence, OSCE developers should consider combining analytic and holistic scales when assessing communication skills.Key words
Full text:
1
Collection:
01-internacional
Database:
MEDLINE
Main subject:
Psychometrics
/
Clinical Competence
/
Communication
/
Educational Measurement
/
Internal Medicine
/
Internship and Residency
Type of study:
Diagnostic_studies
/
Prognostic_studies
Limits:
Humans
Language:
En
Journal:
Patient Educ Couns
Year:
2017
Document type:
Article