Development and Assessment of a Rubric for Evaluating Teaching Portfolios Developed by Teaching and Learning Curriculum (TLC) Program Participants.

ABSTRACT

OBJECTIVE: To describe the validation and reliability assessment of a rubric designed to assess the participants' teaching portfolios and identify teaching excellence among teaching and learning curriculum (TLC) program participants. METHODS: Following focus groups with program leadership at a single TLC program, an initial rubric was developed, consisting of criteria mapped to 5 domains, to be rated on a 4-point scale. The rubric was then redistributed to the TLC program leadership and external stakeholders for evaluation of face and content validity. The rubric was piloted using teaching portfolios from 3 cycles of the program. Cronbach αwas used to measure internal consistency and a 2-way random-effects model was used to assess the inter-rater reliability. RESULTS: A total of 18 portfolios were independently evaluated by 4 raters. The overall mean Cronbach α for internal consistency was 0.90 and ranged from 0.65 to 0.84 by domain. The overall mean intraclass correlation coefficient for inter-rater reliability was 0.95 and ranged from 0.57 to 0.91 by domain. CONCLUSION: The rubric evaluates characteristics of teaching portfolios important to internal and external stakeholders and had good to excellent internal consistency and inter-rater reliability. It can be adapted and applied by TLC programs to identify teaching excellence.