Nailed it! Doctor of pharmacy learners' self-awareness of performance on authentic assessments.

INTRODUCTION: The primary objective of this pilot study was to determine the reliability and validity of an objective structured clinical examination (OSCE) self-awareness survey. A secondary objective was to assess learner self-awareness in OSCE performance. METHODS: A retrospective review of OSCE data from a single cohort in a three-year accelerated doctor of pharmacy program occurred. The cohort completed four OSCEs with a total of 23 discrete cases. At the end of each case, before receiving feedback, learners completed an evaluation of their skills, communication, overall performance, and identified if they believed the patient/provider would return ("nailed or failed"). These self-evaluations were then compared with respective learner performance scores for each case. Content validity was assessed for the self-awareness survey via the Lawshe method. Reliability of the OSCE cases and self-awareness survey were conducted utilizing Cronbach's alpha and the Spearman-Brown formula, respectively. RESULTS: The OSCE self-awareness survey was found to be face valid and reliable for the overall survey. The subsections of return to pharmacist and overall learner performance were also found to be valid and reliable. A statistical relationship was found between learner self-assessment of pass/fail and their actual grade for overall performance and return to pharmacist. CONCLUSIONS: Results from this study may guide remediation plans related to OSCE performance and development of other self-awareness activities in practice settings. Further studies are needed to evaluate the broader application and use of self-awareness tools.

Evaluating a measure of the research training environment in a sample of pharmacy residents.

INTRODUCTION: While an important predictor of future research involvement, formal assessment of the quality of the research training environment in pharmacy residency training has not been examined. The objective of this study was to evaluate the psychometric properties of a modified version of the shortened form of the revised Research Training Environment Scale (RTES-R-S) in pharmacy residents. METHODS: The original wording of the 18 items from RTES-R-S was modified to reflect pharmacy residency training. Data were collected as part of a larger study involving a random sample of pharmacy residents in the United States during the 2016-2017 residency year. Internal consistency was assessed with Cronbach's α. Confirmatory factor analysis was used to assess the fit of the factor structures examined previously during the original RTES-R-S development. RESULTS: Based on confirmatory factor analysis, the preferred model proposed a single, second-order factor predicting nine factors reflecting Gelso's previously described ingredients of the research training environment. This model had acceptable fit indices and was statistically better than models without second-order factors. Cronbach's α for the global scale was 0.86, with the two subscales >0.7. CONCLUSIONS: Minor rewording of an existing instrument to measure residents' perceptions of the quality of the research training environment demonstrated acceptable internal consistency. The factor structure of the recommended use proposed by the original developers of the RTES-R-S was supported in this sample of pharmacy residents. Continued work with additional samples is needed to generate additional validity evidence supporting the use of this tool in pharmacy residents.