The cognitive apprenticeship model as a framework for desired precepting qualities to inform preceptor development.

INTRODUCTION: Positive learner perceptions of learning experiences have been linked to better learning outcomes. More information is needed on learners' desired qualities of preceptors and learning experiences to inform preceptor development. Aligning learners' perceptions with a teaching framework, such as the Cognitive Apprenticeship (CA) framework, may be useful to guide preceptor self-assessment and development. However, information is lacking regarding whether the CA framework is consistent with learners' expectations. The purpose of this study was to determine pharmacy learner perspectives on desired preceptor behaviors and qualities and to evaluate their alignment with the CA framework to inform preceptor development. METHODS: Twenty-two learners (nine residents and 13 introductory and advanced pharmacy practice students) participated in nine focus group interviews. Data were analyzed qualitatively by inductive coding and pattern coding and then condensed into themes. After initial analysis, the CA framework was adapted into codes and applied to the data to explore the alignment of quality preceptor characteristics with CA. RESULTS: Learners identified desired general preceptor characteristics, teaching behaviors, and qualities of sites and experience structure in their discussion. All four CA dimensions (Methods, Sociology, Sequencing, and Content) were represented in the described desired preceptor qualities. Most comments were connected to the Methods dimension. CONCLUSIONS: This study supports the use of CA as a framework to guide preceptor development and assessment for desired precepting qualities, preceptor behaviors, and learning environments. Additional research is needed for best practices in implementing CA in preceptor assessment and professional development.

Lessons learned from a non-traditional application process: Integrating user experiences to inform admission application design.

PURPOSE: The research aim was to gather applicant expectations and perspectives with a user-centered approach during a pilot application process for a new dental school. The goal was to inform future admissions practices at our institution and for programs broadly. METHODS: The High Point University admission team developed an electronic application external to current centralized services that was launched from May to August 2022. Participants who completed the application were requested to complete an electronic survey of closed- and open-ended questions about the factors influencing their application decisions, feedback about how to optimize the experience, and their overall satisfaction using a Net Promoter Score. RESULTS: A total of 303 applicants (79% response rate) completed the survey about their application experience. The most influential reason for applying to the program was the optional Dental Admissions Test requirement (43%). Participants frequently referenced a need for a more human-centered process focused on the candidate rather than their numerical values and that the process should be smooth, straight-forward, short, easy, and simple. Participants requested that experiences should be weighed the most in the application process followed by letters of recommendation. Most applicants were "promoters" (68%) for the program with a Net Promoter Score of 62. Moreover, 32% and 45% rated the application process as the best or better application experience compared to other programs. CONCLUSION: Programs can benefit from collecting applicant's (i.e., users) opinions and feedback about their expectations and experiences to inform more user-centered application processes.

Exploring user experience (UX) research methods in health professions education.

OUR ISSUE: Researchers often design interventions or experiences to meet the needs of a specific user. However, the user's perspective is often excluded in the process, which can minimize effectiveness due to a lack of understanding about the user, their perspective, and/or their needs. User experience (UX) research methods offer an opportunity to capture additional information about the user that can inform the design of these solutions. METHODOLOGICAL LITERATURE REVIEW: UX research became prominent during the 1990s to describe the complexity of human interactions with technological solutions. UX research methods emerged in other disciplines as strategies for describing user perspectives, needs, challenges, and potential impact of proposed solutions-these may be useful in health professions education research. OUR RECOMMENDATIONS AND THEIR APPLICATIONS: Those interested in UX research methods should define the user, establish focused research questions, and select applicable strategies cognizant of time and resource constraints. UX research often requires a clear outline of research activities, how they address the questions, and techniques to engage representative users for data collection. Researchers can optimize data collection by creating inclusive spaces that emphasize active listening. Researchers should compile insights regularly and remember UX research emphasizes an iterative approach to design. POTENTIAL IMPACT: UX research can support deeper insights into users, their perspectives, their needs, and offers opportunities to co-construct solutions with their experiences in mind. These methods may help educators design impactful experiences that better achieve targeted outcomes in collaboration with their users (e.g., learners, faculty, teams).

Design, Implementation, and Outcomes of an Interprofessional Mobile Web Application for Preceptors for Challenging Issues.

OBJECTIVE: The purpose of this study was to develop and assess an easily accessible interprofessional mobile web application to assist preceptors with challenging teaching and learning situations. METHODS: Phase 1 was a modified Delphi process of 48 advanced practice nursing, dentistry, medicine, and pharmacy preceptors to determine the content of the application. Phase 2 consisted of 12 preceptors from the 4 disciplines piloting a prototype to refine the tool using design-thinking principles. Feedback was analyzed using inductive coding and thematic analysis. Phase 3 evaluated the impact of the final tool on 80 preceptors' satisfaction, knowledge, self-efficacy, and perception of behavior change. RESULTS: Consensus on 10 topics was reached in the following 3 themes: feedback and communication, clinical and professional development of learners, and precepting efficiency. Preceptors rated the tool as efficient and applicable. Features perceived as useful included concise and applicable content that was easy to navigate with practical video examples. Features to improve included academic jargon, length of content, and lack of connectivity with other preceptors. Knowledge and self-efficacy improved after the use of the refined tool. Change in perceptions of behavior after 1 month was mixed, with a significant change in accessing resources to address challenging situations and regularly reflecting on challenging situations and no significant change in awareness, frequency, or success in managing challenging situations. CONCLUSION: An interprofessional mobile web application for challenging teaching and learning situations developed through a modified Delphi process was deemed efficient and relevant and demonstrated positive knowledge and self-efficacy change.

Assessing the Use of Microlearning for Preceptor Development.

The objective of this study was to evaluate microlearning as a preceptor development method compared to a traditional method of learning. Twenty-five preceptor participants volunteered to engage in a learning intervention about two preceptor development topics. Participants were randomized 1:1 to either a thirty-minute traditional learning experience or a fifteen-minute microlearning experience; participants then crossed over to the other intervention for comparison. Primary outcomes were satisfaction, changes in knowledge, self-efficacy, and perception of behavior, confidence scale, and self-reported frequency of behavior, respectively. One-way repeated measures ANOVA and Wilcoxon paired t-tests were used to analyze knowledge and self-efficacy, and Wilcoxon paired t-tests were utilized to assess satisfaction and perception of behavior. Most participants preferred microlearning over the traditional method (72% vs. 20%, p = 0.007). Free text satisfaction responses were analyzed using inductive coding and thematic analysis. Participants reported that microlearning was more engaging and efficient. There were no significant differences in knowledge, self-efficacy, or perception of behavior between microlearning and the traditional method. Knowledge and self-efficacy scores for each modality increased compared to the baseline. Microlearning shows promise for educating pharmacy preceptors. Further study is needed to confirm the findings and determine optimal delivery approaches.

Lessons From Using Design Thinking to Develop the 2021 AACP Teachers' Seminar.

Objective. To describe and evaluate how a design thinking approach aided the creation of the 2021 American Association of Colleges of Pharmacy (AACP) Teachers' Seminar.Methods. The design thinking framework (ie, inspiration, ideation, and implementation) was used to structure the seminar development process from July 2020 to July 2021. Nine committee members engaged in a persona activity (ie, inspiration), a brainstorming activity (ie, ideation), and a prototyping activity (ie, implementation) to create a user-centered learning experience. Twenty-five small group facilitators were then recruited to create and deliver breakout session content. After the seminar, the team was invited to debrief their experience in a focus group and an electronic survey to evaluate the perceived impact of using design thinking in the planning process.Results. Twenty-one (62%) of the 34 committee members and small group facilitators attended the focus group, and 28 (82%) completed the electronic survey. Most agreed that design thinking was a useful approach to support the Teachers' Seminar, and they were generally positive about the experience. There was a significant increase in self-reported creative self-efficacy for coming up with novel ideas, ability to solve problems, and helping expand others' ideas. Team members identified positive attributes about the seminar and planning process as well as areas for improvement. Team members also acknowledged challenges and potential solutions for professional organizations and program developers to consider when creating user-centered experiences.Conclusion. Design thinking can be a useful framework for seminar planning and implementation to create engaging, meaningful, and valuable educator development experiences.

Simulation-Based Training to Improve Clinical Pharmacist Self-Efficacy in the Management of a Rapidly Decompensating Patient.

Background: The optimal training method to prepare pharmacists as an integral rapid response team or cardiopulmonary arrest responders is poorly described. This study assessed the utility of simulation-based training (SBT) as a training technique for clinical pharmacists. Objective: This study aimed to determine if attending SBT is associated with an improvement in self-efficacy. Methods: This single-center, prospective, interventional cohort study offered three simulations to clinical pharmacists over the course of seven months at a 957-bed quaternary care academic medical center. Pharmacists who participated in at least one simulation were categorized in the intervention group and were compared to pharmacists who did not attend a simulation. All participants were asked to complete a 19-question self-efficacy survey in the form of a 100-point scale, a 15-question multiple-choice knowledge assessment, and a perception survey in the form of 4-point Likert scale administered at baseline and following the conclusion of the SBT. Results: Forty-four clinical pharmacists participated; 20 in the intervention group and 24 in the control group. Median change in self-efficacy score improved significantly in the intervention group compared to the control group (14.3 vs 2.3, P = .009). Median change in perception score improved significantly (2 vs 0, P = .046). Knowledge score did not change significantly from baseline. Conclusion: Simulation-based training improved clinical pharmacist self-efficacy and perceptions in the care of rapidly decompensating patients. These findings support SBT as a viable modality of training clinical pharmacists for the management of rapidly decompensating patients.

Considerations for writing and including demographic variables in education research.

OUR SITUATION: At many points within pharmacy education, we collect demographic information about faculty, staff, students, and trainees. Admissions procedures and surveys, for example, typically categorize participants based on various identities such as race and ethnicity as well as gender and sex. In this article, we interrogate the complex nature of capturing participant identities through demographic variables, using race and ethnicity, gender and sex, sexual orientation and identity, and disability status as specific examples. METHODOLOGICAL LITERATURE REVIEW: Various approaches to defining demographic characteristics can be seen in pharmacy education research. To help readers understand why certain characteristics and categories are commonly utilized, we describe common influences of demographic variables, such as federal laws and professional guidelines. We also review several common demographic variables to illustrate the complexity of this issue. OUR RECOMMENDATIONS AND THEIR APPLICATION(S): Seven recommendations for improving demographic data collection and use are provided. Examples include respecting and honoring lived student experiences, defining demographic data more precisely, and channeling institutional resources toward specific student needs. POTENTIAL IMPACT: Collecting demographic information more accurately and thoughtfully could help pharmacy education improve its educational practice and research. The provided recommendations provided will help researchers and educators measure and utilize demographic variables more effectively.

Designing an Inclusive Learning Training Series for Pharmacy Educators.

This article describes the design, implementation, and evaluation of five faculty development sessions focused on inclusive teaching strategies in pharmacy education. Inclusive strategies ensure that every student can clearly understand and engage in meaningful learning opportunities. Three sessions were implemented in fall 2020 and two in spring 2021. Sessions focused on experiential, didactic, and graduate education. A convergent parallel mixed methods evaluation was conducted using descriptive statistics and thematic analysis. Sessions were highly rated, and participants provided suggestions for curriculum improvement (e.g., creating resources, surveying students, and peer auditing syllabi for aspects of inclusiveness). Given the increasing emphasis on inclusion in pharmacy education, this work is timely for sharing strategies aimed at faculty development and teaching practices.

Item response theory in high-stakes pharmacy assessments.

OUR SITUATION: Classical test theory (CTT) and item response theory (IRT) are two measurement models used to evaluate results from examinations, questionnaires, and instruments. To illustrate the benefits of IRT, we compared how results from multiple-choice tests can be interpreted using CTT and IRT. METHODOLOGICAL LITERATURE REVIEW: IRT encompasses a collection of statistical models that estimate the probability of providing a correct response for a test item. The models are non-linear and generate item characteristic curves that illustrate the relationship between the examinee's ability level and whether they answered the item correctly. Several models can be used to estimate parameters such as item difficulty, discrimination, and guessing. In addition, IRT can generate item and test information functions to illustrate the accuracy of ability estimates. OUR RECOMMENDATIONS AND THEIR APPLICATIONS: Researchers interested in IRT should gather the necessary resources early in the research process and collaborate with those experienced in quantitative and advanced statistical models. Researchers should confirm IRT is the optimal choice and select the model ideal for their needs. Once data are acquired, confirm model assumptions are met and model fit is appropriate. Lastly, researchers should consider disseminating the findings with accompanying visuals. POTENTIAL IMPACT: IRT can be a valuable approach in assessment design and evaluation. Potential opportunities include supporting the design of computer adaptive tests, creating equivalent test forms that evaluate a range of examinee abilities, and evaluating whether items perform differently for examinee sub-groups. Further, IRT can have noteworthy visuals such as test information and functions.

Interested in situational judgment tests? Preparing pharmacy educators for potential challenges.

PURPOSE: There is a growing interest in situational judgment tests (SJTs) for assessments and the authors offer their insights to help others develop and utilize these instruments while navigating common challenges. DESCRIPTION: This manuscript provides important considerations to those interested in SJTs using the "What? So What? Now What?" reflection framework. The authors offer their experience developing, implementing, and evaluating SJTs designed to measure empathy, adaptability, and integrity. ANALYSIS/INTERPRETATION: Effective SJT design can be organized around five key steps: attribute definition, item design, key development and scoring, test administration, and data analysis and interpretation. There are general recommendations to optimize SJT design to enhance reliability and validity of the results; however, there are often other challenges that have not been previously described. CONCLUSIONS: SJTs offer an exciting tool for measuring social and behavioral attributes; however, their infancy in health professions education warrants caution. Like any assessment, there are several challenges to consider when designing, implementing, and evaluating SJTs in pharmacy education. IMPLICATIONS: Individuals interested in SJTs are encouraged to ensure the construct measured is well-defined, there are clear and supported decisions around item and key design, and the appropriate psychometric analyses are conducted.

Responding to emerging oral healthcare needs with CARE: Design of the CARE curriculum model.

The release of the National Institutes of Health report, Oral Health in America: Advances and Challenges, in 2022 identified a pressing concern about oral healthcare and that "the job is far from finished." The High Point University Workman School of Dental Medicine utilized this report as inspiration to facilitate the design of the Clinician-Advocate-Researcher-Entrepreneur Curriculum. The aspiration was to create a novel curriculum and experiential model to prepare learners for the future of dental medicine, including addressing these public oral healthcare needs. This article describes how we conducted this process in three steps: defining the curriculum philosophy, outcomes, and structure. We present this example for those interested in curriculum design and modifications. Our goal is to encourage educators to explore opportunities to enhance dental education as a critical component of the complex system that impacts oral healthcare broadly.

Safe to speak: Fostering psychological safety among incoming predoctoral dental students.

PURPOSE: To describe the implementation and impact of an introductory session on psychological safety for incoming first-year Doctor of Dental Surgery (DDS) learners. METHODS: In August 2020, first-year DDS students participated in a 2-h online instructional session focused on defining psychological safety, the behaviors that promote and disrupt it, and setting expectations for learner and faculty behavior to foster it. Learners worked in small groups to discuss the various components of psychological safety and document their discussion in a collaborative workspace before a large group debriefs. Participants were requested to complete an electronic survey before and after the session to evaluate their change in knowledge, confidence in performing tasks to foster psychological safety, and perceptions about their level of control. Six months after the session, learners were requested to participate in a focus group for additional feedback and reflections. RESULTS: Seventy-four participants completed the before and after experience survey. There was a statistically significant increase in knowledge of the components of psychological safety and perceptions of control. In addition, self-reported measures of confidence to perform the tasks increased from an average rating of 36% confident before the session to 86% confident afterward. In the experience, learners were active in their discussion and generated many ideas about behaviors that promote and disrupt psychological safety, which informed the creation of learner and faculty classroom expectations. CONCLUSION: Engaging learners in discussions about psychological safety can be a useful activity to establish expectations and create group norms that may support collaborative learning environments.

Does the ability to identify the construct on an empathy situational judgment test relate to performance? Exploring a new concept in assessment.

INTRODUCTION: The ability to identify criteria (ATIC) refers to an examinee's capacity to distinguish the construct being evaluated. Previous research indicates ATIC can be predictive of performance on some assessments. This exploratory study investigated the relationship between a participant's ability to identify criteria and their performance on an empathy situational judgment test (SJT), an assessment format used to measure social and behavioral attributes. METHODS: A 12-item empathy SJT was completed by 15 students and 15 pharmacists. During a cognitive interview, participants were asked what they believed each exam question measured. Responses were coded to determine whether participants stated "empathy" (indication of ATIC). The point-biserial correlation coefficient was calculated to explore the relationship of ATIC (correctly or not correctly identifying the item measured empathy) and performance on the SJT (total score). RESULTS: Participants identified empathy 33.3% of the time, and it was the construct most often identified. Pharmacists (27.5%) identified empathy less often than students (39.2%). When empathy was identified as the construct, it was most often reported for items in a non-healthcare setting (56.3%) rather than a healthcare setting (43.7%) and for questions targeting affective empathy (71.3%) rather than cognitive empathy (28.7%). There were no statistically significant relationships with correctly identifying the construct and performance on individual items and the overall test. CONCLUSIONS: There is inconclusive evidence that ATIC relates to performance on an empathy SJT. Additional research is needed to evaluate the role of ATIC and assessment performance to corroborate study results.

How to use the Delphi method to aid in decision making and build consensus in pharmacy education.

OUR SITUATION: As pharmacy educators, we often encounter situations such as designing new curricula or establishing shared values for an organization that may be ambiguous or controversial. To generate effective solutions, it is often necessary to build group consensus with key stakeholders. The purpose of this paper is to describe and provide recommendations for using the Delphi method, a process for facilitating discussions and aiding in decision making. METHODOLOGICAL LITERATURE REVIEW: An overview of the Delphi method, including its multiple variations, is presented. Steps necessary to complete a Delphi study (building a protocol, developing a research question, defining panelists and panel size, piloting the protocol, round one item creation and analysis, round two and beyond, consensus, increasing clarity, and reporting) is described. OUR RECOMMENDATIONS AND THEIR APPLICATION(S): Practical recommendations are provided to support use the Delphi method to build consensus in research. These recommendations include: (1) clarify the purpose of the Delphi, (2) ensure the research questions are grounded in the literature and are relevant, (3) carefully consider panelist processes, (4) determine any definitions that should be given to or developed by panelists, (5) determine methods for enhancing clarity, and (6) employ methods to reduce attrition. POTENTIAL IMPACT: The Delphi method provides a systematic approach to generating consensus in pharmacy education for commonly encountered situations such as committee meetings, research studies, faculty retreats, classroom activities, and lab meetings.

Exploring the Role of Item Scenario Features on Situational Judgment Test Response Selections.

Objective. To explore pharmacists' and pharmacy students' perceptions regarding the significance of changing the features of test item scenario (eg, switching from a health care to a non-health care context) on their situational judgment test (SJT) responses.Methods. Fifteen Doctor of Pharmacy students and 15 pharmacists completed a 12-item SJT intended to measure empathy. The test included six scenarios in a health care context and six scenarios in a non-health care context; participants had to rank potential response options in order of appropriateness and no two items could be of equal rank. Qualitative data were collected individually from participants using think-aloud and cognitive interview techniques. During the cognitive interview, participants were asked how they selected their final responses for each item and whether they would have changed their answer if features of the scenario were switched (eg, changed to a non-health care context if the original item was in a health care context). Interviews were transcribed and a thematic analysis was conducted to identify the features of the scenario for each item that were perceived to impact response selections.Results. Participants stated that they would have changed their responses on average 51.3% of the time (range 20%-100%) if the features of the scenario for an item were changed. Qualitative analysis identified four pertinent scenario features that may influence response selections, which included information about the examinee, the actors in the scenario, the relationship between examinee and actors, and details about the situation. There was no discernible pattern linking scenario features to the component of empathy being measured or participant type.Conclusion. Results from this study suggest that the features of the scenario described in an SJT item could influence response selections. These features should be considered in the SJT design process and require further research to determine the extent of their impact on SJT performance.

A Qualitative Preceptor Development Needs Assessment to Inform Program Design and Effectiveness.

Objective. To assess preceptor teaching challenges and development programming design preferences through a qualitative needs assessment of Doctor of Pharmacy student and resident preceptors.Methods. In 2018, 148 experiential education stakeholders across North Carolina (eg, preceptors, residency program directors, experiential faculty administrators, and practice site administrators) were invited to participate in a 60-minute semi-structured interview as part of a broad preceptor development needs assessment. Interview questions focused on: precepting challenges, positive and negative features of development programs, and preferences for program design. Interview transcripts were coded using thematic analysis.Results. Forty-two participants completed interviews, including preceptors from various rotation types, residency program directors, experiential faculty administrators, and institution administrators. Participants identified numerous teaching challenges related to learners, preceptors, and institutional level factors. Participants often noted there was inadequate time, resources, and support to effectively teach. Desirable preceptor development program features included practical strategies, collaboration with preceptors, delivery by education and practice experts, and topics specific to precepting experience. Participants identified live, on-demand, and webinar formats as acceptable if collaboration and engagement were included. Participants also desired unique training opportunities such as online platforms, coaching programs, and simulated learning environments.Conclusion. Preceptors for pharmacy students and residents face numerous challenges and require sufficient time, support, and resources to develop their skills. Participants requested training that included on-demand, frequent sessions delivered through various modalities, collaboration opportunities, a choice in topics and delivery formats, and sessions from educational and practice experts.

Role of Knowledge and Experience in Situational Judgment Test Responses of Pharmacists and Pharmacy Students.

Objective. To describe the role of examinee knowledge and experience in situational judgment test (SJT) response processes.Methods. Thirty participants (15 students and 15 pharmacists) completed a 12-item SJT on empathy. Each participant completed a think-aloud interview followed by a cognitive interview to elicit their understanding of the items and factors that influenced their response selections. Interviews were coded to identify references to general and job-specific knowledge and experiences. Utterances were quantified to explore differences in the occurrence based on the individual item, item setting (ie, health care or non-health care setting), participant type (ie, student or pharmacists), and empathy component being assessed (ie, affective or cognitive empathy).Results. Participants made 480 references to knowledge and experiences: 45.2% were job-specific knowledge or experiences, 27.5% were general knowledge or experiences, 17.9% related to a lack of experience, and 9.4% were nondescript and could not be distinguished. There were significant differences in the reference to general and job-specific knowledge or experiences based on whether the item scenario occurred in a health care or non-health care setting and the component of empathy being assessed. Experience references often included comments about location, actors, task, similarity, specificity, and recency; knowledge references were classified by information, strategies, and skills.Conclusion. Findings from this study suggest general and job-specific knowledge and experiences influence response processes in SJTs.

Lessons learned from pharmacy learner and educator experiences during early stages of COVID-19 pandemic.

PURPOSE: To explore pharmacy learner (eg, resident, fellow) and educator (eg, residency program director, preceptor) experiences and lessons learned during the coronavirus disease 2019 (COVID-19) pandemic. METHODS: In May 2020, attendees of the virtual Research in Education and Practice Symposium (REPS) were invited to participate in two 1-hour networking sessions. During these sessions, participants completed individual and group reflection exercises where they were asked questions about their experiences during the initial stages of the COVID-19 pandemic. Participants who volunteered submitted their responses through an electronic survey application. Anonymous responses were coded using thematic analysis to address the research questions. RESULTS: Sixty-eight and 38 participants, respectively, attended the 2 networking discussions. Participant-identified professional impacts of the COVID-19 crisis included unexpected learning experiences, greater adaptability, workflow and learning interruptions, and decreased productivity. Personal impacts included a greater focus on well-being, physical and social isolation, and changes in travel plans. Participants noted positive and negative emotions including acceptance, encouragement, anxiety, stress, and frustration. The main lessons learned focused on adaptability, gratitude, and empathy. Participants shared that they wished they would have known the anticipated duration of the pandemic, associated workflow changes, and reliance on technology prior to the start of the pandemic. In addition, they predicted that pharmacy practice will require changes to workflow flexibility, training expectations, the pharmacist's role, and organizational structure. CONCLUSION: The COVID-19 pandemic has positively and negatively impacted the professional and personal lives of pharmacy learners and educators, with the most notable impacts being in the areas of well-being and adaptability. Future research should explore the experiences of other workforce personnel and evaluate the long-term impact on pharmacy practice, patient outcomes, and workforce well-being and resiliency.