A scoping review of advanced pharmacy practice experience readiness assessment measures across schools of pharmacy.

BACKGROUND: The development and implementation of Advanced Pharmacy Practice Experience (APPE) readiness assessments has been an area of increased interest to the academy since the publication of the Accreditation Council for Pharmacy Education (ACPE) standards in 2016. This scoping literature review aims to provide an updated summary of current APPE readiness assessment practices among ACPE-accredited institutions in the United States (US). METHODS: A literature search was conducted between 2022 and 2024 using the terms "APPE student readiness," "APPE readiness assessment," "APPE preparedness," "APPE student preparedness," "pharmacy" AND "readiness assessment", "pharmacy" AND "practice readiness," and "pharmacy" AND "student practice readiness" in Pubmed and Embase. The websites for the American Journal of Pharmaceutical Education and Currents in Pharmacy Teaching and Learning were also searched using these terms. Abstracts for all results were reviewed. Abstract only and poster presentations were excluded, as well as articles centered on non-US and/or non-ACPE-accredited institutions. Data was collected regarding the outcomes assessed, type of assessment activity, and how the assessment was implemented. Results were reviewed by a second author to ensure consistency in reporting. RESULTS: A total of 289 unique abstracts were reviewed by author pairs for inclusion based on relevance to the review objectives. A total of 13 articles were included in the final analysis. Each institution measured different knowledge areas, abilities, and entrustable professional activities (EPAs). Most programs had a summative component, with APPE readiness being primarily assessed in the final didactic year of the pharmacy curriculum. Most programs report at least one source of validity. IMPLICATIONS: Given the lack of specific APPE readiness assessment requirements in the ACPE Standards 2016, the different assessment methods among programs were not surprising. However, the commonalities identified can be leveraged with the release of the Curricular Outcomes and Entrustable Professional Activities (COEPA) 2022 to promote a standardized definition of APPE readiness. Future research should focus on formative assessment methods embedded throughout the didactic curriculum to identify "at-risk" students prior to a "high-stakes" summative assessment at the end of the didactic curriculum that impedes student progression to the APPE year.

An assessment of interprofessional education in schools/colleges of pharmacy in the United States.

INTRODUCTION: Pharmacy educational standards provide requirements for interprofessional education (IPE). However, there has not been a comparison of IPE structure between doctor of pharmacy programs in the United States (US). The purpose of this study was to gather information regarding current IPE programs and curricula in schools/colleges of pharmacy (S/COP) across the US. METHODS: A survey was developed and emailed to at least one faculty member or administrator from each S/COP responsible for IPE or experiential education. The survey gathered information on IPE status, structure, and oversight. It also explored mechanisms promoting IPE success, pieces of advice for starting or expanding IPE, and hurdles for IPE implementation. RESULTS: Eighty-five S/COP representing 59.9% of programs in the US responded to the survey. All respondents felt IPE was either very important or important to pharmacy education. Mechanisms that promoted IPE success included partnerships, faculty interest in IPE, having an IPE center, being located on an academic medical center, administrative support, and integrating IPE during curricular development. IPE hurdles included logistics, faculty buy-in, sustainability, distance from other programs, and differing accreditation standards across programs. Themes of advice for others were to be innovative/patient/flexible, collaborative, identify key initial partners, develop a director or coordinator of IPE position, and receive administrative support. CONCLUSIONS: IPE is a vital component to training the next generation of health professionals, but the process for can be daunting. Building on the successes of others and predicting barriers can assist S/COP in developing effective IPE.

An initial environmental scan of APPE readiness assessment.

INTRODUCTION: Standards 2016 require schools/colleges of pharmacy (s/cop) to assess students' readiness to enter advanced pharmacy practice experiences (APPEs). However, literature describing how schools are meeting this standard is limited. The purpose of this study was to conduct an environmental scan to describe how s/cop assess student readiness to enter APPEs. METHODS: A web-based survey was distributed to assessment leads at United States s/cop, regardless of accreditation status. Respondents answered questions related to their current approach to assessing student APPE readiness, existence of intentional assessment plans, competencies used, assessment methods, benchmarks, and remediation strategies. Aggregate data were analyzed using descriptive statistics. RESULTS: Fifty-two S/COP (36.1%) responded. The majority (90.1%) were fully accredited schools. Most respondents have an intentional APPE readiness plan (73.5%), although the duration since implementation varied. There was no consensus among schools on which competencies informed APPE readiness with 67.3% listing Center for the Advancement of Pharmacy Education (CAPE) 2013 outcomes, 61.2% Guidance for Standards 2016 Appendix A, 53.1% pre-APPE domains (Standards 2007), and 30.6% Entrustable Professional Activities. Twenty-eight S/COP (57.1%) reported having individual student-level data to assess student APPE readiness. The most common methods for validating student APPE readiness were preceptor (48.9%) and student (44.9%) surveys. CONCLUSIONS: This environmental scan begins to identify trends in how S/COP is approaching the assessment of student readiness to begin APPEs. Further research is needed to identify best practices and practical methods to ensure compliance with current accreditation standards.

Assessment of contraceptive curricula in US pharmacy programs.

INTRODUCTION: Pharmacists prescribe contraception in some states following expansions in scope of practice. Adequate education on contraception in pharmacy curricula is crucial to effectively deliver these services. METHODS: A 26-item survey assessing contraception curricula regarding was administered by email to instructors and administrators at 139 pharmacy schools in the United States. The survey assessed teaching methods, hours taught, topic content, and opinion of adequacy of contraceptive education provided by the program. RESULTS: The survey achieved a response rate of 40% (n = 56). All programs that responded offer emergency contraception and hormonal contraception content, 96% offer non-hormonal over-the-counter contraception content, and 91% offer long-acting reversible hormonal contraception content. Average number of hours taught were as follows: non-hormonal over-the-counter contraception 2.0 hours, emergency contraception 0.9 hours, hormonal contraception 3.0 hours, long-acting reversible hormonal contraception 0.8 hours, and non-reversible hormonal contraception 0.5 hours. Patient cases were most used to supplement didactic content in all topics. Standardized patient interviews were used less frequently for both hormonal contraception (25%) and emergency contraception (7%). About 68% of programs agreed or strongly agreed that the contraceptive education provided by the program was adequate. A majority (70%) indicated interest in a standardized contraceptive curriculum. CONCLUSIONS: Contraceptive education is broadly covered in didactic curricula within pharmacy education. Further assessment and development of curricula standards may be warranted to assess quality and adequacy of contraceptive education in pharmacy.

Inclusion of Socioeconomic Status as an Additional Criterion in Holistic Pharmacy School Admission Processes.

Objective. To investigate the impact of socioeconomic status as an admissions criterion for pharmacy school admissions. Methods. Using de-identified data from two PharmCAS application cycles (2012-2013 and 2015-2016), a new socio-economic status indicator was crafted based on complex parental education and occupation information provided by applicants. The indicator was calibrated on federal standards to ensure uniform composition regardless of the applicant's location. The indicator was then used to describe acceptance patterns within each pharmacy application cycle, explore correlations with other demographic and academic indicators, and develop a regression model to understand the impact of this indicator on applicants' acceptance into pharmacy school. Results. The new education and occupation (EO) indicator for an applicant's parent(s) provided new insight into complex applicant demographic information. The number of academically qualified applicants that could have received further admission consideration based on identification of a low socioeconomic status for parent one were 2016; n=435, 13%, 2103; n=767, 14%. The EO indicator by itself had limited value for predicting acceptance rates and became less predictive when academic performance variables were incorporated, demonstrating socioeconomic status was not accounted for in admissions. Conclusion. Based on the Accreditation Council of Pharmacy Education Standards 2016, schools are required to admit a diverse group of students. The addition of the EO indicator will be valuable in identifying diverse and promising future pharmacists during each pharmacy school's holistic review process.

Characterization of Use of the Pharmacy Curriculum Outcomes Assessment Across Accredited Colleges of Pharmacy.

Objective. To characterize use of the Pharmacy Curriculum Outcomes Assessment (PCOA) in terms of timing, manner of delivery, and application of the results by accredited colleges of pharmacy. Methods. Accredited pharmacy programs were surveyed regarding PCOA administration, perceived benefits, and practical application of score reports. Survey items were comprised of new items developed from a literature review and items from prior studies. The survey addressed five domains: program demographics, administration, student preparation, use of results, and recommendations to improve the utility of the PCOA. Results. Responses were received from 126 of 139 (91%) surveyed programs. The majority of respondent programs administered PCOA in one session on a single campus. Most indicated PCOA results had limited use for individual student assessment. Almost half reported that results were or could be useful in curriculum review and benchmarking. Considerable variability existed in the preparation and incentives for PCOA performance. Differences in some results were found based on prior PCOA experience and between new vs older programs. Open-ended responses provided suggestions to enhance the application and utility of PCOA. Conclusion. The intended uses of PCOA results, such as for student assessment, curricular review, and programmatic benchmarking, are not being implemented across the academy. Streamlining examination logistics, providing additional examination-related data, and clarifying the purpose of the examination to faculty members and students may increase the utility of PCOA results.

Development and assessment of an academic performance enrichment program for low-performing, first-year pharmacy students.

Pharmacy school applications have steadily declined over the past several years. Thus pharmacy schools are not only searching for effective means to increase enrollment of qualified candidates, but are also focusing on the development of programs to improve academic performance and retention of enrolled students. To address the needs of struggling first-year pharmacy students enrolled in an Integrated Biological Sciences (BSI) course, an academic performance enrichment program (APEP) was developed. The program was designed to improve academic success by engaging low-performing students with the aims of improving their time management skills, study skills, metacognition, and understanding of BSI course material. The APEP consisted of structured tutoring sessions twice per week, which were required for all students with a course grade ≤73.5% at any point during the semester. To assess program effectiveness, performance improvement on BSI exams by the APEP students were compared with that of non-APEP students in the same class and to those in the previous 3 yr. Student perceptions of the program were also evaluated via an online survey. The APEP was deemed effective in that a greater percentage of students were able to improve their exam scores and to a greater extent by attending the APEP sessions compared with non-APEP students in the same class and with low-performing students in previous years when the APEP did not exist. Furthermore, APEP students believed the program was effective in meeting its aims. In conclusion, the APEP was effective in improving academic performance of low-performing students in BSI.

Surveying the current landscape of assessment structures and resources in US schools and colleges of pharmacy.

INTRODUCTION: Expectations for assessment in higher education have increased in recent decades, prompting institutions to invest additional resources in this area. This study aimed to determine the resources, structure, and perception of assessment resources in United States schools and colleges of pharmacy (S/COPs). METHODS: Assessment personnel in S/COPs were surveyed electronically. Information collected included S/COP demographics, composition of assessment positions, experience and training of assessment personnel, and structure and responsibilities of committees engaged in assessment. Respondents' perception of their S/COPs having sufficient assessment personnel, recent changes in assessment, and the factors that prompted assessment changes were also surveyed. RESULTS: Respondents included individuals from 113 S/COPs (84% response rate). Most S/COPs had 1-2 assessment positions and 1-2 assessment-related committees. The most common assessment position titles were assistant/associate dean, director, coordinator/specialist, and administrative assistant. Dean-level administrators typically had worked in assessment the longest, whereas directors were more likely to have formal assessment training. Most respondents (75%) agreed they had sufficient assessment personnel to meet the 2007 Accreditation Council for Pharmacy Education Standards. Nearly two-thirds of respondents agreed they had sufficient personnel to meet the 2016 Standards and support their current assessment plan/process. Most S/COPs had a formal assessment committee (93%) and an average of two committees overseeing assessment. CONCLUSION: Most S/COPs reported having sufficient resources to support assessment activities. Although there were some consistent themes, there does not appear to be a single model for structuring assessment resources or committees. Effectiveness of various assessment structures represents an area for future research.

Factors associated with cultures of assessment at US schools and colleges of pharmacy.

INTRODUCTION: Pharmacy education has standards for assessment that must be met, prompting many programs to develop cultures of assessment. This study assesses the extent to which assessment personnel perceive that a culture of assessment has been established within their schools/colleges of pharmacy (S/COPs) and determines whether a relationship exists between assessment structure and the perceived strength of the culture of assessment. METHODS: An electronic survey was administered to assessment personnel at 134 S/COPs. Respondents were asked about changes within the past five years pertaining to assessment personnel/committees and factors prompting these changes. They were also asked to rate items related to support for institutional assessment (SIA scale) and faculty engagement with assessment (FEA scale). The reliability of each scale was estimated using Cronbach alpha and bivariate correlations were conducted to estimate the relationships among the two scales and other survey items. RESULTS: The S/COP response rate was 84% (113/134). Most (58%) noted the formation or re-structuring of the assessment office, with 26% creating one or more new assessment positions. The most common source of change was new accreditation standards (55%) followed by a new leadership position created in assessment (40%). Cronbach alpha for the SIA scale was 0.79 and 0.84 for the FEA scale. Program size was strongly negatively correlated with school age (rs = -0.58) and the FEA scale (r = -0.21). CONCLUSIONS: There was a strong relationship between SIA and FEA. Future studies are warranted to determine the interdependency of support for assessment and faculty engagement with assessment.

Assessing self-assessment practices: A survey of U.S. colleges and schools of pharmacy.

OBJECTIVE: This study quantifies and describes student self-assessment approaches in colleges of pharmacy across the United States. METHODS: Faculty members identified as assessment directors from college websites at U.S. colleges of pharmacy were electronically surveyed. Prior to distribution, feedback and question validation was sought from select assessment directors. Surveys were distributed and recorded, via Qualtrics® survey software and analyzed in Microsoft Excel®. RESULTS: Responses were received from 49 colleges of pharmacy (n = 49/134, 37% response rate). The most commonly used strategies were reflective essays (n = 44/49, 90%), portfolios (n = 40/49, 82%), student self-evaluations (n = 35/49, 71%) and questionnaires/surveys/checklists (n = 29/49, 59%). Out of 49 submitted surveys, 35 programs noted students received feedback on self-assessment. Feedback came most commonly from faculty (n = 31/35, 88%). Thirty-four programs responded regarding self-assessment integration including fifteen colleges (n = 15/34, 44%) that integrated self-assessment both into the curriculum and co-curricular activities, while 14 (n = 14/34, 41%) integrated self-assessment exclusively into the curriculum, and five (n = 5/34, 15%) used self-assessment exclusively in co-curricular activities. DISCUSSION AND CONCLUSIONS: Student self-assessment is a critical first step of the Continuing Professional Development (CPD) process. Colleges and schools of pharmacy use a wide variety of methods to develop this skill in preparing future practitioners.

Mapping of the biomedical literature evaluation competencies based on pharmacy students' feedback.

BACKGROUND: This study aims to map the learned curriculum based on students' feedback regarding the biomedical literature evaluation competencies in a pharmacy curriculum, to evaluate teaching methods and to report students' longitudinal self-assessment of their achievement of related learning outcomes as they progress from didactic to experiential courses. METHODS: The biomedical literature evaluation competencies were mapped in three courses delivered during different pharmacy professional years (PPY): Drug Information and Literature Evaluation (PHA421) offered in the second PPY, Pharmacoeconomics (PHA557) and Professional Pharmacy Practice Experience-Hospital/Drug Information Services (PHA570) offered in the third PPY. A unified survey was developed to collect information from students at the beginning and completion of these courses. Survey results were then compared to school assessment data of identified courses for triangulation of findings. RESULTS: Listed student learning outcomes are consistently achieved through all three courses with more assertion from the students at the completion of the applied experiential course PHA 570 (>90 % agree or strongly agree). In terms of delivery methods, 84 % of students perceived the benefits of active learning methods in reinforcing acquired skills and increasing confidence in knowledge and critical thinking in a less stressful learning environment. Results shown at the end of each course indicate a favorable student response from one course to the next where almost all students replied with 'agree to strongly agree' to survey questions assessing their readiness to critically evaluating trials (72 %, 96 % and 92 %) in PHA421, PHA557 and PHA570, respectively. Study findings are in congruence with school assessment database of the selected courses. CONCLUSION: Formative assessment results demonstrated acquisition of required analytical skills, and completion of course learning outcomes as students progressed from introductory to advanced courses covering the biomedical literature component.

Identifying Best Practices for and Utilities of the Pharmacy Curriculum Outcome Assessment Examination.

Objective. A review was conducted to determine implementation strategies, utilities, score interpretation, and limitations of the Pharmacy Curriculum Outcome Assessment (PCOA) examination. Methods. Articles were identified through the PubMed and American Journal of Pharmaceutical Education, and International Pharmaceutical Abstracts databases using the following terms: "Pharmacy Curriculum Outcomes Assessment," "pharmacy comprehensive examination," and "curricular assessment." Studies containing information regarding implementation, utility, and predictive values for US student pharmacists, curricula, and/or PGY1/PGY2 residents were included. Publications from the Academic Medicine Journal, the Accreditation Council for Pharmacy Education (ACPE), and the American Association of Colleges of Pharmacy (ACCP) were included for background information and comparison of predictive utilities of comprehensive examinations in medicine. Results. Ten PCOA and nine residency-related publications were identified. Based on published information, the PCOA may be best used as an additional tool to identify knowledge gaps for third-year student pharmacists. Conclusion. Administering the PCOA to students after they have completed their didactic coursework may yield scores that reflect student knowledge. Predictive utility regarding the North American Pharmacy Licensure Examination (NAPLEX) and potential applications is limited, and more research is required to determine ways to use the PCOA.

Assessment of Communications-related Admissions Criteria in a Three-year Pharmacy Program.

OBJECTIVE: To determine if there is a correlation between TOEFL and other admissions criteria that assess communications skills (ie, PCAT variables: verbal, reading, essay, and composite), interview, and observational scores and to evaluate TOEFL and these admissions criteria as predictors of academic performance. METHODS: Statistical analyses included two sample t tests, multiple regression and Pearson's correlations for parametric variables, and Mann-Whitney U for nonparametric variables, which were conducted on the retrospective data of 162 students, 57 of whom were foreign-born. RESULTS: The multiple regression model of the other admissions criteria on TOEFL was significant. There was no significant correlation between TOEFL scores and academic performance. However, significant correlations were found between the other admissions criteria and academic performance. CONCLUSION: Since TOEFL is not a significant predictor of either communication skills or academic success of foreign-born PharmD students in the program, it may be eliminated as an admissions criterion.

Development and Assessment of a Horizontally Integrated Biological Sciences Course Sequence for Pharmacy Education.

OBJECTIVE: To design and assess a horizontally integrated biological sciences course sequence and to determine its effectiveness in imparting the foundational science knowledge necessary to successfully progress through the pharmacy school curriculum and produce competent pharmacy school graduates. DESIGN: A 2-semester course sequence integrated principles from several basic science disciplines: biochemistry, molecular biology, cellular biology, anatomy, physiology, and pathophysiology. Each is a 5-credit course taught 5 days per week, with 50-minute class periods. ASSESSMENT: Achievement of outcomes was determined with course examinations, student lecture, and an annual skills mastery assessment. The North American Pharmacist Licensure Examination (NAPLEX) results were used as an indicator of competency to practice pharmacy. CONCLUSION: Students achieved course objectives and program level outcomes. The biological sciences integrated course sequence was successful in providing students with foundational basic science knowledge required to progress through the pharmacy program and to pass the NAPLEX. The percentage of the school's students who passed the NAPLEX was not statistically different from the national percentage.

Recommendations for the successful pursuit of scholarship by pharmacy practice faculty members.

Scholarship has long been a basic expectation of faculty members at institutions of higher learning in the United States and elsewhere. This expectation is no less assumed in academic pharmacy. A number of organizations have verbalized and enforced this precept over the years.(1-3) For example, this expectation is spoken to directly in the American Council for Pharmacy Education's Accreditation Standards and Guidelines.(4) This expectation is further emphasized in the draft document of the accreditation standards to be implemented in 2016, in Standard 20. Specifically, Element 20.2 states: "The college or school must create an environment that both requires and promotes scholarship, and must also develop mechanisms to assess both the quantity and quality of faculty scholarly productivity."(5) The successful pursuit of scholarship by clinical faculty members (those engaged in both clinical practice and teaching, without regard to tenure or clinical track status) is challenging. (6-10) Thus, faculty member job descriptions or models should be designed so clinical faculty members can successfully meet all academic job expectations, including productive and meaningful scholarship. In 2012, an AACP Section of Teachers of Pharmacy Practice task force was charged with examining this issue and providing recommendations for models for clinical faculty members that would allow the successful pursuit of scholarship. The task force gathered information relating to the current state of affairs at a number of colleges and reviewed relevant literature. This information, along with personal experiences and much discussion and contemplation, led to some general observations as well as specific recommendations. This paper reiterates the task force's observations and recommendations and provides further detail regarding our interpretation of the findings and basis for the eventual recommendations to the section.

A qualitative analysis of common concerns about challenges facing pharmacy experiential education programs.

OBJECTIVE: To qualitatively analyze free-text responses gathered as part of a previously published survey in order to systematically identify common concerns facing pharmacy experiential education (EE) programs. METHODS: In 2011, EE directors at all 118 accredited pharmacy schools in the US were asked in a survey to describe the most pressing issues facing their programs. Investigators performed qualitative, thematic analysis of responses and compared results against demographic data (institution type, class size, number of practice sites, number and type of EE faculty member/staff). Expert and novice investigators identified common themes via an iterative process. To check validity, additional expert and novice reviewers independently coded responses. The Cohen kappa coefficient was calculated and showed good agreement between investigators and reviewers. RESULTS: Seventy-eight responses were received (66% response rate) representing 75% of publicly funded institutions and 71% of schools with class sizes 51-150. Themes identified as common concerns were site capacity, workload/financial support, quality assurance, preceptor development, preceptor stipends, assessment, onboarding, and support/recognition from administration. Good agreement (mean percent agreement 93%, ƙ range=0.59-0.92) was found between investigators and reviewers. CONCLUSION: Site capacity for student placements continues to be the foremost concern for many experiential education programs. New concerns about preceptor development and procedures for placing and orienting students at individual practice sites (ie, "onboarding") have emerged and must be addressed as new accreditation standards are implemented.

Qualitative analysis of common definitions for core advanced pharmacy practice experiences.

OBJECTIVE: To determine how colleges and schools of pharmacy interpreted the Accreditation Council for Pharmacy Education's (ACPE's) Standards 2007 definitions for core advanced pharmacy practice experiences (APPEs), and how they differentiated community and institutional practice activities for introductory pharmacy practice experiences (IPPEs) and APPEs. METHODS: A cross-sectional, qualitative, thematic analysis was done of survey data obtained from experiential education directors in US colleges and schools of pharmacy. Open-ended responses to invited descriptions of the 4 core APPEs were analyzed using grounded theory to determine common themes. Type of college or school of pharmacy (private vs public) and size of program were compared. RESULTS: Seventy-one schools (72%) with active APPE programs at the time of the survey responded. Lack of strong frequent themes describing specific activities for the acute care/general medicine core APPE indicated that most respondents agreed on the setting (hospital or inpatient) but the student experience remained highly variable. Themes were relatively consistent between public and private institutions, but there were differences across programs of varying size. CONCLUSION: Inconsistencies existed in how colleges and schools of pharmacy defined the core APPEs as required by ACPE. More specific descriptions of core APPEs would help to standardize the core practice experiences across institutions and provide an opportunity for quality benchmarking.