Design and Evaluation of an Interprofessional Preceptor Development Mini-Fellowship Program.

INTRODUCTION: Health professions preceptors require skills and knowledge to effectively meet the educational needs of interprofessional students in clinical environments. We implemented a mini-fellowship program to enhance the knowledge, skills, and self-efficacy of preceptors teaching students and applying quality improvement (QI) methods across disciplines and patient care settings. METHOD: The design, implementation, and evaluation of the program were informed by the faculty development literature, principles of adult learning, and preceptor needs. The 3-day program included workshops on curriculum design, clinical teaching methods, QI, social determinants of health, cultural humility, and interprofessional teamwork. Quantitative and qualitative evaluation methods were used including preprogram and postprogram knowledge and self-efficacy surveys, along with end-of-session and program evaluations. RESULTS: Five annual cohorts involving 41 preceptors with varied demographics, professions, and clinical practices completed the mini-fellowship program. Participants' percentage of items answered correctly on a QI knowledge test increased from 79.2% (pretest) to 85.5% (post-test), a gain of 6.3% (90% CI: 2.9-9.7%; P < .003). The average QI self-efficacy scores improved from 2.64 to 3.82, a gain of 1.18 points on a five-point scale (P < .001). The average education/teaching self-efficacy increased from 2.79 to 3.80 on a five-point scale (P < .001). Ultimately, 94% would recommend the program to other preceptors. DISCUSSION: An interprofessional preceptor development program designed to train clinicians to effectively teach in the clinical setting and to conduct QI projects with students was achievable and effective. This program can serve as a model for academic centers charged with training future health care workers and supporting their community-based preceptors' training needs.

An Exploration of Doctor of Physical Therapy Students' Belongingness in Clinical Education: A Validation Study.

INTRODUCTION: The aim of this study was to adapt and validate the Belongingness Scale-Clinical Placement Experience (BES-CPE) for Doctor of Physical Therapy (DPT) students in the United States. REVIEW OF LITERATURE: Belongingness is vital to one's mental, emotional, and physical health. Research has shown that belongingness is positively correlated with students' academic performance and achievement. An absence of belongingness may hinder students' full participation in clinical experiences and compromise clinical achievement. SUBJECTS: Respondents were current or former DPT students at least 18 years of age who had either completed the midterm evaluation of their final terminal full-time clinical education experience (TCE) in their DPT program or were no more than 1 year from the completion of their final TCE. METHODS: The BES-CPE was adapted for DPT students, and the scale was completed electronically by those who met the inclusion criteria. Principal component analysis with promax rotation and Cronbach's α were used to determine construct validity and reliability. RESULTS: One hundred fifty-nine respondents completed all items on the BES-CPE and demographic survey. A 3-component structure was identified (esteem, connectedness, and efficacy), which was aligned to the original BES-CPE scale. One item was discarded, and the final version of the BES-CPE for DPT students is a 33-item scale with satisfactory internal consistency. DISCUSSION AND CONCLUSION: This study adapted and provided evidence for validity of the first known scale to measure belongingness in DPT students during their clinical education experiences (CEEs) in the United States. The 33-item BES-CPE provided valid and reliable measures of belongingness in DPT students during CEEs that can be used to provide a better understanding of the student experience in the clinical learning environment.

Validation of a Novel Statistical Method to Identify Aberrant Patient Logging: A Multi-Institutional Study.

INTRODUCTION: Student patient encounter logging informs the quality of supervised clinical practice experiences (SCPEs). Yet, it is unknown whether logs accurately reflect patient encounters, and the faculty resources necessary to review for potential aberrant logging are significant. The purpose of this study was to identify a statistical method to identify aberrant logging. METHODS: A multi-institutional (n = 6) study examined a statistical method for identifying potentially aberrant logging behavior. An automated statistical Mahalanobis Distance (MD) measurement was used to categorize student logs as aberrant if they were identified as probable multivariate outliers. This approach was validated using a gold standard for aberrant logging behavior with manual review by 4 experienced faculty ("faculty consensus") and then comparing interrater agreement between faculty and MD-based categorization. In secondary analyses, we compared the relative accuracy of MD-based categorization to individual faculty categorizing data from their own program ("own program" categorization). RESULTS: 323 student logging records from 6 physician assistant (PA) programs were included. Compared to "faculty consensus" (the gold standard), MD-based categorization was highly sensitive (0.846, 95% CI: 0.650, 1.000) and specific (0.766, 95% CI: 0.645, 0.887). Additionally, there was no significant difference in sensitivity, specificity, positive predictive value, or negative predictive value between MD-based categorization and "own program" categorization. DISCUSSION: The MD-based method of identifying aberrant and nonaberrant student logging compared favorably to the more traditional, faculty-intensive approach of reviewing individual student logging records. This supports MD-based screening as a less labor-intensive alternative to individual faculty review to identify aberrant logging. Identification of aberrant logging may facilitate early intervention with students to improve clinical exposure logging during their SCPEs.

Do Physician Assistant Students Use Retrieval-Based Strategies While Studying?

INTRODUCTION: Research on learning indicates that active retrieval of information (ie, testing) enhances student retention of knowledge, yet, it is underutilized by learners. This research investigated physician assistant (PA) students' study strategies and the extent to which retrieval-based strategies (RBS) are used. METHODS: A survey instrument adapting items from Hartwig and Dunlosky's Study Habits Survey was administered to first-year PA students to investigate their study behaviors over a 4-week time frame in preparation for multiple-choice exams. RESULTS: Seventy-eight students (88%) completed the survey. The most common study strategies were reviewing lecture slide decks (83%), reading notes (78%), and taking notes (73%). Retrieval-based study strategies most often used were practicing test questions (38%) and creating questions and responding to them (12%). DISCUSSION: Many PA students are underutilizing retrieval-based study strategies and might benefit from more awareness of this learning approach as well as faculty guidance about how to incorporate it into their exam preparation.

Perceived Benefits of Training Clinicians in Community Engagement for a Leadership Development Program.

BACKGROUND AND OBJECTIVES: Community engagement (CE), including community-engaged research, is a critical tool for improving the health of patients and communities, but is not taught in most medical curricula, and is even rarer in leadership training for practicing clinicians. With the growth of value-based care and increasing concern for health equity, we need to turn our attention to the benefits of working with communities to improve health and health care. The objective of this brief report is to increase understanding of the perceived benefits of CE training for primary care clinicians, specifically those already working. METHODS: We assessed perceived benefits of CE training for primary care clinicians participating in health care transformation leadership training through analysis of learner reflection papers. RESULTS: Clinicians (n=12) reported transformational learning and critical shifts of perspective. Not only did they come to value and understand CE, but the training changed their perception of their roles as clinicians and leaders. CONCLUSIONS: Educating primary care clinicians in CE as a foundational principle can orient them to the criticality of stakeholder engagement for daily practice, practice transformation, and population health improvement, and provides them with a new understanding of their roles as clinicians and leaders.

Considerations Around Predicting Physician Assistant National Certifying Exam Scores From PACKRAT®: A Multiprogram Study.

PURPOSE: The Physician Assistant Clinical Knowledge Rating and Assessment Tool (PACKRAT®) is a known predictor of performance on the Physician Assistant National Certifying Exam (PANCE). It is unknown, however, whether these associations (1) vary across programs; (2) differ by PACKRAT metrics (first-year [PACKRAT 1], second-year [PACKRAT 2], and composite score [arithmetic mean of PACKRAT 1 and PACKRAT 2]); or (3) are modified by demographic or socioeconomic variables. METHODS: Linear and logistic hierarchical regression models (HRMs) were used to evaluate associations between PACKRAT metrics and (1) continuous PANCE scores and (2) odds of low PANCE performance (LPP), respectively. Likelihood ratio tests were used to evaluate differences in associations between programs and effect modification by demographic and socioeconomic variables. Receiver operating characteristic (ROC) curves were used to examine the sensitivity, specificity, positive predictive values, and negative predictive values for various PACKRAT metrics/cut points. Models were adjusted for demographic and socioeconomic variables. The PACKRAT scores were standardized for each year to the national mean and SD. RESULTS: Adjusted HRMs across 5 programs (n = 1014) found the composite score to have the strongest association, with a 10-percentile-point increase associated with a 22-point (95% confidence interval [CI]: 19-26) increase in PANCE score. The composite score also strongly predicted decrements in odds of LPP (odds ratio: 0.46; 95% CI: 0.38-0.55). Hierarchical regression models and ROC curves identified significant variability in associations among programs. Effect modification was not observed by any investigated variable. CONCLUSIONS: The composite score had the largest magnitudes of association with PANCE scores and odds of LPP. The significant difference in association identified between programs suggests that the predictive ability of the exam is not uniform. The lack of effect modification by demographic and socioeconomic variables suggests that associations do not significantly differ by these metrics.

Use of design effects and sample weights in complex health survey data: a review of published articles using data from 3 commonly used adolescent health surveys.

OBJECTIVES: We assessed how frequently researchers reported the use of statistical techniques that take into account the complex sampling structure of survey data and sample weights in published peer-reviewed articles using data from 3 commonly used adolescent health surveys. METHODS: We performed a systematic review of 1003 published empirical research articles from 1995 to 2010 that used data from the National Longitudinal Study of Adolescent Health (n=765), Monitoring the Future (n=146), or Youth Risk Behavior Surveillance System (n=92) indexed in ERIC, PsycINFO, PubMed, and Web of Science. RESULTS: Across the data sources, 60% of articles reported accounting for design effects and 61% reported using sample weights. However, the frequency and clarity of reporting varied across databases, publication year, author affiliation with the data, and journal. CONCLUSIONS: Given the statistical bias that occurs when design effects of complex data are not incorporated or sample weights are omitted, this study calls for improvement in the dissemination of research findings based on complex sample data. Authors, editors, and reviewers need to work together to improve the transparency of published findings using complex sample data.

Making treatment effect inferences from multiple-baseline data: the utility of multilevel modeling approaches.

Multiple-baseline studies are prevalent in behavioral research, but questions remain about how to best analyze the resulting data. Monte Carlo methods were used to examine the utility of multilevel models for multiple-baseline data under conditions that varied in the number of participants, number of repeated observations per participant, variance in baseline levels, variance in treatment effects, and amount of autocorrelation in the Level 1 errors. Interval estimates of the average treatment effect were examined for two specifications of the Level 1 error structure (sigma(2)I and first-order autoregressive) and for five different methods of estimating the degrees of freedom (containment, residual, between-within, Satterthwaite, and Kenward-Roger). When the Satterthwaite or Kenward-Roger method was used and an autoregressive Level 1 error structure was specified, the interval estimates of the average treatment effect were relatively accurate. Conversely, the interval estimates of the treatment effect variance were inaccurate, and the corresponding point estimates were biased.