Performance of Progressive Generations of GPT on an Exam Designed for Certifying Physicians as Certified Clinical Densitometrists.

BACKGROUND: Artificial intelligence (AI) large language models (LLMs) such as ChatGPT have demonstrated the ability to pass standardized exams. These models are not trained for a specific task, but instead trained to predict sequences of text from large corpora of documents sourced from the internet. It has been shown that even models trained on this general task can pass exams in a variety of domain-specific fields, including the United States Medical Licensing Examination. We asked if large language models would perform as well on a much narrower subdomain tests designed for medical specialists. Furthermore, we wanted to better understand how progressive generations of GPT (generative pre-trained transformer) models may be evolving in the completeness and sophistication of their responses even while generational training remains general. In this study, we evaluated the performance of two versions of GPT (GPT 3 and 4) on their ability to pass the certification exam given to physicians to work as osteoporosis specialists and become a certified clinical densitometrists. The CCD exam has a possible score range of 150 to 400. To pass, you need a score of 300. METHODS: A 100-question multiple-choice practice exam was obtained from a 3rd party exam preparation website that mimics the accredited certification tests given by the ISCD (International Society for Clinical Densitometry). The exam was administered to two versions of GPT, the free version (GPT Playground) and ChatGPT+, which are based on GPT-3 and GPT-4, respectively (OpenAI, San Francisco, CA). The systems were prompted with the exam questions verbatim. If the response was purely textual and did not specify which of the multiple-choice answers to select, the authors matched the text to the closest answer. Each exam was graded and an estimated ISCD score was provided from the exam website. In addition, each response was evaluated by a rheumatologist CCD and ranked for accuracy using a 5-level scale. The two GPT versions were compared in terms of response accuracy and length. RESULTS: The average response length was 11.6 ±19 words for GPT-3 and 50.0±43.6 words for GPT-4. GPT-3 answered 62 questions correctly resulting in a failing ISCD score of 289. However, GPT-4 answered 82 questions correctly with a passing score of 342. GPT-3 scored highest on the "Overview of Low Bone Mass and Osteoporosis" category (72 % correct) while GPT-4 scored well above 80 % accuracy on all categories except "Imaging Technology in Bone Health" (65 % correct). Regarding subjective accuracy, GPT-3 answered 23 questions with nonsensical or totally wrong responses while GPT-4 had no responses in that category. CONCLUSION: If this had been an actual certification exam, GPT-4 would now have a CCD suffix to its name even after being trained using general internet knowledge. Clearly, more goes into physician training than can be captured in this exam. However, GPT algorithms may prove to be valuable physician aids in the diagnoses and monitoring of osteoporosis and other diseases.

Assessment of pressure injury knowledge using PZ-PUKT questionnaire among Indian nurses and the evaluation of impact of an educational intervention on the knowledge scores: A quasi-experimental study.

AIM: To determine the knowledge of pressure injury among Indian nurses using PZ-PUKT questionnaire and to evaluate the effect of an educational intervention on knowledge scores. MATERIALS AND METHODS: A Quasi-experimental study design was used to test the Pressure injury knowledge of 273 nurses in a tertiary care teaching hospital. The Pieper Zulkowski Pressure Ulcer Knowledge Test questionnaire was given as a pre-test prior to education session and re-administered after the activity was completed. RESULTS: The mean score of the pre-test was 48.58 ± 6.75 (C·I 47.8-49.4) and post-test 54.14 ± 7.69 (C·I 53.3-55.1), which showed a statistically significant improvement. In the pre-test, nurses had highest score in the prevention subset while wound subset had the greatest improvement in the post-test. Female participants had a better understanding about Pressure injuries when compared to males. Also, the Assistant Nursing Superintendents and ward in Charge nurses had better knowledge as compared with other nurses. CONCLUSIONS: The knowledge of pressure injury among nurses is limited. Knowledge deficits should be identified and targeted educational interventions should be administered to all the nurses irrespective of their educational level and work experience. Wound certification courses should be instituted so that it gives the nurses a better opportunity to learn about pressure injuries at a certified level. All nurses should undergo periodic training in this ever-evolving field so as to provide the best care to their patients.

prepOSCE: A Virtual, Scalable Solution to Prepare Residents for Their OSCE Examination.

The Royal College Comprehensive Objective Examination in Neurology provides certification for Canadian neurologists and consists of a written examination and the Observed Structured Clinical Encounter (OSCE). The OSCE portion of the certification involves residents visiting several patient stations where they address case scenarios with an examiner. Unfortunately, residents lack exam preparation time due to demanding work hours. In response to resident needs, we created a novel, virtual preparation OSCE program - "prepOSCE" - and evaluated its efficacy. The prepOSCE program employed a proprietary virtual solution from CTC Communications Corp. Ten virtual sessions accommodated 70 residents totally. Seven Canadian physicians and two co-chairs created case scenarios for the stations. On session day, seven residents arrived in a virtual plenary room for briefing followed by assignment to a station by CTC. Residents then moved virtually through prepOSCE stations a different examiner and case scenario in each. Following their session, residents and evaluators were surveyed to capture experiences. The average program rating was 4.22 out of 5 (n = 36 residents of 70 residents who participated in the program) and 4.35 (n = 17 evaluators). Ninety-two percent of residents agreed or strongly agreed that they would recommend this program to their peers; they would like prepOSCE to continue next year; and the program was relevant and added value to their studies. The positive feedback received from prepOSCE participants indicates there is a need for a program like prepOSCE. This model has potential for expansion and it is hoped that specialties outside of neurology could benefit from a similar program.

Selection tools and student diversity in health professions education: a multi-site study.

Student diversity in health professions education (HPE) can be affected by selection procedures. Little is known about how different selection tools impact student diversity across programs using different combinations of traditional and broadened selection criteria. The present multi-site study examined the chances in selection of subgroups of applicants to HPE undergraduate programs with distinctive selection procedures, and their performance on corresponding selection tools. Probability of selection of subgroups (based on gender, migration background, prior education, parental education) of applicants (N = 1935) to five selection procedures of corresponding Dutch HPE undergraduate programs was estimated using multilevel logistic regression. Multilevel linear regression was used to analyze performance on four tools: prior-education grade point average (pe-GPA), biomedical knowledge test, curriculum-sampling test, and curriculum vitae (CV). First-generation Western immigrants and applicants with a foreign education background were significantly less likely to be selected than applicants without a migration background and with pre-university education. These effects did not vary across programs. More variability in effects was found between different selection tools. Compared to women, men performed significantly poorer on CVs, while they had higher scores on biomedical knowledge tests. Applicants with a non-Western migration background scored lower on curriculum-sampling tests. First-generation Western immigrants had lower CV-scores. First-generation university applicants had significantly lower pe-GPAs. There was a variety in effects for applicants with different alternative forms of prior education. For curriculum-sampling tests and CVs, effects varied across programs. Our findings highlight the need for continuous evaluation, identifying best practices within existing tools, and applying alternative tools.

Examining the diversity of MRCS examiners.

BACKGROUND: MRCS examiners are the face of the Royal College of Surgeons for early-career surgeons and should therefore represent the workforce they are examining as not to marginalise or negatively impact on the assessment experience of candidates from minoritised groups. This study aimed to explore the diversity of MRCS examiners and whether they represent the demographics of the MRCS candidates. METHODS: A retrospective observational study including all active examiners and examination candidates who attempted MRCS Part A or Part B between January 2020 and July 2021. Self-declared demographic data collected by the Intercollegiate Committee for Basic Surgical Examinations (ICBSE) included gender, sexual orientation, disability status and ethnicity. Following data anonymisation, total group response frequencies were made available to the research team for statistical analysis. RESULTS: Chi-squared analyses showed statistically significant differences in the representation of gender, disability and ethnicity between candidates and examiners (all p < 0.001). Men (83.9% (n = 1121) vs 70.9% (n = 6017) respectively), individuals without disability (98.7% (n = 917) vs 96.1% (n = 6847)) and individuals of White ethnicity (36.6% (n = 346) vs 20.4% (n = 1223)) were significantly overrepresented in the examiners compared to the examination candidates. There was no statistically significant difference in sexual orientation between examiners and candidates (p = 0.712). CONCLUSIONS: Broadly speaking, the socio-demographic profile of MRCS examiners reflects that seen in senior and leadership positions in surgery in the UK - that is, predominantly male and White - but not that seen in early-career surgeons. Positive action is now required in examiner recruitment by the Royal Colleges to ensure that the cohort of MRCS examiners reflects the modern surgical workforce.

"We just don't have the resources": Supervisor perspectives on introducing workplace-based assessments into medical specialist training in South Africa.

BACKGROUND: South Africa (SA) is on the brink of implementing workplace-based assessments (WBA) in all medical specialist training programmes in the country. Despite the fact that competency-based medical education (CBME) has been in place for about two decades, WBA offers new and interesting challenges. The literature indicates that WBA has resource, regulatory, educational and social complexities. Implementing WBA would therefore require a careful approach to this complex challenge. To date, insufficient exploration of WBA practices, experiences, perceptions, and aspirations in healthcare have been undertaken in South Africa or Africa. The aim of this study was to identify factors that could impact WBA implementation from the perspectives of medical specialist educators. The outcomes being reported are themes derived from reported potential barriers and enablers to WBA implementation in the SA context. METHODS: This paper reports on the qualitative data generated from a mixed methods study that employed a parallel convergent design, utilising a self-administered online questionnaire to collect data from participants. Data was analysed thematically and inductively. RESULTS: The themes that emerged were: Structural readiness for WBA; staff capacity to implement WBA; quality assurance; and the social dynamics of WBA. CONCLUSIONS: Participants demonstrated impressive levels of insight into their respective working environments, producing an extensive list of barriers and enablers. Despite significant structural and social barriers, this cohort perceives the impending implementation of WBA to be a positive development in registrar training in South Africa. We make recommendations for future research, and to the medical specialist educational leaders in SA.

Establishing the predictive validity of the intercollegiate membership of the Royal Colleges of surgeons written examination: MRCS Part A.

Successful completion of the Intercollegiate Membership of the Royal Colleges of Surgeons (MRCS) examination is mandatory for surgical trainees entering higher specialist training in the United Kingdom. Despite its international reputation, and the value placed on the examination in surgical training, there has been little evidence of its predictive validity until recently. In this review, we present a summary of findings of four recent Intercollegiate studies assessing the predictive validity of the MRCS Part A (written) examination. Data from all four studies showed statistically significant positive correlations between the MRCS Part A and other written examinations taken by surgical trainees over the course of their education. The studies summarised in this review provide compelling evidence for the predictive validity of this gatekeeping examination. This review will be of interest to trainees, training institutions and the Royal Colleges given the value placed on the examination by surgical training programmes.

Assessing Operative Skill in the Competency-based Education Era: Lessons From the UK and Ireland.

INTRODUCTION: Decisions regarding the operative competence of surgical residents in the United Kingdom and Ireland are informed by operative workplace-based assessments (WBAs) and operative number targets for index procedures. This review seeks to outline the validity evidence of these assessment methods. METHODS: A review of the MEDLINE (Pubmed), EMBASE and Cochrane Library databases was undertaken in accordance with the Joanna Briggs Institute Protocol for Scoping Reviews (2020). Articles were included if they provided evidence of the validity of procedure-based assessments, direct observation of procedural skills, or indicative operative number targets. The educational impact of each article was evaluated using a modified Kirkpatrick model. RESULTS: Twenty-eight articles outlining validity evidence of WBAs and operative number targets were synthesised by narrative review. Five studies documented users' views on current assessment methods (Kirkpatrick level 1). Two articles recorded changes in attitudes towards current operative assessments (level 2a). Ten studies documented the ability of current assessments to record improvements in operative competence (level 2b). Ten studies measured a change in behaviour as a result of the introduction of these assessments (level 3). One article studied the ability of operative assessments to predict clinical outcomes (level 4b). CONCLUSIONS: Operative WBAs are reliable. Scores achieved correlate with both time spent in training and recorded operative experience. Trainers and residents have concerns regarding the subjectivity of these assessments and the opportunistic nature in which they are used. Operative number targets are not criterion-referenced, lack validity evidence, and may be set too low to ensure operative competence.

Resident Autonomy and Performance Independence in Surgical Training Are Time- and Skill-Dependent.

INTRODUCTION: Appropriate faculty supervision and conditional independence of residents during training are required for autonomous and independent postgraduate practice. However, there is a growing concern that competence for transition to independent practice is not universally met. We hypothesize that surgery residents play a significant and active role in achieving their own independent status. METHODS: Over seven academic years (July 2014 through June 2021), 46 surgeons supervised and intraoperatively assessed the performance of 51 residents using validated Objective Structured Assessment of Technical Skill (OSATS) and Zwisch Operative Autonomy (ROA) assessments. Resident readiness to perform procedures independently (RRI) was graded as yes, no, or not applicable. Data were analyzed using descriptive statistics with categorical variables reported as frequencies and percentages. RESULTS: A total of 1657 elective procedures were performed by residents supervised by faculty. Association between RRI and postgraduate year (PGY), OSATS scores, ROA, resident and faculty gender, and case complexity was analyzed. Results indicated positive correlation between RRI and summative OSATS score (r = 0.510, P < 0.001), PGY (r = 0.535, P < 0.001) and ROA (r = 0.473, P < 0.001). Percentage of overall RRI increased from 7% at PGY1 to 87.4% at PGY5. Meaningful autonomy ratings increased from 23.6% at PGY1 to 92.5% at PGY5. Variations in ratings was observed when considering case category and complexity. CONCLUSIONS: RRI increases with years of training with variation when considering the specialty/The Accreditation Council for Graduate Medical Education procedure category and the complexity of cases. Specialty fellowships are a viable option to address the gap in The Accreditation Council for Graduate Medical Education categories when residency alone cannot reach appropriate independence. Residents' technical skills play a crucial role in evaluating RRI and granting operative autonomy.

Analysis of Performance Trends from 2010-2019 on the American Board of Internal Medicine Nephrology Certifying Exam.

BACKGROUND: The pass rate on the American Board of Internal Medicine (ABIM) nephrology certifying exam has declined and is among the lowest of all internal medicine (IM) subspecialties. In recent years, there have also been fewer applicants for the nephrology fellowship match. METHODS: This retrospective observational study assessed how changes between 2010 and 2019 in characteristics of 4094 graduates of US ACGME-accredited nephrology fellowship programs taking the ABIM nephrology certifying exam for the first time, and how characteristics of their fellowship programs were associated with exam performance. The primary outcome measure was performance on the nephrology certifying exam. Fellowship program pass rates over the decade were also studied. RESULTS: Lower IM certifying exam score, older age, female sex, international medical graduate (IMG) status, and having trained at a smaller nephrology fellowship program were associated with poorer nephrology certifying exam performance. The mean IM certifying exam percentile score among those who subsequently took the nephrology certifying exam decreased from 56.7 (SD, 27.9) to 46.1 (SD, 28.7) from 2010 to 2019. When examining individuals with comparable IM certifying exam performance, IMGs performed less well than United States medical graduates (USMGs) on the nephrology certifying exam. In 2019, only 57% of nephrology fellowship programs had aggregate 3-year certifying exam pass rates ≥80% among their graduates. CONCLUSIONS: Changes in IM certifying exam performance, certain trainee demographics, and poorer performance among those from smaller fellowship programs explain much of the decline in nephrology certifying exam performance. IM certifying exam performance was the dominant determinant.

BEST of Surgical Training: the pan-London Core Surgical Training induction programme: "There's no I in team, but there is a me."

BACKGROUND: In the UK, Core Surgical Training (CST) marks the start of a surgical career, but previous experience and skills vary widely. Whilst Individual hospital Trusts offer local inductions, these are generally of a generic, administrative nature rather than advising trainees on how best to harness training opportunities. We designed a regional induction programme, 'Building Excellence in Surgical Training' (BEST) to address this, develop essential technical and non-technical skills, and engender support networks. METHODS: All incoming London Core Surgical Trainees (annual cohort size 90) were invited to participate, during the week prior to commencement of training. Trainees undertook 3 modules (portfolio, surgical skills and human factors-based simulation) and a research paper presentation day. We collected qualitative and quantitative data through a structured evaluation form, pre and post course Likert-scale scores and self-assessment utilising the non-technical skills for surgeons (NOTSS) framework. RESULTS: 972 CSTs have completed BEST over the past 12 years. In 2019, significant improvements were seen in: confidence for starting CST, 45% (n = 22/49)-83% (n = 33/40,p = 0.00045); feeling the core programme cared about them, 55% (n = 27/49)-98% (n = 41/42, p =< 0.00001); getting to know peers 16% (n = 8/49)-88% (n = 35/40, p =< 0.00001); understanding human factors 82% (n = 40/49)-95% (n = 36/38, p = 0.00427); gaining confidence of trainers, 49% (n = 23/47)-86% (n = 31/36, p = 0.00114), and ability to speak out over patient safety concerns, 78% (n = 38/49)-97% (n = 37/38,p = 0.00019). NOTSS assessments showed significant improvement across all 4 criteria (n = 142, p =< 0.00001). CONCLUSION: BEST equips trainees with the fundamental skills and confidence to safely embark on surgical training and provides tools to navigate the challenges training presents. The ethos of collaboration and support will aid the development of more resilient and empowered surgeons, vital in this era.

A Teaching Model to Promote NCLEX-RN Success.

ABSTRACT: Nurse educators in RN programs are called to address the nursing shortage, assist increasingly diverse learners to succeed on the National Council Licensure Examination the first time, and prepare the new nurse graduate to transition into an ever-changing health care system. This article purposes a teaching model to empower both the nurse educator and the student. As graduates have gained locus of control, National Council Licensure Examination pass rates have increased from 82 percent to 93 percent. The discussion examines a variety of strategies to facilitate nurse graduate success.

A competency roadmap for advanced practice nursing education using PRIME-NP.

BACKGROUND: Clinical competency validation is essential for nurse practitioner (NP) education and public accountability. While there has been robust discussion around what constitutes clinical competency and assessment, clear and consistent definitions and measurements remain elusive. PURPOSE: This article describes the PRIME-NP clinical competency model that is scalable, reproducible and accurately documents NP student competency across clinical courses. METHODS: To develop the model, work in 5 discrete domains was necessary: (a) model development, (b) assessment tool to be used in Objective Structured Clinical Exams (OSCE), (c) rubrics to accompany the OSCE exam, (d) faculty education, and (e) evaluating the model use. FINDINGS: Faculty and student outcomes reveal that the model and assessment tool acceptability and effectiveness of the model, especially for early identification for at risk students. CONCLUSION: The PRIME-NP offered faculty the opportunity to identify at-risk students, identify a more nuanced remediation plan, and assess student competency in simulated environments.

Key dimensions of innovations in workplace-based assessment for postgraduate medical education: a scoping review.

BACKGROUND: Specialist training bodies continue to devise innovative methods of gathering information on trainee workplace performance to meet the requirements of competency-based medical education. We reviewed recent innovations in workplace-based assessment (WBA) tools to identify strengths, weaknesses, and trade-offs inherent in their design and use. METHODS: In this scoping review, using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines, we systematically searched databases between 2009 and 2019 for WBA tools with novel characteristics not typically seen in traditional WBAs. These included innovations in rating scales, ways of collecting information, technological innovations, ways of triggering WBAs, and approaches to compiling and using information. RESULTS: We identified 30 innovative WBA tools whose characteristics could be categorised into seven dimensions: frequency of assessment, granularity (unit of performance assessed), coverage of the curriculum, rating method, initiation of the WBA, information use, and incentives. These dimensions had multiple interdependencies and trade-offs, often balancing generating assessment data with available resources. Philosophical stance on assessment also influenced WBA choice, for example prioritising trainee-centred learning (i.e. initiation of WBA and transparency of assessment data), perceptions of assessment and feedback as burdensome or beneficial, and holistic vs reductionist views on assessment of performance. CONCLUSIONS: Our synthesis of the literature on innovative WBAs provides a framework for categorising tool characteristics across seven dimensions, systematically teasing apart the considerations in design and use of workplace assessments. It also draws attention to the trade-offs inherent in tool design and selection, and enables a more deliberate consideration of the tool characteristics most appropriate to the local context.

The Anesthesiology Milestones 2.0: An Improved Competency-Based Assessment for Residency Training.

The evolution of medical education, from a time-based to a competency-based platform, began nearly 30 years ago and continues to slowly take shape. The development of valid and reproducible assessment tools is the first step. Medical educators across specialties acknowledge the challenges and remain motivated to develop a relevant, generalizable, and measurable system. The Accreditation Council for Graduate Medical Education (ACGME) remains committed to its responsibility to the public by assuring that the process and outcome of graduate medical education in the nation's residency programs produce competent, safe, and compassionate doctors. The Milestones Project is the ACGME's current strategy in the evolution to a competency-based system, which allows each specialty to develop its own set of subcompetencies and 5-level progression, or milestones, along a continuum of novice to expert. The education community has now had nearly 5 years of experience with these rubrics. While not perfect, Milestones 1.0 provided important foundational information and insights. The first iteration of the Anesthesiology Milestones highlighted some mismatch between subcompetencies and current and future clinical practices. They have also highlighted challenges with assessment and evaluation of learners, and the need for faculty development tools. Committed to an iterative process, the ACGME assembled representatives from stakeholder groups within the Anesthesiology community to develop the second generation of Milestones. This special article describes the foundational data from Milestones 1.0 that was useful in the development process of Milestones 2.0, the rationale behind the important changes, and the additional tools made available with this iteration.

Entrustable professional activities in entry-level health professional education: A scoping review.

Entrustable professional activities (EPAs) are a recent enhancement to competency-based health professional education that describe the observable work done by a competent health professional. Through defining education outcomes in a work-based context, EPAs offer potential to identify skill gaps in individual or student cohorts and focus improvements. Entrustable professional activities have been pioneered and gained rapid acceptance in postgraduate medical education; however, less is known about their application and use in undergraduate or entry-level health professional education. The Joanna Briggs Institute scoping review methodology was used to explore how and in what context EPAs are being used in entry-level health professional education. Databases searched include CINAHL, EMBASE, MEDLINE, Web of Science and PsycINFO. A total of 748 abstracts were returned after duplicates removed, and 127 full-text articles were screened with 30 included for data extraction. Publications in this area have recently accelerated with disciplines of professions of medicine, pharmacy, dietetics and physician assistants reporting on EPA development, implementation and evaluation. EPA use has been reported in the United States, Canada, Europe Australia and Central America. Major motivation reported for EPA use is to improve patient safety by aligning performance and expectations and to improve student assessment. Several studies report on the use of EPAs to evaluate different curriculum models or identify curriculum gaps representing potential application in education research.

A 15-Year Trend Study of Internationally Educated Nurses' NCLEX-RN Performance.

AIM: The aim of the study was to describe trends in internationally educated nurses' (IEN) National Council Licensure Examination-Registered Nurses (NCLEX®-RN) performance from 2003 to 2017 and to determine the odds of passing the exam based on country of nursing education. BACKGROUND: IEN comprise 5.6 percent of US nurses; more than half come from the Philippines. There is a lack of research on IEN NCLEX-RN performance. METHOD: Correlational research was used to determine the performance and likelihood of passing the NCLEX-RN based on country of nursing education using secondary data analysis. Odds ratios were estimated to express the odds of passing. RESULTS: IEN NCLEX-RN applications and pass rates are decreasing. The odds of passing the NCLEX-RN among Philippine-educated nurses are lower compared to all other IEN. The odds of passing the Canadian NCLEX-RN are higher for all IEN. CONCLUSION: The low NCLEX-RN pass rate of IEN reflects differences in nursing education and practice across countries.

Development of an Objective Structured Clinical Examination as a Component of Assessment for Initial Board Certification in Anesthesiology.

With its first administration of an Objective Structured Clinical Examination (OSCE) in 2018, the American Board of Anesthesiology (ABA) became the first US medical specialty certifying board to incorporate this type of assessment into its high-stakes certification examination system. The fundamental rationale for the ABA's introduction of the OSCE is to include an assessment that allows candidates for board certification to demonstrate what they actually "do" in domains relevant to clinical practice. Inherent in this rationale is that the OSCE will capture competencies not well assessed in the current written and oral examinations-competencies that will allow the ABA to judge whether a candidate meets the standards expected for board certification more properly. This special article describes the ABA's journey from initial conceptualization through first administration of the OSCE, including the format of the OSCE, the process for scenario development, the standardized patient program that supports OSCE administration, examiner training, scoring, and future assessment of reliability, validity, and impact of the OSCE. This information will be beneficial to both those involved in the initial certification process, such as residency graduate candidates and program directors, and others contemplating the use of high-stakes summative OSCE assessments.

Training junior faculty to become clinical teachers: The value of personalized coaching.

Background: Junior clinical faculty require institutional support in the acquisition of feedback and clinical supervision skills of trainees. We tested the effectiveness of a personalized coaching versus guided self-reflection format of a faculty development program at improving faculty skills and self-efficacy.Methods: Participants were evaluated both before and after the program using a four-station Objective Structured Teaching Exercise (OSTE). A gain-score analysis, one-way ANOVA, and paired t-tests were used to evaluate both groups. The impact on the learning environment was measured by resident ratings of the Maastricht Clinical Teaching Questionnaire.Results: One hundred and twenty-seven participants completed the study over a three-year period. Both groups had significant improvements in self-efficacy. Participants in the coaching group demonstrated superior performance in encouraging learner self-reflection, teaching effectiveness, verifying learner understanding, exploring feelings/needs, and defining learning objectives. Over a 5-year period, the overall institutional learning climate significantly improved concerning faculty role-modeling, coaching, articulation, and explorations skills.Conclusion: Offering a contextualized faculty-development program using OSTEs that provides multiple opportunities for feedback and is focused on creating a community of practice is an effective method to facilitate the transfer of skills to the clinical environment, supports teacher identity development, and favorably impacts the learning climate.

Fostering teaching-learning through workplace based assessment in postgraduate chemical pathology residency program using virtual learning environment.

BACKGROUND: The principle of workplace based assessment (WBA) is to assess trainees at work with feedback integrated into the program simultaneously. A student driven WBA model was introduced and perception evaluation of this teaching method was done subsequently by taking feedback from the faculty as well as the postgraduate trainees (PGs) of a residency program. METHODS: Descriptive multimethod study was conducted. A WBA program was designed for PGs in Chemical Pathology on Moodle and forms utilized were case-based discussion (CBD), direct observation of practical skills (DOPS) and evaluation of clinical events (ECE). Consented assessors and PGs were trained on WBA through a workshop. Pretest and posttest to assess PGs knowledge before and after WBA were conducted. Every time a WBA form was filled, perception of PGs and assessors towards WBA, time taken to conduct single WBA and feedback were recorded. Faculty and PGs qualitative feedback on perception of WBA was taken via interviews. WBA tools data and qualitative feedback were used to evaluate the acceptability and feasibility of the new tools. RESULTS: Six eligible PGs and seventeen assessors participated in this study. A total of 79 CBDs (assessors n = 7 and PGs n = 6), 12 ECEs (assessors n = 6 and PGs n = 5), and 20 DOPS (assessors n = 6 and PGs n = 6) were documented. PGs average pretest score was 55.6%, which was improved to 96.4% in posttest; p value< 0.05. Scores of annual assessment before and after implementation of WBA also showed significant improvement, p value 0.039, Overall mean time taken to evaluate PG's was 12.6 ± 9.9 min and feedback time 9.2 ± 7.4 min. Mean WBA process satisfaction of assessors and PGs on Likert scale of 1 to 10 was 8 ± 1 and 8.3 ± 0.8 respectively. CONCLUSION: Both assessors and fellows were satisfied with introduction and implementation of WBA. It gave the fellows opportunity to interact with assessors more often and learn from their rich experience. Gain in knowledge of PGs was identified from the statistically significant improvement in PGs' assessment scores after WBA implementation.