The impact of repeated item development training on the prediction of medical faculty members' item difficulty index.

BACKGROUND: Item difficulty plays a crucial role in assessing students' understanding of the concept being tested. The difficulty of each item needs to be carefully adjusted to ensure the achievement of the evaluation's objectives. Therefore, this study aimed to investigate whether repeated item development training for medical school faculty improves the accuracy of predicting item difficulty in multiple-choice questions. METHODS: A faculty development program was implemented to enhance the prediction of each item's difficulty index, ensure the absence of item defects, and maintain the general principles of item development. The interrater reliability between the predicted, actual, and corrected item difficulty was assessed before and after the training, using either the kappa index or the correlation coefficient, depending on the characteristics of the data. A total of 62 faculty members participated in the training. Their predictions of item difficulty were compared with the analysis results of 260 items taken by 119 fourth-year medical students in 2016 and 316 items taken by 125 fourth-year medical students in 2018. RESULTS: Before the training, significant agreement between the predicted and actual item difficulty indices was observed for only one medical subject, Cardiology (K = 0.106, P = 0.021). However, after the training, significant agreement was noted for four subjects: Internal Medicine (K = 0.092, P = 0.015), Cardiology (K = 0.318, P = 0.021), Neurology (K = 0.400, P = 0.043), and Preventive Medicine (r = 0.577, P = 0.039). Furthermore, a significant agreement was observed between the predicted and actual difficulty indices across all subjects when analyzing the average difficulty of all items (r = 0.144, P = 0.043). Regarding the actual difficulty index by subject, neurology exceeded the desired difficulty range of 0.45-0.75 in 2016. By 2018, however, all subjects fell within this range. CONCLUSION: Repeated item development training, which includes predicting each item's difficulty index, can enhance faculty members' ability to predict and adjust item difficulty accurately. To ensure that the difficulty of the examination aligns with its intended purpose, item development training can be beneficial. Further studies on faculty development are necessary to explore these benefits more comprehensively.

Burnout of Faculty Members of Medical Schools in Korea.

BACKGROUND: There is no national survey on medical school faculty members' burnout in Korea. This study aimed to investigate burnout levels and explore possible factors related to burnout among faculty members of Korean medical schools. METHODS: An anonymous online questionnaire was distributed to 40 Korean medical schools from October 2020 to December 2020. Burnout was measured by a modified and revalidated version of the Maslach Burnout Inventory-Human Service Survey. RESULTS: A total of 996 faculty members participated in the survey. Of them, 855 answered the burnout questions, and 829 completed all the questions in the questionnaire. A significant number of faculty members showed a high level of burnout in each sub-dimension: 34% in emotional exhaustion, 66.3% in depersonalization, and 92.4% in reduced personal accomplishment. A total of 31.5% of faculty members revealed a high level of burnout in two sub-dimensions, while 30.5% revealed a high level of burnout in all three sub-dimensions. Woman faculty members or those younger than 40 reported significantly higher emotional exhaustion and depersonalization. Long working hours (≥ 80 hours/week) showed the highest reduced personal accomplishment scores (F = 4.023, P = 0.018). The most significant stressor or burnout source was "excessive regulation by the government or university." The research was the most exasperating task, but the education was the least stressful. CONCLUSION: This first nationwide study alerts that a significant number of faculty members in Korean medical schools seem to suffer from a high level of burnout. Further studies are necessary for identifying the burnout rate, related factors, and strategies to overcome physician burnout.

Challenges and solutions in developing an objective and structured clinical examination for complementary and integrative medicine: A mixed-method approach.

This study explores the challenges encountered by developers when creating objective and structured clinical examination (OSCE) modules specifically for Korean medicine (KM). The complexity of developing pattern identification (PI) items, due to the lack of standardized materials and ambiguity in KM, was a primary focus. A mixed-method approach was utilized, including a survey, importance-performance analysis, and focus group interviews. Seven developers participated, creating a total of 21 OSCE modules. The main difficulties identified were in developing PI items, selecting appropriate cases, crafting realistic examination situations, determining scoring criteria, setting up checklists, and writing scenarios. Challenges were categorized into "case," "examination situation," "postexamination notes," "checklist," "scenario," "format," and "PI." The importance-performance analysis revealed improvements in module development capabilities with each iteration. For the future development of efficient OSCE modules, standardization of KM diagnostic methods and PI is essential. The study highlights the need for social and academic efforts, as well as support from the KM education community and schools, to address these challenges and enhance the development process.

Clinical performance of medical students in Korea in a whole-task emergency station in the objective structured clinical examination with a standardized patient complaining of palpitations

This study assessed the clinical performance of 150 third-year medicalstudents in Busan, Korea in a whole-task emergency objective structured clinical examination station that simulated a patient with palpitations visiting the emergency department. The examination was conducted from November 25 to 27, 2019. Clinical performance was assessed as the number and percentage of students who performed history-taking (HT), a physical examination (PE), an electrocardiography (ECG) study, patient education (Ed), and clinical reasoning (CR), which were items on the checklist. It was found that 18.0% of students checked the patient's pulse, 51.3% completed an ECG study, and 57.9% explained the results to the patient. A sizable proportion (38.0%) of students did not even attempt an ECG study. In a whole-task emergency station, students showed good performance on HT and CR, but unsatisfactory results for PE, ECG study, and Ed. Clinical skills educational programs for subjected student should focus more on PE, timely diagnostic tests, and sufficient Ed.

Medical students' clinical performance of dealing with patients in the context of domestic violence.

PURPOSE: The aim of this study was to inquire about the clinical performance and determine the performance pattern of medical students in standardized patient (SP) based examinations of domestic violence (DV). METHODS: The clinical performance sores in DV station with SP of third-year (n=111, in 2014) and 4th-year (n=143, in 2016) medical students of five universities in the Busan-Gyeongnam Clinical Skills Examination Consortium were subjected in this study. The scenarios and checklists of DV cases were developed by the case development committee of the consortium. The students' performance was compared with other stations encountered in SP. The items of the checklists were categorized to determine the performance pattern of students investigating DV into six domains: disclosure strategy (D), DV related history taking (H), checking the perpetrator's psychosocial state (P), checking the victim's condition (V), negotiating and persuading the interviewee (N), and providing information about DV (I). RESULTS: Medical students showed poorer performance in DV stations than in the other stations with SP in the same examination. Most students did confirm the perpetrator and commented on confidentiality but ignored the perpetrator's state and patient's physical and psychological condition. The students performed well in the domains of D, H, and I but performed poorly in domains P, V, and N. CONCLUSION: Medical students showed poor clinical performance in the DV station. They performed an 'event oriented interview' rather than 'patient centered' communication. An integrated educational program of DV should be set to improve students' clinical performance.

Authenticity, acceptability, and feasibility of a hybrid gynecology station for the Papanicolaou test as part of a clinical skills examination in Korea.

PURPOSE: The objective of this study was to evaluate the authenticity, acceptability, and feasibility of a hybrid station that combined a standardized patient encounter and a simulated Papanicolaou test. METHODS: We introduced a hybrid station in the routine clinical skills examination (CSE) for 335 third-year medical students at 4 universities in Korea from December 1 to December 3, 2014. After the tests, we conducted an anonymous survey on the authenticity, acceptability, and feasibility of the hybrid station. RESULTS: A total of 334 medical students and 17 professors completed the survey. A majority of the students (71.6%) and professors (82.4%) agreed that the hybrid station was more authentic than the standard CSE. Over 60 percent of the students and professors responded that the station was acceptable for assessing the students' competence. Most of the students (75.2%) and professors (82.4%) assessed the required tasks as being feasible after reading the instructions. CONCLUSION: Our results showed that the hybrid CSE station was a highly authentic, acceptable, and feasible way to assess medical students' performance.

Assessing clinical reasoning abilities of medical students using clinical performance examination.

PURPOSE: The purpose of this study is to investigate the reliability and validity of new clinical performance examination (CPX) for assessing clinical reasoning skills and evaluating clinical reasoning ability of the students. METHODS: Third-year medical school students (n=313) in Busan-Gyeongnam consortium in 2014 were included in the study. One of 12 stations was developed to assess clinical reasoning abilities. The scenario and checklists of the station were revised by six experts. Chief complaint of the case was rhinorrhea, accompanied by fever, headache, and vomiting. Checklists focused on identifying of the main problem and systematic approach to the problem. Students interviewed the patient and recorded subjective and objective findings, assessments, plans (SOAP) note for 15 minutes. Two professors assessed students simultaneously. We performed statistical analysis on their scores and survey. RESULTS: The Cronbach α of subject station was 0.878 and Cohen κ coefficient between graders was 0.785. Students agreed on CPX as an adequate tool to evaluate students' performance, but some graders argued that the CPX failed to secure its validity due to their lack of understanding the case. One hundred eight students (34.5%) identified essential problem early and only 58 (18.5%) performed systematic history taking and physical examination. One hundred seventy-three of them (55.3%) communicated correct diagnosis with the patient. Most of them had trouble in writing SOAP notes. CONCLUSION: To gain reliability and validity, interrater agreement should be secured. Students' clinical reasoning skills were not enough. Students need to be trained on problem identification, reasoning skills and accurate record-keeping.

A pilot study on the evaluation of medical student documentation: assessment of SOAP notes.

PURPOSE: The purpose of this study was evaluation of the current status of medical students' documentation of patient medical records. METHODS: We checked the completeness, appropriateness, and accuracy of 95 Subjective-Objective-Assessment-Plan (SOAP) notes documented by third-year medical students who participated in clinical skill tests on December 1, 2014. Students were required to complete the SOAP note within 15 minutes of an standard patient (SP)-encounter with a SP complaining rhinorrhea and warring about meningitis. RESULTS: Of the 95 SOAP notes reviewed, 36.8% were not signed. Only 27.4% documented the patient's symptoms under the Objective component, although all students completed the Subjective notes appropriately. A possible diagnosis was assessed by 94.7% students. Plans were described in 94.7% of the SOAP notes. Over half the students planned workups (56.7%) for diagnosis and treatment (52.6%). Accurate documentation of the symptoms, physical findings, diagnoses, and plans were provided in 78.9%, 9.5%, 62.1%, and 38.0% notes, respectively. CONCLUSION: Our results showed that third-year medical students' SOAP notes were not complete, appropriate, or accurate. The most significant problems with completeness were the omission of students' signatures, and inappropriate documentation of the physical examinations conducted. An education and assessment program for complete and accurate medical recording has to be developed.