Challenges and potential improvements in the Accreditation Standards of the Korean Institute of Medical Education and Evaluation 2019 (ASK2019) derived through meta-evaluation: a cross-sectional study

PURPOSE: This study aimed to identify challenges and potential improvements in Korea's medical education accreditation process according to the Accreditation Standards of the Korean Institute of Medical Education and Evaluation 2019 (ASK2019). Meta-evaluation was conducted to survey the experiences and perceptions of stakeholders, including self-assessment committee members, site visit committee members, administrative staff, and medical school professors. METHODS: A cross-sectional study was conducted using surveys sent to 40 medical schools. The 332 participants included self-assessment committee members, site visit team members, administrative staff, and medical school professors. The t-test, one-way analysis of variance and the chi-square test were used to analyze and compare opinions on medical education accreditation between the categories of participants. RESULTS: Site visit committee members placed greater importance on the necessity of accreditation than faculty members. A shared positive view on accreditation's role in improving educational quality was seen among self-evaluation committee members and professors. Administrative staff highly regarded the Korean Institute of Medical Education and Evaluation's reliability and objectivity, unlike the self-evaluation committee members. Site visit committee members positively perceived the clarity of accreditation standards, differing from self-assessment committee members. Administrative staff were most optimistic about implementing standards. However, the accreditation process encountered challenges, especially in duplicating content and preparing self-evaluation reports. Finally, perceptions regarding the accuracy of final site visit reports varied significantly between the self-evaluation committee members and the site visit committee members. CONCLUSION: This study revealed diverse views on medical education accreditation, highlighting the need for improved communication, expectation alignment, and stakeholder collaboration to refine the accreditation process and quality.

Experiences and perspectives on patient-centered education of medical students in Korea.

PURPOSE: This study analyzed the current status of and correlations between Korean medical students' experiences and perspectives surrounding patient-centered medical education (PCME). METHODS: A structured PCME questionnaire composed of three categories, understanding patients within social and cultural contexts, understanding patients' individual health contexts through communication, and placement of patients at the center of medical education, was used. The students were stratified into pre-medical (Pre-med), medical (Med), and policlinic (PK) groups because of curriculum differences by grade. The χ2 test was applied to analyze the association between students' experiences with and perspectives on PCME. A Cramer's V of 0.200 was considered a large effect size for any association between experiences with and perspectives on PCME. RESULTS: Among the respondents, 50.6% answered that they did not know about patient-centered medicine before the survey. With increasing school years went up from Pre-med to PK, fewer students agreed that PCME should be added to pre-clinical medicine curricula (p<0.001), that patients should be in the center throughout medical education (p=0.011), and that patients' personal histories, values, and objectives are important PCME (p=0.001). Students who said they learned PCME for each category were more likely to consider PCME important (Cramer's V was 0.219 and 0.271 for "with," and "for the patients" respectively, p<0.001 for "about/with/for the patients"). Students in all groups chose clinical practice as the best method for PCME (p=0.021). Med group chose the lectures as the most effective tool to learn about the importance of communication (p<0.001). CONCLUSION: Students who experienced PCME were likely to perceive PCME as important and it showed that experiences of PCME had positive effects on PCME perceptions. Despite students' preferences for clinical practice as the best method for PCME, PK reported that they did not learn PCME, and regarded PCME as less important compared to students at earlier stages of their medical education. Therefore, more intensive and holistic PCME curricula rather than only clinical practice exposure may be necessary.

Development and validation of a portfolio assessment system for medical schools in Korea.

PURPOSE: Consistent evaluation procedures based on objective and rational standards are essential for the sustainability of portfolio-based education, which has been widely introduced in medical education. We aimed to develop and implement a portfolio assessment system, and to assess its validity and reliability. METHODS: We developed a portfolio assessment system from March 2019 to August 2019 and confirmed its content validity through expert assessment by an expert group comprising 2 medical education specialists, 2 professors involved in education at medical school, and a professor of basic medical science. Six trained assessors conducted 2 rounds of evaluation of 7 randomly selected portfolios for the "Self-Development and Portfolio II" course from January 2020 to July 2020. These data are used inter-rater reliability was evaluated using intra-class correlation coefficients (ICCs) in September 2020. RESULTS: The portfolio assessment system is based on the following process; assessor selection, training, analytical/comprehensive evaluation, and consensus. Appropriately trained assessors evaluated portfolios based on specific assessment criteria and a rubric for assigning points. In the analysis of inter-rater reliability, the first round of evaluation grades was submitted, and all assessment areas except "goal-setting" showed a high ICC of 0.81 or higher. After the first round of assessment, we attempted to standardize objective assessment procedures. As a result, all components of the assessments showed close correlations, with ICCs of 0.81 or higher. CONCLUSION: We confirmed that when assessors with an appropriate training conduct portfolio assessment based on specified standards through a systematic procedure, the results are reliable.

The relationship between students' perception of the educational environment and their subjective happiness.

BACKGROUND: Happiness, a subjective judgment about one's quality of life, is influenced by environmental factors and should be considered as an important goal of medical education, which should support each learner's development as a person as well as a professional. However, although several studies have reported on the correlation between Dundee Ready Educational Environment Measure (DREEM) scores and students' academic achievement, few have investigated the relationship between DREEM scores and students' subjective happiness. This study examined different perceptions of the educational environment between phases of the curriculum and determined which DREEM subscales affect the overall level of happiness. METHODS: We used the Korean version of the DREEM questionnaire and a single item measure of happiness on a scale of 0 to 10. First we analyzed student perceptions of the educational environment according to their demographic characteristics using independent sample t-tests and one-way analysis of variance. A multiple regression analysis was performed to reveal which subscales affect the overall level of happiness while controlling for grade point average (GPA) and other demographic characteristics. RESULTS: The subjects were 239 medical school students across all stages of the curriculum. The students' overall perception was more positive for the educational environment during Phase 3 (clerkship) than Phase 1 (pre-medical). Among the DREEM subscales, this difference was especially prominent in Students' Perception of Learning and Students' Academic Self-Perceptions. In contrast, no difference in the subjective perception of happiness was found between phases. The effect of GPA on happiness became insignificant under the control of other variables, but the influence of the Students' Social Self-Perceptions (SSSP) subscale remained significant. CONCLUSIONS: The students' overall perception of the educational environment was more positive during the clerkship period than in the pre-medical period. Based on our finding that the SSSP correlates significantly with subjective happiness, we suggest that institutions promote not only students' academic development but also their happiness by fostering an appropriate educational environment.

Satisfaction with and suitability of the problem-based learning program at the Catholic University of Korea College of Medicine.

PURPOSE: This study was conducted to identify suggestions for improving the effectiveness and promoting the success of the current Problem-based learning (PBL) program at the Catholic University of Korea College of Medicine according to professor and student awareness survey result. METHODS: A survey was carried out by sending out mobile Naver Form survey pages via text messages 3 times in December 2018, to 44 medical students and 74 professors of College of Medicine, The Catholic University of Korea. In addition, relevant official documents from the school administration were reviewed. The collected data were analyzed to identify the achievement of educational goals, overall satisfaction with, and operational suitability of the PBL. RESULTS: The overall satisfaction scores for the PBL program were neutral range (students: 3.27 ± 0.95 vs. professors: 3.58 ± 1.07, P=0.118). Regarding the achievement of educational goals, the integration of basic and clinical medicine and encouragement of learning motivation was ranked lowest. Many respondents expressed negative opinions about the modules (students: 25.0%, professors: 39.2%) and tutors (students: 54.5%, professors: 24.3%). The students and professors agreed that the offering timing of program and length of phase were suitable, while opinions expressed in greater detail pointed to issues such as the classes being held too close to exams and their alignment with regular course units. CONCLUSION: Issues with modules and tutors were the most pressing. Detailed and appropriate modules should be developed on the basis of advice from professors with experience in PBL tutoring. Inconsistencies in tutoring should be reduced by standardization and retraining.

Evaluation of a portfolio-based course on self-development for pre-medical students in Korea.

PURPOSE: We have developed and operated a portfolio-based course aimed at strengthening pre-medical students' capabilities for self-management and self-improvement. In order to determine the effectiveness of the course and to establish future operational strategies, we evaluated the course and the students' learning experience. METHODS: The subjects of this study were 97 students of a pre-medical course "Self-development and portfolio I" in 2019. Their learning experience was evaluated through the professor's assessment of portfolios they had submitted, and the program was evaluated based on the responses of 68 students who completed a survey. The survey questionnaire included 32 items. Descriptive statistics were reported for quantitative data, including the mean and standard deviation. Opinions collected from the open-ended question were grouped into categories. RESULTS: The evaluation of students' portfolios showed that only 6.2% of the students' portfolios were well-organized, with specific goals, strategies, processes, and self-reflections, while most lacked the basic components of a portfolio (46.4%) or contained insufficient content (47.4%). Students' responses to the survey showed that regular portfolio personality assessments (72.1%), team (64.7%), and individual (60.3%) activities were felt to be more appropriate as educational methods for this course, rather than lectures. Turning to the portfolio creation experience, the forms and components of the portfolios (68.2%) and the materials provided (62.2%) were felt to be appropriate. However, students felt that individual autonomy needed to be reflected more (66.7%) and that this course interfered with other studies (42.5%). CONCLUSION: The findings of this study suggest that standardized samples, guidelines, and sufficient time for autonomous portfolio creation should be provided. In addition, education on portfolio utilization should be conducted in small groups in the future.

Etiquette for medical students' email communication with faculty members: a single-institution study.

BACKGROUND: Email is widely used as a means of communication between faculty members and students in medical education because of its practical and educational advantages. However, because of the distinctive nature of medical education, students' inappropriate email etiquette may adversely affect their learning as well as faculty members' perception of them. Little data on medical students' competency in professional email writing is available; therefore, this study explored the strengths and weaknesses of medical students' email etiquette and factors that contribute to professional email writing. METHODS: A total of 210 emails from four faculty members at Seoul National University College of Medicine were collected. An evaluation criteria and a scoring rubric were developed based on the various email-writing guidelines. The rubric comprised 10 items, including nine items for evaluation related to the email components and one item for the assessment of global impression of politeness. Three evaluators independently assessed all emails according to the criteria. RESULTS: Students were identified as being 61.0% male and 52.8% were in the undergraduate-entry program. The sum of each component score was 62.21 out of 100 and the mean value for global impression was 2.6 out of 4. The results demonstrated that students' email etiquettes remained low-to-mediocre for most criteria, except for readability and honorifics. Three criteria, salutation (r=0.668), closing (r=0.653), and sign-off (r=0.646), showed a strong positive correlation with the global impression of politeness. Whether a student entered a graduate-entry program or an undergraduate-entry program significantly contributed to professional email writing after other variables were controlled. CONCLUSIONS: Although students in the graduate-entry program demonstrated a relatively superior level of email etiquette, the majority of medical students did not write emails professionally. Educating all medical students in email etiquette may well contribute to the improvement of student-faculty relationships as well as their email writing.

Evaluation of an international faculty development program for developing countries in Asia: the Seoul Intensive Course for Medical Educators.

BACKGROUND: The issue of collaboration in medical education is becoming prominent. Some faculty development programs have suggested an approach for promoting collaboration on a global level. However, non-English-speaking developing countries in Asia, especially in Southeast Asia, do not take advantage of them due to their unique context, such as language and culture. To address these issues, Seoul National University College of Medicine initiated a 6-week international faculty development program called the "Seoul Intensive Course for Medical Educators" for 16 fellows from five Asian countries (Cambodia, Laos, Mongolia, Myanmar, and Vietnam). The aim of this study is to report the evaluation results of the outcome of the program and discuss better ways of collaborating with developing countries. METHODS: Three levels of collaboration-intraorganizational, intranational, and international-were central initiatives of the program. Prior to setting up the program details, we first established four design principles, following which the contents, materials, and facilitators were determined to maintain consistency with the design principles. The evaluation of the program was done with Kirkpatrick's four-level model. Most of the evaluation data for level 1 were collected by two questionnaires, the post-module survey and the post-program survey. Portfolios and progress reports were mainly used to collect outcome data for levels 2 and 3, respectively. RESULTS: The reaction was generally positive throughout the program and there was a significant correlation between satisfaction and relevance to one's job or needs. Despite the fellows' propensity for overestimating themselves, both the evaluators and fellows reported that there was significant improvement in learning. Opinions on the impact or urgency of the topics were slightly different from country to country; however, the answers regarding feasibility were fairly similar. Moreover, we could observe from the post-program progress reports that the transfer of learning was actively in progress, mainly for topics that were highly feasible. CONCLUSIONS: These results show that the program was successful in terms of its effectiveness. Consistent and timely support is essential for the sustainable development of the medical education systems in these countries. Further understanding of the underlying factors on transfer (level 3) could improve the effectiveness of faculty development programs for developing countries.

[Research and development of evaluation criteria for premedical curriculum].

PURPOSE: The purpose of this study was to develop criteria to evaluate a premedical curriculum to ultimately improve the quality of premedical education. METHODS: The first draft of the evaluation criteria was developed through a literature review and expert consultation. The Delphi survey was conducted to ensure the validity of the draft. RESULTS: The final premedical curriculum criteria consisted of three evaluation areas (curriculum development, curriculum implementation, and curriculum outcome), five evaluation items (educational objective, organization of curriculum, instructional method, class management,and educational outcome), and 18 evaluation indicators. CONCLUSION: There should be further discussion on the evaluation questionnaire and the content for each evaluation indicator with regard to its practical application. Also, a concrete evaluation system, including evaluation standards and rating scales, should be developed.

[Peer review analysis of lectures using video recordings in an integrated curriculum].

PURPOSE: The purpose of this study is to evaluate the effect of the peer review in an integrated curriculum and to guide further improvements of curriculum. METHODS: In 2012, Seoul National University College of Medicine implemented a peer review system for 11 courses in an integrated curriculum. For each lecture, two reviewers conducted the rating using a 10-item questionnaire on a 4-point scale. We analyzed the correlation between total scores and each item and the inter-rater reliability between the two reviewers by Pearson correlation. Further, the link between peer review scores and the student lecture evaluation was analyzed. RESULTS: The mean total score for the checklist rating was 31.3 (out of 40.0), and the mean score for each item ranged from 2.65 to 3.35 (out of 4.00). The correlation coefficient between the total score and each item was high, ranging from 0.656 to 0.849, except for three items. The mean of difference scores between reviewers was 5.03, and the correlation coefficient was significantly high, which ranged from 0.968 to 0.999. The peer reviews scores and student lecture evaluations generally correlated, but there were some outlying exceptions; the correlation coefficient was 0.105 and 0.093. CONCLUSION: Peer review is a useful method for improving the quality of lectures in an integrated curriculum by monitoring the objectives, contents, and methods of the lectures and providing feedback to the professors.

[Equating scores using bridging stations on the clinical performance examination].

PURPOSE: This study examined the use of the Tucker linear equating method in producing an individual student's score in 3 groups with bridging stations over 3 consecutive days of the clinical performance examination (CPX) and compared the differences in scoring patterns by bridging number. METHODS: Data were drawn from 88 examinees from 3 different CPX groups-DAY1, DAY2, and DAY3-each of which comprised of 6 stations. Each group had 3 common stations, and each group had 2 or 3 stations that differed from other groups. DAY1 and DAY3 were equated to DAY2. Equated mean scores and standard deviations were compared with the originals. DAY1 and DAY3 were equated again, and the differences in scores (equated score-raw score) were compared between the 3 sets of equated scores. RESULTS: By equating to DAY2, DAY1 decreased in mean score from 58.188 to 56.549 and in standard deviation from 4.991 to 5.046, and DAY3 fell in mean score from 58.351 to 58.057 and in standard deviation from 5.546 to 5.856, which demonstrates that the scores of examinees in DAY1 and DAY2 were accentuated after use of the equation. The patterns in score differences between the equated sets to DAY1, DAY2, and DAY3 yielded information on the soundness of the equating results from individual and overall comparisons. CONCLUSION: To generate equated scores between 3 groups on 3 consecutive days of the CPX, we applied the Tucker linear equating method. We also present a method of equating reciprocal days to the anchoring day as much as bridging stations.