Tutor assessment of medical students in problem-based learning sessions.

BACKGROUND: Problem-based learning (PBL) is a method of learning that has been adopted in different curricula of different disciplines for more than 30 years; the assessment of the students in PBL sessions in medical schools is fundamental to ensure students' attainment of the expected outcomes of conducting PBL sessions and in providing the students with the feedback that help them to develop and encourage their learning. This study investigated the inter-rater reliability of the tutor assessment in assessing medical students' performance in their PBL tutorial sessions. MATERIALS AND METHODS: This study was conducted in the College of Medicine (COM), in the academic year 2021-2022. The study involved ten raters (tutors) of two genders who assessed 33 students in three separate PBL tutorial sessions. The PBL sessions were prerecorded and shown to the 10 raters for their assessment of PBL sessions. RESULTS: This study showed that male raters gave higher scores to students compared with female raters. In addition, this investigation showed low inter-rater reliability and poor agreement among the raters in assessing students' performance in PBL tutorial sessions. CONCLUSION: This study suggests that PBL tutor assessment should be reviewed and evaluated; this should be performed with consideration of using assessment domains and criteria of performance. Thus, we recommend that 360-degree assessment including tutor, self, and peer assessment should be used to provide effective feedback to students in PBL tutorial sessions.

The influence of context on diagnostic reasoning: A narrative synthesis of experimental findings.

AIMS AND OBJECTIVES: Contextual information which is implicitly available to physicians during clinical encounters has been shown to influence diagnostic reasoning. To better understand the psychological mechanisms underlying the influence of context on diagnostic accuracy, we conducted a review of experimental research on this topic. METHOD: We searched Web of Science, PubMed, and Scopus for relevant articles and looked for additional records by reading the references and approaching experts. We limited the review to true experiments involving physicians in which the outcome variable was the accuracy of the diagnosis. RESULTS: The 43 studies reviewed examined two categories of contextual variables: (a) case-intrinsic contextual information and (b) case-extrinsic contextual information. Case-intrinsic information includes implicit misleading diagnostic suggestions in the disease history of the patient, or emotional volatility of the patient. Case-extrinsic or situational information includes a similar (but different) case seen previously, perceived case difficulty, or external digital diagnostic support. Time pressure and interruptions are other extrinsic influences that may affect the accuracy of a diagnosis but have produced conflicting findings. CONCLUSION: We propose two tentative hypotheses explaining the role of context in diagnostic accuracy. According to the negative-affect hypothesis, diagnostic errors emerge when the physician's attention shifts from the relevant clinical findings to the (irrelevant) source of negative affect (for instance patient aggression) raised in a clinical encounter. The early-diagnosis-primacy hypothesis attributes errors to the extraordinary influence of the initial hypothesis that comes to the physician's mind on the subsequent collecting and interpretation of case information. Future research should test these mechanisms explicitly. Possible alternative mechanisms such as premature closure or increased production of (irrelevant) rival diagnoses in response to context deserve further scrutiny. Implications for medical education and practice are discussed.

Relationship between admission selection tools and student attrition in the early years of medical school.

Objectives: Placement in medical schools is highly sought after worldwide with fierce competition among applicants. However, some of the best students withdraw after being accepted to medical school. The aim of this study was to investigate early student attrition within the first 2 years of medical school and determine its relationship to admission selection tools. Methods: Quantitative research was conducted at the College of Medicine and Health Sciences from 2016 until 2020, during which time routine admission data and students' examination results for the first 2 years were collected and analyzed. Results: The attrition rate during the study period was 31.7%. High school and college written examination scores were significantly related to completing the premedical program (p = 0.001 and p = 0.002, respectively). Female students scored significantly higher in multiple mini interviews (MMIs) compared with male counterparts (p < 0.001). However, the difference in MMI score was not related to student attrition (p = 0.148). Conclusion: The cause of early attrition is complex and cannot be attributed to a single factor.Undergraduate high school score and written admission examination results were statistically significant factors in relation to student attrition rate and low academic performance. The results of this study showed that the female students scored significantly higher in the multiple MMI tests compared to male students. However, MMI score alone was not significantly related to student attrition.

Overconfidence, Time-on-Task, and Medical Errors: Is There a Relationship?

Background: Literature suggest that physicians' high level of confidence has a negative impact on medical decisions, and this may lead to medical errors. Experimental research is lacking; however, this study investigated the effects of high confidence on diagnostic accuracy. Methods: Forty internal medicine residents from different hospitals in Saudi Arabia were divided randomly into two groups: A high-confidence group as an experimental and a low-confidence group acting as a control. Both groups solved each of eight written complex clinical vignettes. Before diagnosing these cases, the high-confidence group was led to believe that the task was easy, while the low-confidence group was presented with information from which it could deduce that the diagnostic task was difficult. Level of confidence, response time, and diagnostic accuracy were recorded. Results: The participants in the high-confidence group had a significantly higher confidence level than those in the control group: 0.75 compared to 0.61 (maximum 1.00). However, neither time on task nor diagnostic accuracy significantly differed between the two groups. Conclusion: In the literature, high confidence as one of common cognitive biases has a strong association with medical error. Even though the high-confidence group spent somewhat less time on the cases, suggesting potential premature decision-making, we failed to find differences in diagnostic accuracy. It is suggested that overconfidence should be studied as a personality trait rather than as a malleable characteristic.

Integrating the social determinants of health into curriculum: AMEE Guide No. 162.

The World Health Organization (WHO) defines the Social Determinants of Health (SDOH) as the non-medical factors influencing health outcomes. SDOH is associated with conditions in which people are born, grow, work, and live. Medical schools and licensing bodies are increasingly recognizing the need for doctors and healthcare professionals to be aware of their patient's social context and how it impacts their states of health and disease. However, there is considerable variation in the approaches of different institutions and countries to incorporating SDOH into their curricula. In order to allow clinicians to adopt a holistic approach to patient health, equipping them with extensive knowledge of SDOH would give learners the confidence, skills, knowledge, and attitudes needed to effectively engage with patients and their families. This approach aids health professionals with knowledge of the influence of the social context and cultural factors that affect patients' behaviors in relation to health. Incorporating the SDOH in medical and health professional school curricula would contribute towards adequately preparing future healthcare practitioners to provide effective, comprehensive, and equitable care, especially to marginalized and underserved populations. The Guide will take an evidence-based approach grounded in the available contemporary literature and case studies. The focus will be on integrating SDOH into undergraduate and postgraduate medical curricula to promote an understanding of the social factors that influence patients' and communities' health. Ultimately, this guide seeks to contribute to the reduction of inequalities in health.

Practical Guide to Undergraduate Medical Curriculum Alignment and Mapping.

Objective: Curriculum development and reviews are of paramount importance for academic programs curriculum management. Medical curricula need proper construction and frequent updates to suits learning outcomes considering their integrative nature and rapid scientific advancement. Curriculum alignment and mapping are fundamental for proper integration and planning of medical curriculum. This process is beneficial in detecting gaps and redundancies, and ensuring proper instruction, integration, and transparency. However, there is a paucity in the literature of a practical guidance to such process. Hence, this manuscript provides a practical guidance that was adopted in our institutes. Methods: A detailed description of twelve step-by-step guidance to curriculum alignment and mapping was provided. The process needed for each step and the flow chart of work was detailed. Results: This guide was developed and implemented successfully. Among many benefits encountered, many gaps in the contents, learning outcomes and assessment methods were detected and rectified. Additionally, better curriculum integration has been achieved. The current mapping will make any future curriculum reviews easier. Conclusion: This guide could be utilized by newly developed and existing programs for curriculum alignment and mapping. It fills a gap in academic literature through stepwise workflow which has been tested and implemented.

Assessing relational coordination and its impact on perceived mental health of students, teachers and staff in a clinical skills program during the COVID-19 pandemic.

BACKGROUND: The global spread of the COVID-19 virus caused unprecedented interruptions in medical education. This paper evaluates Relational Coordination (RC): communicating and relating for task integration; between the distinct stakeholders responsible for scheduling,delivering and receiving clinical teaching in the wake of the pandemic. METHODOLOGY: Using a cross-sectional design, the level of Relational Coordination was assessed between twelve groups within a Clinical Skills Program at a Medical School in the United Arab Emirates. It also measures three relevant mental health factors: namely, Job satisfaction, Work Engagement, and Burnout. RESULTS: Overall, RC scores were moderate (3.65 out of 5.00). Controlling for participants' position, RC was found to positively and significantly increase both job satisfaction (ß = 1.10, p < 0.001) and work engagement (ß = 0.78, p < 0.01)., Additionally, RC was significantly associated with lower burnout (ß = -0.56, p = 0.05). Fifty percent of participants experienced high job satisfaction, with a mean score of 5.0 out of 7.0, while 73% reported being enthusiastic about their job, with a mean score of 6.0 out of 7.0. About a third of participants (27%) reported feeling burnout. CONCLUSIONS: During times of disruption and crisis, medical education can benefit from higher levels of relational coordination. Our study shows the significant impact of relational coordination on mental health measures like job satisfaction and work engagement. To achieve the full potential and benefits of excellent levels of relational coordination in this program, we recommend six interventions focusing on improving communication, work processes, regular meetings, education innovations, capacity building, and the establishment of coaching and counseling programs for students and faculty.

Failure to demonstrate effects of interruptions on diagnostic reasoning: three experiments.

BACKGROUND: Diagnostic error is a major source of patient suffering. Researchshows that physicians experience frequent interruptions while being engaged with patients and indicate that diagnostic accuracy may be impaired as a result. Since most studies in the field are observational, there is as yet no evidence suggesting a direct causal link between being interrupted and diagnostic error. Theexperiments reported in this article were intended to assess this hypothesis. METHODS: Three experiments were conducted to test the hypothesis that interruptions hurt diagnostic reasoning and increase time on task. In the first experiment (N = 42), internal medicine residents, while diagnosing vignettes of actual clinical cases were interrupted halfway with a task unrelated to medicine, solving word-spotting puzzles and anagrams. In the second experiment (N = 78), the interruptions were medically relevant ones. In the third experiment (N = 30), we put additional time pressure on the participants. In all these experiments, a control group diagnosed the cases without interruption. Dependent variables were diagnostic accuracy and amount of time spent on the vignettes. RESULTS: In none of the experiments interruptions were demonstrated to influence diagnostic accuracy. In Experiment 1: Mean of interrupted group was 0.88 (SD = 0.37) versus non- interrupted group 0.91 (SD = 0.32). In Experiment 2: Mean of interrupted group was 0.95 (SD = 0.32) versus non-interrupted group 0.94 (SD = 0.38). In Experiment 3: Mean of interrupted group was 0.42 (SD = 0.12) versus non-interrupted group 0.37 (SD = 0.08). Although interrupted residents in all experiments needed more time to complete the diagnostic task, only in Experiment 2, this effect was statistically significant. CONCLUSIONS: These three experiments, taken together, failed to demonstrate negative effects of interruptions on diagnostic reasoning. Perhaps physicians who are interrupted may still have sufficient cognitive resources available to recover from it most of the time.

Conducting online OSCEs aided by a novel time management web-based system.

BACKGROUND: Objective Structured Clinical Examinations (OSCEs) are an essential part of the assessment process for medical students. They have traditionally been face-to-face assessments, however, due to the COVID-19 pandemic, medical schools have been forced to attempt to carry them out remotely. OSCEs are difficult to carry out online due to rotation management aspects which make it difficult to synchronize movement of students from one station to another. METHODS: The authors have developed a dynamic OSCE time management website which aids in solving the movement synchronization issue. This secure website enables participants to view the list of stations they are allowed to enter, and the start and end time of each station. OSCE administrators can control time of entry and monitor progress of the OSCE remotely. RESULTS: The authors have used the system to conduct several exams successfully, showing the feasibility and cost effectiveness of this method, as well as user acceptance and satisfaction. In contrast to traditional OSCEs, students are set up in individual virtual rooms for the whole exam while examiners, simulated patients and proctors rotate between them. CONCLUSIONS: This online OSCE implementation shows feasibility, cost effectiveness and acceptance of this method. The authors found that student outcomes are comparable to traditional OSCEs conducted in the past. There was no significant difference in student marks in one exam compared to last year, while marks were slightly higher in two exams, potentially due to lack of physical exam stations. An unresolved drawback is the inability to assess physical exam stations online, although having students verbally describe what they would do in physical exam situations may be a partial solution.

Introducing free response short answer questions in anatomy spot tests: experiment study.

PURPOSE: A spot test is commonly used to assess practical knowledge through a series of specific questions related to the marked anatomical structures on cadavers. However, the continuous reviewing and developing of the medical curricula demands the need to improve the assessment method of the practical examination of anatomical knowledge. Therefore, the aim of the study was to examine the effectiveness of a free response short-answer questions (FRSAQs) test and traditional questions on medical students' performance. METHODS: This is an experimental study using a Randomized Posttest-Only Control Group Design to compare the mean of students' performance in addition to their perceptions about the two versions of the spot tests. Two hundred and ninety preclinical-year medical students were invited to participate in this study. RESULTS: Only 109 (38%) students participated in this study. The data analysis showed a significant improvement in students' performance in the FRSAQs test (5.3 ± 2.7) than the traditional spot test (4.7 ± 2.6), (P = 0.04). The majority of the students (70%) preferred the FRSAQs spot test, citing that it is good test to assess practical knowledge. The psychometric analysis revealed that the FRSAQs produced more ideal stations (60%) when compared to the traditional spot test (10%). CONCLUSION: The current study found that the FRSAQs spot test had a significant impact on students' performance, and it was considered an ideal, flexible, and stress-free assessment method when compared to the traditional spot test. Implementation of the strategy in teaching and in assessment is recommended.

Do poor patients suffer from inaccurate diagnoses more than well-to-do patients? A randomized control trial.

BACKGROUND: Poor patients have greater morbidity and die up to 10 years earlier than patients who have higher socio-economic status. These findings are often attributed to differences in life-style between groups. The present study aimed at investigating the extent to which physicians contribute to the effect by providing relative poorer care, resulting in relative neglect in terms of time spent with a poor patient and more inaccurate diagnoses. METHODS: A randomised experiment with 45 internal medicine residents. Doctors diagnosed 12 written clinical vignettes that were exactly the same except for the description of the patients' socio-economic status. Each participant diagnosed four of the vignettes in a poor-patient version, four in a rich-patient version, and four in a version that did not contain socio-economic markers, in a balanced within-subjects incomplete block design. Main measurements were: diagnostic accuracy scores and time spent on diagnosis. RESULTS: Mean diagnostic accuracy scores (range 0-1) did not significantly differ among the conditions of the experiment (for poor patients: 0.48; for rich patients: 0.52; for patients without socio-economic markers: 0.54; p > 0.05). While confronted with patients not presenting with socio-economic background information, the participants spent significantly less time-to-diagnosis ((for poor patients: 168 s; for rich patients: 176 s; for patients without socio-economic markers: 151 s; p < 0.01), however due to the fact that the former vignettes were shorter. CONCLUSION: There is no reason to believe that physicians are prejudiced against poor patients and therefore treat them differently from rich patients or patients without discernible socio-economic background.

Reliability and validity of the faculty evaluation instrument used at King Saud bin Abdulaziz University for Health Sciences: Results from the Haematology Course.

OBJECTIVE: To assess reliability and validity of evaluation tool using Haematology course as an example. METHODS: The cross-sectional study was conducted at King Saud Bin Abdul Aziz University of Health Sciences, Riyadh, Saudi Arabia, in 2012, while data analysis was completed in 2013. The 27-item block evaluation instrument was developed by a multidisciplinary faculty after a comprehensive literature review. Validity of the questionnaire was confirmed using principal component analysis with varimax rotation and Kaiser normalisation. Identified factors were combined to get the internal consistency reliability of each factor. Student's t-test was used to compare mean ratings between male and female students for the faculty and block evaluation. RESULTS: Of the 116 subjects in the study, 80(69%) were males and 36(31%) were females. Reliability of the questionnaire was Cronbach's alpha 0.91. Factor analysis yielded a logically coherent 7 factor solution that explained 75% of the variation in the data. The factors were group dynamics in problem-based learning (alpha0.92), block administration (alpha 0.89), quality of objective structured clinical examination (alpha 0.86), block coordination (alpha 0.81), structure of problem-based learning (alpha 0.84), quality of written exam (alpha 0.91), and difficulty of exams (alpha0.41). Female students' opinion on depth of analysis and critical thinking was significantly higher than that of the males (p=0.03). CONCLUSIONS: The faculty evaluation tool used was found to be reliable, but its validity, as assessed through factor analysis, has to be interpreted with caution as the responders were less than the minimum required for factor analysis.

Does Time Pressure Have a Negative Effect on Diagnostic Accuracy?

PURPOSE: Studies suggest time pressure has negative effects on physicians' working conditions and may lead to suboptimal patient care and medical errors. Experimental evidence supporting this is lacking, however. This study investigated the effect of time pressure on diagnostic accuracy. METHOD: In 2013, senior internal medicine residents at three hospitals in Saudi Arabia were divided randomly into two groups: a time-pressure condition and a control condition without time pressure. Both groups diagnosed eight written clinical cases presented on computers. In the time-pressure condition, after completing each case, participants received information that they were behind schedule. Response time was recorded, and diagnostic accuracy was scored. RESULTS: The 23 participants in the time-pressure condition spent significantly less time diagnosing the cases (mean = 96.00 seconds) than the 19 control participants (mean = 151.97 seconds) (P < .001). Participants under time pressure had a significantly lower diagnostic accuracy score (mean = 0.33; 95% CI, 0.23-0.43) than participants without time pressure (mean = 0.51; 95% CI, 0.42-0.60) (F[1, 41] = 6.90, P = .012, η = 0.15). This suggests participants in the time-pressure condition made on average 37% more errors than control participants. CONCLUSIONS: Time pressure has a negative impact on diagnostic performance. The authors propose that the effect of time pressure on diagnostic accuracy is moderated by both the case difficulty level and the physician's level of experience. Post hoc analyses demonstrated that time pressure affects diagnostic accuracy only if cases are not too difficult and physicians' expertise level is intermediate.

Time for an Adolescent Health Surveillance System in Saudi Arabia: Findings From "Jeeluna".

PURPOSE: With the increasing burden of noncommunicable disease, adolescence is viewed as an opportune time to prevent the onset of certain behaviors and promote healthy states. Although adolescents comprise a considerable portion of Saudi Arabia's population, they have received insufficient attention and indicators of their health status, as a first step in a prevention cycle are unavailable. This study was carried out with the aim of identifying the health risk behaviors and health status of adolescents in Saudi Arabia. METHODS: This cross-sectional, school-based study was carried out in all 13 regions of Saudi Arabia. Through multistage, cluster, random sampling, intermediate, and secondary school students were invited to participate. Data were collected by means of a self-administered questionnaire addressing health risk behaviors and health status, clinical anthropometric measurements, and laboratory investigations. RESULTS: A total of 12,575 adolescents participated. Various health risk behaviors, including dietary and sedentary behaviors, lack of safety measures, tobacco use, bullying, and violence were highly prevalent. Twenty-eight percent of adolescents reported having a chronic health condition, 14.3% reported having symptoms suggestive of depression, 30.0% were overweight/obese, and 95.6% were vitamin D deficient. CONCLUSION: Behaviors and conditions known to persist into adulthood and result in morbidity and premature mortality are prevalent among adolescents in Saudi Arabia. Preventive measures and local health policies are urgently needed and can impact adolescents and future adults. Establishing adolescent health surveillance is necessary to monitor trends and impacts of such measures.

Effect of course coordinator behavior and motivation on students' achievement: Results from five curriculum blocks of two undergraduate student cohorts at King Saud bin Abdulaziz University of Health Sciences.

OBJECTIVE: The purpose of the study was to assess the relationship between students' perception of course/block coordinators performance and attributes with students' assessment scores in respective courses. METHODS: This retrospective data based study was conducted at the College of Medicine, King Saud bin Abdulaziz University of Health Sciences (KSAU-HS). It was started in March 2013 and completed in June 2013 after the graduation of the fourth cohort. Exam score of 3(rd) and 4(th) cohort of students from the courses taught in the last two years of medical school were correlated with faculty and block evaluation done by the students. Scores from mid-block MCQs, portfolio scores, OSCEs and end-of-block MCQs were obtained. RESULTS: The Mean scores of all the assessments for all five blocks were not significantly different for both batches. There was significant difference between block coordinators for students' score on portfolio, midterm exam and the final written exam. The students' Score in OSCE had significantly strong correlation with quality of station monitors, coverage of content and flow between stations. Student's perception of the commitment and motivation of the coordinator was strongly correlated with block organization, availability of clinical cases, performance of block coordinator, cooperation with students, and organization of clinical activities. CONCLUSIONS: Block coordinator's motivation and commitment affects quality of block organization and student`s success. Faculty training programs should include block management competencies and components identified through self-determination theory for improving the intrinsic motivation for students success.

Evaluation of off-service rotations at National Guard Health Affairs: Results from a perception survey of off-service residents.

CONTEXT: "Off-service" clinical rotations are part of the necessary requirements for many residency training programs. Because these rotations are off-service, little attention is given to their structure and quality of training. This often leads to suboptimal educational experience for the residents on these rotations. AIMS: The aim of this study was to assess medical residents' perceptions, opinions, and levels of satisfaction with their "off-service" rotations at a major residency training site in Saudi Arabia. It was also to evaluate the reliability and validity of a questionnaire used for quality assurance in these rotations. Improved reliability and validity of this questionnaire may help to improve the educational experience of residents in their "off-service" rotations. MATERIALS AND METHODS: A close-ended questionnaire was developed, Pilot tested and distributed to 110 off-service residents in training programs of different specializations at King Fahad Naitonal Guard Hospital and King Abdulziz Medical City, Riyadh, Saudi Arabia, between September 2011 and December 2011. RESULTS: A total of 80 out of 110 residents completed and returned the questionnaire. Only 33% of these residents had a clear set of goals and educational learning objectives before the beginning of their off-service rotations to direct their training. Surgical specializations had low satisfaction mean scores of 57.2 (11.9) compared to emergency medicine, which had 70.7 (16.2), P value (0.03). The reliability of the questionnaire was Cronbach's alpha 0.57. The factor analysis yielded a 4-factor solution (educational environment, educational balance, educational goals and objectives, and learning ability); thus, accounting for 51% variance in the data. CONCLUSION: Our data suggest that there were significant weaknesses in the curriculum for off-service clinical rotations in KAMC and that residents were not completely satisfied with their training.

Constructing a question bank based on script concordance approach as a novel assessment methodology in surgical education.

BACKGROUND: Script Concordance Test (SCT) is a new assessment tool that reliably assesses clinical reasoning skills. Previous descriptions of developing SCT-question banks were merely subjective. This study addresses two gaps in the literature: 1) conducting the first phase of a multistep validation process of SCT in Plastic Surgery, and 2) providing an objective methodology to construct a question bank based on SCT. METHODS: After developing a test blueprint, 52 test items were written. Five validation questions were developed and a validation survey was established online. Seven reviewers were asked to answer this survey. They were recruited from two countries, Saudi Arabia and Canada, to improve the test's external validity. Their ratings were transformed into percentages. Analysis was performed to compare reviewers' ratings by looking at correlations, ranges, means, medians, and overall scores. RESULTS: Scores of reviewers' ratings were between 76% and 95% (mean 86% ± 5). We found poor correlations between reviewers (Pearson's: +0.38 to -0.22). Ratings of individual validation questions ranged between 0 and 4 (on a scale 1-5). Means and medians of these ranges were computed for each test item (mean: 0.8 to 2.4; median: 1 to 3). A subset of test items comprising 27 items was generated based on a set of inclusion and exclusion criteria. CONCLUSION: This study proposes an objective methodology for validation of SCT-question bank. Analysis of validation survey is done from all angles, i.e., reviewers, validation questions, and test items. Finally, a subset of test items is generated based on a set of criteria.

Integration of evidence based medicine into the clinical years of a medical curriculum.

Teaching Evidence Based Medicine (EBM) helps medical students to develop their decision making skills based on current best evidence, especially when it is taught in a clinical context. Few medical schools integrate Evidence Based Medicine into undergraduate curriculum, and those who do so, do it at the academic years only as a standalone (classroom) teaching but not at the clinical years. The College of Medicine at King Saud bin Abdulaziz University for Health Sciences was established in January 2004. The college adopted a four-year Problem Based Learning web-based curriculum. The objective of this paper is to present our experience in the integration of the EBM in the clinical phase of the medical curriculum. We teach EBM in 3 steps: first step is teaching EBM concepts and principles, second is teaching the appraisal and search skills, and the last step is teaching it in clinical rotations. Teaching EBM at clinical years consists of 4 student-centered tutorials. In conclusion, EBM may be taught in a systematic, patient centered approach at clinical rounds. This paper could serve as a model of Evidence Based Medicine integration into the clinical phase of a medical curriculum.

Toward an objective assessment of technical skills: a national survey of surgical program directors in Saudi Arabia.

BACKGROUND: After almost a decade of implementing competency-based programs in postgraduate training programs, the assessment of technical skills remains more subjective than objective. National data on the assessment of technical skills during surgical training are lacking. We conducted this study to document the assessment tools for technical skills currently used in different surgical specialties, their relationship with remediation, the recommended tools from the program directors' perspective, and program directors' attitudes toward the available objective tools to assess technical skills. METHODS: This study was a cross-sectional survey of surgical program directors (PDs). The survey was initially developed using a focus group and was then sent to 116 PDs. The survey contains demographic information about the program, the objective assessment tools used, and the reason for not using assessment tools. The last section discusses the recommended tools to be used from the PDs' perspective and the PDs' attitude and motivation to apply these tools in each program. The associations between the responses to the assessment questions and remediation were statistically evaluated. RESULTS: Seventy-one (61%) participants responded. Of the respondents, 59% mentioned using only nonstandardized, subjective, direct observation for technical skills assessment. Sixty percent use only summative evaluation, whereas 15% perform only formative evaluations of their residents, and the remaining 22% conduct both summative and formative evaluations of their residents' technical skills. Operative portfolios are kept by 53% of programs. The percentage of programs with mechanisms for remediation is 29% (19 of 65). CONCLUSION: The survey showed that surgical training programs use different tools to assess surgical skills competency. Having a clear remediation mechanism was highly associated with reporting remediation, which reflects the capability to detect struggling residents. Surgical training leadership should invest more in standardizing the assessment of surgical skills.