Quality assurance procedures in assessment - a descriptive study of medical colleges in Pakistan.

OBJECTIVE: To identify the quality assurance procedures being implemented in the assessment system of medical colleges in Pakistan. METHODS: The cross-sectional study was conducted from March 2015 to December 2017 in medical training institutions recognised by the Pakistan Medical and Dental Council across Pakistan and Azad Jammu and Kashmir, and comprised individuals designated by the respective institutional administrations. The mixed method technique was employed using a semi-structured questionnaire. Data was analysed using SPSS 21. RESULTS: Of the 49 institutions, 20(41%) were in the public sector and 29(59%) were in the private sector. Overall, 35(71.4%) institutions followed a written assessment policy provided by the affiliated university, 9(18%) never did so, 22(44.8 %) had content experts checking if the questions matched the objectives, 42(85.7%) took strict steps to prevent cheating in exams, and 26(53.1%) analysed theory exam statistically. Discrimination index, difficulty index, reliability, and point biserial were 14(28.6%), 13(26.5%), 12(24.4%), and 7(14.3%) of the medical colleges respectively. Only 12(24.5%) institutions provided written feedback on the results, and 15(30.6%) conducted annual internal audit. CONCLUSION: General issues related to quality assurance procedures in assessments were found to be in place in majority of the colleges. However, a large proportion did not have them.

Identification and comparison of factors related to academic failure in medical colleges; A Bi-centre, comparative study.

OBJECTIVE: To evaluate student's perceptions related to their academic failure, and to compare these perceptions with their nationality. METHODS: The non-interventional, bi-national, comparative study was conducted in medical colleges of Pakistan and Saudi Arabia from 2015 to 2017 and comprised medical students who had scored less than 50% in their professional examinations. Data was collected using a pre-designed 22-item questionnaire that was scored by the students on a 5-point scale. Data was analysed using SPSS 23. RESULTS: Of the 210 students, 115(55%) were Pakistanis and 95(45%) were Saudis. Common reasons of failure identified were information overload 114(50%), difficult examination 101(48%), poor teaching skills of teachers 82(39%), system of education 75(36%) and unfair examination 78(37%). In terms of nationality, 11 items were found significant (p<0.05). CONCLUSIONS: Regardless of cultural and environmental differences in factors contributing towards failure in students of different countries, the risk factors were found to be common, with most students blaming external factors rather than internal ones.

Difference in memory recall among medical students after reading printed text (hard copy) vs. on screen text (soft copy).

OBJECTIVE: To compare the difference in "memory recall" among undergraduates (UGs) and postgraduates (PGs) medical students reading printed text (hard copy) vs. on-screen text (soft copy). METHODS: Final year UGs medical students (Group A) were divided into two batches, one batch was given hard copy of 1258 worded text and other group was given a soft copy of the same text. Likewise PGs (Group B) were divided into two batches, each batch was given hard copy and soft copy of the same text material with identical physical attributes. All batches were asked to read the text in 15 minutes. Text was then removed and a proforma containing statements pertaining to recall of the text material was administered to all. RESULTS: The scores of students reading soft copy text were significantly higher compared to student reading text from hard copy (p<.008) in the proforma assessing memory recall. CONCLUSIONS: The memory recall from reading on screen text is better than reading printed (paper) text, implying that with the advancement of technology and early exposure to computer screen, students are developing the habit of reading on screen text more conveniently.

Comparison of OSCE scores of surgical clinical education after face-to-face vs. blended learning methods among undergraduate medical students.

OBJECTIVES: To compare the efficacy of blended learning with contemporary face-to-face teaching among medical students. METHODS: The quasi-experimental, cross-over study was conducted at the Dow University of Health Sciences, Karachi, from March to August 2014, and comprised third year medical students posted in the surgical units who were divided into two groups. In the first month, group A was taught by blended learning (BL), while group B was taught by face-to-face (f2f) teaching. Both groups were assessed using objective structured clinical examination. In the second month, the teaching method was flipped for the groups with new learning contents. Assessment tool remained the same. Data was analysed using SPSS 23. RESULTS: Thirty-one students completed OSCE in group A (BL) and 28 in group B (f2f). The mean score in group B was 78.01±13.29 compared to 85.12±13.77 in group A (p=0.49). CONCLUSIONS: Blended surgical learning was found to be more effective compared to face-to-face teaching.

Attitudes of faculty and residents of surgical specialties towards professionalism at a tertiary care hospital of Islamabad.

OBJECTIVES: To determine the attitudes of faculty and residents of surgical specialties towards professionalism and to test the validity and reliability of a tool developed in USA for Pakistan. METHODS: An exploratory validation study was carried out at Pakistan Institute of Medical Sciences (PIMS), Islamabad from 01-Aug-2016 to 31-Jan-2017. Penn State College of Medicine Professionalism Questionnaire (PSCOM) was used, being a reliable and valid survey tools. Exploratory Factor Analysis of the inter-correlations of responses for 36 items was done using SPPS v 21 to give a factor solution to reflect the perceptions regarding attitudinal elements. RESULTS: There were 209 respondents including 172 residents and 37 faculty members. Response rate was 81.32%. Exploratory Factor Analyses of responses gave a seven factor solution of professionalism: accountability, honour and integrity, excellence, duty, altruism, equity and respect. Six of the factors that emerged reflect the six elements of professionalism reported by American Board of Internal Medicine (ABIM), except for 'equity', which was the new factor that emerged. Cronbach's alpha (Internal Consistency Reliability) for each element of professionalism was between 0.88-0.98. CONCLUSION: PSCOM is a valid, reliable, feasible and acceptable tool to assess attitudes of faculty and residents towards professionalism in Pakistan. 'Equity' emerged as a new factor which needs to be studied further.

Student perception and post-exam analysis of one best MCQs and one correct MCQs: A comparative study.

OBJECTIVE: To find differences between One-Correct and One-Best multiple-choice questions with relation to student scores, post-exam item analyses results and student perception. METHODS: This comparative cross-sectional study was conducted at the Dow University of Health Sciences, Karachi, from November 2010 to April 2011, and comprised medical students. Data was analysed using SPSS 18. RESULTS: Of the 207 participants, 16(7.7%) were boys and 191(92.3%) were girls. The mean score in Paper I was 18.62±4.7, while in Paper II it was 19.58±6.1. One-Best multiple-choice questions performed better than One-Correct. There was no statistically significant difference in the mean scores of the two papers or in the difficulty indices. Difficulty and discrimination indices correlated well in both papers. Cronbach's alpha of paper I was 0.584 and that of paper II was 0.696. Point-biserial values were better for paper II than for paper I. Most students expressed dissatisfaction with paper II. CONCLUSIONS: One-Best multiple-choice questions showed better scores, higher reliability, better item performance and correlation values.

Simulation based medical education; teaching normal delivery on intermediate fidelity simulator to medical students.

OBJECTIVE: To assess the effectiveness of medium fidelity simulator in teaching normal vaginal delivery to medical students. METHODS: The quasi-experimental study was conducted at the professional development centre of the Jinnah Sindh Medical University, Karachi, from June to December 2015, and comprised medical students. Third-year medical students were included. They were divided into two groups. Group A was taught normal delivery through traditional PowerPoint and group B through simulator. The instruments used for assessing knowledge were pre-test and post-test, for skills of labour/delivery checklist of performance was used, and perception forms were filled to evaluate workshops/learning environment by students. RESULTS: Of the 76 participants, there were 36(47.4%) in group A and 40(52.6%) in group B. The overall mean age of the participants was 20.86±0.76 years in group B and 20.60±0.95 years in group A (p=0.19). The mean grade point average of the participants was 2.89±0.47 in group A and 2.87±0.48 in group B (p=0.81).Group B performed much better in skill of delivery having a mean score of 8.91±3.20compared to group A which had mean of 5.67±1.84 (p<0.01). CONCLUSIONS: Simulation-based skill learning showed significantly better results.

Formative assessment in undergraduate medical education: concept, implementation and hurdles.

Formative assessment, described as "the process of appraising, judging or evaluating students' work or performance and using this to shape and improve students' competence", is generally missing from medical schools of Pakistan. Progressive institutions conduct "formative assessment" as a fleeting part of the curriculum by using various methods that may or may not include feedback to learners. The most important factor in the success of formative assessment is the quality of feedback, shown to have the maximum impact on student accomplishment. Inclusion of formative assessment into the curriculum and its implementation will require the following: Enabling Environment, Faculty and student Training, Role of Department of Medical Education (DME). Many issues can be predicted that may jeopardize the effectiveness of formative assessment including faculty resistance, lack of motivation from students and faculty and paucity of commitment from the top administration. For improvement in medical education in Pakistan, we need to develop a system considered worthy by national and international standards. This paper will give an overview of formative assessment, its implications and recommendations for implementation in medical institutes of Pakistan.

'E-learning' modalities in the current era of Medical Education in Pakistan.

There are a number of e-Learning modalities, some or all of which may be used throughout a medical, dental, nursing or any other health related undergraduate curriculum. The purpose of this paper is to briefly describe what e-learning is along with some of the modalities, their common advantages and limitations. This publication ends with practical implications of these modalities for Pakistan.

Effect of Team-based Learning on Second Year Students' Academic Performance.

OBJECTIVE: To determine if there was any significant difference in scores of students who acquired knowledge through team-based learning (TBL) versus those who learned through small group discussion (SGD). STUDY DESIGN: Quasi-experimental cross-over study. PLACE AND DURATION OF STUDY: Shifa College of Medicine, Islamabad, from March to December 2015. METHODOLOGY: All students of second year MBBS were assigned to either TBL or SGD, based on non-probability sampling, for 8 themes in a five-week module. The groups were flipped after 4 themes. TBL format recommended by the AMEE guide No: 65 was followed. The impact of both learning methods was assessed by comparing end-of-module written examination scores of the two groups through independent sample t-test. SPSS version 23 was used for data analysis. A p-value of <0.05 was considered statistically significant. RESULTS: End-of-module written examination scores showed there was no significant difference in student academic performance when they studied with TBL or SGD in themes 1-4 (TBL 61.90 ±13.01 versus SGD 59.65 ±11.56, p=0.41). However, for themes 5-8 there was a significant difference in scores (TBL 65.36 ±11.62 versus SGD 59.92 ±10.65, p=0.03) A trend towards higher scores was observed in content learned with TBL in all eight themes. CONCLUSION: TBL is equally effective as an instructional method as SGD. However, it could not be established that it results in significant improvement in learning.

The need for national medical licensing examination in Saudi Arabia.

BACKGROUND: Medical education in Saudi Arabia is facing multiple challenges, including the rapid increase in the number of medical schools over a short period of time, the influx of foreign medical graduates to work in Saudi Arabia, the award of scholarships to hundreds of students to study medicine in various countries, and the absence of published national guidelines for minimal acceptable competencies of a medical graduate. DISCUSSION: We are arguing for the need for a Saudi national medical licensing examination that consists of two parts: Part I (Written) which tests the basic science and clinical knowledge and Part II (Objective Structured Clinical Examination) which tests the clinical skills and attitudes. We propose this examination to be mandated as a licensure requirement for practicing medicine in Saudi Arabia. CONCLUSION: The driving and hindering forces as well as the strengths and weaknesses of implementing the licensing examination are discussed in details in this debate.