BEHAVIOURAL SCIENCES AT UNIVERSITY OF HEALTH SCIENCES: THE WAY FORWARD.

BACKGROUND: The association of medical ethics with teaching and training and health profession has been informal, largely dependent on role modelling and the social contract of the physicians with the community that they abide by. This study was conducted to examine the effect, if any, of introducing the subject of Behavioural Sciences on students' performance in the clinical years' 'viva voce' and 'patient interactions' components of the examinations. METHODS: A prospective study on four cohorts of students at UHS from 2007 to 2012 (8,155 candidates). Reliability was calculated through Cronbach's Alpha. Linear Regression Analysis was applied to determine the relationship between the scores of Basic Medical Sciences, Behavioural Sciences and Forensic medicine with the viva voce and Structured Stations marks of the Clinical Sciences in OSCE. Gender and demographics analysis was also done. RESULTS: Cronbach's Alpha was 0.47, 0.63, 0.67 and 0.53 for the Papers of Behavioural Sciences from 2007 to 2010 respectively. Poor predictive value of Behavioural Sciences for performance in tlhe clinical years' viva voce and OSCE was identified. Basic Medical Sciences and Forensic Medicine were statistically significant predictors for the performance of female candidates in all four cohorts of the study (p < 0.05). In Central Punjab, Behavioural Sciences statistically significantly predicted for better performance in all four cohorts of the study (p < 0.05). CONCLUSION: It is premature to understand the results of Behavioural Sciences teaching at University of Health Sciences (UHS). We can still safely conclude that it can only have a positive sustained effect or the healthcare delivery systems and patient care in Pakistan if it is integrated within each subject and taught and learned not as a theoretical construct but rather an evaluation of one's values within the code of conduct of medical professionalism in the larger context of the societal and cultural norms.

Predictive validity of medical and dental colleges' entrance test in Punjab: the way forward.

OBJECTIVE: To identify the strengths and weaknesses of the current selection process for admissions to public medical colleges being followed in the province of Punjab and to suggest changes. METHODS: The prospective study comprised candidates sitting the Medical and Dental Colleges Entrance Tests administered by the University of Health Sciences, Lahore, from 2008 to 2011. The marks of pre- and post-admission examinations were entered in SPSS 17 and performance of candidates in different examinations up to 2012 was compared using parametric statistical tests. P ≤ 0.05 was considered statistically significant. Reliability of the entrance test question paper was calculated through Cronbach's Alpha. RESULTS: Overall 1,01,273 candidates sat in the entrance test during the study period, and of them 14, 995 (14.8%) students were admitted to public and private medical colleges in Punjab affiliated with the University of Health Sciences, Lahore. Cronbach's Alpha showed excellent internal consistency reliability of the question paper of 220 items for all these four years. Overall, students scored significantly lower marks (p < 0.05) in all their MBBS Professional Examinations than their pre-admission tests. A similar pattern of difference was seen in entrance test marks in different geographical localities as in the case with MBBS Professional Examination. Students from Islamabad and Rawalpindi regions scored significantly higher marks in all MBBS examinations as well as in the entrance test and aptitude test, but lower in the Higher Secondary School Certificate examination. Internal consistency reliability of the aptitude test was found to be 0.83. Linear regression analysis showed that Estimated Regression Coefficient of the Higher Secondary School Certificate marks was negative, but positive for entrance test and indicated that the latter results were a good positive predictor for the post-selection performance. In the sub-components, only the Biology and Chemistry regression coefficient section was found to be positive. CONCLUSIONS: Entrance and aptitude tests in Punjab were found to be valid and reliable with incremental predictive validity at least for performance in the pre-clinical years. It is suggested that Higher Secondary School Certificate-marks should have little, if any, weightage in the selection process.

The effect on student performance of scrambling questions and their stems in medical colleges admission tests.

Assessment is an indispensable part of an educational program. Multiple Choice Questions (MCQs) is an objective tool of assessment provided cheating is controlled. A method employed to reduce the chance of cheating is to scramble the sequence of the MCQs and responses in multiple papers having the same content. It is assumed that the performance of students is mainly dependent on the difficulty of the items and not the order in which they are placed within the instrument. The marks obtained by 1,02,211 candidates sitting in Medical Colleges Admission Test (MCAT) from 2008 to 2011 and given similar-content but scrambled-sequence question paper codes were analyzed using parametric tests. A significant difference amongst the mean marks of candidates in the different codes of MCAT 2008 (F = 22.15, p < 0.001) and MCAT 2011 (F = 3.85, p = 0.009) was identified. No significant difference was found in the mean marks of the candidates' each year for different codes in each centre.

Comparison of pre-medical academic achievement, entrance test and aptitude test scores in admission selection process.

OBJECTIVE: To compare the awards of various pre-medical academic achievements with aptitude test scores obtained by candidates taking the Entrance Test 2011, and to identify demographic differences in the trend, if any. METHODS: The cross-sectional study involved a 22-item aptitude test administered to all the students taking the Medical College Admission Test conducted by the University of Health Sciences, Lahore, on September 13, 2011. The Matriculation, Intermediate, EntranceTest and AptitudeTest scores of all the students along with their demographic variables were entered into SPSS 16 and statistically analysed by using parametric tests. A p value of < 0.05 was considered statistically significant. RESULTS: Out of the 32,746 students, 22,935 (70%) were females. Students who scored more than 75% marks (n = 4,723; 14%) also scored higher marks in the Aptitude Test (p < 0.05). Female candidates in general scored higher marks in the Entrance Test (p < 0.05) and in the Aptitude Test (p < 0.05). Overall, students from the economically and academically underdeveloped districts of Punjab scored less marks in the Entrance Test (p < 0.05) and the Aptitude Test (p < 0.05).The difference in Entrance Test and Aptitude Test marks of students from underdeveloped districts was greater than that of students for the developed districts (p < 0.05). However, in Matriculation and Intermediate examinations, students from the underdeveloped districts scored higher marks than the developed districts (p < 0.05). Students scoring higher marks in Matric and Intermediate, scored low marks in the Aptitude Test (p < 0.05). CONCLUSION: The Entrance Test and Aptitude Test scores correlated poorly with Intermediate and Matriculation scores. There is a need to strengthen the underdeveloped areas academically. Besides, the predictive value of the Aptitude Test scores related to future performance of candidates selected needs to be ascertained with further studies.

Medical colleges admission test in Punjab, Pakistan.

BACKGROUND: Nearly 18,000 candidates securing 60% and above marks in Higher Secondary School Certificate (HSSC) examination contest for admission in Medical Colleges, in Punjab, Pakistan by sitting in the Medical College Admission Test (MCAT) each summer. This cross-sectional study was conducted to identify patterns related to demographic, economic and educational backgrounds, over a two-year-period, in this population, and how HSSC and MCAT marks predict future performance of the selected candidates. METHODS: Marks obtained by candidates in HSSC, MCAT, and 1st Professional MBBS (Part-I) Examinations over two years 2008-2009, were analysed using parametric tests in SPSS. RESULTS: Total 18,090 candidates in 2008 and 18,486 in 2009 sat in the MCAT. National IHSSC candidates scored higher marks in HSSC and MCAT but lower marks than their foreign qualified HSSC counterparts (e.g., Advanced-levels from Cambridge University, UK) in Part-I overall and in all its subcomponents individually (p < 0.05). Female students scored higher marks than males in HSSC (p > 0.05). MCAT (p > 0.05) and Part-I theory, practical, viva voce, continuous assessment and Objective-Structured Performance Evaluation (OSPE) components (p < 0.05). In both years, students from the Dera Ghazi Khan District scored the highest marks in the HSSC Examinations (p < 0.05) but least marks in MCAT in 2008 (p < 0.05) and in Part-I in 2008 and 2009 (p < 0.05). Students from 'tougher' Boards like Rawalpindi in 2008 and the Federal Board in 2009 who scored least marks in HSSC scored highest marks in MCAT. and in Part-I Examinations (p < 0.05). Linear regression on Part-I by taking HSSC and MCAT marks as independent variables showed that the MCAT marks exerted the greatest positive influence consistently at 0.104 (2008) and 0.106 (2009). In 2009 HSSC marks were shown to exert a negative influence (-0.08) on Part-I. CONCLUSION: There is need to standardise HSSC education and examination across all Intermediate Boards. MCAT is a better predictor of Medical Students' future performance.

Medical professionalism: a panoramic view through the kaleidoscope of stakeholder perspectives.

BACKGROUND: Recently, over the last couple of decades, great emphasis is being placed on the role of professionalism in medical education. This interest has intensified following the positive relationship identified between unprofessional behaviour in medical schools and subsequent practice. This paper aims to develop an understanding of the perceptions of various stakeholders regarding the subject in the local setting and tries to relate it to the global context. METHODS: This was a qualitative study conducted in 2011 involving the faculty and students of the University of Health Sciences Lahore. An open-ended questionnaire was fashioned following brainstorming and utilising Delphi technique involving representatives of the students, faculty and the public. Responses from all the respondents were entered into Microsoft Excel data sheet and then imported into Qualitative Data Analysis Software 'NVIVO 9'. Themes were extracted from the responses. RESULTS: Overall 650 questionnaires were distributed amongst the faculty, students and general public. Response rates were 74%, 68% and 59% respectively. Commonalities and differences in the perceptions of the various stakeholders of medical profession including the medical practitioner, public and the medical students were identified. CONCLUSION: The product of the healthcare professional education system needs to conform to the global standards applied within local settings. It is the identification of the local setting that is critical to devising a cost-effective and efficient curriculum, which amongst others includes teaching/training/learning and practice of professionalism.

DREEM on: validation of the Dundee Ready Education Environment Measure in Pakistan.

OBJECTIVES: To validate DREEM in medical education environment of Punjab, Pakistan. METHODS: The DREEM questionnaire was anonymously collected from Final year Baccalaureate of Medicine; Baccalaureate of Surgery students in the private and public medical colleges affiliated with the University of Health Sciences, Lahore. Data was analyzed using Principal Component Analysis with Varimax Rotation. RESULTS: The response rate was 84.14 %. The average DREEM score was 125. Confirmatory and Exploratory Factor Analysis was applied under the conditions of eigenvalues >1 and loadings > or = 0.3. In CONFIRMATORY FACTOR ANALYSIS, Five components were extracted accounting for 40.10% of variance and in EXPLORATORY FACTOR ANALYSIS, Ten components were extracted accounting for 52.33% of variance. Total 50 items had internal consistency reliability of 0.91 (Cronbach's Alpha). The value of Spearman-Brown was 0.868 showing the reliability of the analysis. In both analyses the subscales produced were sensible but the mismatch from the original was largely due to the English-Pakistan contextual and cultural differences. CONCLUSION: DREEM is a generic instrument that will do well with regional modifications to suit individual, contextual and cultural settings.

Developing the outcomes of a baccalaureate of dental surgery programme.

BACKGROUND: Curriculum broadly falls into two categories, prescriptive and outcome-based. In the prescriptive curricula emphasis is placed on teaching with generally little integration between subjects and across disciplines. Currently, universities in Pakistan are undergoing a cultural change in the curricular design in order to apply an outcome-based learning instead of prescriptive teaching. Regionally, the need for change was recognised on account of the vast body of evidence available internationally. In order to bring about a shift towards an outcome based curriculum in the 4-year BDS programme, we first need to specify the outcomes/traits that the dental health professionals should be able to demonstrate upon leaving the programme. This paper describes the process and outcome of arriving at the desired consensus through a series of workshops involving all stakeholders including students, community members, teaching faculty, programme directors and representatives of the dental health industry. METHODS: A series of workshops were conducted between September 2009 to February 2010 in all of the 18 disciplines of basic and dental sciences individually and then collectively. A questionnaire sought responses from the participants regarding their perception about the status of the current BDS curriculum and their understanding of an outcome-based integrated curriculum, as well as whether such an integrated curriculum should be adopted or not? In the second half of the workshop through brainstorming and Delphi technique, the outcomes in terms of measurable traits that should be possessed by a graduate dental health professional entering community service were enlisted. The pre- and post-workshop questionnaire scores were entered into SPSS-16 and paired sample t-test as well as chi-square test were applied. Cron back alpha value of < 0.05 was taken as statistically significant. Secondly, the outcomes developed in each workshop were entered into Ethnograph and common outcomes of the 4-year BDS programme were extracted. RESULTS: In total, 234 participants attended the workshop over a period of five months in 18 discipline-wise workshops and four integrated workshops involving faculty members of all disciplines. Results indicate clearly that the workshop resulted in an attitude shift of the participants and their perception of the current curriculum and the need and rationale for a move towards an outcome-based curriculum. The 30 outcomes identified were grouped under two categories namely 'clinical skills' and 'professional behaviours'. CONCLUSION: Defining the final programme outcomes is only the initial step in developing an outcome-based, objective, integrated curriculum which will require considerable work in the future.

Relationship of awards in multiple choice questions and structured answer questions in the undergraduate years and their effectiveness in evaluation.

BACKGROUND: A number of evaluation tools for assessing the cognitive and affective domains in accordance with Bloom's taxonomy are available for summative assessment. At the University of Health Sciences, Lahore, Multiple Choice Questions (MCQs) and Structured Answer Questions (SAQs) are used for the evaluation of the cognitive domain at all six hierarch levels of taxonomy using the tables of specifications to ensure content validity. The rationale of having two evaluation tools seemingly similar in their evaluative competency yet differing in feasibility of construction, administration and marking is being challenged in this study. METHODS: The MCQ and SAQ awards of the ten percent sample population amounting to 985 students in fifteen Medical and Dental Colleges across Punjab were entered into SPSS-15 and correlated according to the cognitive and affective level of assessment in relation to the Bloom's taxonomy and their grouping in the Tables of Specifications, using parametric tests. 3494 anonymously administered questionnaires were analyzed using ethnograph. RESULTS: No statistically significant difference was found in the mean marks obtained by the students when MCQs and SAQs were compared according to their groupings in the Tables of Specifications at all levels of cognitive hierarchical testing. End-of-yearcognitive level testing targets set were not met and more questions were set at the lower cognitive testing levels. Expenses incurred in setting MCQs and SAQs were comparable but conduct and assessment costs for MCQs and SAQs were 6% and 94% of the total respectively. In both MCQs and SAQs students performed better at higher cognitive testing levels whereas the SAQs and MCQs were able to marginally test the lower levels of affective domain only. Student's feedback showed that attempting MCQs required critical thinking, experience and practice. CONCLUSION: MCQs are more cost effective means at levels of cognitive domain assessment.

Determination of medical education environment in Punjab private and public medical colleges affiliated with University of Health Sciences, Lahore-Pakistan.

BACKGROUND: The main purpose of this study was to identify differences, if any, in the Medical Education Climate between the Private and Public Medical Colleges in the Province of Punjab affiliated with the University of Health Sciences, Lahore and to gather recommendations from students on measures that need to be taken to improve the environment. METHODS: This Mixed Quantitative and Qualitative Prospective Study was conducted in 2008. The population of the study consisted of 1612 MBBS Final Year Medical Students of both Private and Public Medical Colleges. Stratified Random Sampling was done to ensure representation of both Sectors. Dundee Ready Education Environment Measure (DREEM) was used to assimilate Quantitative Data and a Questionnaire consisting of 10 items was used to accumulate Qualitative Data. To analyse Quantitative Data, t-test and Chi-square tests were used. Common themes were identified in the Qualitative Data. RESULTS: All the SIX Research Hypotheses were rejected and Null Hypotheses were upheld. Analysis of Qualitative Data indicated a number of Examination, Curriculum, Teaching Methodology, Teacher and Peer related Stressors without discrimination in students of both Private and Public Sectors. Solutions by students focused on improving co-ordination between Institutions and University of Health Sciences as well as developing and delivering Clinically-Centred, Community-Oriented and Problem-Based Education through development of appropriate Teaching Methodologies. CONCLUSIONS: Even though there is no difference in the Medical Education Climate between the Private and Public Medical Education Sectors, the Environment is less than Ideal. However, this can be improved through shifting the onus of Education from Teacher-Centred Didactic Approach to a more Student-Centred Self-Learning Strategy. In this paradigm shift the UHS, Lahore needs to play a pivotal role in order to effectively train the Trainers and standardise this change throughout Punjab.

Knowledge rather than language proficiency affects achievement scores in interactive objectively structured performance evaluation.

BACKGROUND: A common belief is that language proficiency might have affected the outcome scores of the Oral Structured Performance Evaluation (OSPE) especially at the interactive stations. The objective of this study was to explore this postulation. METHODS: The subject of Behavioural Sciences was examined for the first time as a component of undergraduate medical curriculum in 2nd Professional Medical Examinations in 2007. Equal weightage was given to written and OSPE components in the examination. The OSPE scores in the interactive component of all (1457) candidates were compared with their written scores in the subject of Behavioural Sciences as well as their Higher Secondary School Certificate (HSSC) scores in the subject of English using Multiple Linear Regression Analysis in SPSS v.12. RESULTS: Overall as well as in the Public Medical Colleges, relative to each other, knowledge of the subject as indicated by marks obtained in the written component of the examination exerted a positive and statistically significant (p < 0.05), and command over language as indicated by marks obtained in the HSSC examination exerted a small positive but statistically insignificant (p = 0.231 and 0.639 respectively) influence on the performance of students in the interactive OSPE component. In the private medical colleges command over language exerted a small negative but statistically insignificant (p = 0.936) influence on the performance of students in the interactive OSPE component of the examination. CONCLUSION: Command over the subject content is the best indicator of achievement in OSPE.