Accreditation of medical schools in Saudi Arabia: A qualitative study.

CONTEXT: The accreditation of undergraduate medical education is a universal undertaking. Despite the widespread adoption of accreditation processes and an increasing focus on accreditation as a mechanism to ensure minimum standards are met in various fields, there is little evidence to support the effectiveness of accreditation. Traditionally, accreditation has worked toward achieving two ends: assuring and improving quality. Many recent articles emphasize the need for continuous quality improvement mechanisms to work, as well as the quality assurance role of accreditation. METHODS: The aim of the study was to examine the purposes and outcomes of accreditation, and stakeholders' experience of accreditation in Saudi Arabia. Triangulation of data was achieved through literature review, analysis of accreditation documents, examined the outcome of accreditation process (pre and post) through stakeholders' experience of accreditation (learner, teacher, and academic leader perspectives). Data were interrogated using thematic analysis approach involving identifying, analyzing, and reporting repeated patterns (themes) of meaning within data. RESULTS: Three themes emerged from the three phase study: "Passing the exam" versus long-term benefit, generic versus specialized accreditation standards, and internal quality assurance and self-evaluation. The data revealed a number of strategies that stakeholders can employ to achieve a balance between an "accreditation threat" and a quality improvement approach that is likely to have a lasting effect on educational outcomes. DISCUSSION: This empirical study revealed strong parallels between assessment and accreditation purpose, engagement, and outcomes. Like an increasing number of commentaries in the literature, this study suggests that accreditation bodies would do well to shift toward a holistic approach to quality management in medical education; implementation of quality improvement by an external "other"-described by some participants as the "policeman approach"-is not ideal for promoting sustainable quality education. Sustainable accreditation for long-term education improvement is not presented as a method, but as a way of thinking about important, and often overlooked, aspects of accreditation practice. Sustainable accreditation means that there is a need to meet both the immediate accreditation standards ("the exam") as well as establishing a basis for continuing quality improvement.

Assessing the technical skills of surgical trainees.

BACKGROUND: The aims were to determine whether tests of technical skill on simple simulations can predict competence in the operating theatre and whether objective assessment in the operating theatre by direct observation and video recording is feasible and reliable. METHODS: Thirty-three general surgical trainees undertook five simple skill simulations (knotting, skin incision and suturing, tissue dissection, vessel ligation and small bowel anastomosis). The operative competence of each trainee was then assessed during two or three saphenofemoral disconnections (SFDs) by a single surgeon. Video recordings of the operations were also assessed by two surgeons. RESULTS: The inter-rater reliability between direct observation and blinded videotape assessment was high (alpha = 0.96 (95 per cent confidence interval 0.92 to 0.98)). Backward stepwise regression analysis revealed that the best predictors of operative competence were the number of SFDs performed previously plus the simulation scores for dissection and ligation, the key components of SFD (64 per cent of variance explained; P = 0.001). CONCLUSION: Deconstruction of operations into their component parts enables trainees to practise on simple simulations representing each component, and be assessed as competent, before undertaking the actual operation. Assessment of surgical competence by direct observation and video recording is feasible and reliable; such assessments could be used for both formative and summative assessment.

The relationship between competence and performance: implications for assessing practice performance.

OBJECTIVE: This paper aims to describe current views of the relationship between competence and performance and to delineate some of the implications of the distinctions between the two areas for the purpose of assessing doctors in practice. METHODS: During a 2-day closed session, the authors, using their wide experiences in this domain, defined the problem and the context, discussed the content and set up a new model. This was developed further by e-mail correspondence over a 6-month period. RESULTS: Competency-based assessments were defined as measures of what doctors do in testing situations, while performance-based assessments were defined as measures of what doctors do in practice. The distinction between competency-based and performance-based methods leads to a three-stage model for assessing doctors in practice. The first component of the model proposed is a screening test that would identify doctors at risk. Practitioners who 'pass' the screen would move on to a continuous quality improvement process aimed at raising the general level of performance. Practitioners deemed to be at risk would undergo a more detailed assessment process focused on rigorous testing, with poor performers targeted for remediation or removal from practice. CONCLUSION: We propose a new model, designated the Cambridge Model, which extends and refines Miller's pyramid. It inverts his pyramid, focuses exclusively on the top two tiers, and identifies performance as a product of competence, the influences of the individual (e.g. health, relationships), and the influences of the system (e.g. facilities, practice time). The model provides a basis for understanding and designing assessments of practice performance.

Is insight important? measuring capacity to change performance.

BACKGROUND: Some doctors who perform poorly appear not to be aware of how their performance compares with accepted practice. The way that professionals maintain their existing expertise and acquire new knowledge and skills - that is, maintain their 'currency' of practice - requires a capacity to change. This capacity to change probably requires the individual doctor to possess insight into his or her performance as well as motivation to change. There may be a range of levels of insight in different individuals. At some point this reaches a level which is inadequate for effective self-regulation. Insight and performance may be critically related and there are instances where increasing insight in the presence of decreasing performance can also cause difficulties. OBJECTIVE: This paper presents an exploration into the nature of insight, its relationship to professional performance and its measurement as part of performance, reflecting the combined experiences of a group of experienced education researchers and the results of literature searches on insight and performance. CONCLUSION: There may be individuals in whom insight is so lacking that they are beyond remediation. If there is a dichotomy between adequate and inadequate levels of insight, testing this could be a cost-effective way of determining where efforts for remediation should be focussed.

Teaching in the new NHS: the specialised ward based teacher.

OBJECTIVES: To evaluate the effects of introducing specialised ward based teachers (WBTs) who had a broad remit to improve third year medical undergraduates' clinical experience. DESIGN: Quantitative and qualitative methods including interviews with WBTs, participating consultants and SIFT co-ordinator; student questionnaire and evaluations; analysis of Objective Structured Clinical Examination (OSCE) scores to ascertain if exposure to WBTs affected OSCE scores. SETTING: Two university teaching hospitals. PARTICIPANTS: Third year undergraduate medical students from one school of medicine; four WBTs; 25 consultants; SIFT co-ordinator. MAIN OUTCOME MEASURES: Student evaluations. Student questionnaires. Student OSCE scores. Interview data. RESULTS: WBTs had a demonstrable effect on student performance in OSCE examinations. 94% of students either agreed or strongly agreed that WBTs had helped them develop their examination skills and 87% either agreed or strongly agreed that WBTs had helped them develop their history taking skills. Interview data indicated that the consultants and SIFT co-ordinator considered that WBTs made an important contribution to clinical teaching. CONCLUSIONS: This study suggests that specialised WBTs are one way to manage clinical experience and enhance learning of undergraduate medical students. As clinical teaching moves into earlier parts of the undergraduate curriculum and into the community there is potential for this role to be developed.

The reproducibility of assessing radiological reporting: studies from the development of the General Medical Council's Performance Procedures.

OBJECTIVES: To investigate the reproducibility of peer ratings of consultant radiologists' reports, as part of the new General Medical Council (GMC) Performance Procedures. DESIGN: An evaluation protocol was piloted, used in a blocked, balanced, randomized generalizability analysis with three blocks of three judges (raters), each rating 30 reports from 10 radiologists, and re-rated to estimate intrarater reliability with conventional statistics (kappa). SETTING: Rating was performed at the Royal College of Radiologists. Volunteers were sampled from 23 departments of radiology in university teaching and district general hospitals. PARTICIPANTS: A nationally drawn non-random sample of 30 consultant radiologists contributing a total of 900 reports. Three trained and six non-trained judges were used in the rating analysis. RESULTS: A protocol was generated that was usable by judges. Generalizable results would be obtained with not less than three judges all rating the same 60 reports from a radiologist. CONCLUSIONS: Any assessment of performance of technical abilities in this field will need to use multiple assessors, basing judgements on an adequate sample of reports.

Effective supervision in clinical practice settings: a literature review.

CONTEXT: Clinical supervision has a vital role in postgraduate and, to some extent, undergraduate medical education. However it is probably the least investigated, discussed and developed aspect of clinical education. This large-scale, interdisciplinary review of literature addressing supervision is the first from a medical education perspective. PURPOSE: To review the literature on effective supervision in practice settings in order to identify what is known about effective supervision. CONTENT: The empirical basis of the literature is discussed and the literature reviewed to identify understandings and definitions of supervision and its purpose; theoretical models of supervision; availability, structure and content of supervision; effective supervision; skills and qualities of effective supervisors; and supervisor training and its effectiveness. CONCLUSIONS: The evidence only partially answers our original questions and suggests others. The supervision relationship is probably the single most important factor for the effectiveness of supervision, more important than the supervisory methods used. Feedback is essential and must be clear. It is important that the trainee has some control over and input into the supervisory process. Finding sufficient time for supervision can be a problem. Trainee behaviours and attitudes towards supervision require more investigation; some behaviours are detrimental both to patient care and learning. Current supervisory practice in medicine has very little empirical or theoretical basis. This review demonstrates the need for more structured and methodologically sound programmes of research into supervision in practice settings so that detailed models of effective supervision can be developed and thereby inform practice.

Relationships between students' clinical experiences in introductory clinical courses and their performances on an objective structured clinical examination (OSCE).

PURPOSE: To elucidate the link between the quantity and quality of clinical exposure gained by first-year clinical students in hospital settings and their performance on a subsequent comprehensive assessment of clinical skills (the objective structured clinical examination, or OSCE). METHOD: Data relating to educational activities and workload were collected for the second introductory clinical attachment undertaken by 152 (of 246) students in two British medical colleges prior to a joint comprehensive 22-station OSCE administered in May 1994. Pearson correlation coefficients were used as the main analytical tool to study the relationships between measures of clinical activity and total OSCE scores. RESULTS: In general, of 43 indices of the amount, nature, and quality of bedside, ward-based, or outpatient experience, only six correlated with OSCE scores. The strongest links were for whether students examined out-patients on their own (r = .2), whether the objectives had been made clear (r = .19) and the number of clinics attended (r = .18). Variables meeting the criteria were entered into a backwards stepwise regression analysis to predict total OSCE scores, but they explained only 23% of the variance. CONCLUSION: The association between clinical experience and educational outcomes remains poorly understood.

Rheumatology and medical education in Great Britain.

Rheumatology has been relatively under-represented in UK medical school curricula to date. The incidence of rheumatic disease in the community is not reflected by the amount of time spent on it in undergraduate medicine. In addition, the emphasis in medical colleges is on the less common conditions like systemic lupus erythematosus and vasculitis, rather than the commoner treatment of sore shoulders and backs. This article reviews the current changes in the philosophy of medical education in the UK and the response of the General Medical Council of Great Britain towards updating curricula. It explains some of the new teaching and assessment methods being increasingly used in today's medical colleges, and encourages rheumatologists to become actively involved in teaching and curricular reform.

CPR and the RCP (2). Training of students and doctors in UK medical schools.

We asked British medical schools and teaching hospitals about the training they offer to medical students and hospital doctors in cardiopulmonary resuscitation. The response rate was 96%. Training that is practical and consistent with guidelines is offered to nearly all students and house officers, often by consultants. Training for other junior doctors and consultants is much less common. The organisation of training is haphazard, and many hospitals have no resuscitation training officers. As a result, few doctors receive the frequent retraining needed to maintain competence in managing cardiopulmonary arrest.

The effect of formal instruction in ophthalmoscopy on medical student performance.

Ophthalmoscopy is an essential part of a complete clinical examination of a patient. However specific formal instruction in fundoscopy is rarely given to medical students. We decided to determine the value of explicit teaching of ophthalmoscopy and devised and validated a rating scale for assessing performance which was used to evaluate 29 first year clinical medical student volunteers at St. Bartholomew's Hospital Medical College, before and after formal instruction in fundoscopy. The competence of this group at ophthalmoscopy was then compared to the rest of their year (109 medical students) during the objective structured clinical end-of-year examination. Students formally instructed in ophthalmoscopy showed an improved score after instruction (from 53% to 77%). They also performed better at fundoscopy than the rest of their year (mean average score 64%), when tested 2 months later, suggesting persistence of the training effect. However as a group they performed no better than their peers at clinical skills other than ophthalmoscopy. We suggest that as fundoscopy is such an important clinical skill, medical students should be given specific teaching, preferably in their ophthalmology firm attachment, as we have shown that it results in a persistent improvement in performance.

Community pharmacy: a method of assessing quality of care.

Generalisability theory is presented as a research method for assessing the quality of health advice. The theory is applied to make a comprehensive assessment of primary health care advice given by community pharmacists. A random sample of pharmacies from all London postal districts were selected and visited for set periods throughout one year. During these visits all consultations on health or the use of drugs between pharmacists and clients were tape-recorded. Nine characteristics were derived by an external criterion panel on which the quality of advice in a random sample of consultations (50) was assessed using rating scales. Following the application of generalisability theory, operational conditions for the assessment were selected to obtain a generalisability coefficient of around 0.8. Nearly half the consultations achieved satisfactory scores on at least three-quarters of the criteria on which they were assessed. However almost a third were considered satisfactory on less than a quarter of the criteria, most of these being satisfactory on none of them. Performance regarding disease prevention/health promotion activity was poorest. Thus the quality of most consultations was found to be either very good or very poor, relatively few occupying the middle ground.

Education for practice: the role of practical experience in undergraduate and general clinical training.

A questionnaire and interview study in two London medical schools reviewed the undergraduate curricula and house year from the perspectives of graduates one year after qualification. Data from 113 house officers (44%), obtained by questionnaire and interview, showed general satisfaction with training. However, they also saw undergraduate preparation as deficient in exposure to some practical procedures and common conditions, and several difficult communication skills were not taught. Graduates reported that they would have liked more experience as undergraduates in resuscitation, lumbar puncture, inserting nasogastric tubes, endotracheal intubation, and in managing diabetic keto-acidosis, asthmatic attacks, myocardial infarcts and respiratory failure. The purposes of the house year and the articulation between it and the undergraduate years require clarification.

Towards the identification of student learning problems: the development of a diagnostic inventory.

An understanding of how students approach their learning has important implications for medical education. Of particular interest is the fact that the approach students use in their study has a significant impact on both the quality of the learning and their academic success. It would clearly be of value to identify students whose approach to learning was predictive of unsatisfactory performance. This paper describes the initial development of two versions of an inventory (questionnaire) which was designed to aid in the diagnosis of student learning problems. Preliminary information is given on the reliability and validity of these instruments. Factor analyses support the underlying design. Correlation of subscales with academic performance has provided encouraging evidence of the potential of these inventories in identifying students with specific learning or study problems.

Graduates of foreign medical schools: demographic and personal predictors of success on an OSCE-format internship programme entrance examination.

A Discriminant Function Analysis on 233 graduates of foreign medical schools, from whom a group of 24 were selected that although several variables correlated significantly with selection, no combination of demographic variables could act as an efficient screen of clinically deficient candidates.