Medical students' learning processes the first time they role-play in psychiatry: A grounded theory study.

BACKGROUND: Recent changes in psychiatric care and teaching that limit patient contact for medical students can be overcome in part by simulation-based education. Understanding the learning processes of medical students involved in psychiatric simulation-based programmes could usefully inform efforts to improve this teaching. This study explored the learning processes of medical students the first time they role-play in psychiatry. METHODS: We used constructivist grounded theory to analyse semi-structured interviews of 13 purposively sampled medical students and the six psychiatrists who trained them. To improve the triangulation process, the results of this analysis were compared with those of the analyses of the role-play video and the debriefing audio-tapes. RESULTS: Five organising themes emerged: improving the students' immediate perception of patients with mental disorders; cultivating clinical reasoning; managing affect; enhancing skills and attitudes and fostering involvement in learning psychiatry. CONCLUSION: Results suggest that psychiatric role-playing can improve students' progressive understanding of psychiatry through the development of intuition and by allaying affects. Emotional elaboration and student involvement appear to be key features.

A review of educational strategies to improve nurses' roles in recognizing and responding to deteriorating patients.

BACKGROUND: Physiological instability leading to clinical deterioration often precedes cardiopulmonary arrest. Nurses, who have the most frequent patient contact and responsibility for ongoing monitoring of patients, play a crucial role in recognizing and responding to clinical deterioration. The importance of education in supporting such a role has been addressed in numerous studies. AIM: This study aimed to identify nurses' educational needs and explore educational strategies to enhance their ability in recognizing and managing wards with deteriorating patients. METHODS: A literature search from databases (2000-2010) was undertaken to include papers that identified the educational needs of ward nurses and existing educational programmes related to the care of deteriorating patients. FINDINGS: Twenty-six papers were included in this review. Findings identified the educational need to empower nurses with the appropriate knowledge and skills in recognizing, reporting and responding to patient deterioration. The review of existing educational programmes and their outcomes identified valuable teaching information and strategies, and areas that could be improved in meeting nurses' educational needs. CONCLUSION: The review has highlighted important aspects of patient safety in clinical deterioration that could be further addressed by educational strategies targeting the role of ward nurses. These strategies include: utilizing clinical decision-making models to develop nurses' decision making skills; developing a standardized tool for systematic nursing assessment and management of clinical deterioration; incorporating training in clinical deterioration as a core competence of pre-registered nursing education; providing vital signs training to nursing assistants; and conducting more rigorous studies to evaluate the effectiveness of the educational programmes.

The role of a computerised case-based testing procedure in practice performance assessment.

INTRODUCTION: For postgraduate training of doctors there is a need for valid and reliable instruments to assess their daily performance. Various instruments have been suggested, some of which use incognito simulated patients (SPs). These methods are resource intensive. Computerised Case-based testing (CCT) is logistically simpler and may still predict performance well. The research question was to evaluate the predictive validity of CCT for performance. METHODS: Seventeen rheumatologists were each visited by eight incognito SPs presenting various rheumatological complaints, and scoring the performance of the rheumatologists using a predefined checklist. From this checklist a panel of experts identified essential items. In addition the rheumatologists sat a CCT test containing 55 cases with a total of 121 items. RESULTS: Negative correlations were found between the SP scores and the CCT scores. This was unexpected. Therefore, background variables on experience were used to compare both methods. The correlation between these and CCT were high and positive and with the SP scores high and negative. This pattern did not differ when using the essential items of the checklist. Reliabilities of the SP scores were markedly high. DISCUSSION: Although CCT was not predictive of SP scores, it was related to working experience. There are good reasons to assume that although SP-scores were more authentic, they were less valid than CCT scores, mainly because they focussed more on thoroughness than on efficiency in data gathering. The results underpin the assumption that for valid performance assessment the most important issue is what information about the candidate is collected and now how authentic the method is.

Setting priorities and identifying barriers for general practice research in Europe. Results from an EGPRW meeting.

BACKGROUND: In spring 2002, WONCA Europe, the European Society of General Practice/Family Medicine and its Network organizations reached consensus on a 'new' European definition of general practice. Subsequently, the European General Practice Research Workshop (EGPRW) started working on a European General Practice Research Agenda. This topic was addressed during the 2002 EGPRW autumn meeting. OBJECTIVE: Our aim was to explore the views of European general practice researchers on needs and priorities as well as barriers for general practice research in Europe. METHODS: In seven discussion groups, 43 general practice researchers from 18 European countries had to answer the following questions. (i) What major topics should be included in a research agenda for general practice in your country? (ii) What are the barriers to adequate implementation of general practice research in your country? Group answers were listed and subsequently categorized by two authors. RESULTS: Research on 'clinical issues' (common diseases, chronic diseases, etc.), including diagnostic strategies, was considered to be the core content of general practice research, with primary care-based morbidity registration essential for surveillance of disease, clinical research and teaching in general practice. There was also consensus on the need for research on education and teaching. 'Insufficient funding opportunities' was perceived to be the major barrier to the development of general practice research. CONCLUSIONS: These findings could be used as a basis for national checklists of 'content of' and 'conditions for' general practice research. European general practice research training programmes should be developed further.

The relationship between competence and performance: implications for assessing practice performance.

OBJECTIVE: This paper aims to describe current views of the relationship between competence and performance and to delineate some of the implications of the distinctions between the two areas for the purpose of assessing doctors in practice. METHODS: During a 2-day closed session, the authors, using their wide experiences in this domain, defined the problem and the context, discussed the content and set up a new model. This was developed further by e-mail correspondence over a 6-month period. RESULTS: Competency-based assessments were defined as measures of what doctors do in testing situations, while performance-based assessments were defined as measures of what doctors do in practice. The distinction between competency-based and performance-based methods leads to a three-stage model for assessing doctors in practice. The first component of the model proposed is a screening test that would identify doctors at risk. Practitioners who 'pass' the screen would move on to a continuous quality improvement process aimed at raising the general level of performance. Practitioners deemed to be at risk would undergo a more detailed assessment process focused on rigorous testing, with poor performers targeted for remediation or removal from practice. CONCLUSION: We propose a new model, designated the Cambridge Model, which extends and refines Miller's pyramid. It inverts his pyramid, focuses exclusively on the top two tiers, and identifies performance as a product of competence, the influences of the individual (e.g. health, relationships), and the influences of the system (e.g. facilities, practice time). The model provides a basis for understanding and designing assessments of practice performance.

Is insight important? measuring capacity to change performance.

BACKGROUND: Some doctors who perform poorly appear not to be aware of how their performance compares with accepted practice. The way that professionals maintain their existing expertise and acquire new knowledge and skills - that is, maintain their 'currency' of practice - requires a capacity to change. This capacity to change probably requires the individual doctor to possess insight into his or her performance as well as motivation to change. There may be a range of levels of insight in different individuals. At some point this reaches a level which is inadequate for effective self-regulation. Insight and performance may be critically related and there are instances where increasing insight in the presence of decreasing performance can also cause difficulties. OBJECTIVE: This paper presents an exploration into the nature of insight, its relationship to professional performance and its measurement as part of performance, reflecting the combined experiences of a group of experienced education researchers and the results of literature searches on insight and performance. CONCLUSION: There may be individuals in whom insight is so lacking that they are beyond remediation. If there is a dichotomy between adequate and inadequate levels of insight, testing this could be a cost-effective way of determining where efforts for remediation should be focussed.

Psoriatic arthritis: performance of rheumatologists in daily practice.

OBJECTIVES: To assess, using standardised patients (SPs), how rheumatologists diagnose psoriatic arthritis, whether the diagnostic efficiency is influenced by specific characteristics of the rheumatologists, and to study the relationship with costs. METHODS: Twenty three rheumatologists were each visited by one of two SPs (one male, one female) presenting as a patient with psoriatic arthritis. SPs remained incognito for all meetings for the duration of the study. Immediately after the encounter, SPs completed case-specific checklists on the medical content of the encounter. Information on ordered laboratory and imaging tests was obtained from each hospital. RESULTS: Fourteen rheumatologists diagnosed psoriatic arthritis correctly. They inspected the skin for psoriatic lesions more often than those rheumatologists who established other diagnoses. Rheumatologists diagnosing psoriatic arthritis spent more on additional laboratory and imaging investigations. These were carried out after the diagnosis to confirm it and to record the extent and severity of the disease. No differences in type of practice, number of outpatients seen each week, working experience, or sex were found between rheumatologists who made the correct diagnosis and those who made other diagnoses. The correct diagnosis was more often missed by rheumatologists who saw the male SP, who presented with clear distal interphalangeal DIP joint arthritis only, causing confusion with osteoarthritis of the DIP joints. CONCLUSION: There is a considerable amount of variation in the delivery of care among rheumatologists who see an SP with psoriatic arthritis. Rheumatologists focusing too much on the most prominent features (DIP joint arthritis) sometimes seem to forget "the hidden (skin) symptoms".

On the relationship between the efficiency and the quality of the consultation. A validity study.

OBJECTIVES: The aim of this study was to determine how the medical performance of physicians during consultations is related to doctor-patient communication and satisfaction of patients, taking into account the actual length of the consultations. In addition, we studied the validity of the 'efficiency-per-time score' as a measure of competence. METHODS: General practice trainees participated in a test situation in which they were confronted with six consultations with standardized (simulated) patients (SPs). All consultations were videotaped and evaluated by multiple observers, using national guidelines on medical content and on communication. The SPs scored satisfaction with the consultation using a satisfaction checklist. Forty GP-trainees were invited, of whom 34 participated. The main outcome measures were the number of obligatory actions undertaken by the GP-trainees, total number of actions undertaken, consultation time, efficiency-per-time score, patient satisfaction and quality of communication score, and the Pearson correlations between these measures. RESULTS: There was a negative correlation between the 'efficiency-per-time score' of the GP-trainees and the satisfaction of the SPs in five of the six consultations [Pearson r from -0.29 (P < 0.05) to -0.58 (P < 0.001)] and between the 'efficiency-per-time score' and the quality of the communication in three of the six consultations [Pearson r from -0.34 (P <.05) to -0.51 (P < 0.001)]. CONCLUSIONS: Short consultations with high technical medical efficiency seem to be related to bad communication and dissatisfied patients, thus questioning the validity of the 'efficiency-per-time score' as a measure of competence.

Rhemulatologists' performance in dailiy practice.

OBJECTIVE: To assess rheumatologists' performance for 8 rheumatologic conditions and to explore possible explanatory factors. METHODS: After written informed consent was obtained, 27 rheumatologists (21% of all Dutch rheumatologists) practicing in 16 outpatient departments were each visited by 8 incognito "standardized patients" (SPs). The diagnoses of these 8 cases account for about 23% of all new referred patients in the Netherlands. Results for ordered lab tests as well as real radiographs with corresponding results from a radiologist were simulated. Information from the visits was obtained from the SPs, who completed predefined case-specific checklists, and by collecting data on resource utilization. Feedback was provided. RESULTS: Altogether 254 encounters took place, of which 201 were first visits and 53 were followup visits. SPs were unmasked twice during a visit. There was considerable variation in resource utilization (lab tests and imaging) between cases and between rheumatologists. Mean costs per rheumatologist ranged from US $ 4.67 to $ 65.36 per visit for lab tests and from US $ 33.15 to $ 226.84 per visit for imaging tests. No significant correlations were seen between resource utilization costs and number of years of clinical experience or performance on checklist scores. Rheumatologists with longer experience had lower total item checklist scores (r = -0.47; P < 0.05). CONCLUSION: A considerable variation in resource utilization was found among 27 Dutch rheumatologists. The information obtained is an excellent source for discussion on the appropriateness of care.

Teaching and assessing communication skills in Maastricht: the first twenty years.

In order to share the issues involved in setting up a communication skills training programme in a medical school, the development of such a programme at the Skillslab of Maastricht Medical School, the Netherlands, is described and the problems encountered are addressed. A multidisciplinary working group developed teaching goals for communication skills, focused on observable behaviour to be displayed by students. These teaching goals were incorporated in a generic model for doctor-patient communication. A longitudinal training programme was created, throughout the four years of the preclinical curriculum. Students meet in small groups of 10, once every 2 weeks. In between group sessions they practise consultation skills in simulated patient contacts. Communication skills are assessed in the annual multiple station examination. In the development of this programme the following consecutive actions were taken: teaching material was produced, and an assessment tool was developed, as were instruments for programme evaluation. The programme evaluation allowed student feedback to teachers, the teachers' departments, and the administration of the medical school. Finally, teacher training was professionalized.

Developing case-specific checklists for standardized-patient-based assessments in internal medicine: A review of the literature.

PURPOSE: To review the literature on the methods used in writing case-specific checklists for studies of internal medicine physicians' performances that were assessed by standardized patients. METHOD: The authors searched Medline, Embase, Psychlit, and ERIC for articles in English published between 1966 and February 1998. The following search string was used: "[(standardi(*) or simulat(*) or programm(*)) near (patient(*) or client(*) or consultati(*))] and internal medicine." The authors then searched the reference lists of papers retrieved from the database searches, as well as those from seven proceedings of the International Ottawa Conference on Medical Education and Assessment. RESULTS: The procedure yielded 29 relevant articles: database searches yielded 14 published reports dealing with case-specific checklists, 11 articles were culled from the reference lists of these papers, and the Ottawa Conference proceedings yielded four articles. Only 12 articles reported specifically on the development of checklists. In general, there were three sources used for developing checklists: panels of experts, the investigators themselves, and responses from expert physicians to written protocols. No article indicated that researchers had relied exclusively on data from the literature to compose their checklists. Only three articles indicated that literature sources had informed their checklist development. All articles except one relied on explicit criteria for the inclusion of items on the checklists. In 21 of the 29 articles, the checklists had been scored by SPs, but the scoring of specific items on the checklists varied according to the purpose of the SP-physician encounter. Only four of the articles made the checklists available or indicated that the checklists could be obtained from the authors. CONCLUSION: The development of case-specific checklists for SP examinations of physicians' performance has received little attention. To judge the validity of studies of physicians' performances that use SPs, the development processes for the checklists need to be more fully described to enable readers to evaluate the validity and reliability of the studies.

Effect of a short skills training course on competence and performance in general practice.

OBJECTIVES: Evaluation of the efficacy of a short course of technical clinical skills to change performance in general practice. DESIGN: Subjects were self-selected general practitioners (n=59) who were unaware of the study design. They were assigned to the intervention group (n=31) or control group (n=28) according to their preference for course date. The course covered four different technical clinical skills (shoulder injection technique, PAP-smear, laboratory examination of vaginal discharge, ophthalmoscopic control in diabetes mellitus). Main outcome measures used were pre- and post-training scores on a knowledge test of skills (60 multiple choice items), and pre- and post-training performance of procedures in practice using a log-diary covering 20 days. SETTING: University of Maastricht, The Netherlands. SUBJECTS: Self-selected general practitioners. RESULTS: Competence, as measured by the knowledge test of skills, improved significantly as a result of the training and skills test scores were satisfactory after training. A significant effect on performance in practice was found for PAP-smear and shoulder injection technique, whereas no effect could be demonstrated for examination of vaginal discharge and ophthalmoscopic control in diabetes mellitus. CONCLUSIONS: A good degree of competence is a necessary but not always sufficient condition for a physician to change his performance in practice. While some skills training seems adequate to bring about desired changes, for other skills more complex interventions are probably needed.

The NECTAR-study: development of nutrition modules for general practice vocational training; determinants of nutrition guidance practices of GP-trainees. Nutrition Education by Computerized Training And Research.

OBJECTIVE: To identify determinants of nutrition guidance practices of general practitioner-trainees (GP-trainees), to investigate whether these determinants differ from those found by experienced general practitioners; to reveal educational directions towards the development of computer-based instruction on nutrition. DESIGN: Cross-sectional study by means of validated questionnaires. SUBJECTS: All GP-trainees in training at the eight university departments for vocational training in the Netherlands in September, 1998 (n = 985). MAIN OUTCOME MEASURES: Reliability of determinants of nutrition guidance practices was calculated by means of Crohnbach's alpha. The mechanism of action of determinants was identified by means of linear structural relationship analysis (LISREL) using a model developed for GPs. RESULTS: Crohnbach's alphas for factors ranged from 0.58-0.90. The empirical GP-trainee-data fitted with the corresponding GP-model on the mechanism of action. CONCLUSIONS: The same predisposing factors, driving forces and barriers as found with GPs were identified with GP-trainees. Comparing the GP-and GP-trainee-models, only minor differences were found in the path coefficients between factors. Lack of nutrition training and education proved to be of great influence on the extent of nutrition information given. The GP-trainee-model will be of use in developing computer-based instruction on nutrition. It is expected that GPs may also benefit from this instruction.

Assessment of general practitioners by video observation of communicative and medical performance in daily practice: issues of validity, reliability and feasibility.

OBJECTIVES: To develop a video assessment method for General Practitioners (GPs) by analysing issues of validity, reliability and feasibility of observation of videotaped regular consultations. DESIGN: In a cross-sectional study consultations of 93 GPs were video recorded in the practice during 1 week. The GPs registered consultation and patient data in a logbook; 16 consultations per GP were selected using preset criteria. The quality of communicative and medical performance of these consultations was assessed by GP observers with a validated instrument. The validity of the procedure was evaluated by checking the content of each GP's sample using specific sample criteria. Selection bias was estimated by multiple regression analysis, with sample characteristics as independent variables and scores on communication and medical performance as dependent variables. The influence of observation on GPs and patients was assessed by a questionnaire. Generalizability theory was used to estimate reliability. Feasibility was assessed by conducting a questionnaire, by keeping accounts, and by checking the technical quality of the videotaped consultations. SETTING: Universities of Nijmegen and Maastricht, The Netherlands. SUBJECTS: General Practitioners (GPs). RESULTS: The domain of general practice was well covered in the samples; content validity was satisfactory. With regard to the sample characteristics, only the total duration of consultations appeared to correlate significantly with both the score on communication and the score on medical performance. A majority (71%) of GPs reported not being influenced by the observation, except in the first cases, and recognizing their usual daily performance in the videotaped consultations. An acceptable level of reliability was reached after 2.5 hours of observation, i.e. 12 cases by a single observer. The method was well accepted by both GPs and patients. The costs were pound250 per GP. CONCLUSIONS: Video assessment of GPs in daily practice according to the procedures described is a valid and reliable method, one which is useful for education and quality improvement. There is a trade-off between feasibility on one hand and validity, reliability and credibility on the other hand. Compared to investments in observation methods in standardized settings, the costs of video observation of GPs' actual performance are acceptable.

Assessment in general practice: the predictive value of written-knowledge tests and a multiple-station examination for actual medical performance in daily practice.

This study compares the predictive values of written-knowledge tests and a standardized multiple-station examination for the actual medical performance of general practitioners (GPs) in order to select effective assessment methods to be used in quality-improvement activities. A comprehensive assessment was performed in four phases. First, 100 GPs from the southern part of the Netherlands were assessed by a general medical knowledge test and by a knowledge test on technical skills. Second, in order to check for time-order effects, participants were randomly divided into two groups of 50 each, comparable on scores of both knowledge tests and on professional characteristics. Finally, both groups went through a multiple station examination using standardized patients and a practice video assessment of real surgery, but in opposite orders. Consultations were videotaped and assessed by well-trained peer observers. The drop-out rate was 10%. In both groups the predictive value of medical knowledge tests, ranging from 0.43 to 0.56 (Pearson correlation disattenuated), proved to be comparable with the predictive value of the multiple-station examination for actual performance (0.33-0.59). The overall explained variance of scores of the practice video assessment, measured by multiple regression analysis with performance scores as dependent variables and scores on the knowledge tests and the multiple-station examination as independent variables was moderate (19%). A time-order effect showed in only one direction: from practice video assessment to the multiple-station examination. The GP's professional characteristics did not contribute to the explanation of variation in performance. Medical knowledge tests can predict actual clinical performance to the same extent as a multiple-station examination. Compared with a station examination, a knowledge test may be a good alternative method for assessment the procedures of a large number of practising GPs.

Assessment of practicing family physicians: comparison of observation in a multiple-station examination using standardized patients with observation of consultations in daily practice.

PURPOSE: Looking for a valid, reliable, and feasible method to collect data on the performances of practicing family physicians, the authors compare the measurement characteristics of a multiple-station examination (MSE) using standardized patients with those of a video assessment of regular consultations in daily practice (practice video assessment, PVA). METHOD: In a cross-sectional study, consultations of 90 family physicians were videotaped both in an MSE and in their daily practices. Peer-observers used a validated instrument (MAAS-Global) to assess the physicians' communication with patients and their medical performances. The physicians were randomly divided into two groups, comparable for demographic characteristics, and half underwent the assessments in reverse order to test for time-order effects. Content validity, criterion validity, reliability, and feasibility of the two methods were compared. RESULTS: Content validity of the PVA was superior to that of the MSE, since the domain of general family practice care was better covered. Observed participants judged the videotaped practice consultations to be "natural," whereas hardly any family physician, after reviewing the videotaped consultations of the MSE, recognized his or her usual working style. Specific criteria made it possible to standardize real practice. Concerning criterion validity, only the medical-performance components of the two methods correlated. No correlation was found for the communication components. Real-practice performance proved to be less influenced by observation than was performance during the MSE. The reliabilities of the two methods, expected to be better in the controlled MSE, were comparable. The administration of the PVA was more flexible, less costly, and better accepted by the family physicians than was that of the MSE. CONCLUSION: Assessment for quality improvement of family physicians' practices by video observation in daily practice is superior to video assessment in a simulated setting using standardized patients.

Structure and process: the relationship between practice management and actual clinical performance in general practice.

OBJECTIVES: The precise relationship between practice management (structure) and the doctor's actual performance (process) in general practice is tenuous. Analysis of their mutual relationship may yield insight into the way they contribute to outcome and into corresponding assessment procedures. METHOD: In a cross-sectional study, consultations of 93 GPs were videotaped in their own practice and assessed by peer-observers on medical performance and on communication with patients, followed by a practice visit by a non-physician observer using a validated Visitation Instrument to assess Practice management and organization (VIP). Pearson correlations (observed and disattenuated for unreliability of the instruments) between scores on 22 practice management dimensions and scores of 16 selected cases on medical performance and communication were calculated. The predictive value of specific practice management aspects for actual performance was determined by multiple regression analysis, with performance scores as dependent variables and scores on the 22 management dimensions and GPs' professional characteristics as independent variables. RESULTS: Nine practice management dimensions correlated significantly with medical performance and so did five dimensions with actual communication. Overall, most associations were weak. Combined with demographic variables (age for medical performance and working single-handedly for communication), 26% of variance in medical performance scores could be explained by only three practice management dimensions. One practice dimension (delegation of medical tasks to the practice assistant) explained 11% of variance in communication with patients. Organization of quality assessment activities explained most of the variation in medical performance. CONCLUSIONS: Practice management (structure) and actual performance (process) seem to be largely autonomous constructs. Quality improvement and assessment activities should emphasize that practice management is different from actual performance. Structure and process may contribute to patient outcome independently of each other.

Do general practitioners act consistently in real practice when they meet the same patient twice? Examination of intradoctor variation using standardised (simulated) patients.

OBJECTIVE: To assess the variation within individual general practitioners facing the same problem twice in actual practice under unbiased conditions. DESIGN: General practitioners were consulted during normal surgery hours by a standardised patient portraying a patient with angina pectoris. Six weeks later the same general practitioners were consulted again by a similar standardised patient portraying a similar case. The patients reported on the consultations. SETTING: Trondheim, Norway. SUBJECTS: Of 87 general practitioners invited by letter, 28 (32%) agreed to participate without hesitation; nine others (10%) wanted more information before consenting. From these 24 were selected and visited. MAIN OUTCOME MEASURES: Number of actions undertaken from a guideline in both rounds of consultations. Duration of consultations. RESULTS: The mean (range, interquartile range) guideline score, total score, and duration of consultation were not significantly different between the first and second patient encounters for the group as a whole. For individual doctors the mean (SD) difference was -0.09 (3.36) for the guideline score, 0.30 (8.1) for the total score, and -0.87 (9.01) for consultation time. CONCLUSIONS: The study shows that assessment of performance in real practice for a group of general practitioners is consistent from the first round of consultations to the second round. However, significant variation occurs in performance of individual physicians.