Should we assess clinical performance in single patient encounters or consistent behaviors of clinical performance over a series of encounters? A qualitative exploration of narrative trainee profiles.

BACKGROUND: A variety of tools have been developed to assess performance which typically use a single clinical encounter as a source for making competency inferences. This strategy may miss consistent behaviors. We therefore explored experienced clinical supervisors' perceptions of behavioral patterns that potentially exist in postgraduate general practice trainees expressed as narrative profiles to aid the grading of clinical performance. METHODS: We conducted semistructured interviews with clinical supervisors who had frequently observed clinical performance in trainees. Supervisors were asked to describe which behavioral patterns they had discerned in excellent and underperforming trainees, during different stages of training, in their careers as clinical supervisor. We analyzed the interviews using a grounded theory approach. RESULTS: The analysis resulted in a conceptual framework that distinguishes between desirable and undesirable narrative profiles. The framework consists of two dimensions: doctor-patient interaction and medical expertise. Personal values appear to be a moderating factor. CONCLUSIONS: According to experienced clinical supervisors, consistent behaviors do exist in GP trainees when observing clinical performance over time. The conceptual framework has to be validated by further observational studies to assess its potential for making robust and fair assessments of clinical performance and monitor the development of consultation performance over time.

New methodology for using incognito standardised patients for telephone consultation in primary care.

Context Many countries now use call centres as an integral part of out-of-hours primary care. Although some research has been carried out on safety issues pertaining to telephone consultations, there has been no published research on how to train and use standardised patients calling for medical advice or on the accuracy of their role-play. Objectives This study aimed to assess the feasibility and validity of using telephone incognito standardised patients (TISPs), the accuracy of their role-play and the rate of detection. Further objectives included exploring the experiences of TISPs and the difficulties encountered in self-recording calls. Methods Twelve TISPs were trained in role-play by presenting their problem to a general practitioner and a nurse. They were also trained in self-recording calls. Calls were made to 17 different out-of-hours centres (OOHCs) from home. Of the four or five calls made per evening, one call was assessed for accuracy of role play. Retrospectively, the OOHCs were asked whether they had detected any calls made by a TISP. The TISPs filled in a questionnaire concerning their training, the self-recording technique and their personal experiences. Results The TISPs made 375 calls over 84 evenings. The accuracy of role-play was close to 100%. A TISP was called back the same evening for additional information in 11 cases. Self-recording caused extra tension for some TISPs. All fictitious calls remained undetected. Conclusions Using the method described, TISPs can be valuable both for training and assessment of performance in telephone consultation carried out by doctors, trainees and other personnel involved in medical services.

Quality of communication during telephone triage at Dutch out-of-hours centres.

OBJECTIVE: To assess the quality of communication skills of triagists, working at out-of-hours (OOH) centres, and to determine the correlation between the communication score and the duration of the telephone consultation. METHODS: Telephone incognito standardised patients (TISPs) called 17 OOH centres presenting different clinical cases. The assessment of communication skills was carried out using the RICE-communication rating list. The duration of each telephone consultation was determined. RESULTS: The mean overall score for communication skills was 35% of the maximum feasible. Triagists usually asked questions about the clinical situation correctly and little about the patients' personal situation, perception of the problem or expectation. Advice about the outcome of triage and self-care advice was usually given without checking for patients' understanding and acceptance of the advice. Calls were often handled in an unstructured way, without summarizing or clarifying the different steps within the consultation. There was a positive correlation of 0.86 (p<0.01) between the overall communication score and the duration of the telephone consultation. CONCLUSION: Assessment of communication skills of triagists revealed specific shortcomings and learning points to improve the quality of communication skills during telephone triage. PRACTICE IMPLICATIONS: Training in telephone consultation should focus more on patient-centred communication with active listening, active advising and structuring the call. Apart from adequate communication skills, triagists need sufficient time for telephone consultation to enable high quality performance.

GPs' patterns of clinical assessment when faced with a patient suspected for spondyloarthritis: a prospective educational intervention study.

BACKGROUND: Timely recognition and referral of patients with spondyloarthritis (SpA) is challenging due to the frequent unawareness of the clinical picture. AIM: To identify clinical assessment patterns of GPs and GP-residents when facing a patient suspected of having SpA, and to determine which components of clinical assessment were most prevalent prior to referral to the rheumatologist and whether targeted education could positively influence pattern recognition. DESIGN & SETTING: Prospective multicentre educational intervention study in primary care practices in the Netherlands. METHOD: GPs and GP-residents were visited in two rounds by standardised patients (SPs) simulating axial or peripheral SpA (dactylitis). Between these rounds, an educational intervention regarding SpA took place for part of the participants. SPs completed a case-specific checklist inquiring about disease-related items and items on physical examination. RESULTS: Sixty-eight participants (30 GPs and 38 GP-residents) were included and 19 (28%) received the educational intervention. In round 1, about half of the participants asked at least one question to differentiate between an inflammatory or mechanical origin of the back pain or peripheral complaint; on average, <15% asked for extra-articular manifestations. After education, GP-residents inquired more about the presence of extra-articular manifestations and family history of axial SpA; this pattern was also observed in the GPs and GP-residents who correctly referred the SP. In the peripheral SpA case, the observed gain was less evident when compared to the axial SpA case. CONCLUSION: Pattern recognition of patients suspected for SpA by GP(-residents) is essential for referral to a rheumatologist and can be improved by education.

Education improves referral of patients suspected of having spondyloarthritis by general practitioners: a study with unannounced standardised patients in daily practice.

OBJECTIVES: To evaluate the practice performance of general practitioners (GPs) and GP residents in recognising and referring patients suspected for having axial or peripheral spondyloarthritis (SpA), and to investigate the influence of education on this performance. METHODS: GP (residents) were visited in two rounds by standardised patients (SPs) simulating axial SpA, peripheral SpA or carpal tunnel syndrome (CTS) with in between an educational intervention on SpA for part of the participants. Participants were unaware of the nature of the medical problem and study purpose. CTS was included as diversionary tactic. The primary outcome was ≥40% improvement in (considering) referral of the SPs with SpA to the rheumatologist after education. Secondary outcomes included ordering additional diagnostic tests, correct recognition of SpA and identification of variables contributing to this. RESULTS: 68 participants (30 GPs and 38 GP residents) were included, of which 19 received education. The primary outcome was met. A significantly higher proportion of GP (residents) from the intervention group referred patients to the rheumatologist compared with the control group after education (change scores, axial SpA +71% vs +15% (p<0.01); peripheral SpA +48% vs 0% (p<0.001)). Participants who received education, more frequently correctly recognised SpA compared with controls (change scores, axial SpA +50% vs -5% (p<0.001); peripheral SpA +21% vs 0% (p=0.01). CONCLUSIONS: Recognition and referral of patients suspected for having SpA by GP (residents) is low, but targeted education markedly improved this. This supports the development of educational initiatives to improve recognition of SpA and hence referral to a rheumatologist.

Controlled trial of effect of computer-based nutrition course on knowledge and practice of general practitioner trainees.

BACKGROUND: Nutrition education is not an integral part of either undergraduate or postgraduate medical education. Computer-based instruction on nutrition might be an attractive and appropriate tool to fill this gap. OBJECTIVE: The study objective was to assess the degree to which computer-based instruction on nutrition improves factual knowledge and practice behavior of general practitioner (GP) trainees. DESIGN: We carried out a controlled experimental study, using a 79-item knowledge test and 3 incognito standardized patients' visits in a pre- and posttest design with 49 first-year GP trainees. The experimental group (n = 25) received an average of 6 h of a newly developed computer-based instruction on nutrition. The control subjects (n = 24) took the standard vocational training program. RESULTS: The percentage of correct answers on the knowledge test increased from 30% at pretest to 42% at posttest in the experimental group, and from 36% to 37% in the control group. Analysis of covariance, with the pretest scores as covariate, showed a significant experimental versus control group difference at posttest: 9.2% (P = 0.002). The mean percentage of correctly performed items during the 3 standardized patients' visits (assessed by checklists) showed an increase in the experimental group from 20% at pretest to 36% at posttest, whereas the control group changed from 20% to 22%. Analysis of covariance, with the pretest scores as covariate, revealed a significant group difference at posttest: 13.7% (P < 0.001). CONCLUSION: The computer-based instruction proved its effectiveness, both by increasing factual knowledge and by substantially enhancing GP trainees' practice behavior on the subject of nutrition.

'Quod scripsi, scripsi.' The quality of the report of telephone consultations at Dutch out-of-hours centres.

OBJECTIVE: To assess the quality of the content of reports of telephone consultations at out-of-hours centres and to investigate to what extent the reports reflect the actual telephone consultation. DESIGN AND SETTING: Cross-sectional qualitative study; 17 out-of-hours centres in The Netherlands. METHOD: To assess the quality of the content of reports, a focus group developed the Reason for calling, Information gathered, Care advice given, Evaluation of the care advice with the patient (RICE) report rating instrument. Telephone Incognito Standardised Patients presented seven different clinical problems three times to 17 out-of-hours centres. All calls were recorded and transcribed. The out-of-hours centres being called were asked for a copy of the report of the call. The authors assessed the quality of the content of the reports and compared this with the transcripts. RESULTS: The out-of-hours centres returned a report for 78% of the 357 calls. For the remaining 22% of the calls, no report was written. Reports contained almost always information about the medical reason for calling but little information about details of the clinical history. Patients' expectation, personal situation or perception of the care advice was seldom documented. In all but one out-of-hours centre, answers to obligatory questions were reported by triagists, although they had not been asked, varying between 1% and 54% of all questions entered. Triagists entered a subjective evaluation of a patients' condition in 12% of the reports. CONCLUSION: Reports of telephone consultations of out-of-hours centres contained little information on patients' clinical and personal condition. This could potentially endanger patients' continuity of care and might pose legal consequences for the triagist.

Quality of clinical aspects of call handling at Dutch out of hours centres: cross sectional national study.

OBJECTIVE: To assess the quality of telephone triage by following the consecutive phases of its care process and the quality of the clinical questions asked about the patient's clinical condition, of the triage outcome, of the content of the home management advice, and of the safety net advice given at out of hours centres. DESIGN: Cross sectional national study using telephone incognito standardised patients. SETTING: The Netherlands. PARTICIPANTS: 17 out of hours centres. MAIN OUTCOME MEASURES: Percentages of clinical obligatory questions asked and items within home management and safety net advice, both in relation to pre-agreed standards, and of care advice given in relation to the required care advice. RESULTS: The telephone incognito standardised patients presented seven clinical cases three times each over a period of 12 months, making a total of 357 calls. The mean percentage of obligatory questions asked compared with the standard was 21%. Answers to questions about the clinical condition were not always correctly evaluated from a clinical viewpoint, either by triagists or by general practitioners. The quality of information on home management and safety net advice varied, but it was consistently poor for all cases and for all out of hours centres. Triagists achieved the appropriate triage outcome in 58% of calls. CONCLUSION: In determining the outcome of the care process, triagists often reached a conclusion after asking a minimal number of questions. By analysing the quality of different phases within the process of telephone triage, evaluation of whether an appropriate triage outcome has been arrived at by means of good clinical reasoning or by an educated guess is possible. In terms of enhancing the overall clinical safety of telephone triage, apart from obtaining an appropriate clinical history, adequate home management and safety net advice must also be given.

GPs' needs for practice-oriented nutrition education; a Delphi study among Dutch GPs.

BACKGROUND: Published guidelines and lists of topics in nutrition education for GPs are affected by practical drawbacks, which prevent them from being easily implemented in vocational training curricula. OBJECTIVE: Our aim was to draw up a concise priority list of disease-related nutrition topics reflecting the needs among Dutch GPs for nutrition education geared to everyday practice, which can be used to decide on the main topics and educational content of computer-based instruction. METHODS: This was a two-round Delphi study, using pilot-tested questionnaires carried out among 41 GPs who attended a national CME meeting on nutrition in The Netherlands. Sum scores of, and summarized comments on nutrition topics and their frequencies in top-10 lists drawn up by respondents were the main outcome measures. RESULTS: The response in both Delphi rounds was high (93 and 95%). Analysis of questionnaires in both rounds yielded a priority list of disease-related nutrition topics, with the 14 highest ranking topics identical in both rounds. A fifteenth topic, only found in the second questionnaire, completed the priority list. A group of experts assessed this list as fully acceptable. CONCLUSION: It was possible to compile a priority list of disease-related nutrition topics reflecting the needs of GPs for nutrition education. Based on the results of the Delphi study, we generated the following list of five priority topics: weight problems; diabetes mellitus; hypercholesterolaemia; intestinal complaints; and hypertension. This list of five and the summarized comments of respondents will determine the main topics and educational content of a computer-based instruction we are currently developing.

Fielding incognito standardised patients as 'known' patients in a controlled trial in general practice.

BACKGROUND: Incognito standardised patients (SPs) have only been used to represent new patients so far. The few trials with incognito SPs provide little detail on the method used for fielding them. OBJECTIVE: To establish the feasibility of introducing SPs as 'known' patients (i.e. patients who have previously visited the practice) into practices, to indicate the required practice preparations, and to describe the various aspects of using SPs in a pretest/post-test design. METHODS: We used incognito SPs as known patients in a controlled trial to assess the practice behaviour of 49 trainees. The SPs received a 2-day training in role playing and completing checklists. We compiled comprehensive practice information folders of each practice to be visited. Real personal data and faked medical data of SPs were inserted in the filing system of each practice to be visited. Apart from SP roles with slightly different reasons-for-encounter and different SPs, the same training protocol, checklists and practice information folders were used in the post-test. RESULTS: The SPs carried out 287 visits in 50 practices. All practices prepared the patient records for the SP visits in a fairly authentic practice style. Trainees detected the SP in 74 visits. The main reasons for detection were imperfections in patient records and aspects of SP roles or role playing. CONCLUSION: Fielding incognito SP visits as known patients was feasible but labour-intensive. Preparing the SP patient records and familiarising SPs with the interior of practices represented new elements in the use of SPs. The pretest/post-test format made their use more efficient instead of complicating it.