Knowledge translation in health: how implementation science could contribute more.

BACKGROUND: Despite increasing interest in research on how to translate knowledge into practice and improve healthcare, the accumulation of scientific knowledge in this field is slow. Few substantial new insights have become available in the last decade. MAIN BODY: Various problems hinder development in this field. There is a frequent misfit between problems and approaches to implementation, resulting in the use of implementation strategies that do not match with the targeted problems. The proliferation of concepts, theories and frameworks for knowledge transfer - many of which are untested - has not advanced the field. Stakeholder involvement is regarded as crucial for successful knowledge implementation, but many approaches are poorly specified and unvalidated. Despite the apparent decreased appreciation of rigorous designs for effect evaluation, such as randomized trials, these should remain within the portfolio of implementation research. Outcome measures for knowledge implementation tend to be crude, but it is important to integrate patient preferences and the increased precision of knowledge. CONCLUSIONS: We suggest that the research enterprise be redesigned in several ways to address these problems and enhance scientific progress in the interests of patients and populations. It is crucially important to establish substantial programmes of research on implementation and improvement in healthcare, and better recognize the societal and practical benefits of research.

Development of a tailored strategy to improve postpartum hemorrhage guideline adherence.

BACKGROUND: Despite the introduction of evidence based guidelines and practical courses, the incidence of postpartum hemorrhage shows an increasing trend in developed countries. Substandard care is often found, which implies an inadequate implementation in high resource countries. We aimed to reduce the gap between evidence-based guidelines and clinical application, by developing a strategy, tailored to current barriers for implementation. METHODS: The development of the implementation strategy consisted of three phases, supervised by a multidisciplinary expert panel. In the first phase a framework of the strategy was created, based on barriers to optimal adherence identified among professionals and patients together with evidence on effectiveness of strategies found in literature. In the second phase, the tools within the framework were developed, leading to a first draft. In the third phase the strategy was evaluated among professionals and patients. The professionals were asked to give written feedback on tool contents, clinical usability and inconsistencies with current evidence care. Patients evaluated the tools on content and usability. Based on the feedback of both professionals and patients the tools were adjusted. RESULTS: We developed a tailored strategy to improve guideline adherence, covering the trajectory of the third trimester of pregnancy till the end of the delivery. The strategy, directed at professionals, comprehending three stop moments includes a risk assessment checklist, care bundle and time-out procedure. As patient empowerment tools, a patient passport and a website with patient information was developed. The evaluation among the expert panel showed all professionals to be satisfied with the content and usability and no discrepancies or inconsistencies with current evidence was found. Patients' evaluation revealed that the information they received through the tools was incomplete. The tools were adjusted accordingly to the missing information. CONCLUSION: A usable, tailored strategy to implement PPH guidelines and practical courses was developed. The next step is the evaluation of the strategy in a feasibility trial. TRIAL REGISTRATION: Clinical trial registration: The Fluxim study, registration number: NCT00928863 .

From Postpartum Haemorrhage Guideline to Local Protocol: A Study of Protocol Quality.

Objective Postpartum hemorrhage (PPH) has a continuously rising incidence worldwide, suggesting suboptimal care. An important step in optimizing care is the translation of evidence-based guidelines into comprehensive hospital protocols. However, knowledge about the quality of these protocols is lacking. The objective of this study was to evaluate the quality of PPH-protocols on structure and content in the Netherlands. Methods We performed an observational multicenter study. Eighteen PPH-protocols from 3 University Hospitals (UH), 8 Teaching Hospitals (TH) and 7 Non-Teaching hospitals (NTH) throughout the Netherlands were acquired. The structure of the PPH-protocols was assessed using the Appraisal of Guidelines for Research and Evaluation (AGREE-II) Instrument. The content was appraised using previously developed quality indicators, based on international guidelines and Advance-Trauma-Life-Support (ATLS)-based course instructions. Results The quality of the protocols for postpartum hemorrhage for both structure and content varied widely between different hospitals, but all of them showed room for improvement. The protocols scored mainly below average on the different items of the AGREE-II instrument (8 of the 10 items scored <4 on a 1-7 scale). Regarding the content, adoption of guideline recommendations in protocols was 46 %. In addition, a timely indication of 'when to perform' a recommendation was lacking in three-fourths of the items. Conclusion This study shows that the quality of the PPH-protocols for both structure and content in the Netherlands is suboptimal. This makes adherence to the guideline and ATLS-based course instructions difficult.

Influencing factors for high quality care on postpartum haemorrhage in the Netherlands: patient and professional perspectives.

BACKGROUND: Postpartum haemorrhage (PPH) remains a major contributor to maternal morbidity even in high resource settings, despite the development and dissemination of evidence-based guidelines and Advance-Trauma-Life-Support (ATLS) based courses for optimal management of PPH. We aimed to assess current influencing factors (obstacles and facilitators) for the delivery of high quality PPH-care from both patient and professional perspective. METHODS: We qualitatively explored influencing factors for delivering high quality PPH-care, by having individual interviews with PPH-patients and focus group interviews with the different types of professionals working in the delivery room. For both perspectives, the theoretical frameworks of Grol and Cabana were used to classify the influencing factors for optimal PPH-care (factors of the guidelines, of professionals, of patients, of the social setting and of the organisation). In order to assess the importance of the influencing factors found among the professionals, we quantified these factors in a web-based questionnaire. RESULTS: A total of 12 patients and 41 professionals participated in the interviews, and 315 complete surveys were analyzed. The main obstacle for high quality PPH-care identified by patients was the lack of information given by the professionals to the patient and partner before, during and after the PPH event. An informative patient website, a patient leaflet and a follow-up consultation were mentioned as facilitators. The main obstacles according to the professionals were: lack of clarity of the guidelines, lack of knowledge and failing team-communication. Team training and checklists/ flowcharts were considered facilitators. CONCLUSIONS: Different obstacles to the delivery of high quality PPH-care were identified by both patients and professionals. These data can be used to develop a focused strategy to improve PPH-care. TRIAL REGISTRATION: NCT 00928863.

Guideline-based development of quality indicators for prevention and management of postpartum hemorrhage.

INTRODUCTION: To systematically develop a set of guideline-based quality indicators for postpartum hemorrhage (PPH) as a tool to measure guideline adherence in actual PPH care. MATERIAL AND METHODS: A Rand-modified Delphi procedure was used to systematically achieve consensus among a panel of 22 experts on PPH care on recommendations extracted from evidence-based guidelines, Managing-Obstetrics-Emergencies-Trauma (MOET) instructions and international literature. The selected recommendations were individually rated on health gain (prevention of maternal mortality and morbidity) and overall efficiency by the expert panel. Subsequently, consensus about the most important recommendations to measure quality of PPH care among the panel members was reached, followed by a final approval. Last, definition of the final set by critical appraisal of the recommendations regarding measurability took place. The main outcome measure was a set of valid quality indicators for prevention and management of PPH. RESULTS: From the 69 extracted recommendations, 50 were selected and translated into 22 quality indicators on professional performance (n = 17) and organization of PPH care (n = 5). The professional performance indicators covered all fields of PPH care, such as prevention (n = 2) and management of PPH, including communication and documentation (n = 4), monitoring and prevention of shock (n = 3), use of blood products (n = 3) and treatment of PPH (n = 5). Organizational indicators (n = 5) were clustered into protocols and agreements, audit, accessibility and documentation. CONCLUSIONS: This study describes a stepwise systematic development of 22 performance and organizational indicators to use for measuring the whole care process of prevention and management of PPH.

Understanding resident ratings of teaching in the workplace: a multi-centre study.

Providing clinical teachers with feedback about their teaching skills is a powerful tool to improve teaching. Evaluations are mostly based on questionnaires completed by residents. We investigated to what extent characteristics of residents, clinical teachers, and the clinical environment influenced these evaluations, and the relation between residents' scores and their teachers' self-scores. The evaluation and feedback for effective clinical teaching questionnaire (EFFECT) was used to (self)assess clinical teachers from 12 disciplines (15 departments, four hospitals). Items were scored on a five-point Likert scale. Main outcome measures were residents' mean overall scores (MOSs), specific scale scores (MSSs), and clinical teachers' self-evaluation scores. Multilevel regression analysis was used to identify predictors. Residents' scores and self-evaluations were compared. Residents filled in 1,013 questionnaires, evaluating 230 clinical teachers. We received 160 self-evaluations. 'Planning Teaching' and 'Personal Support' (4.52, SD .61 and 4.53, SD .59) were rated highest, 'Feedback Content' (CanMEDS related) (4.12, SD .71) was rated lowest. Teachers in affiliated hospitals showed highest MOS and MSS. Medical specialty did not influence MOS. Female clinical teachers were rated higher for most MSS, achieving statistical significance. Residents in year 1-2 were most positive about their teachers. Residents' gender did not affect the mean scores, except for role modeling. At group level, self-evaluations and residents' ratings correlated highly (Kendall's τ 0.859). Resident evaluations of clinical teachers are influenced by teacher's gender, year of residency training, type of hospital, and to a lesser extent teachers' gender. Clinical teachers and residents agree on strong and weak points of clinical teaching.

What are the key ingredients for effective public involvement in health care improvement and policy decisions? A randomized trial process evaluation.

CONTEXT: In the past 50 years, individual patient involvement at the clinical consultation level has received considerable attention. More recently, patients and the public have increasingly been involved in collective decisions concerning the improvement of health care and policymaking. However, rigorous evaluation guiding the development and implementation of effective public involvement interventions is lacking. This article describes those key ingredients likely to affect public members' ability to deliberate productively with professionals and influence collective health care choices. METHOD: We conducted a trial process evaluation of public involvement in setting priorities for health care improvement. In all, 172 participants (including 83 patients and public members and 89 professionals) from 6 Health and Social Services Centers in Canada participated in the trial. We video-recorded 14 one-day meetings, and 2 nonparticipant observers took structured notes. Using qualitative analysis, we show how public members influenced health care improvement priorities. FINDINGS: Legitimacy, credibility, and power explain the variations in the public members' influence. Their credibility was supported by their personal experience as patients and caregivers, the provision of a structured preparation meeting, and access to population-based data from their community. Legitimacy was fostered by the recruitment of a balanced group of participants and by the public members' opportunities to draw from one another's experience. The combination of small-group deliberations, wider public consultation, and a moderation style focused on effective group process helped level out the power differences between professionals and the public. The engagement of key stakeholders in the intervention design and implementation helped build policy support for public involvement. CONCLUSIONS: A number of interacting active ingredients structure and foster the public's legitimacy, credibility, and power. By paying greater attention to them, policymakers could develop and implement more effective public involvement interventions.

Implementing guidelines for depression on antidepressant prescribing in general practice: a quasi-experimental evaluation.

BACKGROUND: Internationally, guidelines for depression recommend a stepped care approach, implying that antidepressant medication should not be offered as a first step treatment to patients with sub-threshold or mild depression. In the Netherlands, antidepressant prescribing rates in general practice as a first treatment step are considered to be high. The aim of this study was to evaluate the implementation of guideline recommendations on antidepressant prescribing. METHODS: A quasi-experimental study with a non-equivalent naturalistic control group and three years follow-up was performed in the general practice setting in the Netherlands. General Practitioners (GPs) participated in a national Quality Improvement Collaborative (QIC), focusing on the implementation of a guideline based model for a stepped care approach to depression. The model consisted of self-help and psychological treatment options for patients with milder symptoms as an alternative to antidepressants in general practice. Changes in antidepressant prescription rates of GPs were documented for a three-year period and compared to those in a control group of GPs, selected from an ongoing national registration network. RESULTS: A decrease of 23.3% (49.4%-26.1%) in antidepressant prescription rates for newly diagnosed patients with depressive symptoms was found within the intervention group, whereas no difference occurred in the reference group (50.3%-52.6%). The decrease over time was significant, compared to the usual care group (OR 0.44, 95% CI: 0.21-0.92). CONCLUSIONS: An implementation program using stepped care principles for the allocation of depression interventions resulted in reduced antidepressant prescription rates in general practice. GPs can change prescribing behaviour within the context of a QIC.

Assessment of a pay-for-performance program in primary care designed by target users.

BACKGROUND: Evidence for pay-for-performance (P4P) has been searched for in the last decade as financial incentives increased to influence behaviour of health care professionals to improve quality of care. The effectiveness of P4P is inconclusive, though some reviews reported significant effects. OBJECTIVE: To assess changes in performance after introducing a participatory P4P program. DESIGN: An observational study with a pre- and post-measurement. Setting and subjects. Sixty-five general practices in the south of the Netherlands. Intervention. A P4P program designed by target users containing indicators for chronic care, prevention, practice management and patient experience (general practitioner's [GP] functioning and organization of care). Quality indicators were calculated for each practice. A bonus with a maximum of 6890 Euros per 1000 patients was determined by comparing practice performance with a benchmark. MAIN OUTCOME MEASURES: Quality indicators for clinical care (process and outcome) and patient experience. RESULTS: We included 60 practices. After 1 year, significant improvement was shown for the process indicators for all chronic conditions ranging from +7.9% improvement for cardiovascular risk management to +11.5% for asthma. Five outcome indicators significantly improved as well as patients' experiences with GP's functioning and organization of care. No significant improvements were seen for influenza vaccination rate and the cervical cancer screening uptake. The clinical process and outcome indicators, as well as patient experience indicators were affected by baseline measures. Smaller practices showed more improvement. CONCLUSIONS: A participatory P4P program might stimulate quality improvement in clinical care and improve patient experiences with GP's functioning and the organization of care.

Minimal improvement of nurses' motivational interviewing skills in routine diabetes care one year after training: a cluster randomized trial.

BACKGROUND: The effectiveness of nurse-led motivational interviewing (MI) in routine diabetes care in general practice is inconclusive. Knowledge about the extent to which nurses apply MI skills and the factors that affect the usage can help to understand the black box of this intervention. The current study compared MI skills of trained versus non-trained general practice nurses in diabetes consultations. The nurses participated in a cluster randomized trial in which a comprehensive program (including MI training) was tested on improving clinical parameters, lifestyle, patients' readiness to change lifestyle, and quality of life. METHODS: Fifty-eight general practices were randomly assigned to usual care (35 nurses) or the intervention (30 nurses). The ratings of applying 24 MI skills (primary outcome) were based on five consultation recordings per nurse at baseline and 14 months later. Two judges evaluated independently the MI skills and the consultation characteristics time, amount of nurse communication, amount of lifestyle discussion and patients' readiness to change. The effect of the training on the MI skills was analysed with a multilevel linear regression by comparing baseline and the one-year follow-up between the interventions with usual care group. The overall effect of the consultation characteristics on the MI skills was studied in a multilevel regression analyses. RESULTS: At one year follow up, it was demonstrated that the nurses improved on 2 of the 24 MI skills, namely, "inviting the patient to talk about behaviour change" (mean difference=0.39, p=0.009), and "assessing patient's confidence in changing their lifestyle" (mean difference=0.28, p=0.037). Consultation time and the amount of lifestyle discussion as well as the patients' readiness to change health behaviour was associated positively with applying MI skills. CONCLUSIONS: The maintenance of the MI skills one year after the training program was minimal. The question is whether the success of MI to change unhealthy behaviour must be doubted, whether the technique is less suitable for patients with a complex chronic disease, such as diabetes mellitus, or that nurses have problems with the acquisition and maintenance of MI skills in daily practice. Overall, performing MI skills during consultation increases, if there is more time, more lifestyle discussion, and the patients show more readiness to change. TRIAL REGISTRATION: Current Controlled Trials ISRCTN68707773.

No identifiable Hb1Ac or lifestyle change after a comprehensive diabetes programme including motivational interviewing: a cluster randomised trial.

OBJECTIVE: To study the effectiveness of a comprehensive diabetes programme in general practice that integrates patient-centred lifestyle counselling into structured diabetes care. Design and setting. Cluster randomised trial in general practices. INTERVENTION: Nurse-led structured diabetes care with a protocol, record keeping, reminders, and feedback, plus training in motivational interviewing and agenda setting. SUBJECTS: Primary care nurses in 58 general practices and their 940 type 2 diabetes patients with an HbA1c concentration above 7%, and a body mass index (BMI) above 25 kg/m². Main outcome measures. HbA1c, diet, and physical activity (medical records and patient questionnaires). RESULTS: Multilevel linear and logistic regression analyses adjusted for baseline outcomes showed that despite active nurse participation in the intervention, the comprehensive programme was no more effective than usual care after 14 months, as shown by HbA1c levels (difference between groups = 0.13; CI 20.8-0.35) and diet (fat (difference between groups = 0.19; CI 20.82-1.21); vegetables (difference between groups = 0.10; CI-0.21-0.41); fruit (difference between groups = 20.02; CI 20.26-0.22)), and physical activity (difference between groups = 21.15; CI 212.26-9.97), or any of the other measures of clinical parameters, patient's readiness to change, or quality of life. CONCLUSION: A comprehensive programme that integrated lifestyle counselling based on motivational interviewing principles integrated into structured diabetes care did not alter HbA1c or the lifestyle related to diet and physical activity. We thus question the impact of motivational interviewing in terms of its ability to improve routine diabetes care in general practice.

Residents provide feedback to their clinical teachers: reflection through dialogue.

BACKGROUND: Physicians play a crucial role in teaching residents in clinical practice. Feedback on their teaching performance to support this role needs to be provided in a carefully designed and constructive way. AIMS: We investigated an evaluation system for evaluating supervisors and providing formative feedback. METHOD: In a design based research approach, the 'Evaluation and Feedback For Effective Clinical Teaching System' (EFFECT-S) was examined by conducting semi-structured interviews with residents and supervisors of five departments in five different hospitals about feedback conditions, acceptance and its effects. Interviews were analysed by three researchers, using qualitative research software (ATLAS-Ti). RESULTS: Principles and characteristics of the design are supported by evaluating EFFECT-S. All steps of EFFECT-S appear necessary. A new step, team evaluation, was added. Supervisors perceived the feedback as instructive; residents felt capable of providing feedback. Creating safety and honesty require different actions for residents and supervisors. Outcomes include awareness of clinical teaching, residents learning feedback skills, reduced hierarchy and an improved learning climate. CONCLUSIONS: EFFECT-S appeared useful for evaluating supervisors. Key mechanism was creating a safe environment for residents to provide honest and constructive feedback. Residents learned providing feedback, being part of the CanMEDS and ACGME competencies of medical education programmes.

Experiences of general practices with a participatory pay-for-performance program: a qualitative study in primary care.

The involvement of target users in the design choices of a pay-for-performance program may enhance its impact, but little is known about the views of participants in these programs. To explore general practices' experiences with pay-for-performance in primary care we conducted a qualitative study in general practices in the Netherlands. Thirty out of 65 general practices participating in a pay-for-performance program, stratified for bonus, were invited for a semistructured interview on feasibility, feedback and the bonus, spending of the bonus, unintended consequences, and future developments. Content analysis was used to process the resulting transcripts. We included 29 practices. The feasibility of the pay-for-performance program was questioned due to the substantial time investment. The feedback on clinical care, practice management and patient experience was mostly discussed in the team, and used for improvement plans, but was also qualified as annoying for one GP and for another GP it brought feelings of insecurity. Most practices considered the bonus a stimulus to improve quality of care, in addition to compensation for their effort and time invested. Distinctive performance features were not displayed, for instance, on a website. The bonus was mainly spent on new equipment or team building. Practices referred to gaming and focusing on those aspects that were incentivised ('tunnel vision') as unintended consequences. Future developments should be directed to absolute thresholds, new indicators to keep the process going, and an independent audit. Linking a part of the bonus to innovation was also suggested. The participants thought the pay-for-performance program was a labour-intensive positive breakthrough to stimulate quality improvement, but warned of unintended consequences of the program and the sustainability of the indicator set.

Retrospective evaluation of control measures for contacts of patient with Marburg hemorrhagic fever.

After an imported case of Marburg hemorrhagic fever was reported in 2008 in the Netherlands, control measures to prevent transmission were implemented. To evaluate consequences of these measures, we administered a structured questionnaire to 130 contacts classified as either having high-risk or low-risk exposure to body fluids of the case-patient; 77 (59.2%) of 130 contacts responded. A total of 67 (87.0%) of 77 respondents agreed that temperature monitoring and reporting was necessary, significantly more often among high-risk than low-risk contacts (p<0.001). Strict compliance with daily temperature monitoring decreased from 80.5% (62/77) during week 1 to 66.2% (51/77) during week 3. Contacts expressed concern about development of Marburg hemorrhagic fever (58.4%, 45/77) and infecting a family member (40.2%, 31/77). High-risk contacts had significantly higher scores on psychological impact scales (p<0.001) during and after the monitoring period. Public health authorities should specifically address consequences of control measures on the daily life of contacts.

Evaluation of physicians' professional performance: an iterative development and validation study of multisource feedback instruments.

BACKGROUND: There is a global need to assess physicians' professional performance in actual clinical practice. Valid and reliable instruments are necessary to support these efforts. This study focuses on the reliability and validity, the influences of some sociodemographic biasing factors, associations between self and other evaluations, and the number of evaluations needed for reliable assessment of a physician based on the three instruments used for the multisource assessment of physicians' professional performance in the Netherlands. METHODS: This observational validation study of three instruments underlying multisource feedback (MSF) was set in 26 non-academic hospitals in the Netherlands. In total, 146 hospital-based physicians took part in the study. Each physician's professional performance was assessed by peers (physician colleagues), co-workers (including nurses, secretary assistants and other healthcare professionals) and patients. Physicians also completed a self-evaluation. Ratings of 864 peers, 894 co-workers and 1960 patients on MSF were available. We used principal components analysis and methods of classical test theory to evaluate the factor structure, reliability and validity of instruments. We used Pearson's correlation coefficient and linear mixed models to address other objectives. RESULTS: The peer, co-worker and patient instruments respectively had six factors, three factors and one factor with high internal consistencies (Cronbach's alpha 0.95 - 0.96). It appeared that only 2 percent of variance in the mean ratings could be attributed to biasing factors. Self-ratings were not correlated with peer, co-worker or patient ratings. However, ratings of peers, co-workers and patients were correlated. Five peer evaluations, five co-worker evaluations and 11 patient evaluations are required to achieve reliable results (reliability coefficient ≥ 0.70). CONCLUSIONS: The study demonstrated that the three MSF instruments produced reliable and valid data for evaluating physicians' professional performance in the Netherlands. Scores from peers, co-workers and patients were not correlated with self-evaluations. Future research should examine improvement of performance when using MSF.

Design choices made by target users for a pay-for-performance program in primary care: an action research approach.

BACKGROUND: International interest in pay-for-performance (P4P) initiatives to improve quality of health care is growing. Current programs vary in the methods of performance measurement, appraisal and reimbursement. One may assume that involvement of health care professionals in the goal setting and methods of quality measurement and subsequent payment schemes may enhance their commitment to and motivation for P4P programs and therefore the impact of these programs. We developed a P4P program in which the target users were involved in decisions about the P4P methods. METHODS: For the development of the P4P program a framework was used which distinguished three main components: performance measurement, appraisal and reimbursement. Based on this framework design choices were discussed in two panels of target users using an adapted Delphi procedure. The target users were 65 general practices and two health insurance companies in the South of the Netherlands. RESULTS: Performance measurement was linked to the Dutch accreditation program based on three domains (clinical care, practice management and patient experience). The general practice was chosen as unit of assessment. Relative standards were set at the 25th percentile of group performance. The incentive for clinical care was set twice as high as the one for practice management and patient experience. Quality scores were to be calculated separately for all three domains, and for both the quality level and the improvement of performance. The incentive for quality level was set thrice as high as the one for the improvement of performance. For reimbursement, quality scores were divided into seven levels. A practice with a quality score in the lowest group was not supposed to receive a bonus. The additional payment grew proportionally for each extra group. The bonus aimed at was on average 5% to 10% of the practice income. CONCLUSIONS: Designing a P4P program for primary care with involvement of the target users gave us an insight into their motives, which can help others who need to discuss similar programs. The resulting program is in line with target users' views and assessments of relevance and applicability. This may enhance their commitment to the program as was indicated by the growing number of voluntary participants after a successfully performed field test during the procedure. The elements of our framework can be very helpful for others who are developing or evaluating a P4P program.

Cardiovascular risk management in patients with coronary heart disease in primary care: variation across countries and practices. An observational study based on quality indicators.

BACKGROUND: Primary care has an important role in cardiovascular risk management (CVRM) and a minimum size of scale of primary care practices may be needed for efficient delivery of CVRM . We examined CVRM in patients with coronary heart disease (CHD) in primary care and explored the impact of practice size. METHODS: In an observational study in 8 countries we sampled CHD patients in primary care practices and collected data from electronic patient records. Practice samples were stratified according to practice size and urbanisation; patients were selected using coded diagnoses when available. CVRM was measured on the basis of internationally validated quality indicators. In the analyses practice size was defined in terms of number of patients registered of visiting the practice. We performed multilevel regression analyses controlling for patient age and sex. RESULTS: We included 181 practices (63% of the number targeted). Two countries included a convenience sample of practices. Data from 2960 CHD patients were available. Some countries used methods supplemental to coded diagnoses or other inclusion methods introducing potential inclusion bias. We found substantial variation on all CVRM indicators across practices and countries. We computed aggregated practice scores as percentage of patients with a positive outcome. Rates of risk factor recording varied from 55% for physical activity as the mean practice score across all practices (sd 32%) to 94% (sd 10%) for blood pressure. Rates for reaching treatment targets for systolic blood pressure, diastolic blood pressure and LDL cholesterol were 46% (sd 21%), 86% (sd 12%) and 48% (sd 22%) respectively. Rates for providing recommended cholesterol lowering and antiplatelet drugs were around 80%, and 70% received influenza vaccination. Practice size was not associated to indicator scores with one exception: in Slovenia larger practices performed better. Variation was more related to differences between practices than between countries. CONCLUSIONS: CVRM measured by quality indicators showed wide variation within and between countries and possibly leaves room for improvement in all countries involved. Few associations of performance scores with practice size were found.

Quality of after-hours primary care in the Netherlands: a narrative review.

Many Western countries are seeking an organizational model for after-hours primary care that is safe, efficient, and satisfactory for patients and health care professionals. Around the year 2000, Dutch primary care physicians (PCPs) reorganized their after-hours primary care and shifted from small rotation groups to large-scale PCP cooperatives. This article provides a narrative review of studies on a range of issues about after-hours primary care in the Netherlands, including experiences of health care professionals and patients, patient-safety incidents, adherence to practice guidelines, waiting times, and quality of telephone triage. Physicians expressed high satisfaction with PCP cooperatives; their workload decreased, and job satisfaction increased compared with the situation before the reorganization. In general, patients were also satisfied, but areas for improvement included telephone consultations, patient education, and distance to a pharmacy. A study identified patient-safety incidents in 2.4% of all contacts, of which most did not result in harm to patients. The average adherence to clinical guidelines by physicians was 77%, with lowest adherence scores for prescribing antibiotics and treatment in emergency cases. The average waiting time for home visits was 30 minutes. Seventy percent of patients with life-threatening problems were visited within the time target of 15 minutes. Telephone triage by nurses had positive effects on care efficiency by increasing the proportion of telephone consultations and decreasing the proportion of clinic consultations and home visits. The after-hours primary care system in the Netherlands might set an example for other countries struggling to find a good solution for the problems they encounter with after-hours primary care. Future developments in the Netherlands include integration and extensive collaboration with the accident and emergency departments of hospitals, in which PCPs take care of self-referring patients.

Evaluation and feedback for effective clinical teaching in postgraduate medical education: validation of an assessment instrument incorporating the CanMEDS roles.

BACKGROUND: Providing clinical teachers in postgraduate medical education with feedback about their teaching skills is a powerful tool to improve clinical teaching. A systematic review showed that available instruments do not comprehensively cover all domains of clinical teaching. We developed and empirically test a comprehensive instrument for assessing clinical teachers in the setting of workplace learning and linked to the CanMEDS roles. METHODS: In a Delphi study, the content validity of a preliminary instrument with 88 items was studied, leading to the construction of the EFFECT (evaluation and feedback for effective clinical teaching) instrument. The response process was explored in a pilot test and focus group research with 18 residents of 6 different disciplines. A confirmatory factor analyses (CFA) and reliability analyses were performed on 407 evaluations of 117 supervisors, collected in 3 medical disciplines (paediatrics, pulmonary diseases and surgery) of 6 departments in 4 different hospitals. RESULTS: CFA yielded an 11 factor model with a good to excellent fit and internal consistencies ranged from 0.740 to 0.940 per domain; 7 items could be deleted. CONCLUSION: The model of workplace learning showed to be a useful framework for developing EFFECT, which incorporates the CanMEDS competencies and proved to be valid and reliable.

Factors predicting doctors' reporting of performance change in response to multisource feedback.

BACKGROUND: Multi-source feedback (MSF) offers doctors feedback on their performance from peers (medical colleagues), coworkers and patients. Researchers increasingly point to the fact that only a small majority of doctors (60-70 percent) benefit from MSF. Building on medical education and social psychology literature, the authors identified several factors that may influence change in response to MSF. Subsequently, they quantitatively studied the factors that advance the use of MSF for practice change. METHODS: This observational study was set in 26 non-academic hospitals in the Netherlands. In total, 458 specialists participated in the MSF program. Besides the collation of questionnaires, the Dutch MSF program is composed of a reflective portfolio and a facilitative interview aimed at increasing the acceptance and use of MSF. All specialists who finished a MSF procedure between May 2008 and September 2010 were invited to complete an evaluation form. The dependent variable was self-reported change. Three categories of independent variables (personal characteristics, experiences with the assessments and mean MSF ratings) were included in the analysis. Multivariate regression analysis techniques were used to identify the relation between the independent variables and specialists' reported change in actual practice. RESULTS: In total, 238 medical specialists (response rate 52 percent) returned an evaluation form and participated in the study. A small majority (55 percent) of specialists reported to have changed their professional performance in one or more aspects in response to MSF. Regression analyses revealed that two variables had the most effect on reported change. Perceived quality of mentoring positively influenced reported change (regression coefficient beta = 0.527, p < 0.05) as did negative scores offered by colleagues. (regression coefficient beta = -0.157, p < 0.05). The explained variance of these two variables combined was 34 percent. CONCLUSIONS: Perceived quality of mentoring and MSF ratings from colleagues seem to be the main motivators for the self-reported change in response to MSF by specialists. These insights could leverage in increasing the use of MSF for practice change by investing in the quality of mentors.