Quality assessment for three common conditions in primary care: validity and reliability of review criteria developed by expert panels for angina, asthma and type 2 diabetes.

OBJECTIVES: To field test the reliability, validity, and acceptability of review criteria for angina, asthma, and type 2 diabetes which had been developed by expert panels using a systematic process to combine evidence with expert opinion. DESIGN: Statistical analysis of data derived from a clinical audit, and postal questionnaire and semi-structured interviews with general practitioners and practice nurses in a representative sample of general practices in England. SETTING: 60 general practices in England. MAIN OUTCOME MEASURES: Clinical audit results for angina, asthma, and type 2 diabetes. General practitioner and practice nurse validity ratings from the postal questionnaire. RESULTS: 54%, 59%, and 70% of relevant criteria rated valid by the expert panels for angina, asthma, and type 2 diabetes, respectively, were found to be usable, valid, reliable, and acceptable for assessing quality of care. General practitioners and practice nurses agreed with panellists that these criteria were valid but not that they should always be recorded in the medical record. CONCLUSION: Quality measures derived using expert panels need field testing before they can be considered valid, reliable, and acceptable for use in quality assessment. These findings provide additional evidence that the RAND panel method develops valid and reliable review criteria for assessing clinical quality of care.

Improving the quality of mental health services in Personal Medical Services pilots: a longitudinal qualitative study.

BACKGROUND: A series of government initiatives in the UK have included strategies to improve the quality of services received by patients, including fundholding, the development of National Service Frameworks, clinical governance, and Personal Medical Services (PMS). PMS represents a new contractual arrangement between government and general practitioners (GPs) which provides new investment in return for more detailed specification of processes and outcomes of care. OBJECTIVES: To evaluate the effects of PMS on the quality of primary mental health care between 1998 and 2001. DESIGN: Multiple longitudinal case studies. Semi-structured interviews with key staff within practices (GPs, nurses, practice managers) and outside (health authority and primary care group/trust managers). SAMPLE: Six first wave PMS sites which had specifically planned to improve their mental health care. RESULTS: Improvements in mental health care were found in some PMS practices and not in others. Five mechanisms associated with successful quality improvement in mental health were identified: clear goals, effective teamwork within the practice, routine use of protocols and audits, additional resources, and effective collaboration with community and secondary care. Sites where these factors were not present struggled to meet their objectives. CONCLUSION: The five mechanisms which resulted in improved mental health care were facilitated by the new contractual arrangements in PMS. The new contracts were not a necessary part of these changes, but they enabled sites with an identified interest and motivation to make the changes. The contractual changes were not in themselves sufficient to improve care.

Implementing clinical governance in English primary care groups/trusts: reconciling quality improvement and quality assurance.

OBJECTIVES: To investigate the concept of clinical governance being advocated by primary care groups/trusts (PCG/Ts), approaches being used to implement clinical governance, and potential barriers to its successful implementation in primary care. DESIGN: Qualitative case studies using semi-structured interviews and documentation review. SETTING: Twelve purposively sampled PCG/Ts in England. PARTICIPANTS: Fifty senior staff including chief executives, clinical governance leads, mental health leads, and lay board members. MAIN OUTCOME MEASURES: Participants' perceptions of the role of clinical governance in PCG/Ts. RESULTS: PCG/Ts recognise that the successful implementation of clinical governance in general practice will require cultural as well as organisational changes, and the support of practices. They are focusing their energies on supporting practices and getting them involved in quality improvement activities. These activities include, but move beyond, conventional approaches to quality assessment (audit, incentives) to incorporate approaches which emphasise corporate and shared learning. PCG/Ts are also engaged in setting up systems for monitoring quality and for dealing with poor performance. Barriers include structural barriers (weak contractual levers to influence general practices), resource barriers (perceived lack of staff or money), and cultural barriers (suspicion by practice staff or problems overcoming the perceived blame culture associated with quality assessment). CONCLUSION: PCG/Ts are focusing on setting up systems for implementing clinical governance which seek to emphasise developmental and supportive approaches which will engage health professionals. Progress is intentionally incremental but formidable challenges lie ahead, not least reconciling the dual role of supporting practices while monitoring (and dealing with poor) performance.

Can health care quality indicators be transferred between countries?

OBJECTIVE: To evaluate the transferability of primary care quality indicators by comparing indicators for common clinical problems developed using the same method in the UK and the USA. METHOD: Quality indicators developed in the USA for a range of common conditions using the RAND-UCLA appropriateness method were applied to 19 common primary care conditions in the UK. The US indicators for the selected conditions were used as a starting point, but the literature reviews were updated and panels of UK primary care practitioners were convened to develop quality indicators applicable to British general practice. RESULTS: Of 174 indicators covering 18 conditions in the US set for which a direct comparison could be made, 98 (56.3%) had indicators in the UK set which were exactly or nearly equivalent. Some of the differences may have related to differences in the process of developing the indicators, but many appeared to relate to differences in clinical practice or norms of professional behaviour in the two countries. There was a small but non-significant relationship between the strength of evidence for an indicator and the probability of it appearing in both sets of indicators. CONCLUSION: There are considerable benefits in using work from other settings in developing measures of quality of care. However, indicators cannot simply be transferred directly between countries without an intermediate process to allow for variation in professional culture or clinical practice.

Assessing the quality of care of multiple conditions in general practice: practical and methodological problems.

OBJECTIVE: To investigate practical and methodological problems in assessing the quality of care of multiple conditions in general practice. SETTING: Sixteen general practices from two socioeconomically diverse regions in the UK. METHOD: Quality of care was assessed in 100 randomly selected patient records in each practice using an established set of quality indicators covering 23 conditions commonly seen in primary care. Inter-rater reliability assessment was carried out for five of the conditions. RESULTS: Conducting simultaneous quality assessment across multiple conditions is highly resource intensive. Poor data quality and the low prevalence of some items of care defined by the indicators are significant problems. Scores for individual indicators require very large samples for reliable assessment. Quality scores are more reliable when reported at a higher unit of analysis. This is particularly true for indicators and conditions with low prevalence where data may need to be aggregated to the level of groups of conditions or organisational providers. There is no single ideal way of aggregating quality scores. CONCLUSION: The study identified some of the practical and methodological difficulties in assessing quality of care across multiple conditions. For improved quality assessment, advances in information technology and improvements in data quality are required for more efficient and reliable data extraction from medical records, together with the development of methods for combining scores across indicators, conditions, and practices. However, electronic data extraction methods will still be based on the assumption that the care recorded reflects the care provided.

Defining quality of care.

This paper defines quality of health care. We suggest that there are two principal dimensions of quality of care for individual patients; access and effectiveness. In essence, do users get the care they need, and is the care effective when they get it? Within effectiveness, we define two key components--effectiveness of clinical care and effectiveness of inter-personal care. These elements are discussed in terms of the structure of the health care system, processes of care, and outcomes resulting from care. The framework relates quality of care to individual patients and we suggest that quality of care is a concept that is at its most meaningful when applied to the individual user of health care. However, care for individuals must placed in the context of providing health care for populations which introduces additional notions of equity and efficiency. We show how this framework can be of practical value by applying the concepts to a set of quality indicators contained within the UK National Performance Assessment Framework and to a set of widely used indicators in the US (HEDIS). In so doing we emphasise the differences between US and UK measures of quality. Using a conceptual framework to describe the totality of quality of care shows which aspects of care any set of quality indicators actually includes and measures and, and which are not included.

Analysis of referral behaviour: responses to simulated case histories may not reflect real clinical behaviour.

In an attempt to develop a measure of the referral behaviour of general practitioners, 21 simulated case histories were constructed which presented between three and six stages at which a decision was required whether or not refer a 'patient' to hospital. Twenty general practitioners completed the case histories and their responses were compared with their referral rates to outpatient departments. No significant correlations were found between the doctors' responses to the vignettes and their actual referral rates, and the repeatability of the vignettes was disappointing. The fact that the case histories appeared realistic to the doctors who completed them was not reflected in objective measures of their validity. Simulated case histories should not be used as a method of measuring doctors' behaviour without establishing their validity and reliability.

Evaluation of a computerized appointment system in general practice.

Computers have gained rapid acceptance in general practice. A recent development has been the use of computers to run practice appointment systems. This study demonstrates the benefits of installing a computerized appointment system, with improved service for patients, and more efficient use of time by both doctors and receptionists.

Dementia in general practice: the practical consequences of a more positive approach to diagnosis.

General practitioners are often reluctant to administer brief cognitive tests, and to question the relatives of patients who appear to be demented, for fear of causing distress. Diagnoses of dementia are therefore often based on guesswork, and non-demented patients may be rated as cognitively impaired in error. A randomly selected sample of 174 general practice patients aged 80 years and over were asked to complete a simple test of orientation and information in order to assess the usefulness and acceptability of such a procedure. If patients scored 10 points or less out of 12, a relative or other knowledgeable informant was questioned about their changes in mental state and behaviour. Assessments proved acceptable to patients, relatives and doctors; diagnoses were revised in 29 cases; and practical initiatives were proposed in 15 cases. Open discussions with patients and their families proved innocuous and have much to commend them.

Research capacity in UK primary care.

BACKGROUND: Moves towards a 'primary care-led' National Health Service (NHS) and towards evidence-based care have focused attention upon the need for evaluative research relating to the structure, delivery, and outcome of primary health care in the United Kingdom (UK). This paper describes work carried out to inform the Department of Health Committee on Research and Development (R&D) in Primary Care (Mant Committee). AIM: To describe the extent and nature of current research capacity in primary care in the UK and to identify future needs and priorities. METHOD: Funding data were requested from NHS National Programmes, NHS Executive Regional Offices, the Department of Health (DoH), Scottish Office, Medical Research Council, and some charities. A postal survey was sent to relevant academic departments, and appropriate academic journals were reviewed from 1992 to 1996. In addition, interviews were conducted with academic and professional leaders in primary care. RESULTS: Overall, total annual primary care R&D spend by the NHS and the DoH was found to be 7% of the total spend, although annual primary care R&D spend differs according to funding source. Journals relating to primary care do not, with some notable exceptions (e.g. British Journal of General Practice, Family Practice), have high academic status, and research into primary care by academic departments is, with perhaps the exception of general practice, on a small scale. The research base of most primary care professions is minimal, and significant barriers were identified that will need addressing if research capacity is to be expanded. CONCLUSION: There are strong arguments for the development of primary care research in a 'primary care-led' NHS in the UK. However, dashes for growth or attempts to expand capacity from the present infrastructure must be avoided in favour of endeavours to foster a sustainable, long-term research infrastructure capable of responding meaningfully to identified needs.

Systematic review of studies of quality of clinical care in general practice in the UK, Australia and New Zealand.

OBJECTIVES: Little is known about the quality of clinical care provided outside the hospital sector, despite the increasingly important role of clinical generalists working in primary care. In this study we aimed to summarise published evaluations of the quality of clinical care provided in general practice in the UK, Australia, and New Zealand. DESIGN: A systematic review of published studies assessing the quality of clinical care in general practice for the period 1995-9. SETTING: General practice based care in the UK, Australia, and New Zealand. Main outcome measures-Study design, sampling strategy and size, clinical conditions studied, quality of care attained for each condition (compared with explicit or implicit standards for the process of care), and country of origin for each study. RESULTS: Ninety papers fulfilled the entry criteria for the review, 80 from the UK, six from Australia, and four from New Zealand. Two thirds of the studies assessed care in self-selected practices and 20% of the studies were based in single practices. The majority (85.5%) examined the quality of care provided for chronic conditions including cardiovascular disease (22%), hypertension (14%), diabetes (14%), and asthma (13%). A further 12% and 2% examined preventive care and acute conditions, respectively. In almost all studies the processes of care did not attain the standards set out in national guidelines or those set by the researchers themselves. For example, in the highest achieving practices 49% of diabetic patients had had their fundii examined in the previous year and 47% of eligible patients had been prescribed beta blockers after an acute myocardial infarction. CONCLUSIONS: This study adopts an overview of the magnitude and the nature of clinical quality problems in general practice in three countries. Most of the studies in the systematic review come from the UK and the small number of papers from Australia and New Zealand make it more difficult to draw conclusions about the quality of care in these two countries. The review helps to identify deficiencies in the research, clinical and policy agendas in a part of the health care system where quality of care has been largely ignored to date. Further work is required to evaluate the quality of clinical care in a representative sample of the population, to identify the reasons for substandard care, and to test strategies to improve the clinical care provided in general practice.

Development of review criteria for assessing the quality of management of stable angina, adult asthma, and non-insulin dependent diabetes mellitus in general practice.

OBJECTIVE: To develop review criteria to assess the quality of care for three major chronic diseases: adult asthma, stable angina, and non-insulin dependent diabetes mellitus. SUBJECTS AND METHODS: Modified panel process based upon the RAND/UCLA (University College of Los Angeles) appropriateness method. Three multiprofessional panels made up of general practitioners, hospital specialists, and practice nurses. RESULTS: The RAND/UCLA appropriateness method of augmenting evidence with expert opinion was used to develop criteria for the care of the three conditions. Of those aspects of care which were rated as necessary by the panels, only 26% (16% asthma, 10% non-insulin dependent diabetes, 40% angina) were subsequently rated by the panels as being based on strong scientific evidence. CONCLUSION: The results show the importance of a systematic approach to combining evidence with expert opinion to develop review criteria for assessing the quality of three chronic diseases in general practice. The evidence base for the criteria was often incomplete, and explicit methods need to be used to combine evidence with expert opinion where evidence is not available.

A critical review of the evidence for a pain-spasm-pain cycle in spinal disorders.

The existence of a pain-spasm-pain cycle in musculoskeletal disorders has been debated for over 40 years. This paper reviews critically the evidence for such a cycle in patients with back pain. Clinical studies indicate that a substantial proportion of patients with back pain have muscle spasm. Patients with acute back pain have increased muscular activity on electromyography (EMG). EMG studies of patients with chronic back pain show, on the whole, an increase in activity in static postures, and a reduction of muscle activity during movement. Experimental evidence shows that pain may cause muscle spasm and that muscular activity can be painful. Further evidence for the existence of a pain-spasm-pain cycle comes from studies which show that analgesics can reduce muscle spasm, and that a variety of muscle relaxant techniques can reduce pain. There are substantial methodological problems in many of the studies cited, but the evidence reviewed provides general support for the existence of a pain-spasm-pain cycle. However further work is required to determine the nature of spasm and to evaluate methods for its detection, particularly in acute back pain syndromes. The value of therapeutic intervention may then be assessed.

Should general practitioners refer more patients to hospital?

Twenty-two patients with skin problems were identified on the basis that their general practitioner was satisfied with their management and had no intention of referring them to hospital. These patients were reviewed by one of two dermatologists who made treatment recommendations in 14 cases, and these suggestions were taken up by 11 patients. Six patients reported definite subjective improvement in their skin condition six weeks later. The general practitioners found the consultation valuable in 17 cases. Although these patients might have improved without the dermatologists' intervention, a group of patients may exist who would benefit from specialist advice, even though that advice has not been sought. Restriction on hospital referrals could therefore be detrimental to patient care. Controlled trials could be conducted to determine the benefit of specialist advice on patients not currently seen in hospital.

How much variation in referral rates among general practitioners is due to chance?

A 20-fold variation in referral rates from general practitioners to hospital outpatient departments has been shown in studies published to date. Most of the hypotheses proposed to account for this variation have not been supported by these studies. A simple model was constructed, which showed that a significant part of the variation may be due to the fairly small numbers of referrals in most studies. Real differences may have been swamped by random variations in the small numbers of referrals. The statistical power of the studies may not have been high enough to determine which factors were significant in describing the variation and how much of the variation was due to differing clinical practice. There remains a substantial part of the variation that cannot be accounted for.

Out of hours primary care centres: characteristics of those attending and declining to attend.

OBJECTIVES: To study the number, demography, and clinical details of patients who agreed or refused to attend centralised primary care centres for out of hours medical care and to study the satisfaction with the service of those who attended. DESIGN: Data collected by telephonists and doctors. Satisfaction questionnaires given to patients who attended. SETTING: Five out of hours primary care centres in the United Kingdom. SUBJECTS: All patients contacting the deputising service to request medical help out of hours who were asked to attend a primary care centre. The study terminated when 1000 patients had agreed to attend (200 from each centre). 1000 patients not agreeing to attend were also sampled. RESULTS: The attendance rate varied from 8.9% to 52.3% in the five centres. The overall standardised attendance rate was 22.4%. The attendance rate was highest in children under 5 (465/2380, 19.5%) and fell with increasing age. Of the 1000 sampled non-attenders, 403 said that they had no transport and 345 said that they were too ill to attend. Those who attended were seen by the doctor more quickly. There was no significant difference between the groups in the number who received a prescription (810 attenders v 820 non-attenders, P = 0.57) or who were admitted to hospital (59 v 52, P = 0.5). Satisfaction with the service among those who attended was very high; 95% (694/731) said that they would be prepared to attend under similar circumstances in the future. CONCLUSION: Most patients are not able or prepared to attend a central facility for primary care out of hours. Substantial cultural change will be necessary and careful consideration given to planning if such centres are to provide a major part of out of hours care.

General practitioner outpatient referrals: do good doctors refer more patients to hospital?

OBJECTIVE: To investigate the relation between general practitioners' referral rates to individual specialties and the individual areas of expertise of the referring doctors. DESIGN: Data collected on referral patterns in one group practice over nine months. SETTING: General practice in suburban Birmingham consisting of five partners and a trainee. RESULTS: In 395 referrals there were large differences in referral patterns among partners for otorhinolaryngology, ophthalmology, general surgery, and dermatology. The doctors with particular expertise in otorhinolaryngology and ophthalmology had high referral rates to those specialties, and these differences persisted after allowing for case mix. CONCLUSION: A high referral rate does not necessarily imply a high level of inappropriate referral.

Questionnaire study and audit of use of angiotensin converting enzyme inhibitor and monitoring in general practice: the need for guidelines to prevent renal failure.

OBJECTIVES: To determine the current pattern of use of angiotensin converting enzyme inhibitor and monitoring of renal function in general practice and to audit all admissions to a regional renal unit for uraemia related to use of these drugs. DESIGN: Postal questionnaire sent to 400 general practitioners; audit of clinical notes of all patients receiving these drugs in one large general practice; audit of all cases of uraemia (creatinine concentration >500 micromol/l) related to treatment presenting to hospital renal services over 12 months. SETTING: General practices in the North Wales health authority and one in central Manchester. Regional renal unit in Salford. MAIN OUTCOME MEASURES: Proportion of general practitioners who regularly monitored renal function before and after initiation of angiotensin converting enzyme inhibitors. Indications for treatment and details of monitoring of renal function in patients receiving these drugs. Incidence of related uraemia and evidence of comorbid disease, other aetiological factors, delayed detection, and patient outcome. RESULTS: 277 (69%) general practitioners replied; 235 (85%) checked renal function before but only 93 (34%) after the start of treatment, and 42 (15%) never checked renal function. Angiotensin converting enzyme inhibitors were prescribed for 162 patients from a total of 3625 aged >35 years (mean age 66.4 (SD 15.9) years). Monitoring of renal function occurred before treatment in 55 (45%) and after start of treatment in 35 (29%) of the 122 patients treated in general practice. Angiotensin converting enzyme inhibitors could be causally implicated in 9 (7%) of 135 admissions for uraemia (mean age 74.2 (7. 2) v 62.1 (2.1) years; P<0.01). 3 patients had renovascular disease and 6 had congestive cardiac failure with another intercurrent illness. Renal function had not been checked in any patient after the start of treatment; mean duration of illness before admission was 10.5 (3.2) days. Mean length of hospital stay was 20.9 (10.4) days; there were 8 survivors. CONCLUSION: Cases of uraemia related to treatment with angiotensin converting enzyme inhibitors are still encountered and are often detected late because of lack of judicious monitoring of renal function in vulnerable, often elderly, patients, especially at times of intercurrent illness. Guidelines for appropriate monitoring of renal function may help to minimise the problem.

Understanding hospital referral rates: a user's guide.

Detailed referral information from one practice was used to investigate the effect of calculating referral rates in several different ways. Referral rates for individual general practitioners should be related to the number of consultations carried out and not to the number of registered patients; for whole practices list size may be used as the denominator. Most doctors will not need to control for age and sex of patients when comparing referral rates but may need to control for case mix when comparing referral rates to individual specialties. In addition, a method is described for distinguishing systematic variation between the referral rates of individual doctors from the random variation that may arise from data based on fairly small numbers of referrals. The method indicates whether systematic variation is greater than would be expected by chance, and it can be extended to indicate whether variability in referral rates is greater in one specialty than another. Because of random variation with time a year's data may not be sufficient to allow reliable interpretation of referral rates to individual specialties, except for the largest. At present there is no known relation between high or low referral rates and quality of care. Nevertheless, if doctors are to interpret their own rates of referral they need those rates to be reliable and valid. Use of the 10 guidelines described in this paper will help to prevent unwarranted conclusions being drawn from information on general practitioners' rates of referral to hospital.

Identifying predictors of high quality care in English general practice: observational study.

OBJECTIVES: To assess variation in the quality of care in general practice and identify factors associated with high quality care. DESIGN: Observational study. SETTING: Stratified random sample of 60 general practices in six areas of England. OUTCOME MEASURES: Quality of management of chronic disease (angina, asthma in adults, and type 2 diabetes) and preventive care (rates of uptake for immunisation and cervical smear), access to care, continuity of care, and interpersonal care (general practice assessment survey). Multiple logistic regression with multilevel modelling was used to relate each of the outcome variables to practice size, routine booking interval for consultations, socioeconomic deprivation, and team climate. RESULTS: Quality of clinical care varied substantially, and access to care, continuity of care, and interpersonal care varied moderately. Scores for asthma, diabetes, and angina were 67%, 21%, and 17% higher in practices with 10 minute booking intervals for consultations compared with practices with five minute booking intervals. Diabetes care was better in larger practices and in practices where staff reported better team climate. Access to care was better in small practices. Preventive care was worse in practices located in socioeconomically deprived areas. Scores for satisfaction, continuity of care, and access to care were higher in practices where staff reported better team climate. CONCLUSIONS: Longer consultation times are essential for providing high quality clinical care. Good teamworking is a key part of providing high quality care across a range of areas and may need specific support if quality of care is to be improved. Additional support is needed to provide preventive care to deprived populations. No single type of practice has a monopoly on high quality care: different types of practice may have different strengths.