Practical guidance on undertaking a service evaluation.

This article describes the basic principles of evaluation, focusing on the evaluation of healthcare services. It emphasises the importance of evaluation in the current healthcare environment and the requirement for nurses to understand the essential principles of evaluation. Evaluation is defined in contrast to audit and research, and the main theoretical approaches to evaluation are outlined, providing insights into the different types of evaluation that may be undertaken. The essential features of preparing for an evaluation are considered, and guidance provided on working ethically in the NHS. It is important to involve patients and the public in evaluation activity, offering essential guidance and principles of best practice. The authors discuss the main challenges of undertaking evaluations and offer recommendations to address these, drawing on their experience as evaluators.

'Calling executives and clinicians to account': user involvement in commissioning cancer services.

BACKGROUND: English NHS guidance emphasizes the importance of involving users in commissioning cancer services. There has been considerable previous research on involving users in service improvement, but not on involvement in commissioning cancer services. OBJECTIVE: To identify how users were involved as local cancer service commissioning projects sought to implement good practice and what has been learned. DESIGN: Participatory evaluation with four qualitative case studies based on semi-structured interviews with project stakeholders, observation and documentary analysis. Users were involved in every stage from design to analysis and reporting. SETTING AND PARTICIPANTS: Four English cancer network user involvement in commissioning projects, with 22 stakeholders interviewed. RESULTS: Thematic analysis identified nine themes: initial involvement, preparation for the role, ability to exercise voice, consistency and continuity, where decisions are made, closing the feedback loop, assessing impact, value of experience and diversity. DISCUSSION: Our findings on the impact of user involvement in commissioning cancer services are consistent with other findings on user involvement in service improvement, but highlight the specific issues for involvement in commissioning. Key points include the different perspectives users and professionals may have on the impact of user involvement in commissioning, the time necessary for meaningful involvement, the importance of involving users from the beginning and the value of senior management and PPI facilitator support and training. CONCLUSIONS: Users can play an important role in commissioning cancer services, but their ability to do so is contingent on resources being available to support them.

Understanding the concept of the key worker: do focus groups help?

The concept of the 'key-worker role' within paediatric haematology and oncology services is recognised in the UK through inclusion in published policies and guidance. Such guidance originates from both statutory and voluntary sector organisations. Within the policy direction itself, references are made to both 'designated' and 'non-designated' key workers, and there remains ongoing confusion within the professional field about the exact nature of the process of 'key-working' and how this should operate. This confusion therefore also exists for parents, carers and service users. The project described here aimed to examine the concept of the key-worker role through consultation with users as part of local service development. Focus group discussion was identified as the methodology of choice. Careful planning and delivery ensured that meaningful data emerged. Active participation by those attending the focus group discussion was observed. The focus group was in two sessions, both of which were digitally recorded and transcribed, with contemporaneous notes taken. These were subjected to thematic analysis and clear themes emerged regarding the importance of terminology, communication, skill mix and the use of technology. This local project achieved greater clarity about how to develop the key-worker role to best meet the needs of users through highlighting the need to include both the key-worker role, and the process of key-working. It is concluded that the use of focus groups is both a valid and valuable mechanism of consultation, as user consultation regarding service design and evaluation of care delivered is high on the wider agenda of the NHS.

Decisions to use complementary and alternative medicine (CAM) by male cancer patients: information-seeking roles and types of evidence used.

BACKGROUND: Complementary and Alternative Medicine (CAM) is increasingly popular with cancer patients and yet information provision or discussion about CAM by health professionals remains low. Previous research suggests that patients may fear clinicians' 'disapproval' if they raise the subject of CAM, and turn to other sources to acquire information about CAM. However, little empirical research has been conducted into how cancer patients acquire, and, more importantly evaluate CAM information before deciding which CAM therapies to try. METHODS: Qualitative study, comprising semi-structured interviews with 43 male cancer patients of varying ages, cancer type and stage of illness, 34 of whom had used CAM. They were recruited from a range of NHS and non-NHS settings in Bristol, England. RESULTS: As a result of the lack of CAM information from health professionals, men in this study became either 'pro-active seekers' or 'passive recipients' of such information. Their main information resource was the 'lay referral' network of family, friends and acquaintances, especially females. 'Traditional' information sources, including books, magazines, leaflets and the media were popular, more so in fact than the internet. Views on the internet ranged from enthusiasm or healthy scepticism through to caution or disinterest. CAM information was generally regarded as 'empowering' as it broadened treatment and self-care options. A minority of participants were information averse fearing additional choices that might disrupt their fragile ability to cope. There was general consensus that CAM information should be available via the NHS, to give it a 'stamp of approval', which combined with guidance from informed health professionals, could help patients to make 'guided' choices. However, a small minority of these men valued the independence of CAM from the NHS and deliberately sought 'alternative' information sources and treatment options. Men were selective in identifying particular therapies to use and sceptical about others, basing their choices on forms of 'evidence' that were personally meaningful: personal stories of individuals who had been helped by CAM; the long history and enduring popularity of some therapies; the plausibility of the mechanism of action; a belief or trust in individual therapies or their providers; scientific evidence. Scientific evidence ranked low in the men's personal decision-making about CAM, while it was recognised as important for NHS support for CAM. CONCLUSION: These male cancer patients valued the support and guidance of 'trusted individuals' in making choices about CAM. Trusted health professionals could also play a significant role in helping patients to make informed choices. Any such dialogue must, however, acknowledge the different standards of evidence used by patients and clinicians to evaluate the benefits or otherwise of CAM therapies. Such open communication could help to foster an environment of mutual trust where patients are encouraged to discuss their interest in CAM, rather than perpetuate covert, undisclosed use of CAM with its attendant potential hazards.

Collaboration, user involvement and education: a systematic review of the literature and report of an educational initiative.

Collaboration is advocated widely through government policy as part of enormous change within the Health Service (Department of Health 1998, 2000b). Directives from policy regarding collaboration impact onto organizations, professions and individuals including users of the service. A literature review suggests that there would appear to be limited anecdotal, discursive or rigorous evidence available on collaboration at all levels including involving users. However, literature does demonstrate a mounting body of evidence that collaboration with users is being promoted as a way of working. This paper reviews the literature around collaboration and user involvement in the context of cancer care. Findings suggest that there is confusion of terminology around collaboration and user involvement. Benefits of and barriers to user involvement are identified and these are explored in the context of caring for the patient with cancer. An evaluation of a team-based educational initiative designed to help health-care professionals working within the cancer arena to explore ways to collaborate with users is presented. Findings suggest that education may be one way to develop collaboration between health-care professionals and service users.

Divided care and the Third Way: user involvement in statutory and voluntary sector cancer services.

In health care, as in much of the public sphere, the voluntary sector is playing an increasingly large role in the funding, provision and delivery of services and nowhere is this more apparent than in cancer care. Simultaneously the growth of privatisation, marketisation and consumerism has engendered a rise in the promotion of 'user involvement' in health care. These changes in the organisation and delivery of health care, in part inspired by the 'Third Way' and the promotion of public and citizen participation, are particularly apparent in the British National Health Service. This paper presents initial findings from a three-year study of user involvement in cancer services. Using both case study and survey data, we explore the variation in the definition, aims, usefulness and mechanisms for involving users in the evaluation and development of cancer services across three Health Authorities in South West England. The findings have important implications for understanding shifts in power, autonomy and responsibility between patients, carers, clinicians and health service managers. The absence of any common definition of user involvement or its purpose underlines the limited trust between the different actors in the system and highlights the potentially negative impact of a Third Way health service.

A randomised controlled trial of clinics in secondary schools for adolescents with asthma.

AIM: To compare a nurse-led clinic in schools versus care in general practice for adolescents with asthma. DESIGN OF STUDY: Randomised controlled trial in four schools; parallel observational study in two schools. SETTING: Six comprehensive schools. METHOD: In the randomised trial, pupils were invited to attend asthma review at a nurse-led clinic either in school, or in general practice. The parallel observational study compared pupils invited to practice care within and outside the randomised trial. Primary outcome measures were attendance for asthma review, symptom control, and quality of life. Secondary outcomes were knowledge, attitudes, inhaler technique, use of steroids, school absence, peak flow rate, preference for future care, health service utilisation, and costs. RESULTS: School clinic pupils were more likely to attend an asthma review than those randomised to practice care (90.8% versus 51.0% overall [P < 0.001, not consistent across schools]). No differences were observed in symptom control (P = 0.42) or quality of life (P = 0.63). Pupils attending school clinics had greater knowledge of asthma (difference = +0.38, 95% CI = 0.19 to 0.56), more positive attitudes (difference = +0.21, 95% CI = 0.05 to 0.36), and better inhaler technique (P < 0.001, not consistent across all schools). No differences were observed in school absence or peak flow rate. A majority (63%) of those who had received care at school preferred this model in future. Median costs of providing care at school and at the practice were 32.10 Pounds and 19.80 Pounds, respectively. No differences were observed between the groups in the observational comparison on any outcome. CONCLUSIONS: The schools asthma clinic increased uptake of asthma reviews. There were improvements in various process measures, but not in clinical outcomes.

Dementia and the Over-75 Check: the role of the primary care nurse.

Primary care nurses are very likely to provide a substantial part of the routine care for patients with dementia. In order to examine the knowledge and attitudes of the primary care nurses who undertake the Over-75 Check, towards assessing and managing patients with symptoms of dementia, and to assess their level of support for a clinical practice guideline, a postal questionnaire survey was undertaken of primary care nurses responsible for the Over-75 Check in 209 general practices in Gloucester, Avon and Somerset. The questionnaire ascertained some demographic information about the nurses, their training and the practice routine for the Over-75 Check. It also included a case vignette describing a typical presentation of dementia at an Over-75 Check. A 65% response rate was achieved. Only one-fifth of respondents ever used formal validated cognitive tests as part of the Over-75 Check. In response to the vignette, nearly 10% took no action at all and a further 25% simply referred the patient on. Amongst the remainder there was considerable variation regarding the tasks considered to be within their remit. The respondents strongly supported the introduction of guidelines. Given the variation in professional responsibilities between practices, it is proposed that a clinical practice guideline covering diagnosis, assessment and management of dementia should address the practice as a whole rather than be targeted to specific professionals. The guideline should prompt professionals carrying out an Over-75 Check to recognize symptoms of dementia and raise awareness of the range of tasks which need to be undertaken in confirming diagnosis, assessing needs and managing patients. Results from this study suggest that improved training and increased autonomy for primary care nurses would improve access to services for these patients and their carers.