Evaluating a quality improvement collaborative: a hybrid approach.

PURPOSE: Quality improvement collaboratives (QICs) are a popular approach to improving healthcare services and patient outcomes. This paper evaluates a QIC implemented by a large, integrated healthcare organisation in Wales in the UK. DESIGN/METHODOLOGY/APPROACH: This evaluation study draws on two well-established evaluation frameworks: Kirkpatrick's approach to gather data on participant satisfaction and learning and Stake's approach to gather data and form judgements about the impact of the intervention. A mixed methods approach was taken which included documentary analysis, surveys, semi-structured interviews, and observation of the QIC programme. FINDINGS: Together the two frameworks provide a rounded interpretation of the extent to which the QIC intervention was fit-for-purpose. Broadly the evaluation of the QIC was positive with some areas of improvement identified. RESEARCH LIMITATIONS/IMPLICATIONS: This study is limited to a QIC conducted within one organisation. Further testing of the hybrid framework is needed that extends to different designs of QICs. PRACTICAL IMPLICATIONS: A hybrid framework is provided to assist those charged with designing and evaluating QICs. ORIGINALITY/VALUE: Evaluation studies are limited on QICs and if present tend to adopt one framework. Given the complexities of undertaking quality improvement within healthcare, this study uniquely takes a hybrid approach.

Rhetoric or Reform? Changing Health and Social Care in Wales.

Throughout the United Kingdom, the National Health Service (NHS) struggles to meet demand and achieve performance targets. Services need to work with individuals and communities to reduce avoidable disease and dependence. All four UK nations have separately realised the need for change but 20 years' experience suggests that vision and rhetoric are not enough. Success requires reformed systems and changed leadership behaviour to enable frontline staff to break the status quo. Top down, target driven behaviour must be replaced with a real focus on improvement, championing those who have the knowledge to deliver it.

Looking for the "Little Things": A Multi-Disciplinary Approach to Medicines Monitoring for Older People Using the ADRe Resource.

Advances in medicines have increased the effectiveness of treatments and the social and cultural authority of doctors. However, as prescribing has become the dominant modality of treatment, the "pharmaceuticalization" of medical practice has often resulted in treatment "at a distance", with doctors having limited contact with patients. Older and poorer people, who are socially distanced from medical prescribers, suffer more adverse drug reactions (ADRs) than the general population. A team approach to checking patients systematically for ADRs, as detailed in manufacturers' literature, can minimise medication errors, but regular review is rare. This paper explains the benefits of medicines monitoring to protect older patients from iatrogenic harm, such as over-sedation, falls, or drug-induced Parkinsonism. We show how multidisciplinary initiatives to optimise prescribing can be supported by using a recognised resource-the adverse drug reaction profile (ADRe). The profile identifies and documents patients' signs and symptoms of putative ADRs. Better monitoring allows professionals to adjust prescribing and respond to identified problems with agility. Implementation of systematic monitoring will require changes to the regulatory regime and better inter-professional cooperation. Providing carers, nurses and pharmacists with a structured system to monitor patients would democratise relevant medical knowledge and help address ageism and the socio-economic health divide.

Exploring the role of communications in quality improvement: A case study of the 1000 Lives Campaign in NHS Wales.

INTRODUCTION: Effective communication is critical to successful large-scale change. Yet, in our experience, communications strategies are not formally incorporated into quality improvement (QI) frameworks. The 1000 Lives Campaign ('Campaign') was a large-scale national QI collaborative that aimed to save an additional 1000 lives and prevent 50 000 episodes of harm in Welsh health care over a 2-year period. We use the Campaign as a case study to describe the development, application, and impact of a communications strategy embedded in a large-scale QI initiative. METHODS: A comprehensive communications strategy guided communications work during the Campaign. The main aims of the communications strategy were to engage the hearts and minds of frontline National Health Service (NHS) staff in the Campaign and promote their awareness and understanding of specific QI interventions and the wider patient safety agenda. We used qualitative and quantitative measures to monitor communications outputs and assess how the communications strategy influenced awareness and knowledge of frontline NHS staff. RESULTS: The communications strategy facilitated clear and consistent framing of Campaign messages and allowed dissemination of information related to the range of QI interventions. It reaffirmed the aim and value of the Campaign to frontline staff, thereby promoting sustained engagement with Campaign activities. The communications strategy also built the profile of the Campaign both internally with NHS organizations across Wales and externally with the media, and played a pivotal role in improving awareness and understanding of the patient safety agenda. Ultimately, outcomes from the communications strategy could not be separated from overall Campaign outcomes. CONCLUSION AND RECOMMENDATIONS: Systematic and structured communications can support and enhance QI initiatives. From our experience, we developed a 'communications bundle' consisting of six core components. We recommend that communications bundles be incorporated into existing QI methodology, though details should be tailored to the specific context and available resource.

Evaluation of a web based tool to improve health behaviours in healthcare staff.

BACKGROUND: A web-based tool was developed and piloted by being made available to healthcare staff in Wales from September 2012 to March 2013. This evaluation included two primary outcome measures: general health and mental well-being, and six secondary outcome measures: sickness absence, alcohol use, healthy eating, smoking, physical activity and maintaining a healthy BMI. The aim was to assess the feasibility of a web-based tool to improve health behaviours in healthcare staff. METHODS: Healthcare staff joined via a website, chose two of five challenges, and recorded their health behaviours using an online tool on a regular basis. Evaluation was undertaken by comparing baseline and follow up questionnaires. RESULTS: 1708 individuals explored the programme's website, of whom 1320 selected two lifestyle challenges to address. Of these 346 individuals (26.2%; 346/1320) completed the end of project evaluation questions for the main outcome and provided the basis of the evaluation. Comparing pre:post data among respondents who engaged with the programme as a whole, self-reported general health status improved in 35.3% (n = 122, p = 0.001); mental health status improved in 33% (n = 110, p = 0.02); alcohol consumption score (AUDIT-C classification) fell in 27.2% (n = 71, p = 0.001); reported fruit and vegetable consumption (7 day recall) increased (p = 0.001); average time spent on vigorous exercise increased from 40.6 minutes a week to 67.6 minutes a week (p = 0.001); and 41 individuals noted a positive change to their BMI classification category (p = 0.001). CONCLUSIONS: Combining interactive web-based tools as part of a multi-media programme is feasible, increases health behaviours and generates interest among a proportion of the healthcare workforce. Further work is required to improve maintenance of engagement over time.

Investigating the organisational impacts of quality improvement: a protocol for a realist evaluation of improvement approaches drawing on the Resource Based View of the Firm.

INTRODUCTION: Little is understood about the role of quality improvement in enabling health organisations to survive and thrive in the contemporary context of financial and economic challenges. We will draw on the theoretical foundations of the 'Resource Based View of the Firm' (RBV) to develop insights into why health organisations engage in improvement work, how impacts are conceptualised, and 'what works' in delivering these impacts. Specifically, RBV theorises that the mix and use of resources across different organisations may explain differences in performance. Whether improvement work influences these resources is unclear. METHODS AND ANALYSIS: Case study research will be conducted across health organisations participating in four approaches to improvement, including: a national improvement programme; a multiorganisational partnership around implementation; an organisational strategy for quality improvement; and a coproduction project designed to enhance the experience of a clinical service from the perspective of patients. Data will comprise in-depth interviews with key informants, observation of key events and documents; analysed within and then across cases. Adopting a realist perspective, the core tenets of RBV will be evaluated as a programme theory, focusing on the interplay between organisational conditions and behavioural or resource responses that are reported through engagement in improvement. ETHICS AND DISSEMINATION: The study has been approved by Bangor University Ethics Committee. The investigation will not judge the relative merits of different approaches to healthcare quality improvement. Rather, we will develop unique insights into the organisational consequences, and dependencies of quality improvement, providing an opportunity to add to the explanatory potential of RBV in this and other contexts. In addition to scientific and lay reports of the study findings, research outputs will include a framework for constructing the economic impacts of quality improvement and practical guidance for health service managers that maximises the impacts of investment in quality improvement.

Developing the Diagnostic Adherence to Medication Scale (the DAMS) for use in clinical practice.

BACKGROUND: There is a need for an adherence measure, to monitor adherence services in clinical practice, which can distinguish between different types of non-adherence and measure changes over time. In order to be inclusive of all patients it needs to be able to be administered to both patients and carers and to be suitable for patients taking multiple medications for a range of clinical conditions. A systematic review found that no adherence measure met all these criteria. We therefore wished to develop a theory based adherence scale (the DAMS) and establish its content, face and preliminary construct validity in a primary care population. METHODS: The DAMS (consisting of 6 questions) was developed from theory by a multidisciplinary team and the questions were initially tested in small patient populations. Further to this, patients were recruited when attending a General Practice and interviewed using the DAMS and two other validated self-reported adherence measures, the Morisky-8 and Lu questionnaires. A semi-structured interview was used to explore acceptability and reasons for differences in responses between the DAMS and the other measures. Descriptive data were generated and Spearman rank correlation tests were used to identify associations between the DAMS and the other adherence measures. RESULTS: One hundred patients completed the DAMS in an average of 1 minute 28 seconds and reported finding it straightforward to complete. An adherence score could not be calculated for the 4(4%) patients only taking 'when required' medication. Thirty six(37.5%) of the remaining patients reported some non-adherence. Adherence ratings of the DAMS were significantly associated with levels of self reported adherence on all other measures Spearman Rho 0.348-0.719, (p < 0.01). Differences in trends could generally be explained by qualitative data. CONCLUSION: The DAMS has been developed for routine monitoring of adherence in clinical practice. It was acceptable to patients taking single or multiple medication and valid when tested against other adherence measures. However, 'when required' medication needs to be excluded. Further tests of the DAMS against objective measures such as MEMS are in progress and reliability needs to be established. Further investigation of the carers' version of the DAMS is required.

Suitability of measures of self-reported medication adherence for routine clinical use: a systematic review.

BACKGROUND: There is a recognised need to build primary care medication adherence services which are tailored to patients' needs. Continuous quality improvement of such services requires a regular working method of measuring adherence in order to monitor effectiveness. Self report has been considered the method of choice for clinical use; it is cheap, relatively unobtrusive and able to distinguish between intentional and unintentional non-adherence, which have different underlying causes and therefore require different interventions. A self report adherence measure used in routine clinical practice would ideally be brief, acceptable to patients, valid, reliable, have the ability to distinguish between different types of non-adherence and be able to be completed by or in conjunction with carers where necessary. METHODS: We systematically reviewed the literature in order to identify self report adherence measures currently available which are suitable for primary care and evaluate the extent to which they met the criteria described above. We searched the databases Medline, Embase, International Pharmaceutical Abstracts, Pharmline, CINAHL, PsycINFO and HaPI to identify studies reporting the development, validation or reliability of generic adherence measures. One reviewer screened all abstracts and assessed all relevant full text articles obtained and a second reviewer screened/assessed 10% to check reliability. RESULTS: Fifty eight measures were identified. While validation data were presented in support of the vast majority of self reported measures (54/58), data for a relatively small number of measures was presented for reliability (16/58) and time to complete (3/58). Few were designed to have the ability to be completed by or in conjunction with carers and few were able to distinguish between different types of non-adherence, which limited their ability be used effectively in the continuous improvement of targeted adherence enhancing interventions. The data available suggested that patients find it easier to estimate general adherence than to report a specific number of doses missed. Visual analogue scales can be easier for patients than other types of scale but are not suitable for telephone administration. CONCLUSIONS: There is a need for a measure which can be used in the routine continual quality monitoring of adherence services.

Quality of medication use in primary care--mapping the problem, working to a solution: a systematic review of the literature.

BACKGROUND: The UK, USA and the World Health Organization have identified improved patient safety in healthcare as a priority. Medication error has been identified as one of the most frequent forms of medical error and is associated with significant medical harm. Errors are the result of the systems that produce them. In industrial settings, a range of systematic techniques have been designed to reduce error and waste. The first stage of these processes is to map out the whole system and its reliability at each stage. However, to date, studies of medication error and solutions have concentrated on individual parts of the whole system. In this paper we wished to conduct a systematic review of the literature, in order to map out the medication system with its associated errors and failures in quality, to assess the strength of the evidence and to use approaches from quality management to identify ways in which the system could be made safer. METHODS: We mapped out the medicines management system in primary care in the UK. We conducted a systematic literature review in order to refine our map of the system and to establish the quality of the research and reliability of the system. RESULTS: The map demonstrated that the proportion of errors in the management system for medicines in primary care is very high. Several stages of the process had error rates of 50% or more: repeat prescribing reviews, interface prescribing and communication and patient adherence. When including the efficacy of the medicine in the system, the available evidence suggested that only between 4% and 21% of patients achieved the optimum benefit from their medication. Whilst there were some limitations in the evidence base, including the error rate measurement and the sampling strategies employed, there was sufficient information to indicate the ways in which the system could be improved, using management approaches. The first step to improving the overall quality would be routine monitoring of adherence, clinical effectiveness and hospital admissions. CONCLUSION: By adopting the whole system approach from a management perspective we have found where failures in quality occur in medication use in primary care in the UK, and where weaknesses occur in the associated evidence base. Quality management approaches have allowed us to develop a coherent change and research agenda in order to tackle these, so far, fairly intractable problems.