Development and first application of an audit system for screening programs based on the PRECEDE-PROCEED model: an experience with breast cancer screening in the region of Lombardy (Italy).

BACKGROUND: High participation and performance are necessary conditions for the effectiveness of breast cancer screening programs. Here we describe the process to define and test a planning software application and an audit cycle based on the PRECEDE-PROCEED model applied to improving breast cancer screening. We developed a planning software application following the phases of the PRECEDE-PROCEED model. The application was co-designed by local cancer screening program coordinators. An audit model was also developed. The revised application and the audit model were tested by all the coordinators of 15 breast cancer screening programs in the region of Lombardy in a 3-day workshop. The project plans produced using the application were compared with those produced in the previous year for clarity and completeness. RESULTS: The 9 phases of the PRECEDE-PROCEED model were adapted to screening as follows: 1) identification of program goals (i.e., participation, sensitivity, false positive); 2) epidemiological issues; 3) best practices analysis; 4) evidence-based actions to be implemented in the screening center and the relationships with partners and stakeholders; 5) priority setting and identification of solutions for each issue; 6) definition of indicators; 7) monitoring; 8) evaluation; 9) impact assessment. The application automatically generated reports for each phase. During the audit cycle, the regional health authority negotiated the targets to be reached with local authorities and collected the improvement plans generated by the application. The plans produced after the application was adopted were more standardized and had clearer indicators for monitoring and evaluation compared to those produced in the previous year. CONCLUSIONS: The software application helps standardize criteria for planning interventions to improve screening programs and facilitates the implementation of the audit cycle.

[Audit and Feedback approach in the CoViD-19 emergency: adaptation of the EASY-NET project in the AOU Città della Salute e della Scienza of Turin.] / Audit e Feedback nell'emergenza CoViD-19: adattamento del progetto EASY-NET nell'AOU Città della Salute e della Scienza di Torino.

CoViD-19 pandemic heavily impacted most on-going research activities, causing delays and need of re-programming. EASY-NET (NET-2016-02364191) is a network project, started in April 2019, co-funded by the Italian Ministry of Health and the participating regions. Within the general project, centred on the evaluation of Audit and Feedback (A&F) strategies in improving quality and equity in different health care contexts, the Piedmont region is responsible of the work package 3 (WP3) on specific oncology pathways and procedures. After a thorough evaluation of the impact of the CoViD-19 emergency on the WP3 activities, at the beginning of March 2020, the decision was to continue, with some adaptations, the audits already started, and to delay those in the early planning phase. The provisional availability of part of the time-persons involved in EASY-NET on one side, and the urgency of acquiring data on the management of the large number of CoViD-19 patients admitted to the study coordinator hospital on the other side, determined the personnel responsible of the WP3, in accordance with the hospital management, to invest these resources in monitoring the CoViD-19 hospitalized patients with both A&F activity and research objectives. Besides periodic reports, a web site, with restricted access to the involved health care personnel, was developed to allow a direct and timely consultation of graphics describing the flow of the patients, their management, and outcomes. This experience was made possible thanks to a favourable combination of different factors: the presence within the hospital of a group of experienced epidemiologists in A&F, the availability of extra resources, the strong support and collaboration by the hospital management and the readiness for authorisation by the Ethics Committee. We underline the need to provide a certain degree of flexibility in the long-term projects funded by the Ministry of Health, the extraordinary adaptability of the A&F approach also to emergency situations and the possibility of combining audit activities and research objectives in the same project.

Impetus to change: a multi-site qualitative exploration of the national audit of dementia.

BACKGROUND: National audit is a key strategy used to improve care for patients with dementia. Audit and feedback has been shown to be effective, but with variation in how much it improves care. Both evidence and theory identify active ingredients associated with effectiveness of audit and feedback. It is unclear to what extent national audit is consistent with evidence- and theory-based audit and feedback best practice. METHODS: We explored how the national audit of dementia is undertaken in order to identify opportunities to enhance its impact upon the improvement of care for people with dementia. We undertook a multi-method qualitative exploration of the national audit of dementia at six hospitals within four diverse English National Health Service organisations. Inductive framework analysis of 32 semi-structured interviews, documentary analysis (n = 39) and 44 h of observations (n = 36) was undertaken. Findings were presented iteratively to a stakeholder group until a stable description of the audit and feedback process was produced. RESULTS: Each organisation invested considerable resources in the audit. The audit results were dependent upon the interpretation by case note reviewers who extracted the data. The national report was read by a small number of people in each organisation, who translated it into an internal report and action plan. The internal report was presented at specialty- and organisation-level committees. The internal report did not include information that was important to how committee members collectively decided whether and how to improve performance. Participants reported that the national audit findings may not reach clinicians who were not part of the specialty or organisation-level committees. CONCLUSIONS: There is considerable organisational commitment to the national audit of dementia. We describe potential evidence- and theory-informed enhancements to the enactment of the audit to improve the local response to performance feedback in the national audit. The enhancements relate to the content and delivery of the feedback from the national audit provider, support for the clinicians leading the organisational response to the feedback, and the feedback provided within the organisation.

50 Years of the IAEA/WHO postal dose audit programme for radiotherapy: what can we learn from 13756 results?

Background: The IAEA/WHO postal dose audit programme has been operating since 1969 with the aim of improving the accuracy and consistency of dosimetry in radiotherapy in low-income and middle-income countries world-wide. This study summarises the 50 years' experience of audits and explores the quality of reference dosimetry in participating radiotherapy centres throughout the years.Material and methods: During the IAEA/WHO postal audits the dose determined from the mailed dosimeter is compared with that stated by the participant. Agreement to within ±5% is regarded acceptable whilst deviations outside ±5% limits trigger follow-up actions. Of particular interest in this study was the dependence of clinical dosimetry quality on factors related to the centre infrastructure and expertise in dosimetry of its staff.Results: The IAEA/WHO dose audit programme noted great increase in the overall percentage of acceptable results from about 50% in its early years to 99% at present, although there is some variability of results amongst participating countries. Whereas results for younger radiotherapy machines show the agreement rate between the measured and the stated doses well above 90%, for those over 20 years old the rate dropped to <80%. Linac dosimetry was always better than 60Co dosimetry and multi-machine centres generally performed better than single machine centres equipped with cobalt alone. Second and subsequent participation in audits showed higher quality dosimetry than the first participation. The implementation of modern dosimetry protocols resulted in more accurate dosimetry than the use of the older protocols.Conclusions: Over the 50 years that the IAEA has accumulated dosimetry audit data, practices in radiotherapy centres have significantly improved. Higher quality dosimetry confirmed in audits is generally associated with better infrastructure and adequate dosimetry expertise of medical physicists in participating centres.

An IAEA survey of radiotherapy practice including quality assurance extent and depth.

Background: The IAEA recommends a quality assurance program in radiotherapy to ensure safe and effective treatments. In this study, radiotherapy departments were surveyed on their current practice including the extent and depth of quality assurance activities.Methods: Radiotherapy departments were voluntarily surveyed in three stages, firstly, in basic facility information, secondly, in quality assurance activities and treatment techniques, and thirdly, in a snapshot of quality assurance, departmental and treatment activities.Results: The IAEA received completed surveys from 381 radiotherapy departments throughout the world with 100 radiotherapy departments completing all three surveys. Dominant patterns were found in linac-based radiotherapy with access to treatment planning systems for 3D-CRT and 3D imaging. Staffing levels for major staff groups were on average in the range recommended by the IAEA. The modal patient workload per EBRT unit was as expected in the range of 21-30 patients per day, however significant instances of high workload (more than 50 patients per day per treatment unit) were reported. Staffing levels were found to correlate with amount of treatment equipment and patient workload. In a self-assessment of quality assurance performance, most radiotherapy departments reported that they would perform at least 60% of the quality assurance activities itemized in the second survey, with particular strength in equipment quality control. In a snapshot survey of quality assurance performance, again equipment quality control practice was well developed, particularly for the treatment equipment.Conclusions: The IAEA surveys provide a snapshot of current radiotherapy practice including quality assurance activities.

First-line managers' experience of the use of audit and feedback cycle in specialist mental health care: A qualitative case study.

BACKGROUND: Audit and feedback was the main strategy to facilitate implementation of The National Guideline for Persons with Concurrent Substance Use Disorders and Mental Disorders in specialist mental health services. Studies have shown that leadership support contributes to implementation success. The aim of the study was to explore how first-line managers in a District Psychiatric Centre experienced using audit and feedback cycle. METHOD: The study had a qualitative case study design with individual interviews with five first-line managers from a District Psychiatric Centre in Norway. Qualitative content analysis was conducted. RESULTS: First-line managers were positive to contribute to better practice for the patient group and apply available tools. Four themes emerged: 1) Lack of endurance, where first-line managers saw their role as being process leaders, but failed to persist, 2) Lack of support in the process, where first-line managers called for a stronger organisational focus 3) Lack of ownership, where first-line managers felt the process was imposed on them, and 4) Lack of leader autonomy, where first-line managers seemed insecure about their role between professional leadership and own management. CONCLUSION: First-line managers were not sufficiently experienced or equipped to solve the implementation process satisfactorily. They were torn between different commitments, without the autonomy to act as process drivers or facilitators, and without taking the necessary leadership role. The potential impact of the use of audit and feedback may thus not be fully realized, in part, because of limited organisational support and capacity to respond effectively.

Hospital staff participation in a national hip fracture audit: facilitators and barriers.

To ensure meaningful results in a clinical audit, as many hospitals as possible should participate. To optimise participation, the data collection process should either be performed by additional staff or be automated. Active participation may be promoted by offering relevant external parties insight into the actual quality of care. PURPOSE: The aim of the study was to identify which facilitators and barriers experienced by hospital staff are associated with participation in the ongoing nationwide multidisciplinary Dutch Hip Fracture Audit (DHFA). METHODS: A survey including questions about the respondents' characteristics, hospital level of participation and factors of influence on DHFA participation was sent to hip fracture surgeons. The factors were based on results of semi-structured interviews held with hospital staff involved in hip fracture care. Univariable and multivariable logistic regression analyses were used to establish which respondent characteristics and factors were associated with participation and active participation (≥ 80% of patients registered) in the DHFA. Factors significantly increasing the (active) participation in the DHFA were classified as facilitators, and factors significantly decreasing the (active) participation in the DHFA as barriers. RESULTS: One hundred nine surgeons filled out the questionnaire. The factors most agreed on were availability of staffing capacity for data collection and automated data import. A lower intention to participate was associated with being an academic surgeon (odds ratio, 0.15; 95% confidence interval, 0.04-0.52) and an orthopaedic surgeon (odds ratio, 0.30; 95% confidence interval, 0.10-0.90). Data sharing with relevant external parties was associated with active participation (odds ratio, 3.19; 95% confidence interval, 1.14-8.95). CONCLUSIONS: To improve participation in a nationwide clinical audit, it seems that the data collection should either be performed by additional staff or be automated. Active participation is facilitated if audit data is made available to other parties, such as insurers, healthcare authorities or policymakers.

Facilitating action planning within audit and feedback interventions: a mixed-methods process evaluation of an action implementation toolbox in intensive care.

BACKGROUND: Audit and feedback (A&F) is more effective if it facilitates action planning, but little is known about how best to do this. We developed an electronic A&F intervention with an action implementation toolbox to improve pain management in intensive care units (ICUs); the toolbox contained suggested actions for improvement. A head-to-head randomised trial demonstrated that the toolbox moderately increased the intervention's effectiveness when compared with A&F only. OBJECTIVE: To understand the mechanisms through which A&F with action implementation toolbox facilitates action planning by ICUs to increase A&F effectiveness. METHODS: We extracted all individual actions from action plans developed by ICUs that received A&F with (n = 10) and without (n = 11) toolbox for 6 months and classified them using Clinical Performance Feedback Intervention Theory. We held semi-structured interviews with participants during the trial. We compared the number and type of planned and completed actions between study groups and explored barriers and facilitators to effective action planning. RESULTS: ICUs with toolbox planned more actions directly aimed at improving practice (p = 0.037) and targeted a wider range of practice determinants compared to ICUs without toolbox. ICUs with toolbox also completed more actions during the study period, but not significantly (p = 0.142). ICUs without toolbox reported more difficulties in identifying what actions they could take. Regardless of the toolbox, all ICUs still experienced barriers relating to the feedback (low controllability, accuracy) and organisational context (competing priorities, resources, cost). CONCLUSIONS: The toolbox helped health professionals to broaden their mindset about actions they could take to change clinical practice. Without the toolbox, professionals tended to focus more on feedback verification and exploring solutions without developing intentions for actual change. All feedback recipients experienced organisational barriers that inhibited eventual completion of actions. TRIAL REGISTRATION: ClinicalTrials.gov, NCT02922101 . Registered on 26 September 2016.

Validation of data submitted by the treating surgeon in the Victorian Audit of Surgical Mortality.

BACKGROUND: The Victorian Audit of Surgical Mortality (VASM) seeks to peer review all deaths associated with surgical care in Victoria, Australia. The effectiveness of the VASM as an educational and quality improvement tool is dependent on the accuracy of source data it receives. We aimed to examine the accuracy and quality of source data provided by the treating surgeon for peer review, and the inter-rater concordance level between the external validator findings and the treating surgeon. METHODS: Of the 629 cases that completed the VASM audit second-line peer review process over a 4-year period (from 1 July 2012 to 30 June 2016), a total of 32 (5%) were randomly selected for the external validation process. The blinded external validator was impartial to the VASM audit, and was provided only de-identified patient medical records. The analysis for the checked and validated data points and their concordance was determined using Gwet's agreement coefficient, which provides a stable inter-rater reliability coefficient not affected by prevalence and marginal probability. RESULTS: The inter-rater concordance analysis suggested that there is a high level of agreement (82.9% overall) between the treating surgeon and external validator. The use of thromboembolism deterrent stockings was the only variable where agreement was poor (52.4%) with a Gwet score of 0.10 (-0.40 to 0.60). CONCLUSION: The inter-rater concordance analysis results support the validity of the VASM process, which is dependent on the accuracy of data submitted by the treating surgeon.

Development of a quality improvement audit tool for the primary care of children with chronic wet cough using a modified Delphi consensus approach.

AIM: In the absence of quality indicators (QIs) for the management of chronic wet cough, our study's aim was to determine whether consensus on QIs reflecting good primary health care, prior to referral for children with chronic wet cough, can be achieved. METHODS: A questionnaire consisting of 10 QIs was developed by a clinical working group based on current evidence and guidelines on the management of chronic wet cough in children. Each indicator reflected the quality of care provided to children with chronic wet cough in primary care prior to referral. A modified Delphi consensus questionnaire was undertaken involving expert paediatric respiratory clinicians and general paediatricians who graded the importance of each indicator for the purposes above. We a priori defined that consensus was considered achieved if >75% agreed on the indicator. RESULTS: Twenty-two specialists (from Brisbane, Melbourne, Perth and Canberra) participated in the survey. The cumulative number of years of their respiratory experience was 324 and that of general clinical practice was 504. Consensus was achieved in all 10 QIs, with 6 reaching 100% agreement. Mean agreement for the 10 items was 97%. CONCLUSION: As complete consensus was achieved on these QIs, it can be used as a provisional clinical audit tool and can guide the development of a robust audit tool for primary care clinical practice to assist with quality improvement initiatives.

Developing and Testing a Chart Abstraction Tool for ICU Quality Measurement.

Quality measures are increasingly used to measure the performance of providers, hospitals, and health care systems. Intensive care units (ICUs) are an important clinical area in hospitals, given that they generate high costs and present high risks to patients. Yet, currently, few valid and clinically significant ICU-specific outcome measures are reported nationally. This study reports on the creation and evaluation of new abstraction tools that evaluate ICU patients for the following clinically important outcomes: central line-associated bloodstream infection, methicillin-resistant Staphylococcus aureus, gastrointestinal bleed, and pressure ulcer. To allow ICUs and institutions to compare their outcomes, the tools include risk-adjustment variables that can be abstracted from the chart.

The Calgary Audit and Feedback Framework: a practical, evidence-informed approach for the design and implementation of socially constructed learning interventions using audit and group feedback.

BACKGROUND: Audit and feedback interventions may be strengthened using social interaction. The Calgary office of the Alberta Physician Learning Program (CPLP) developed a process for audit and group feedback for physicians. This paper extends previous work in which we developed a conceptual model of physician responses to audit and group feedback based on a qualitative analysis of six audit and group feedback sessions. The present study explored the mediating factors for successfully engaging physician groups in change planning through audit and group feedback. METHODS: To understand why some groups were more interactive than others, we completed a comparative case analysis of the six audit and group feedback projects from the prior study. We used framework analysis to build the case studies, triangulated our observations across data sources to validate findings, compared the case studies for similarities and differences that influenced social interaction (mediating factors), and thematically categorized mediating factors into an organizing framework. RESULTS: Mediating factors for socially interactive AGFS were a pre-existing relationship between the program team and the physician group, projects addressing important, actionable questions, easily interpretable data visualization in the reports, and facilitation of the groups that included reflective questioning. When these factors were in place (cases 1, 2A, 3), the audit and group feedback sessions were dynamic, with physicians sharing and comparing practices, and raising change cues (such as declaring commitments to de-prescribing, planning educational interventions, and improving documentation). In cases 2C-D, the mediating factors were less well established and in these cases, the sessions showed little physician reflection or change planning. We organized the mediating factors into a framework linking the factors for successful sessions to the conceptual model of physician behaviors which these mediating factors drive. CONCLUSIONS: We propose the Calgary Audit and Feedback Framework as a practical tool to help foster socially constructed learning in audit and group feedback sessions. Ensuring that the four factors, relationship, question choice, data visualization, and facilitation, are considered for design and implementation of audit and group feedback will help physicians move from reactions to their data towards planning for change.

Understanding changes in the standardized antimicrobial administration ratio for total antimicrobial use after implementation of prospective audit and feedback.

In this single-center study, the standardized antimicrobial administration ratio (SAAR) for total antimicrobial use decreased in response to a stewardship intervention. Antimicrobial prescribing and clinical outcomes were stable or improved during the period of lower SAARs. Our findings suggest that SAAR values of ~0.8 can be safely achieved.

Incidence and characteristics of ventilator-associated pneumonia in a regional non-tertiary Australian intensive care unit: protocol for a retrospective clinical audit study.

INTRODUCTION: Ventilator-associated pneumonia (VAP) is a medical complication associated with prolonged mechanical ventilation. Most studies looking at VAP originate from major, tertiary intensive care units (ICUs). Our understanding of VAP in regional hospitals is limited. Given that patient characteristics often differ between metropolitan and regional centres, it is important to investigate VAP in a regional non-tertiary ICU. This project will establish and report the incidence, case characteristics and outcomes including mortality and length of stay related to VAP in a regional non-tertiary Australian ICU. Furthermore, it will compare the incidence of VAP in accordance with consultant diagnosed cases in the medical record, and by a post hoc screening of all cases against a list of previously published diagnostic criteria. METHODS AND ANALYSIS: This retrospective clinical audit study will screen medical records from the period 1 January 2013 to 31 December 2016. All cases requiring mechanical ventilation for ≥72 hours will be screened against previously reported diagnostic criteria for VAP. At the same time, their medical records will be screened for a documented diagnosis of VAP. ETHICS AND DISSEMINATION: This study has been granted ethical approval from the Central Queensland Hospital and Health Service (CQHHS) Human Research Ethics Committee (HREC/17/QCQ/11) and the Central Queensland University Human Research Ethics Committee (H17/05-102). This study will be submitted for publication in a peer-reviewed scientific journal and presented at internal workshops (within Queensland Health) and national and/or international scientific conferences.

Evaluating the process and outcomes of child death review in the Solomon Islands.

While maternal and perinatal mortality auditing has been strongly promoted by the World Health Organization (WHO), there has been very limited promotion or evaluation of child death auditing in low/middle-income settings. In 2017, a standardised child death review process was introduced in the paediatric department of the National Hospital in Honiara, Solomon Islands. We evaluated the process and outcomes of child death reviews. The child death auditing process was assessed through systematic observations made at each of the weekly meetings using the following standards for evaluation: (1) adapted WHO tools for paediatric auditing; (2) the five stages of the audit cycle; (3) published principles of paediatric audit; and (4) WHO and Solomon Islands national clinical standards of Hospital Care for Children. Thirty-three child death review meetings were conducted over 6 months, reviewing 66 neonatal and child deaths. Some areas of the process were satisfactory and other areas were identified for improvement. The latter included use of a more systematic classification of causes of death, inclusion of social risk factors and community problems in the modifiable factors and more follow-up with implementation of action plans. Areas for improvement were in communication, clinical assessment and treatment, availability of laboratory tests, antenatal clinic attendance and equipment for high dependency neonatal and paediatric care. Many of the changes recommended by audit require a quality improvement team to implement. Child death auditing can be done in resource-limited settings and yield useful information of gaps which are linked to preventable deaths; however, using the data to produce meaningful changes in practice is the greatest challenge. Audit is an iterative and evolving process that needs a structure, tools, evaluation, and needs to be embedded in the culture of a hospital as part of overall quality improvement, and requires a quality improvement team to follow-up and implement action plans.

New workflows for clinical video recordings in cleft care.

The Cleft team at Cambridge University Hospitals (CUH), Cleft.NET.East (CNE), were experiencing many technical issues with their video recording systems and processes, resulting in poor record keeping and audit performance. It became apparent that the equipment needed to be updated and replaced, with new workflows implemented to improve the quality and consistency in patients' records. Following a review carried out with the Cleft team, equipment was identified and workflows were implemented which improved the processes already in place. These changes have resulted in improved quality of records, better audit performance; time saved for key clinicians, more efficient running of clinics and improved patient experience.

Evaluation of the organisation and effectiveness of internal audits to govern patient safety in hospitals: a mixed-methods study.

OBJECTIVES: Hospital boards are legally responsible for safe healthcare. They need tools to assist them in their task of governing patient safety. Almost every Dutch hospital performs internal audits, but the effectiveness of these audits for hospital governance has never been evaluated. The aim of this study is to evaluate the organisation of internal audits and their effectiveness for hospitals boards to govern patient safety. DESIGN AND SETTING: A mixed-methods study consisting of a questionnaire regarding the organisation of internal audits among all Dutch hospitals (n=89) and interviews with stakeholders regarding the audit process and experienced effectiveness of audits within six hospitals. RESULTS: Response rate of the questionnaire was 76% and 43 interviews were held. In every responding hospital, the internal audits followed the plan-do-check-act cycle. Every hospital used interviews, document analysis and site visits as input for the internal audit. Boards stated that effective aspects of internal audits were their multidisciplinary scope, their structured and in-depth approach, the usability to monitor improvement activities and to change hospital policy and the fact that results were used in meetings with staff and boards of supervisors. The qualitative methods (interviews and site visits) used in internal audits enable the identification of soft signals such as unsafe culture or communication and collaboration problems. Reported disadvantages were the low frequency of internal audits and the absence of soft signals in the actual audit reports. CONCLUSION: This study shows that internal audits are regarded as effective for patient safety governance, as they help boards to identify patient safety problems, proactively steer patient safety and inform boards of supervisors on the status of patient safety. The description of the Dutch internal audits makes these audits replicable to other healthcare organisations in different settings, enabling hospital boards to complement their systems to govern patient safety.