The quality of care delivered to residents in long-term care in Australia: an indicator-based review of resident records (CareTrack Aged study).

BACKGROUND: This study estimated the prevalence of evidence-based care received by a population-based sample of Australian residents in long-term care (LTC) aged ≥ 65 years in 2021, measured by adherence to clinical practice guideline (CPG) recommendations. METHODS: Sixteen conditions/processes of care amendable to estimating evidence-based care at a population level were identified from prevalence data and CPGs. Candidate recommendations (n = 5609) were extracted from 139 CPGs which were converted to indicators. National experts in each condition rated the indicators via the RAND-UCLA Delphi process. For the 16 conditions, 236 evidence-based care indicators were ratified. A multi-stage sampling of LTC facilities and residents was undertaken. Trained aged-care nurses then undertook manual structured record reviews of care delivered between 1 March and 31 May 2021 (our record review period) to assess adherence with the indicators. RESULTS: Care received by 294 residents with 27,585 care encounters in 25 LTC facilities was evaluated. Residents received care for one to thirteen separate clinical conditions/processes of care (median = 10, mean = 9.7). Adherence to evidence-based care indicators was estimated at 53.2% (95% CI: 48.6, 57.7) ranging from a high of 81.3% (95% CI: 75.6, 86.3) for Bladder and Bowel to a low of 12.2% (95% CI: 1.6, 36.8) for Depression. Six conditions (skin integrity, end-of-life care, infection, sleep, medication, and depression) had less than 50% adherence with indicators. CONCLUSIONS: This is the first study of adherence to evidence-based care for people in LTC using multiple conditions and a standardised method. Vulnerable older people are not receiving evidence-based care for many physical problems, nor care to support their mental health nor for end-of-life care. The six conditions in which adherence with indicators was less than 50% could be the focus of improvement efforts.

Is it possible to make 'living' guidelines? An evaluation of the Australian Living Stroke Guidelines.

BACKGROUND: Keeping best practice guidelines up-to-date with rapidly emerging research evidence is challenging. 'Living guidelines' approaches enable continual incorporation of new research, assisting healthcare professionals to apply the latest evidence to their clinical practice. However, information about how living guidelines are developed, maintained and applied is limited. The Stroke Foundation in Australia was one of the first organisations to apply living guideline development methods for their Living Stroke Guidelines (LSGs), presenting a unique opportunity to evaluate the process and impact of this novel approach. METHODS: A mixed-methods study was conducted to understand the experience of LSGs developers and end-users. We used thematic analysis of one-on-one semi-structured interview and online survey data to determine the feasibility, acceptability, and facilitators and barriers of the LSGs. Website analytics data were also reviewed to understand usage. RESULTS: Overall, the living guidelines approach was both feasible and acceptable to developers and users. Facilitators to use included collaboration with multidisciplinary clinicians and stroke survivors or carers. Increased workload for developers, workload unpredictability, and limited information sharing, and interoperability of technological platforms were identified as barriers. Users indicated increased trust in the LSGs (69%), likelihood of following the LSGs (66%), and frequency of access (58%), compared with previous static versions. Web analytics data showed individual access by 16,517 users in 2016 rising to 53,154 users in 2020, a threefold increase. There was also a fourfold increase in unique LSG pageviews from 2016 to 2020. CONCLUSIONS: This study, the first evaluation of living guidelines, demonstrates that this approach to stroke guideline development is feasible and acceptable, that these approaches may add value to developers and users, and may increase guideline use. Future evaluations should be embedded along with guideline implementation to capture data prospectively.

Potentially burdensome care at the end-of-life for cancer decedents: a retrospective population-wide study.

BACKGROUND: Variation persists in the quality of end-of-life-care (EOLC) for people with cancer. This study aims to describe the characteristics of, and examine factors associated with, indicators of potentially burdensome care provided in hospital, and use of hospital services in the last 12 months of life for people who had a death from cancer. METHOD: A population-based retrospective cohort study of people aged ≥ 20 years who died with a cancer-related cause of death during 2014-2019 in New South Wales, Australia using linked hospital, cancer registry and mortality records. Ten indicators of potentially burdensome care were examined. Multinominal logistic regression examined predictors of a composite measure of potentially burdensome care, consisting of > 1 ED presentation or > 1 hospital admission or ≥ 1 ICU admission within 30 days of death, or died in acute care. RESULTS: Of the 80,005 cancer-related deaths, 86.9% were hospitalised in the 12 months prior to death. Fifteen percent had > 1 ED presentation, 9.9% had > 1 hospital admission, 8.6% spent ≥ 14 days in hospital, 3.6% had ≥ 1 intensive care unit admission, and 1.2% received mechanical ventilation on ≥ 1 occasion in the last 30 days of life. Seventeen percent died in acute care. The potentially burdensome care composite measure identified 20.0% had 1 indicator, and 10.9% had ≥ 2 indicators of potentially burdensome care. Compared to having no indicators of potentially burdensome care, people who smoked, lived in rural areas, were most socially economically disadvantaged, and had their last admission in a private hospital were more likely to experience potentially burdensome care. Older people (≥ 55 years), females, people with 1 or ≥ 2 Charlson comorbidities, people with neurological cancers, and people who died in 2018-2019 were less likely to experience potentially burdensome care. Compared to people with head and neck cancer, people with all cancer types (except breast and neurological) were more likely to experience ≥ 2 indicators of potentially burdensome care versus none. CONCLUSION: This study shows the challenge of delivering health services at end-of-life. Opportunities to address potentially burdensome EOLC could involve taking a person-centric approach to integrate oncology and palliative care around individual needs and preferences.

Measuring the quality of skin cancer management in primary care: A scoping review.

Skin cancer is a growing global problem and a significant health and economic burden. Despite the practical necessity for skin cancer to be managed in primary care settings, little is known about how quality of care is or should be measured in this setting. This scoping review aimed to capture the breadth and range of contemporary evidence related to the measurement of quality in skin cancer management in primary care settings. Six databases were searched for relevant texts reporting on quality measurement in primary care skin cancer management. Data from 46 texts published since 2011 were extracted, and quality measures were catalogued according to the three domains of the Donabedian model of healthcare quality (structure, process and outcome). Quality measures within each domain were inductively analysed into 13 key emergent groups. These represented what were deemed to be the most relevant components of skin cancer management as related to structure, process or outcomes measurement. Four groups related to the structural elements of care provision (e.g. diagnostic tools and equipment), five related to the process of care delivery (e.g. diagnostic processes) and four related to the outcomes of care (e.g. poor treatment outcomes). A broad range of quality measures have been documented, based predominantly on articles using retrospective cohort designs; systematic reviews and randomised controlled trials were limited.

Nasal High-Flow Therapy for Newborn Infants in Special Care Nurseries.

BACKGROUND: Nasal high-flow therapy is an alternative to nasal continuous positive airway pressure (CPAP) as a means of respiratory support for newborn infants. The efficacy of high-flow therapy in nontertiary special care nurseries is unknown. METHODS: We performed a multicenter, randomized, noninferiority trial involving newborn infants (<24 hours of age; gestational age, ≥31 weeks) in special care nurseries in Australia. Newborn infants with respiratory distress and a birth weight of at least 1200 g were assigned to treatment with either high-flow therapy or CPAP. The primary outcome was treatment failure within 72 hours after randomization. Infants in whom high-flow therapy failed could receive CPAP. Noninferiority was determined by calculating the absolute difference in the risk of the primary outcome, with a noninferiority margin of 10 percentage points. RESULTS: A total of 754 infants (mean gestational age, 36.9 weeks, and mean birth weight, 2909 g) were included in the primary intention-to-treat analysis. Treatment failure occurred in 78 of 381 infants (20.5%) in the high-flow group and in 38 of 373 infants (10.2%) in the CPAP group (risk difference, 10.3 percentage points; 95% confidence interval [CI], 5.2 to 15.4). In a secondary per-protocol analysis, treatment failure occurred in 49 of 339 infants (14.5%) in the high-flow group and in 27 of 338 infants (8.0%) in the CPAP group (risk difference, 6.5 percentage points; 95% CI, 1.7 to 11.2). The incidences of mechanical ventilation, transfer to a tertiary neonatal intensive care unit, and adverse events did not differ significantly between the groups. CONCLUSIONS: Nasal high-flow therapy was not shown to be noninferior to CPAP and resulted in a significantly higher incidence of treatment failure than CPAP when used in nontertiary special care nurseries as early respiratory support for newborn infants with respiratory distress. (Funded by the Australian National Health and Medical Research Council and Monash University; HUNTER Australian and New Zealand Clinical Trials Registry number, ACTRN12614001203640.).

The complexities, coordination, culture and capacities that characterise the delivery of oncology services in the common areas of ambulatory settings.

BACKGROUND: Relatively little is understood about real-world provision of oncology care in ambulatory outpatient clinics (OPCs). This study aimed to: 1) develop an understanding of behaviours and practices inherent in the delivery of cancer services in OPC common areas by characterising the organisation and implementation of this care; and 2) identify barriers to, and facilitators of, the delivery of this care in OPC common areas. METHODS: A purpose-designed ethnographic study was employed in four public hospital OPCs. Informal field scoping activities were followed by in-situ observations, key informant interviews and document review. A view of OPCs as complex adaptive systems was used as a scaffold for the data collection and interpretation, with the intent of understanding 'work as done'. Data were analysed using an adapted "Qualitative Rapid Appraisal, Rigorous Analysis" approach. RESULTS: Field observations were conducted over 135 h, interviews over 6.5 h and documents were reviewed. Analysis found six themes. Staff working in OPCs see themselves as part of small local teams and as part of a broader multidisciplinary care team. Professional role boundaries could be unclear in practice, as duties expanded to meet demand or to stop patients "falling through the cracks." Formal care processes in OPCs were supported by relationships, social capital and informal, but invaluable, institutional expertise. Features of the clinic layout, such as the proximity of departments, affected professional interactions. Staff were aware of inter- and intra-service communication difficulties and employed strategies to minimise negative impacts on patients. We found that complexity, coordination, culture and capacity underpin the themes that characterise this care provision. CONCLUSIONS: The study advances understanding of how multidisciplinary care is delivered in ambulatory settings and the factors which promote or inhibit effective care practice. Time pressures, communication challenges and competing priorities can pose barriers to care delivery. OPC care is facilitated by: self-organisation of participants; professional acumen; institutional knowledge; social ties and relationships between and within professional groups; and commitment to patient-centred care. An understanding of the realities of 'work-as-done' may help OPCs to sustain high-quality care in the face of escalating service demand.

Designing clinical indicators for common residential aged care conditions and processes of care: the CareTrack Aged development and validation study.

BACKGROUND: People who live in aged care homes have high rates of illness and frailty. Providing evidence-based care to this population is vital to ensure the highest possible quality of life. OBJECTIVE: In this study (CareTrack Aged, CT Aged), we aimed to develop a comprehensive set of clinical indicators for guideline-adherent, appropriate care of commonly managed conditions and processes in aged care. METHODS: Indicators were formulated from recommendations found through systematic searches of Australian and international clinical practice guidelines (CPGs). Experts reviewed the indicators using a multiround modified Delphi process to develop a consensus on what constitutes appropriate care. RESULTS: From 139 CPGs, 5609 recommendations were used to draft 630 indicators. Clinical experts (n = 41) reviewed the indicators over two rounds. A final set of 236 indicators resulted, mapped to 16 conditions and processes of care. The conditions and processes were admission assessment; bladder and bowel problems; cognitive impairment; depression; dysphagia and aspiration; end of life/palliative care; hearing and vision; infection; medication; mobility and falls; nutrition and hydration; oral and dental care; pain; restraint use; skin integrity and sleep. CONCLUSIONS: The suite of CT Aged clinical indicators can be used for research and assessment of the quality of care in individual facilities and across organizations to guide improvement and to supplement regulation or accreditation of the aged care sector. They are a step forward for Australian and international aged care sectors, helping to improve transparency so that the level of care delivered to aged care consumers can be rigorously monitored and continuously improved.

Safety climate, leadership and patient views associated with hip fracture care quality and clinician perceptions of hip fracture care performance.

BACKGROUND: Hip fracture is a major public health concern for older adults, requiring surgical treatment for patients presenting at hospitals across Australia. Although guidelines have been developed to drive appropriate care of hip fracture patients in hospitals, data on health outcomes suggest these are not well-followed. OBJECTIVE: This study aims to examine whether clinician measures of safety, teamwork and leadership, and patient perceptions of care are associated with key indicators of hip fracture care and the extent to which there is agreement between clinician perceptions of hip fracture care performance and actual hospital performance of hip fracture care. METHODS: Retrospective analysis was performed on a series of questionnaires used to assess hospital department- and patient-level measures from the Deepening our Understanding of Quality in Australia study. Data were analysed from 32 public hospitals that encompassed 23 leading hip fracture clinicians, 716 patient medical records and 857 patients from orthopaedic public hospital wards. RESULTS: Aggregated across all hospitals, only 5 of 12 of the key hip fracture indicators had ≥50% adherence. Adherence to indicators requiring actions to be performed within a recommended time period was poor (7.2-25.6%). No Patient Measure of Safety or clinician-based measures of teamwork, safety climate or leadership were associated with adherence to key indicators of hip fracture care. Simple proportionate agreement between clinician perceptions and actual hospital performance was generally strong, but few agreement coefficients were compelling. CONCLUSION: The development of strong quality management processes requires ongoing effort. The findings of this study provide important insights into the relationship between hospital care and outcomes for hip fracture patients and could drive the design of targeted interventions for improved quality assurance of hip fracture care.

Predicting Nasal High-Flow Treatment Success in Newborn Infants with Respiratory Distress Cared for in Nontertiary Hospitals.

OBJECTIVE: To evaluate demographic and clinical variables as predictors of nasal high-flow treatment success in newborn infants with respiratory distress cared for in Australian nontertiary special care nurseries. STUDY DESIGN: A secondary analysis of the HUNTER trial, a multicenter, randomized controlled trial evaluating nasal high-flow as primary respiratory support for newborn infants with respiratory distress who were born ≥31 weeks of gestation and with birth weight ≥1200 g, and cared for in Australian nontertiary special care nurseries. Treatment success within 72 hours after randomization to nasal high-flow was determined using objective criteria. Univariable screening and multivariable analysis was used to determine predictors of nasal high-flow treatment success. RESULTS: Infants (n = 363) randomized to nasal high-flow in HUNTER were included in the analysis; the mean gestational age was 36.9 ± 2.7 weeks and birth weight 2928 ± 782 g. Of these infants, 290 (80%) experienced nasal high-flow treatment success. On multivariable analysis, nasal high-flow treatment success was predicted by higher gestational age and lower fraction of inspired oxygen immediately before randomization, but not strongly. The final model was found to have an area under the curve of 0.65, which after adjustment for optimism was found to be 0.63 (95% CI, 0.57-0.70). CONCLUSIONS: Gestational age and supplemental oxygen requirement may be used to guide decisions regarding the most appropriate initial respiratory support for newborn infants in nontertiary special care nurseries. Further prospective research is required to better identify which infants are most likely to be successfully treated with nasal high-flow. TRIAL REGISTRATION: ACTRN12614001203640.

Assessing appropriateness of paediatric asthma management: A population-based sample survey.

BACKGROUND AND OBJECTIVE: We conducted a comprehensive assessment of guideline adherence in paediatric asthma care, including inpatient and ambulatory services, in Australia. METHODS: National and international clinical practice guidelines (CPG) relating to asthma in children were searched and 39 medical record audit indicator questions were developed. Retrospective medical record review was conducted across hospital inpatient admissions, emergency department (ED) presentations, general practice (GP) and paediatrician consultations in three Australian states for children aged ≤15 years receiving care in 2012 and 2013. Eligibility of, and adherence to, indicators was assessed from medical records by nine experienced and purpose-trained paediatric nurses (surveyors). RESULTS: Surveyors conducted 18 453 asthma indicator assessments across 1600 visits for 881 children in 129 locations. Overall, the adherence for asthma care across the 39 indicators was 58.1%, with 54.4% adherence at GP (95% CI: 46.0-62.5), 77.7% by paediatricians (95% CI: 40.5-97.0), 79.9% in ED (95% CI: 70.6-87.3) and 85.1% for inpatient care (95% CI: 76.7-91.5). For 14 acute asthma indicators, overall adherence was 56.3% (95% CI: 47.6-64.7). Lowest adherences were for recording all four types of vital signs in children aged >2 years presenting with asthma attack (15.1%, 95% CI: 8.7-23.7), and reviewing patients' compliance, inhaler technique and triggers prior to commencing a new drug therapy (20.5%, 95% CI: 10.1-34.8). CONCLUSION: The study demonstrated differences between existing care and CPG recommendations for paediatric asthma care in Australia. Evidence-based interventions to improve adherence to CPG may help to standardize quality of paediatric asthma care and reduce variation of care.

Management of fever in Australian children: a population-based sample survey.

BACKGROUND: Fever in childhood is a common acute presentation requiring clinical triage to identify the few children who have serious underlying infection. Clinical practice guidelines (CPGs) have been developed to assist clinicians with this task. This study aimed to assess the proportion of care provided in accordance with CPG recommendations for the management of fever in Australian children. METHODS: Clinical recommendations were extracted from five CPGs and formulated into 47 clinical indicators for use in auditing adherence. Indicators were categorised by phase of care: assessment, diagnosis and treatment. Patient records from children aged 0 to 15 years were sampled from general practices (GP), emergency departments (ED) and hospital admissions in randomly-selected health districts in Queensland, New South Wales and South Australia during 2012 and 2013. Paediatric nurses, trained to assess eligibility for indicator assessment and adherence, reviewed eligible medical records. Adherence was estimated by individual indicator, phase of care, age-group and setting. RESULTS: The field team conducted 14,879 eligible indicator assessments for 708 visits by 550 children with fever in 58 GP, 34 ED and 28 hospital inpatient settings. For the 33 indicators with sufficient data, adherence ranged from 14.7 to 98.1%. Estimated adherence with assessment-related indicators was 51.3% (95% CI: 48.1-54.6), 77.5% (95% CI: 65.3-87.1) for diagnostic-related indicators and 72.7% (95% CI: 65.3-79.3) for treatment-related indicators. Adherence for children < 3 months of age was 73.4% (95% CI: 58.0-85.8) and 64.7% (95% CI: 57.0-71.9) for children 3-11 months of age, both significantly higher than for children aged 4-15 years (53.5%; 95% CI: 50.0-56.9). The proportion of adherent care for children attending an ED was 77.5% (95% CI: 74.2-80.6) and 76.7% (95% CI: 71.7-81.3) for children admitted to hospital, both significantly higher than for children attending a GP (40.3%; 95% CI: 34.6-46.1). CONCLUSIONS: This study reports a wide range of adherence by clinicians to 47 indicators of best practice for the management of febrile children, sampled from urban and rural regions containing 60% of the Australian paediatric population. Documented adherence was lowest for indicators related to patient assessment, for care provided in GP settings, and for children aged 4-15 years.

Assessing the appropriateness of paediatric antibiotic overuse in Australian children: a population-based sample survey.

BACKGROUND: Infections caused by antibiotic resistant pathogens are increasing, with antibiotic overuse a key contributing factor. OBJECTIVE: The CareTrack Kids (CTK) team assessed the care of children in Australia aged 0-15 years in 2012 and 2013 to determine the proportion of care in line with clinical practice guidelines (CPGs) for 17 common conditions. This study analyses indicators relating to paediatric antibiotic overuse to identify those which should be prioritised by antimicrobial stewardship and clinical improvement programs. METHOD: A systematic search was undertaken for national and international CPGs relevant to 17 target conditions for Australian paediatric care in 2012-2013. Recommendations were screened and ratified by reviewers. The sampling frame comprised three states containing 60% of the Australian paediatric population (South Australia, New South Wales and Queensland). Multi-stage cluster sampling was used to select general practices, specialist paediatric practices, emergency departments and hospital inpatient services, and medical records within these. Medical records were reviewed by experienced paediatric nurses, trained to assess eligibility for indicator assessment and compliance with indicators. Adherence rates were estimated. RESULTS: Ten antibiotic overuse indicators were identified; three for tonsillitis and one each for seven other conditions. A total of 2621 children were assessed. Estimated adherence for indicators ranged from 13.8 to 99.5% while the overall estimate of compliance was 61.9% (95% CI: 47.8-74.7). Conditions with high levels of appropriate avoidance of antibiotics were gastroenteritis and atopic eczema without signs of infection, bronchiolitis and croup. Indicators with less than 50% adherence were asthma exacerbation in children aged > 2 years (47.1%; 95% CI: 33.4-61.1), sore throat with no other signs of tonsillitis (40.9%; 95% CI: 16.9, 68.6), acute otitis media in children aged > 12 months who were mildly unwell (13.8%; 95% CI: 5.1, 28.0), and sore throat and associated cough in children aged < 4 years (14.3%; 95% CI: 9.9, 19.7). CONCLUSION: The results of this study identify four candidate indicators (two for tonsillitis, one for otitis media and one for asthma) for monitoring by antibiotic stewardship and clinical improvement programs in ambulatory and hospital paediatric care, and intervention if needed.

Assessing the appropriateness of the management of otitis media in Australia: A population-based sample survey.

AIM: Acute otitis media (AOM) is the most common infectious disease for which antibiotics are prescribed; its management is costly and has the potential to increase the antimicrobial resistance of this infection. This study measured the levels of adherence to the clinical practice guidelines (CPGs) of AOM and otitis media with effusion (OME) management in Australian children. METHODS: We searched for national and international CPGs relating to AOM and OME in children and created 37 indicators for assessment. We reviewed medical records for adherence to these indicators in 120 locations, across one inpatient and three ambulatory health-care settings. Our review sample was obtained from three Australian states that contain 60% of the nation's children. RESULTS: We reviewed the records of 1063 children with one or more assessments of CPG adherence for otitis media. Of 22 indicators with sufficient data, estimated adherence ranged from 7.4 to 99.1%. Overuse of treatment, particularly overprescribing of antibiotics, was more common than underuse. A frequent lack of adherence with recommended care was observed for children aged between 1 and 2 years with AOM. Adherence varied by health-care setting, with emergency departments and inpatient settings more adherent to CPGs than general practices. CONCLUSIONS: Our assessment of a number of indicators in the common settings in which otitis media is treated found that guideline adherence varied widely between individual indicators. Internationally agreed standards for diagnosis and treatment, coupled with clinician education on the existence and content of CPGs and clinical decision support, are needed to improve the management of children presenting with AOM and OME.

What are the attitudes of health professionals regarding patient reported outcome measures (PROMs) in oncology practice? A mixed-method synthesis of the qualitative evidence.

BACKGROUND: The adoption of Patient Reported Outcome Measures (PROMs) in cancer care has been widely advocated, but little is known about the evidence for the implementation of PROMs in practice. Qualitative research captures the perspectives of health professionals as end-users of PROMs and can be used to inform adoption efforts. This paper presents a systematic review and synthesis of qualitative research conducted to address the question: What are the attitudes of health professionals towards PROMs in oncology, including any barriers and facilitators to the adoption of PROMS, reported in qualitative evidence? METHODS: Systematic searches of qualitative evidence were undertaken in four databases and reviewed using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. Studies published in English between 1998 and 2018, which reported qualitative findings about the attitudes of health professionals working in oncology towards PROMs were eligible. Studies were assessed using the Critical Appraisal Skills Programme's Qualitative Research Checklist. A sentiment analysis was conducted on primary text to examine the polarity (neutral, positive or negative) of health professionals' views of PROMs. Qualitative meta-synthesis was conducted using a constant comparative analysis. RESULTS: From 1227 articles after duplicates were removed, with 1014 excluded against the screening criteria, 213 full text articles remained and were assessed; 34 studies met the inclusion criteria and were included. The majority of studies were of good quality. Sentiment analysis on primary text demonstrated an overall positive polarity from the expressed opinions of health professionals. The meta-synthesis showed health professionals' attitudes in four domains: identifying patient issues and needs using PROMs; managing and addressing patient issues; the care experience; and the integration of PROMs into clinical practice. CONCLUSIONS: From the accounts of health professionals, the fit of PROMs with existing practice, how PROMs are valued, capacity to respond to PROMs and the supports in place, formed the key factors which may impede or promote adoption of PROMs in routine practice. To assist policy-makers and services involved in implementing these initiatives, further evidence is required about the relationship between PROMs data collection and corresponding clinical actions. TRIAL REGISTRATION: International Prospective Register of Systematic Reviews (PROSPERO) CRD42019119447, 6th March, 2019.

Conclusion: the road ahead: where should we go now to improve healthcare quality in acute settings?

This final article in our 12-part series articulating a suite of quality improvement studies completes our report on the Deepening our Understanding of Quality in Australia (DUQuA) program of work. Here, we bring the Supplement's key findings and contributions together, tying up loose ends. Traversing the DUQuA articles, we first argued the case for the research, conducted so that an in-depth analysis of one country's health system, completed 5 years after the landmark Deepening our Understanding of Quality Improvement in Europe (DUQuE), was available. We now provide a digest of the learning from each article. Essentially, we have contributed an understanding of quality and safety activities in 32 of the largest acute settings in Australia, developed a series of scales and tools for use within Australia, modifiable for other purposes elsewhere, and provided a platform for future studies of this kind. Our main message is, despite the value of publishing an intense study of quality activities in 32 hospitals in one country, there is no gold standard, one-size-fits-all methodology or guarantee of success in quality improvement activities, whether the initiatives are conducted at departmental, organization-wide or whole-of-systems levels. Notwithstanding this, armed with the tools, scales and lessons from DUQuA, we hope we have provided many more options and opportunities for others going about strengthening their quality improvement activities, but we do not claim to have solved all problems or provided a definitive approach. In our view, quality improvement initiatives are perennially challenging, and progress hard-won. Effective measurement, evaluating progress over time, selecting a useful suite of quality methods and having the persistence to climb the improvement gradient over time, using all the expertise and tools available, is at the core of the work of quality improvement and will continue to be so.

Organization quality systems and department-level strategies: refinement of the Deepening our Understanding in Quality in Australia (DUQuA) organization and department-level scales.

OBJECTIVE: The aim of this study was to develop and refine indices to measure organization and care pathway-level quality management systems in Australian hospitals. DESIGN: A questionnaire survey and audit tools were derived from instruments validated as part of the Deepening Our Understanding of Quality improvement in Europe (DUQuE) study, adapted for Australian hospitals through expert opinion. Statistical processes were used to explore the factor structure, reliability and non-redundancy and descriptive statistics of the scales. SETTING: Thirty-two large Australian public hospitals. PARTICIPANTS: Audit of quality management processes at organization-level and care pathway processes at department level for three patient conditions (acute myocardial infarction (AMI), hip fracture and stroke) and senior quality manager, at each of the 32 participating hospitals. MAIN OUTCOME MEASURE(S): The degree of quality management evident at organization and care pathway levels. RESULTS: Analysis yielded seven quality systems and strategies scales. The three hospital-level measures were: the Quality Management Systems Index (QMSI), the Quality Management Compliance Index (QMCI) and the Clinical Quality Implementation Index (CQII). The four department-level measures were: Specialised Expertise and Responsibility (SER), Evidence-Based Organisation of Pathways (EBOP), Patient Safety Strategies (PSS) and Clinical Review (CR). For QMCI, and for seven out of eight subscales in QMSI, adequate internal consistency (Cronbach's $\alpha$ >0.8) was achieved. For CQII, lack of variation and ceiling effects in the data resulted in very low internal consistency scores, but items were retained for theoretical reasons. Internal consistency was high for CR (Cronbach's $\alpha$ 0.74-0.88 across the three conditions), and this was supported by all item-total correlations exceeding the desired threshold. For EBOP, Cronbach's $\alpha$ was acceptable for hip fracture (0.80) and stroke (0.76), but only moderate for AMI (0.52). PSS and SER scales were retained for theoretical reasons, although internal consistencies were only moderate (SER) to poor (PSS). CONCLUSIONS: The Deepening our Understanding of Quality in Australia (DUQuA) organization and department scales can be used by Australian hospital managers to assess and measure improvement in quality management at organization and department levels within their hospitals and are readily modifiable for other health systems depending on their needs.

The relationships between quality management systems, safety culture and leadership and patient outcomes in Australian Emergency Departments.

OBJECTIVE: We aimed to examine whether Emergency Department (ED) quality strategies, safety culture and leadership were associated with patient-level outcomes, after controlling for other organization-level factors, in 32 large Australian hospitals. DESIGN: Quantitative observational study, using linear and multi-level modelling to identify relationships between quality management systems at organization level; quality strategies at ED level for acute myocardial infarction (AMI), hip fracture and stroke; clinician safety culture and leadership and patient-level outcomes of waiting time and length of stay. SETTING: Thirty-two large Australian public hospitals. PARTICIPANTS: Audit of quality management processes at organization and ED levels, senior quality manager at each of the 32 participating hospitals, 394 ED clinicians (doctors, nurses and allied health professionals). MAIN OUTCOME MEASURE(S): Within the multi-level model, associations were assessed between organization-level quality measures and ED quality strategies; organization-level quality measures and ED quality strategies and ward-level clinician measures of teamwork climate (TC), safety climate (SC) and leadership for AMI, hip fracture and stroke treatment conditions; and organization-level quality measures and ED quality strategies and ward-level clinician measures of TC, SC and leadership, and ED waiting time and length of stay (performance). RESULTS: We found seven statistically significant associations between organization-level quality systems and ED-level quality strategies; four statistically significant associations between quality systems and strategies and ED safety culture and leadership; and nine statistically significant associations between quality systems and strategies and ED safety culture and leadership, and ED waiting time and length of stay. CONCLUSIONS: Organization-level quality structures influence ED-level quality strategies, clinician safety culture and leadership and, ultimately, waiting time and length of stay for patients. By focusing only on time-based measures of ED performance we risk punishing EDs that perform well on patient safety measures. We need to better understand the trade-offs between implementing safety culture and quality strategies and improving patient flow in the ED, and to place more emphasis on other ED performance measures in addition to time.

Do organization-level quality management systems influence department-level quality? A cross-sectional study across 32 large hospitals in Australia.

OBJECTIVE: Little is known about the influence that hospital quality systems have on quality at department level, in Australia and elsewhere. This study assessed the relationships between organizational-level quality management systems, and the extent to which hospital-level quality management systems and department-level quality management strategies are related. DESIGN: A multi-level, cross-sectional, mixed-method study. SETTING AND PARTICIPANTS: As part of the Deepening our Understanding of Quality in Australia (DUQuA) project, we invited all large hospitals in Australia (~200 or more beds) which provided acute myocardial infarction (AMI), hip fracture and stroke care. The quality managers of these hospitals were the respondents for one of seven measures of hospital quality management systems and strategies. Data across the six remaining measures were collected through site visits by external surveyors assessing the participating hospitals. MAIN OUTCOME MEASURES: Relationships were assessed between three organization-level quality management system measures: a self-report measure assessing organization-level quality activities (quality management systems index, QMSI); externally assessed organization-level compliance to procedures used to plan, monitor and improve quality of care (quality management compliance index, QMCI); and externally assessed implementation of quality systems (clinical quality implementation index, CQII). Associations were also assessed between organization-level quality management systems and department-level quality management strategies: how clinical responsibilities are assigned for a particular condition; whether department organization processes are organized to facilitate evidence-based care recommendations; compliance with selected recommendations of international agencies; and whether clinical reviews are performed systematically. RESULTS: Of 78 invited hospitals, 32 participated in the study. QMSI was positively associated with QMCI and CQII, but after controlling for QMSI, no relationship was found between QMCI and CQII. There appears to be a cluster of relationships between QMSI and department-level measures, but this was not consistent across all departments. CONCLUSION: This is the first national study undertaken in Australia to assess relationships within and between organization-level and department-level quality management systems. These quality management system tools align with many components of accreditation standards and may be useful for hospitals in continuously monitoring and driving improvement.

Do quality management systems influence clinical safety culture and leadership? A study in 32 Australian hospitals.

OBJECTIVE: This study aimed to explore the associations between the organization-level quality arrangements, improvement and implementation and department-level safety culture and leadership measures across 32 large Australian hospitals. DESIGN: Quantitative observational study, using linear and multi-level modelling to identify relationships between quality management systems and clinician safety culture and leadership. SETTING: Thirty-two large Australian public hospitals. PARTICIPANTS: Quality audit at organization level, senior quality manager at each participating hospital, 1382 clinicians (doctors, nurses and allied health professionals). MAIN OUTCOME MEASURES: Associations between organization-level quality measures and department-level clinician measures of teamwork climate, safety climate and leadership for acute myocardial infarction (AMI), hip fracture and stroke treatment conditions. RESULTS: We received 1332 valid responses from participants. The quality management systems index (QMSI, a questionnaire-based measure of the hospitals' quality management structures) was 'positively' associated with all three department-level scales in the stroke department, with safety culture and leadership in the emergency department, but with none of the three scales in the AMI and hip fracture departments. The quality management compliance index (QMCI, an external audit-based measure of the quality improvement activities) was 'negatively' associated with teamwork climate and safety climate in AMI departments, after controlling for QMSI, but not in other departments. There was no association between QMCI and leadership in any department, after controlling for QMSI, and there was no association between the clinical quality implementation index (CQII, an external audit-based measure of the level of implementation of quality activities) and any of the three department-level scales in any of the four departments, after controlling for both QMSI and QMCI. CONCLUSIONS: The influence of organization-level quality management systems on clinician safety culture and leadership varied depending on the hospital department, suggesting that whilst there was some consistency on patient safety attitudes and behaviours throughout the organizations, there were also other factors at play.

Implementation and data-related challenges in the Deepening our Understanding of Quality in Australia (DUQuA) study: implications for large-scale cross-sectional research.

Healthcare organisations vary in the degree to which they implement quality and safety systems and strategies. Large-scale cross-sectional studies have been implemented to explore whether this variation is associated with outcomes relevant at the patient level. The Deepening our Understanding of Quality in Australia (DUQuA) study draws from earlier research of this type, to examine these issues in 32 Australian hospitals. This paper outlines the key implementation and analysis challenges faced by DUQuA. Many of the logistical difficulties of implementing DUQuA derived from compliance with the administratively complex and time-consuming Australian ethics and governance system designed principally to protect patients involved in clinical trials, rather than for low-risk health services research. The complexity of these processes is compounded by a lack of organizational capacity for multi-site health services research; research is expected to be undertaken in addition to usual work, not as part of it. These issues likely contributed to a relatively low recruitment rate for hospitals (41% of eligible hospitals). Both sets of issues need to be addressed by health services researchers, policymakers and healthcare administrators, if health services research is to flourish. Large-scale research also inevitably involves multiple measurements. The timing for applying these measures needs to be coherent, to maximise the likelihood of finding real relationships between quality and safety systems and strategies, and patient outcomes; this timing was less than ideal in DUQuA, in part due to administrative delays. Other issues that affected our study include low response rates for measures requiring recruitment of clinicians and patients, missing data and a design that necessarily included multiple statistical comparisons. We discuss how these were addressed. Successful completion of these projects relies on mutual and ongoing commitment, and two-way communication between the research team and hospital staff at all levels. This will help to ensure that enthusiasm and engagement are established and maintained.