Association between regulator inspection and ratings on primary care prescribing: an observational study in England 2014 to 2019.

BACKGROUND: Healthcare regulators in many countries undertake inspections of healthcare providers and publish inspection outcomes with the intention of improving quality of care. Comprehensive inspections of general practices in England by the Care Quality Commission began for the first time in 2014. It is assumed that inspection and rating will raise standards and improve care, but the presence and extent of any improvements is unknown. We aim to determine if practice inspection ratings are associated with past performance on prescribing indicators and if prescribing behaviour changes following inspection. METHODS: Longitudinal study using a dataset of 6771 general practices in England. Practice inspection date and score was linked with monthly practice-level data on prescribing indicators relating to antibiotics, hypnotics and non-steroidal anti-inflammatory drugs. The sample covers practices receiving their first inspection between September 2014 and December 2018. Regression analysis and the differential timing of inspections is used to identify the impact on prescribing. RESULTS: Better-rated practices had better prescribing in the period before inspections began. In the six months following inspections, no overall change in prescribing was observed. However, the differences between the best and worse rated practices were reduced but not fully. The same is also true when taking a longer-term view. There is little evidence that practices responded in anticipation of inspection or reacted differently once the ratings were made public. CONCLUSION: While some of the observed historic variation in prescribing behaviour has been lessened by the process of inspection and ratings, we find this change is small and appears to come from both improvements among lower-rated practices and deteriorations among higher-rated practices. While inspection and rating no doubt had other impacts, these prescribing indicators were largely unchanged.

Time is a terrible thing to waste: optimising use of intraoperative monitoring practitioner time towards maximising in-house IOM service provision and reducing spend on external provision.

Intraoperative monitoring (IOM) during orthopaedic and neurosurgical operations informs surgeons about the integrity of patients' central and peripheral nervous systems. It is provided by IOM practitioners (IOMPs), who are usually neurophysiology healthcare scientists. Increasing awareness of the benefits for patient safety and surgical outcomes, along with post-COVID-19 service recovery, has resulted in a material increase in demand for IOM provision nationally, and particularly at Salford Royal Hospital (SRH), which is a regional specialist neurosciences centre.There is a shortage of IOMPs in the UK National Health Service (NHS). At SRH, this is exacerbated by staff capacity shortage, requiring £202 800 of supplementary private provision in 2022.At SRH, IOMPs work in pairs. Our productive time is wasted by delays to surgical starts beyond our control and by paired working for much of a surgery session. This quality improvement (QI) project set out to release productive time by: calling the second IOMP to theatre only shortly before start time, the other IOMP returning to the office during significant delays, releasing an IOMP from theatre when appropriate and providing a laptop in theatre for other work.We tested and refined these change ideas over two plan-do-study-act improvement cycles. Compared with complete paired working, we increased the time available for additional productive work and breaks from an average of 102 to 314 min per operating day, not quite achieving our project target of 360 min.The new ways of working we developed are a step towards ability (when staff capacity increases) to test supporting two (simultaneous) operations with three IOMPs (rather than two pairs of IOMPs). Having significantly improved the use of staff time, we then also used our QI project data to make a successful business case for investment in two further IOMP posts with a predicted net saving of £20 000 per year along with other associated benefits.

Reducing first appointment delays for electron radiotherapy patients by improving the treatment planning pathway: a quality improvement project.

Clatterbridge Cancer Centre (CCC) is a specialist hospital trust in England with three sites.Delay to the start of an appointment for radiotherapy, especially the first appointment (a 'New Start') is poor, both for operational efficiency and patient experience, causing stress for both patients and staff. Our aim is for the New Start to begin within 30 min of the allotted appointment time. To this end, we established another aim: for 'Final Checks' to the radiotherapy plan to be completed at least 30 min prior to the New Start appointment time.Prior to this quality improvement (QI) project, only 33% of electron-treatment New Start appointments started within the target 30 min (the average delay was 52.4 min) and only 48% of the corresponding Final Checks had been completed by their 30 min prior target.The treatment pathway for these patients was redesigned, with the aim of 90% of New Start appointments starting within 30 min of the allotted appointment time.By the end of this QI project, 69.2% of New Start appointments started within 30 min of the appointment time (with average delay reduced to 27.2 min), and 92.3% of Final Checks were completed by their 30 min prior target. We also reduced the number of safety (Datix) incidents due to plan not ready from 10 to 0. A year after the project, we have held most of the time improvements and still have had 0 plan-not-ready Datix.The largest improvement was achieved by introducing a proxy (without the patient present) 'day 0' appointment. This takes place in advance of the New Start appointment to enable earlier planning. Subsequent improvements included: automating previously manual planning calculations, making the care path consistent with other external beam radiotherapy care paths at CCC to reduce staff cognitive load and sharing key performance data with staff.

Clinical-scientist-led transoesophageal echocardiography (TOE): using extended roles to improve the service.

At the North West Anglia NHS Foundation Trust, we perform transoesophageal echocardiography (TOE), a semi-invasive diagnostic test using ultrasound for high-quality heart imaging. TOE allows accurate diagnosis of serious heart problems to support high-quality clinical decision-making about treatment pathways. The procedure can be lengthy and is traditionally performed by a consultant cardiologist, who typically has multiple commitments. This constrains patient access to TOE, leading to waits from referral to test, delaying treatment decisions.In this quality improvement project, we improved access by redesigning workforce roles. The clinical scientist, who had been supporting the consultant during TOE clinics, took on performing the procedure as the main operator. We used the Model for Improvement to develop this clinical-scientist-led service-delivery model, and then test and refine it. This increased capacity and frequency of TOE clinics, reducing waits and releasing around 2 days per month of consultant time.Over five plan-do-study-act cycles, we tested six changes/refinements. Our targets were to reduce the maximum waiting time for TOE to 3 working days for inpatients and to 14 working days for outpatients. We succeeded, achieving reductions in mean waiting times from 7.7 days to 3.0 days for inpatients and from 33.2 days to 8.3 days for outpatients.TOE requires intubation; when this fails, TOE is abandoned. We believe light (rather than heavy) sedation is helpful for this intubation. We reduced sedation levels (from a median of 3 mg of midazolam to 1.5 mg) and, as a secondary outcome of this project, reduced the intubation failure rate from 13% to 0% (over 32 postchange patients).Following this project, our TOE service is usually performed by a clinical scientist in echocardiography who has British Society of Echocardiography TOE accreditation and advanced training. We have sustained the improved performance and demonstrated the value of enhanced roles for clinical scientists.

Time to be more efficient: reducing wasted transthoracic echocardiography (TTE) diagnostic appointment slots at Guy's and St Thomas' NHS Trust.

Transthoracic echocardiography (TTE) is one of the most requested non-invasive cardiac imaging diagnostic modalities available in the National Health Service (NHS). There is persistently high demand, but nationally, activity has lagged, producing increasing numbers of breaches of the 6-week waiting time target. This delays patients' diagnosis and treatment.Patients attend hospital for TTE either as a clinic-linked or a standalone appointment. In this quality improvement project, we identified that the clinic-linked slots were a major source of wasted capacity due to both unbooked slots and a high rate of patients not attending their appointments (DNA).DNA is a complex issue, aggravated in our trust by many IT systems, complex clinic-booking pathways and restricted patient communication channels. We parked changing these processes, pending an imminent, unifying IT development programme. We focused instead on unused clinic-linked appointments, with the goal of reducing these from 18% (~31 of ~175 allocated each week) to 5% by the end of the 14 week project period.In close collaboration with service stakeholders, we identified that the primary root causes were related to the clinic-linked TTE booking pathway. The change idea was a 7-day rule: after reminders at 9 and 8 days prior to the clinic date, any appointment slots still unbooked by cardiology sub-specialities for patients attending clinic-linked appointments at 7 days, would be used for booking standalone TTE patients.We refined this process over two plan-do-study-act (PDSA) cycles, reducing unused (wasted) appointment slots, allocated initially to clinic-linked patients, to a sustained level of 5.1%, meaning we could now perform approximately 21 additional TTE tests weekly; we have materially increased activity without increasing capacity.This contributed to a significant reduction in 6-week TTE waiting-time breaches. Over the project, this went from 378 (30%, February 2022) to 71 (8%, September 2022) and latest data show 28 (4%, February 2023).

Successful demand management in diagnostic immunology testing.

AIMS: We investigated whether we could have a material and sustained impact on immunology test ordering by primary care clinicians by building evidence-based and explanatory algorithms into test ordering software. METHODS: A service evaluation revealed cases of over-requesting of antinuclear antibody, allergen-specific IgE and total IgE tests, and under-requesting of urine protein electrophoresis. We conducted a quality improvement programme to address this. We determined the most effective and efficient intervention would be to embed evidence-based and advice-based decision-support algorithms in the ordering software. Consultation with general practitioners revealed lack of knowledge and confidence about testing, and an appetite for support. We iteratively designed and implemented algorithms for the four sets of tests for the primary care practices in our catchment and made them available to other hospital trusts in our region. The ordering system now contains links to advice sheets for clinicians and their patients and to an email address for queries to the lab. RESULTS: We observe large (36% to 88%) reductions in testing activity (workload) for the over-requested tests and large (28%-135%) increases for the under-requested test. We show that these changes are sustained. There have been no complaints from the clinicians and queries to the lab are now minimal (less than one per month on average). CONCLUSIONS: Embedding algorithms in the ordering software can be acceptable to clinicians and have a major and sustained impact on overuse or underuse of tests. The algorithms can be replicated by other hospital trusts.

Doing today's work today: real-time data recording and rolling audit in an IVF clinic.

The assisted conception unit at Sheffield Teaching Hospital NHS Foundation Trust provides in vitro fertilisation treatment. A team of seven embryologists provides a routine clinical laboratory service, involving culture and storage of embryos. This requires a series of management and statutory data administration and communication tasks.We were aware that these were often done many days after clinical tasks, resulting in delays sending patient correspondence and unavailability of clinical notes for multidisciplinary team (MDT) cycle-review meetings. Embryologists also complained that transcribing data were time-consuming and duplicated across our IDEAS software, spreadsheets and paper.We process-mapped our processes and gathered staff views on problems and potential solutions. The baseline average total cycle time (TCT) for completion of all administrative steps was around 17 days; data administration time (DAT, data 'touch time') was around 30 min per patient.We embarked on this Quality Improvemen (QI) project to reduce waste in TCT and DAT, and to have data available for patient communication and MDT deadlines. Exploration of IDEAS' capabilities led to progressive realisation of how much could be transferred to this single data system, removing a lot of off-putting redundancy. Through this we developed a 'to-be' vision of all data entry being real time, as part of the clinical 'jobs'. We conducted five Plan-Do-Study-Act cycles plus two more to test performance and sustainability as changes bedded-in and an external constraint disappeared.We have cut TCT to 0 or 1 days and DAT to around 18 min. All project metrics are reliably within our targets, and data are now always available for timely patient letters and the MDT. Other benefits include easy access for all staff to patient records and removal of paper and spreadsheets. A further, unanticipated, benefit was a switch from a tedious 2 yearly storage tank audit to a more-agreeable and safer rolling audit.

Getting going on time: reducing neurophysiology set-up times in order to contribute to improving surgery start and finish times.

At the Walton Centre we conduct a relatively large number of complex and lengthy elective (booked) spinal operations. Recently, we have had a particular problem with half or more of these sessions finishing late, resulting in staff discontent and greater use of on-call staff.These operations require patient monitoring by neurophysiology clinical scientists. Before the surgeon can start the operation, in-theatre neurophysiological measurements are required to establish a baseline. We reasoned that reducing this set-up time would reduce the risk of surgery starting late, and so the whole session finishing later than expected.In this project we redesigned the neurophysiology parts of in-theatre patient preparation. We conducted five Plan-Do-Study-Act cycles over 3 months, reducing the duration of pre-surgery preparation from a mean of 70 min to around 50 min. We saw improvements in surgical start times and session finish times (both earlier by roughly comparable amounts). The ultimately impact is that we saw on-time session finishes improve from around 50% to 100%. Following this project, we have managed to sustain the changes and the improved performance.The most impactful change was to conduct in-theatre neurophysiology patient preparation simultaneously with anaesthesia, rather than waiting for this to finish; when we performed this with a pair of clinical scientists, we were able to complete neurophysiology patient preparation by the time the anaesthetist was finished, therefore not introducing delays to the start of surgery. A final change was to remove a superfluous preparatory patient-baseline measurement.This is a very challenging and complex environment, with powerful stakeholders and many factors and unpredictable events affecting sessions. Nevertheless, we have shown that we can make improvements within our span of influence that improve the wider process. While using pairs of staff requires greater resource, we found the benefit to be worthwhile.

Improving the turnaround times of infectious disease markers reporting in an NHS stem cell department.

The Stem Cell Donation and Transplantation Department at NHS Blood and Transplant (NHSBT) facilitates unrelated donor haematopoietic stem cell transplantations for patients with life-threatening haematological malignancies or other blood diseases. Donors must be screened for infectious disease markers (IDMs) prior to donation. The purpose of IDM testing is to assess whether the donor currently has, or previously had, an infectious disease that could be transmitted to the recipient. The turnaround time (TaT) from sample collection to the return of IDM results is important to transplant clinicians and their patients. NHSBT has a target TaT of 80% within seven calendar days. Our initial analysis showed us that we failed to meet this in any week in the previous year, and our service was neither efficient nor consistent, so there was considerable improvement potential.This quality improvement (QI) project aimed to improve the TaT of the IDM reporting service. We tested three change ideas through four Plan-Do-Study-Act (PDSA) cycles. We collected data on TaTs from our laboratory information management system (LIMS) and updated our statistical process control charts after each PDSA cycle. Over the course of the project, we reduced the mean TaT from 8.9 days to 5.5 days and increased the proportion of samples reported within the 7-day benchmark from 50% to 89%, reaching the key performance indicator (KPI) target.Conducting this project was a rewarding experience. Although we encountered unanticipated technical issues during PDSA experiments, and we found that some change plans were not as effective in improving the KPIs as we expected, the improvement by the end of the study period was substantial. This QI project enabled us to meet our TaT targets and, ultimately, help ensure that our patients receive timely transplants. It suggests that QI may have wider applications across our part of NHSBT.

The impact of management practices on relative patient mortality: Evidence from public hospitals.

A small, but growing, body of empirical evidence shows that the material and persistent variation in many aspects of the performance of healthcare organisations can be related to variation in their management practices. This study uses public data on hospital patient mortality outcomes, the Summary Hospital-level Mortality Indicator (SHMI) to extend this programme of research. We assemble a five-year dataset combining SHMI with potential confounding variables for all English NHS non-specialist acute hospital trusts. The large number of providers working within a common system provides a powerful environment for such investigations. We find considerable variation in SHMI between trusts and a high degree of persistence of high- or low performance. This variation is associated with a composite metric for management practices based on the NHS National Staff Survey. We then use a machine learning technique to suggest potential clusters of individual management practices related to patient mortality performance and test some of these using traditional multivariate regression. The results support the hypothesis that such clusters do matter for patient mortality, and so we conclude that any systematic effort at improving patient mortality should consider adopting an optimal cluster of management practices.

Summarised, verified and accessible: improving clinical information management for potential haematopoietic stem cell transplantation patients.

The Welsh Transplantation and Immunogenetics Laboratory (WTAIL) is responsible for managing patient work-up for haematopoietic stem cell transplantation (HSCT), the only potentially curative option for many haematological and non-haematological conditions. Work-up requires regular communication between WTAIL and the transplanting clinicians, facilitated by weekly multidisciplinary team (MDT) meetings, to agree decisions and proceed through each work-up stage. Effective communication and minimising error are critical, as transplanting cells from a suboptimal donor could have severe or fatal consequences for the patient. We reviewed our HSCT patient management and identified issues including staff dissatisfaction with the inefficiency of the current (paper-based) system and concern about the potential for incidents caused by errors in manual transcription of patient information and tracking clinical decisions. Another driver for change was the COVID-19 pandemic, which prevented the usual face-to-face MDT meetings in which staff would show clinicians the paper records and reports; the shift to online MDT required new ways of sharing data. In this project we developed a new central reference point for our patient management data along with electronic patient summary sheets, designed with an eye to improving safety and efficiency. Over several improvement cycles we tested and refined the summary sheets with staff and clinicians and experimented with videoconferencing to facilitate data sharing. We conducted interviews with staff from which we concluded that the new process successfully reduced transcription and duplication and improved communication with the clinicians during the pandemic. Despite an increase in workload due to build-up of active patient work-up cases during the pandemic, staff reported that the new summaries enabled them to cope well. A key initiative was creation of a 'Task and Finish' group that helped establish continual improvement culture and identified additional areas for improvement which have been followed up in further improvement projects.

Improving turnaround times for HLA-B*27 and HLA-B*57:01 gene testing: a Barts Health NHS Trust quality improvement project.

Among other tests, Barts Health NHS Trust clinical transplantation laboratory conducts two important gene-detection tests: human leucocyte antigen (HLA)-B*27 ('B27', associated with the diagnosis of ankylosing spondylitis) and HLA-B*57:01 ('B57', associated with prediction of abacavir hypersensitivity disorder). The turnaround time (TaT) from sample receipt to return of results is important to clinicians and their patients but was not monitored. Furthermore, we anticipated an imminent increase in demand from a forthcoming pathology service merger, together with long-term increases with the rise of personalised genetic medicine.In this quality improvement project, we identified current TaT performance and sources of delay. Over three plan-do-study-act (PDSA) cycles, we tested three change ideas, two involving using IT to remove manual administrative steps and alert us to samples needing progressing; both were retained. The other change involved separating out the targeted tests; we judged this not worthwhile with current demand levels, although something to be re-examined when volumes increase. During the project, we reduced mean TaT from 3.8 to 3.3 days and increased the proportion within our 5-day target from 78% to 100%. These have been sustained (at 3.4 days and 97%) for the 3 months following our PDSA cycles and illustrate that reducing variation can be as impactful as reducing the mean.We conducted this project during the COVID-19 disruption, which reduced demand substantially. We took advantage of this to allow staff to spend time on these improvement activities. Another interesting feature of the work is that during the project, we compared changes in performance on our targeted B27/B57 tests with that on another comparable test as a control, to consider the impact of the general increased attention (the Hawthorne effect). We found that performance on this control also increased comparably, but then fell away after our project finished, while it did not for B27/B57.

Making every drop count: reducing wastage of a novel blood component for transfusion of trauma patients.

Recent research demonstrates that transfusing whole blood (WB=red blood cells (RBC)+plasma+platelets) rather than just RBC (which is current National Health Service (NHS) practice) may improve outcomes for major trauma patients. As part of a programme to investigate provision of WB, NHS Blood and Transplant undertook a 2-year feasibility study to supply the Royal London Hospital (RLH) with (group O negative, 'O neg') leucodepleted red cell and plasma (LD-RCP) for transfusion of trauma patients with major haemorrhage in prehospital settings.Incidents requiring such prehospital transfusion occur randomly, with very high variation. Availability is critical, but O neg LD-RCP is a scarce resource and has a limited shelf life (14 days) after which it must be disposed of. The consequences of wastage are the opportunity cost of loss of overall treatment capacity across the NHS and reputational damage.The context was this feasibility study, set up to assess deliverability to RLH and subsequent wastage levels. Within this, we conducted a quality improvement project, which aimed to reduce the wastage of LD-RCP to no more than 8% (ie, 1 of the 12 units delivered per week).Over this 2-year period, we reduced wastage from a weekly average of 70%-27%. This was achieved over four improvement cycles. The largest improvement came from moving near-expiry LD-RCP to the emergency department (ED) for use with their trauma patients, with subsequent improvements from embedding use in ED as routine practice, introducing a dedicated LD-RCP delivery schedule (which increased the units ≤2 days old at delivery from 42% to 83%) and aligning this delivery schedule to cover two cycles of peak demand (Fridays and Saturdays).

The measurement and improvement of maternity service performance through inspection and rating: An observational study of maternity services in acute hospitals in England.

OBJECTIVES: To determine whether the prior performance of maternity services, as measured by Royal College of Obstetricians and Gynaecologists performance indicators, is associated with ratings by the Care Quality Commission at subsequent inspection, and whether performance changes occur after inspection. METHODS: We used hospital activity data from 176 maternity sites inspected between October 2013 and March 2016 to generate a set of performance indicators developed by the Royal College of Obstetricians and Gynaecologists. We linked these data to Care Quality Commission data on inspection dates and rating scores and used regression models, controlling for site level effects, to estimate the relationships between inspection ratings and performance indicators before and after inspections. RESULTS: Coefficients measuring the relationship between indicator performance and subsequent inspection rating score had wide confidence intervals which crossed zero suggesting no statistically significant relationship prior to inspection. The same absence of statistical significance was observed for changes in indicator performance after inspection. CONCLUSIONS: The use of routine data for performance monitoring is becoming increasingly important as regular inspection is costly and regulators require accurate and timely intelligence. However, we found no statistically significant relationships between inspection ratings and performance indicators before or after inspections in maternity services. This calls into question the validity and reliability of the performance indicators, the inspection process and ratings, or both, as measures of performance.

The 85% bed occupancy fallacy: The use, misuse and insights of queuing theory.

Queuing theory can and has been used to inform bed pool capacity decision making, though rarely by managers themselves. The insights it brings are also not widely and properly understood by healthcare managers. These two shortcomings lead to the persistent fallacy of there being a globally applicable optimum average occupancy target, for example 85%, which can in turn lead to over- or under-provision of resources. Through this paper, we aim both to make queuing models more accessible and to provide visual demonstrations of the general insights managers should absorb from queuing theory. Occupancy is a consequence of the patient arrival rate and 'treatment' rate (the number of beds and length of stay). There is a trade-off between the average occupancy and access to beds (measured by, for example, the risk of access block due to all beds being full or the average waiting time for a bed). Managerially, the decision-making input should be the level of access to beds required, and so bed occupancy should be an output. Queuing models are useful to quickly draw the shape of these access-occupancy trade-off curves. Moreover, they can explicitly show the effect that variation (lack of regularity) in the times between arrivals and in the lengths of stay of individual patients has on the shape of the trade-off curves. In particular, with the same level of access, bed pools subject to lower variation can operate at higher average occupancy. Further, to improve access to a bed pool, reducing variation should be considered.

Using quality indicators to predict inspection ratings: cross-sectional study of general practices in England.

BACKGROUND: The Care Quality Commission regulates, inspects, and rates general practice providers in England. Inspections are costly and infrequent, and are supplemented by a system of routine quality indicators, measuring patient satisfaction and the management of chronic conditions. These indicators can be used to prioritise or target inspections. AIM: To determine whether this set of indicators can be used to predict the ratings awarded in subsequent inspections. DESIGN AND SETTING: This cross-sectional study was conducted using a dataset of 6860 general practice providers in England. METHOD: The indicators and first-inspection ratings were used to build ordered logistic regression models to predict inspection outcomes on the four-level rating system ('outstanding', 'good', 'requires improvement', and 'inadequate') for domain ratings and the 'overall' rating. Predictive accuracy was assessed using the percentage of correct predictions and a measure of agreement (weighted κ). RESULTS: The model correctly predicted 79.7% of the 'overall' practice ratings. However, 78.8% of all practices were rated 'good' on 'overall', and the weighted κ measure of agreement was very low (0.097); as such, predictions were little more than chance. This lack of predictive power was also found for each of the individual domain ratings. CONCLUSION: The poor power of performance of these indicators to predict subsequent inspection ratings may call into question the validity and reliability of the indicators, inspection ratings, or both. A number of changes to the way data relating to performance indicators are collected and used are suggested to improve the predictive value of indicators. It is also recommended that assessments of predictive power be undertaken prospectively when sets of indicators are being designed and selected by regulators.

Do performance indicators predict regulator ratings of healthcare providers? Cross-sectional study of acute hospitals in England.

OBJECTIVE: To determine whether a large set of care performance indicators ('Intelligent Monitoring (IM)') can be used to predict the Care Quality Commission's (CQC) acute hospital trust provider ratings. DESIGN: The IM dataset and first-inspection ratings were used to build linear and ordered logistic regression models for the whole dataset (all trusts). This was repeated for subsets of the trusts, with these models then applied to predict the inspection ratings of the remaining trusts. SETTING: The United Kingdom Department of Health and Social Care's Care Quality Commission is the regulator for all health and social care services in England. We consider their first-inspection cycle of acute hospital trusts (2013-2016). PARTICIPANTS: All 156 English NHS acute hospital trusts. INTERVENTION(S): None. MAIN OUTCOME MEASURE(S): Percentage of correct predictions and weighted kappa. RESULTS: Only 24% of the predicted overall ratings for the test sample were correct and the weighted kappa of 0.01 indicates very poor agreement between predicted and actual ratings. This lack of predictive power is also found for each of the rating domains. CONCLUSION: While hospital inspections draw on a much wider set of information, the poor power of performance indicators to predict subsequent inspection ratings may call into question the validity of indicators, ratings or both. We conclude that a number of changes to the way performance indicators are collected and used could improve their predictive value, and suggest that assessing predictive power should be undertaken prospectively when the sets of indicators are being designed and selected by regulators.

Measurement and improvement of emergency department performance through inspection and rating: an observational study of emergency departments in acute hospitals in England.

INTRODUCTION: Hospital inspection and the publication of inspection ratings are widely used regulatory interventions that may improve hospital performance by providing feedback, creating incentives to change and promoting choice. However, evidence that these interventions assess performance accurately and lead to improved performance is scarce. METHODS: We calculated six standard indicators of emergency department (ED) performance for 118 hospitals in England whose EDs were inspected by the Care Quality Commission, the national regulator in England, between 2013 and 2016. We linked these to inspection dates and subsequent rating scores. We used multilevel linear regression models to estimate the relationship between prior performance and subsequent rating score and the relationship between rating score and post-inspection performance. RESULTS: We found no relationship between performance on any of the six indicators prior to inspection and the subsequent rating score. There was no change in performance on any of the six indicators following inspection for any rating score. In each model, CIs were wide indicating no statistically significant relationships. DISCUSSION: We found no association between established performance indicators and rating scores. This might be because the inspection and rating process adds little to the external performance management that EDs receive. It could also indicate the limited ability of hospitals to improve ED performance because of extrinsic factors that are beyond their control.

Combining regression trees and panel regression for exploring and testing the impact of complementary management practices on short-notice elective operation cancellation rates.

Variation in the performance of providers across healthcare systems is pervasive. It is recognised as both a major concern and an opportunity for learning and improvement. Variation between providers is broadly considered to be due to management practices and contextual factors such as catchment-area demographics. However, there is little understanding of the ways in which these impact on performance and how they can be measured. We use recent developments in both regression trees and panel regression techniques to explore and then statistically test complementary alignments of management practices whilst taking into account contextual factors. We apply this to 5 years of NHS hospital trust data, examining performance on short-notice cancellation rates. We find that different alignments of management practices give rise to quite different short-notice cancellation rates between trusts, with some being substantially lower. Our research offers a data-driven approach for identifying optimal clusters of management practices.