Continuous monitoring of emergency admissions of older care home residents to hospital.

BACKGROUND: evidence from inspection programmes suggest that the quality of care provided by individual care homes for older people is very variable. Aside from periodic inspection, there is limited information that is routinely collected and can be used to monitor quality. OBJECTIVES: to describe a method for using routine hospital data on admissions of older people as means for monitoring quality of care within a care home. To explore how this might be applied and used. METHODS: we linked hospital admissions to care homes using postcode matching and analysed hospital admission data as a time series, using the Cumulative Sum (CUSUM) technique to detect unusually high rates of admission. RESULTS: if we develop the CUSUM so that the number of times it falsely signals a high rate of admissions is limited to a rate of 0.1% per year, the chances of successfully detecting a doubling of the admission rate within 2 years will range from 48% for the smaller homes to 96% for the larger homes. CONCLUSION: monitoring tools using data on admissions to hospital are both possible and feasible, particularly for the larger homes. However, due to data limitations, users need to be careful about how they interpret triggers and thus ensure follow-up is appropriate. Some of the problems caused by using routine national data can be overcome if care homes used their own information for local monitoring.

A Cost Evaluation of COVID-19 Remote Home Monitoring Services in England.

BACKGROUND: Remote home monitoring services emerged as critical components of health care delivery from NHS England during the COVID-19 pandemic, aiming to provide timely interventions and reduce health care system burden. Two types of service were offered: referral by community health services to home-based care to ensure the right people were admitted to the hospital at the right time (called COVID Oximetry@home, CO@h); and referral by hospital to support patients' transition from hospital to home (called COVID-19 Virtual Ward, CVW). The information collected for the oxygen levels and other symptoms was provided via digital means (technology-enabled) or over the phone (analogue-only submission mode). This study aimed to evaluate the costs of implementing remote home monitoring for COVID-19 patients across 26 sites in England during wave 2 of the pandemic. Understanding the operational and financial implications of these services from the NHS perspective is essential for effective resource allocation and service planning. METHODS: We used a bottom-up costing approach at the intervention level to describe the costs of setting up and running the services. Twenty-six implementation sites reported the numbers of patients and staff involved in the service and other resources used. Descriptive statistics and multivariable regression analysis were used to assess cost variations and quantify the relationship between the number of users and costs while adjusting for other service characteristics. RESULTS: The mean cost per patient monitored was lower in the CO@h service compared with the CVW service (£527 vs £599). The mean cost per patient was lower for implementation sites using technology-enabled and analogue data submission modes compared with implementation sites using analogue-only modes for both CO@h (£515 vs £561) and CVW (£584 vs £612) services. The number of patients enrolled in the services and the service type significantly affected the mean cost per patient. CONCLUSIONS: Our analysis provides a framework for evaluating the costs of similar services in the future and shows that the implementation of these services benefit from the employment of tech-enabled data submission modes.

Youth violence intervention programme for vulnerable young people attending emergency departments in London: a rapid evaluation.

Background: Youth violence intervention programmes involving the embedding of youth workers in NHS emergency departments to help young people (broadly aged between 11 and 24 years) improve the quality of their lives following their attendance at an emergency department as a result of violent assault or associated trauma are increasing across the NHS. This study evaluates one such initiative run by the charity Redthread in partnership with a NHS trust. Objectives: To evaluate the implementation and impact of a new youth violence intervention programme at University College London Hospital NHS Trust and delivered by the charity Redthread: (1) literature review of studies of hospital-based violent crime interventions; (2) evaluation of local implementation and of University College London Hospital staff and relevant local stakeholders concerning the intervention and its impact; (3) assessment of the feasibility of using routine secondary care data to evaluate the impact of the Redthread intervention; and (4) cost-effectiveness analysis of the Redthread intervention from the perspective of the NHS. Methods: The evaluation was designed as a mixed-methods multiphased study, including an in-depth process evaluation case study and quantitative and economic analyses. The project was undertaken in different stages over two years, starting with desk-based research and an exploratory phase suitable for remote working while COVID-19 was affecting NHS services. A total of 22 semistructured interviews were conducted with staff at Redthread and University College London Hospital and others (e.g. a senior stakeholder involved in NHS youth violence prevention policy). We analysed Redthread documents, engaged with experts and conducted observations of staff meetings to gather more in-depth insights about the effectiveness of the intervention, the processes of implementation, staff perceptions and cost. We also undertook quantitative analyses to ascertain suitable measures of impact to inform stakeholders and future evaluations. Results: Redthread's service was viewed as a necessary intervention, which complemented clinical and other statutory services. It was well embedded in the paediatric emergency department and adolescent services but less so in the adult emergency department. The diverse reasons for individual referrals, the various routes by which young people were identified, and the mix of specific support interventions provided, together emphasised the complexity of this intervention, with consequent challenges in implementation and evaluation. Given the relative unit costs of Redthread and University College London Hospital's inpatient services, it is estimated that the service would break even if around one-third of Redthread interventions resulted in at least one avoided emergency inpatient admission. This evaluation was unable to determine a feasible approach to measuring the quantitative impact of Redthread's youth violence intervention programme but has reflected on data describing the service, including costs, and make recommendations to support future evaluation. Limitations: The COVID-19 pandemic severely hampered the implementation of the Redthread service and the ability to evaluate it. The strongest options for analysis of effects and costs were not possible due to constraints of the consent process, problems in linking Redthread and University College London Hospital patient data and the relatively small numbers of young people having been engaged for longer-term support over the evaluation period. Conclusions: We have been able to contribute to the qualitative evidence on the implementation of the youth violence intervention programme at University College London Hospital, showing, for example, that NHS staff viewed the service as an important and needed intervention. In the light of problems with routine patient data systems and linkages, we have also been able to reflect on data describing the service, including costs, and made recommendations to support future evaluation. Future work: No future work is planned. Funding: National Institute for Health and Care Research Health Services and Delivery Research programme (RSET: 16/138/17).

Youth violence intervention programmes in the NHS embed specialist youth workers into a hospital's paediatric emergency departments. These staff can engage young people and encourage positive change in their lives. Youth violence intervention programmes are part of a broader national strategy to prevent violence among young people. To improve our knowledge of the impact on young people and the cost-effectiveness of youth violence intervention programmes, we carried out an evaluation of a youth violence intervention programme introduced in 2020 at University College London Hospital and run by the charity Redthread. We reviewed the international evidence on youth violence intervention programmes, and other studies of Redthread services but found few studies measuring impact within the NHS. We reviewed documents and conducted 22 interviews with University College London Hospital and Redthread staff among others. We found that the service is viewed positively by NHS staff. We also found that youth workers can help a young person to better engage in their medical care and treatment. Youth violence intervention programmes also provide a link with non­health-care services within the community. Overall, they help NHS staff to better support vulnerable young people following discharge from hospital. We also established the cost of delivering Redthread services per user was £1865. This compares with a cost per inpatient of £5789 for a group of patients similar to those helped by Redthread. The average cost of a Redthread-type patient attending the emergency department was £203. We looked at whether it was possible to measure whether Redthread reduced young people's re-admissions to the hospital's emergency departments. However, we concluded that fully answering this question was not possible over the timescale of the project. This was because of the impact of COVID-19 on Redthread and other paediatric services, the low numbers of young people engaged in a longer-term programme with Redthread (59) and difficulties with linking information from the hospital and Redthread. We have therefore made various recommendations in this report to improve the way that data are collected and linked to aid future evaluations.

Rapid evaluation of service innovations in health and social care: key considerations.

Funding: This project was funded by the National Institute for Health and Care Research (NIHR) Health and Social Care Delivery Research programme (RSET: 16/138/17; BRACE: 16/138/31).

Patient and staff experiences of using technology-enabled and analogue models of remote home monitoring for COVID-19 in England: A mixed-method evaluation.

OBJECTIVE: To evaluate patient and staff experiences of using technology-enabled ('tech-enabled') and analogue remote home monitoring models for COVID-19, implemented in England during the pandemic. METHODS: Twenty-eight sites were selected for diversity in a range of criteria (e.g. pre-hospital or early discharge service, mode of patient data submission). Between February and May 2021, we conducted quantitative surveys with patients, carers and staff delivering the service, and interviewed patients, carers, and staff from 17 of the 28 services. Quantitative data were analysed using descriptive statistics and both univariate and multivariate analyses. Qualitative data were interpreted using thematic analysis. RESULTS: Twenty-one sites adopted mixed models whereby patients could submit their symptoms using either tech-enabled (app, weblink, or automated phone calls) or analogue (phone calls with a health professional) options; seven sites offered analogue-only data submission (phone calls or face-to-face visits with a health professional). Sixty-two patients and carers were interviewed, and 1069 survey responses were received (18 % response rate). Fifty-eight staff were interviewed, and 292 survey responses were received (39 % response rate). Patients who used tech-enabled modes tended to be younger (p = 0.005), have a higher level of education (p = 0.011), and more likely to identify as White British (p = 0.043). Most patients found relaying symptoms easy, regardless of modality, though many received assistance from family or friends. Staff considered the adoption of mixed delivery models beneficial, enabling them to manage large patient numbers and contact patients for further assessment as needed; however, they suggested improvements to the functionality of systems to better fit clinical and operational needs. Human contact was important in all remote home monitoring options. CONCLUSIONS: Organisations implementing tech-enabled remote home monitoring at scale should consider adopting mixed models which can accommodate patients with different needs; focus on the usability and interoperability of tech-enabled platforms; and encourage digital inclusivity for patients.

A rapid mixed-methods evaluation of remote home monitoring models during the COVID-19 pandemic in England.

Background: Remote home monitoring services were developed and implemented for patients with COVID-19 during the pandemic. Patients monitored blood oxygen saturation and other readings (e.g. temperature) at home and were escalated as necessary. Objective: To evaluate effectiveness, costs, implementation, and staff and patient experiences (including disparities and mode) of COVID-19 remote home monitoring services in England during the COVID-19 pandemic (waves 1 and 2). Methods: A rapid mixed-methods evaluation, conducted in two phases. Phase 1 (July-August 2020) comprised a rapid systematic review, implementation and economic analysis study (in eight sites). Phase 2 (January-June 2021) comprised a large-scale, multisite, mixed-methods study of effectiveness, costs, implementation and patient/staff experience, using national data sets, surveys (28 sites) and interviews (17 sites). Results: Phase 1 Findings from the review and empirical study indicated that these services have been implemented worldwide and vary substantially. Empirical findings highlighted that communication, appropriate information and multiple modes of monitoring facilitated implementation; barriers included unclear referral processes, workforce availability and lack of administrative support. Phase 2 We received surveys from 292 staff (39% response rate) and 1069 patients/carers (18% response rate). We conducted interviews with 58 staff, 62 patients/carers and 5 national leads. Despite national roll-out, enrolment to services was lower than expected (average enrolment across 37 clinical commissioning groups judged to have completed data was 8.7%). There was large variability in implementation of services, influenced by patient (e.g. local population needs), workforce (e.g. workload), organisational (e.g. collaboration) and resource (e.g. software) factors. We found that for every 10% increase in enrolment to the programme, mortality was reduced by 2% (95% confidence interval: 4% reduction to 1% increase), admissions increased by 3% (-1% to 7%), in-hospital mortality fell by 3% (-8% to 3%) and lengths of stay increased by 1.8% (-1.2% to 4.9%). None of these results are statistically significant. We found slightly longer hospital lengths of stay associated with virtual ward services (adjusted incidence rate ratio 1.05, 95% confidence interval 1.01 to 1.09), and no statistically significant impact on subsequent COVID-19 readmissions (adjusted odds ratio 0.95, 95% confidence interval 0.89 to 1.02). Low patient enrolment rates and incomplete data may have affected chances of detecting possible impact. The mean running cost per patient varied for different types of service and mode; and was driven by the number and grade of staff. Staff, patients and carers generally reported positive experiences of services. Services were easy to deliver but staff needed additional training. Staff knowledge/confidence, NHS resources/workload, dynamics between multidisciplinary team members and patients' engagement with the service (e.g. using the oximeter to record and submit readings) influenced delivery. Patients and carers felt services and human contact received reassured them and were easy to engage with. Engagement was conditional on patient, support, resource and service factors. Many sites designed services to suit the needs of their local population. Despite adaptations, disparities were reported across some patient groups. For example, older adults and patients from ethnic minorities reported more difficulties engaging with the service. Tech-enabled models helped to manage large patient groups but did not completely replace phone calls. Limitations: Limitations included data completeness, inability to link data on service use to outcomes at a patient level, low survey response rates and under-representation of some patient groups. Future work: Further research should consider the long-term impact and cost-effectiveness of these services and the appropriateness of different models for different groups of patients. Conclusions: We were not able to find quantitative evidence that COVID-19 remote home monitoring services have been effective. However, low enrolment rates, incomplete data and varied implementation reduced our chances of detecting any impact that may have existed. While services were viewed positively by staff and patients, barriers to implementation, delivery and engagement should be considered. Study registration: This study is registered with the ISRCTN (14962466). Funding: This project was funded by the National Institute for Health and Care Research (NIHR) Health and Social Care Delivery Research programme (RSET: 16/138/17; BRACE: 16/138/31) and NHSEI and will be published in full in Health and Social Care Delivery Research; Vol. 11, No. 13. See the NIHR Journals Library website for further project information. The views expressed in this publication are those of the authors and not necessarily those of the National Institute for Health and Care Research or the Department of Health and Social Care.

COVID-19 patients can experience very low oxygen levels, without feeling breathless. Patients may not realise there is a problem until they become extremely unwell, risking being admitted to hospital too late. To address this, COVID-19 remote home monitoring services were developed and later rolled out across England. Patients monitored oxygen levels at home using an 'oximeter' (a small device which clips on to your finger) and sent these readings to providers via phone or technology (e.g. an app). Patients could access further care if needed. We did not know whether these services worked, or what people felt about them. • How services were set up and used in England. • Whether services work (e.g. by reducing deaths and length of hospital stay). • How much they cost. • What patients, carers and staff think about these services (including differences between groups and telephone vs. technology). We looked at available existing evidence and collected data from eight services operating in the first wave of the pandemic. During the second wave of the pandemic, we used data available at a national level and conducted surveys (28 sites) and interviews (17 sites) with staff, patients and individuals involved in developing/leading services nationally. These services have been used worldwide, but they vary considerably. We found many things that help these services to be used (e.g. good communication) but also things that get in the way (e.g. unclear referrals). Our findings did not show that services reduce deaths or time in hospital. But these findings are limited by a lack of data. Staff and patients liked these services, but we found some barriers to delivering and using the service. Some groups found services harder to use (e.g. older patients, those with disabilities and ethnic minorities). Using technology helped with large patient groups, but it did not completely replace phone calls. Better information is needed to know whether these services work. Staff and patients liked these services. However, improvements may make them easier to deliver and use (e.g. further staff training and giving additional support to patients who need it).

Undertaking rapid evaluations during the COVID-19 pandemic: Lessons from evaluating COVID-19 remote home monitoring services in England.

Introduction: Rapid evaluations can offer evidence on innovations in health and social care that can be used to inform fast-moving policy and practise, and support their scale-up according to previous research. However, there are few comprehensive accounts of how to plan and conduct large-scale rapid evaluations, ensure scientific rigour, and achieve stakeholder engagement within compressed timeframes. Methods: Using a case study of a national mixed-methods rapid evaluation of COVID-19 remote home monitoring services in England, conducted during the COVID-19 pandemic, this manuscript examines the process of conducting a large-scale rapid evaluation from design to dissemination and impact, and reflects on the key lessons for conducting future large-scale rapid evaluations. In this manuscript, we describe each stage of the rapid evaluation: convening the team (study team and external collaborators), design and planning (scoping, designing protocols, study set up), data collection and analysis, and dissemination. Results: We reflect on why certain decisions were made and highlight facilitators and challenges. The manuscript concludes with 12 key lessons for conducting large-scale mixed-methods rapid evaluations of healthcare services. We propose that rapid study teams need to: (1) find ways of quickly building trust with external stakeholders, including evidence-users; (2) consider the needs of the rapid evaluation and resources needed; (3) use scoping to ensure the study is highly focused; (4) carefully consider what cannot be completed within a designated timeframe; (5) use structured processes to ensure consistency and rigour; (6) be flexible and responsive to changing needs and circumstances; (7) consider the risks associated with new data collection approaches of quantitative data (and their usability); (8) consider whether it is possible to use aggregated quantitative data, and what that would mean when presenting results, (9) consider using structured processes & layered analysis approaches to rapidly synthesise qualitative findings, (10) consider the balance between speed and the size and skills of the team, (11) ensure all team members know roles and responsibilities and can communicate quickly and clearly; and (12) consider how best to share findings, in discussion with evidence-users, for rapid understanding and use. Conclusion: These 12 lessons can be used to inform the development and conduct of future rapid evaluations in a range of contexts and settings.

The impact of post-hospital remote monitoring of COVID-19 patients using pulse oximetry: A national observational study using hospital activity data.

Background: There was a national roll out of 'COVID Virtual Wards' (CVW) during England's second COVID-19 wave (Autumn 2020 - Spring 2021). These services used remote pulse oximetry monitoring for COVID-19 patients following discharge from hospital. A key aim was to enable rapid detection of patient deterioration. It was anticipated that the services would support early discharge, reducing pressure on beds. This study is an evaluation of the impact of the CVW services on hospital activity. Methods: Using retrospective patient-level hospital admissions data, we built multivariate models to analyze the relationship between the implementation of CVW services and hospital activity outcomes: length of COVID-19 related stays and subsequent COVID-19 readmissions within 28 days. We used data from more than 98% of recorded COVID-19 hospital stays in England, where the patient was discharged alive between mid-August 2020 and late February 2021. Findings: We found a longer length of stay for COVID-19 patients discharged from hospitals where a CVW was available, when compared to patients discharged from hospitals where there was no CVW (adjusted IRR 1·05, 95% CI 1·01 to 1·09). We found no evidence of a relationship between the availability of CVW and subsequent rates of readmission for COVID-19 (adjusted OR 0.97, 95% CI 0.91 to 1·03). Interpretation: We found no evidence of early discharges or changes in readmissions associated with the roll out of COVID Virtual Wards across England. Our analysis made pragmatic use of national-scale hospital data, but it is possible that a lack of specific data (for example, on which patients were enrolled and on potentially important confounders) may have meant that true impacts, especially at a local level, were not ultimately discernible. It is important that future research is able to make use of better quality - preferably linked - data, from multiple sites. Funding: This is independent research funded by the National Institute for Health Research, Health Services & Delivery Research program (RSET Project no. 16/138/17; BRACE Project no. 16/138/31) and NHSE&I. NJF is an NIHR Senior Investigator.

The impact of remote home monitoring of people with COVID-19 using pulse oximetry: A national population and observational study.

BACKGROUND: Remote home monitoring of people testing positive for COVID-19 using pulse oximetry was implemented across England during the Winter of 2020/21 to identify falling blood oxygen saturation levels at an early stage. This was hypothesised to enable earlier hospital admission, reduce the need for intensive care and improve survival. This study is an evaluation of the clinical effectiveness of the pre-hospital monitoring programme, COVID oximetry @home (CO@h). METHODS: The setting was all Clinical Commissioning Group (CCG) areas in England where there were complete data on the number of people enrolled onto the programme between 2nd November 2020 and 21st February 2021. We analysed relationships at a geographical area level between the extent to which people aged 65 or over were enrolled onto the programme and outcomes over the period between November 2020 to February 2021. FINDINGS: For every 10% increase in coverage of the programme, mortality was reduced by 2% (95% confidence interval:4% reduction to 1% increase), admissions increased by 3% (-1% to 7%), in-hospital mortality fell by 3% (-8% to 3%) and lengths of stay increased by 1·8% (-1·2% to 4·9%). None of these results are statistically significant, although the confidence interval indicates that any adverse effect on mortality would be small, but a mortality reduction of up to 4% may have resulted from the programme. INTERPRETATION: There are several possible explanations for our findings. One is that CO@h did not have the hypothesised impact. Another is that the low rates of enrolment and incomplete data in many areas reduced the chances of detecting any impact that may have existed. Also, CO@h has been implemented in many different ways across the country and these may have had varying levels of effect. FUNDING: This is independent research funded by the National Institute for Health Research, Health Services & Delivery Research programme (RSET Project no. 16/138/17; BRACE Project no. 16/138/31) and NHSEI. NJF is an NIHR Senior Investigator.

The implementation of remote home monitoring models during the COVID-19 pandemic in England.

BACKGROUND: There is a paucity of evidence for the implementation of remote home monitoring for COVID-19 infection. The aims of this study were to identify the key characteristics of remote home monitoring models for COVID-19 infection, explore the experiences of staff implementing these models, understand the use of data for monitoring progress against outcomes, and document variability in staffing and resource allocation. METHODS: This was a multi-site mixed methods study conducted between July and August 2020 that combined qualitative and quantitative approaches to analyse the implementation and impact of remote home monitoring models developed during the first wave of the COVID-19 pandemic in England. The study combined interviews (n = 22) with staff delivering these models across eight sites in England with the collection and analysis of data on staffing models and resource allocation. FINDINGS: The models varied in relation to the healthcare settings and mechanisms used for patient triage, monitoring and escalation. Implementation was embedded in existing staff workloads and budgets. Good communication within clinical teams, culturally-appropriate information for patients/carers and the combination of multiple approaches for patient monitoring (app and paper-based) were considered facilitators in implementation. The mean cost per monitored patient varied from £400 to £553, depending on the model. INTERPRETATION: It is necessary to provide the means for evaluating the effectiveness of these models, for example, by establishing comparator data. Future research should also focus on the sustainability of the models and patient experience (considering the extent to which some of the models exacerbate existing inequalities in access to care).

Remote home monitoring (virtual wards) for confirmed or suspected COVID-19 patients: a rapid systematic review.

BACKGROUND: the aim of this review was to analyze the implementation and impact of remote home monitoring models (virtual wards) for confirmed or suspected COVID-19 patients, identifying their main components, processes of implementation, target patient populations, impact on outcomes, costs and lessons learnt. METHODS: we carried out a rapid systematic review on models led by primary and secondary care across seven countries (US, Australia, Canada, The Netherlands, Ireland, China, UK). The main outcomes included in the review were: impact of remote home monitoring on virtual length of stay, escalation, emergency department attendance/reattendance, admission/readmission and mortality. The search was updated on February 2021. We used the PRISMA statement and the review was registered on PROSPERO (CRD: 42020202888). FINDINGS: the review included 27 articles. The aim of the models was to maintain patients safe in the appropriate setting. Most models were led by secondary care and confirmation of COVID-19 was not required (in most cases). Monitoring was carried via online platforms, paper-based systems with telephone calls or (less frequently) through wearable sensors. Models based on phone calls were considered more inclusive. Patient/career training was identified as a determining factor of success. We could not reach substantive conclusions regarding patient safety and the identification of early deterioration due to lack of standardized reporting and missing data. Economic analysis was not reported for most of the models and did not go beyond reporting resources used and the amount spent per patient monitored. INTERPRETATION: future research should focus on staff and patient experiences of care and inequalities in patients' access to care. Attention needs to be paid to the cost-effectiveness of the models and their sustainability, evaluation of their impact on patient outcomes by using comparators, and the use of risk-stratification tools.

Is the performance of cancer services influenced more by hospital factors or by specialization?

The Cancer Plan for England, introduced in 2000, has promoted cancer service specialization. We have investigated how far specialization and general hospital factors each contributed to service performance for four common cancers-breast, colorectal, lung and prostate-at the time of the Cancer Plan. Performance measures of service standards, waiting time to treatment, satisfaction with care, in-hospital mortality and population-level survival were identified from secondary data sets for 167 acute hospitals and 34 cancer networks in England. We correlated rankings of networks and hospitals between the data sets using non-parametric statistics. At cancer network level, peer-review service standards were associated (P < 0.05) with 1-year survival for colorectal and lung cancers, and waiting times for lung cancer. At hospital level, standards were associated (P < 0.01) with waiting time to treatment for breast and colorectal cancers. However, there were stronger associations between specializations within hospitals: rankings of breast, colorectal and prostate cancers were highly associated (P < 0.001) for 5-year survival, patient satisfaction, standards and in-hospital mortality. Hospital-level differences appear to contribute more to variations in cancer performance than specialization differences within hospitals. The findings may be used for planning and commissioning better cancer services.

Comparative indicators for cancer network management in England: availability, characteristics and presentation.

BACKGROUND: In 2000, the national cancer plan for England created 34 cancer networks, new organisational structures to coordinate services across populations varying between a half and three million people. We investigated the availability of data sets reflecting measures of structure, process and outcome that could be used to support network management. METHODS: We investigated the properties of national data sets relating to four common cancers - breast, colorectal, lung and prostate. We reviewed the availability and completeness of these data sets, identified leading items within each set and put them into tables of the 34 cancer networks. We also investigated methods of presentation. RESULTS: The Acute Hospitals Portfolio and the Cancer Standards Peer Review recorded structural characteristics at hospital and cancer service level. Process measures included Hospital Episode Statistics, recording admissions, and Hospital Waiting-List data. Patient outcome measures included the National Survey of Patient Satisfaction for cancer, and cancer survival, drawn from cancer registration. Data were drawn together to provide an exemplar indicator set a single network, and methods of graphical presentation were considered. CONCLUSION: While not as yet used together in practice, comparative indicators are available within the National Health Service in England for use in performance assessment by cancer networks.

Comparison of control charts for monitoring clinical performance using binary data.

BACKGROUND: Time series charts are increasingly used by clinical teams to monitor their performance, but statistical control charts are not widely used, partly due to uncertainty about which chart to use. Although there is a large literature on methods, there are few systematic comparisons of charts for detecting changes in rates of binary clinical performance data. METHODS: We compared four control charts for binary data: the Shewhart p-chart; the exponentially weighted moving average (EWMA) chart; the cumulative sum (CUSUM) chart; and the g-chart. Charts were set up to have the same long-term false signal rate. Chart performance was then judged according to the expected number of patients treated until a change in rate was detected. RESULTS: For large absolute increases in rates (>10%), the Shewhart p-chart and EWMA both had good performance, although not quite as good as the CUSUM. For small absolute increases (<10%), the CUSUM detected changes more rapidly. The g-chart is designed to efficiently detect decreases in low event rates, but it again had less good performance than the CUSUM. IMPLICATIONS: The Shewhart p-chart is the simplest chart to implement and interpret, and performs well for detecting large changes, which may be useful for monitoring processes of care. The g-chart is a useful complement for determining the success of initiatives to reduce low-event rates (eg, adverse events). The CUSUM may be particularly useful for faster detection of problems with patient safety leading to increases in adverse event rates. .

Quality assurance in congenital heart surgery.

OBJECTIVE: The aim of this study was to develop a graphical method of risk-stratified outcome analysis in paediatric cardiac surgery to provide a means of continuous, prospective performance monitoring and allow real-time detection of change in outcomes. METHODS: Risk-adjusted survival following open-heart surgery was prospectively measured over a 15-month period (n=460). Outcomes were charted using variable life-adjusted display (VLAD) charts, which indicate the cumulative difference in observed minus expected survival against the cumulative number of cases performed. Risk stratification was based on RACHS-1 (risk adjustment in congenital heart surgery) risk category and age at surgery, using our previously published risk model. The probability of deviation in performance from the expected baseline level was determined using a mathematical model. RESULTS: By the end of the series, observed survival (443/460=96.3%) exceeded that predicted by the risk model (434.5/460=94.5%), equivalent to a one-third reduction in expected mortality. Mathematical modelling indicated a 1-5% likelihood that this difference would have occurred by random variation alone, suggesting the outcomes represented genuine improvement. CONCLUSIONS: VLAD charts provide an effective, easily visualised display of surgical performance and can be applied to paediatric cardiac surgery. Early detection of change, whether improvement or deterioration, is important for ongoing quality assurance within a cardiac surgery programme.

Real-time monitoring of coronary care mortality: a comparison and combination of two monitoring tools.

BACKGROUND: Quality control in clinical care is becoming increasingly more prevalent, resulting in a need for tools that can be readily used by clinical teams for monitoring their own performance. The aim of this study was to devise a practical and effective scheme for monitoring coronary care mortality in real-time. METHODS: We obtained data for 2153 consecutive patients admitted after acute myocardial infarction between 1st September and 30th November 1995 to one of 20 acute hospitals in West Yorkshire participating in the NHS R and D funded EMMACE-1 study. We developed control charts for each centre to monitor 30-day mortality. These control charts used two complementary tools: the Risk-Adjusted Cumulative Sum chart (CUSUM) and a new 'Rocket Tail' chart based upon the Variable Life-Adjusted Display (VLAD). We also combined information from each of these to devise a further chart. RESULTS: Control charts are shown for two centres in order to illustrate the important features of the different but complimentary monitoring tools. The Risk-Adjusted CUSUM is shown to be useful for detecting isolated runs of unsatisfactory outcome results but is not 'intuitive', and does not give any indication of the preceding history of outcomes. The Rocket Tail chart is shown to give a good summary of outcome history and also how overall performance compares with what would be expected for the case-mix. A chart that combines both approaches appeals to the advantages of each. CONCLUSIONS: We propose a visual approach to health-care monitoring that beneficially combines and extends the different information of the previously used CUSUM and VLAD charts.

A method for detecting runs of good and bad clinical outcomes on Variable Life-Adjusted Display (VLAD) charts.

In recent years there has been a growing need for effective monitoring of clinical outcomes. Two techniques for continuous monitoring that have emerged almost simultaneously are the Variable Life-Adjusted Display (VLAD) and risk-adjusted cumulative sum charts (CUSUM). The VLAD provides clinicians and management with an easily understandable overview of outcome history and is now in routine use in several hospitals. Although it can indicate runs of good and bad outcomes, unlike the CUSUM, it does not provide a quantitative means for assessing whether they merit investigation. This paper introduces a scheme for applying control limits from CUSUM charts onto the VLAD, thus enhancing its role as an effective monitoring tool.