A unified machine learning approach to time series forecasting applied to demand at emergency departments.

BACKGROUND: There were 25.6 million attendances at Emergency Departments (EDs) in England in 2019 corresponding to an increase of 12 million attendances over the past ten years. The steadily rising demand at EDs creates a constant challenge to provide adequate quality of care while maintaining standards and productivity. Managing hospital demand effectively requires an adequate knowledge of the future rate of admission. We develop a novel predictive framework to understand the temporal dynamics of hospital demand. METHODS: We compare and combine state-of-the-art forecasting methods to predict hospital demand 1, 3 or 7 days into the future. In particular, our analysis compares machine learning algorithms to more traditional linear models as measured in a mean absolute error (MAE) and we consider two different hyperparameter tuning methods, enabling a faster deployment of our models without compromising performance. We believe our framework can readily be used to forecast a wide range of policy relevant indicators. RESULTS: We find that linear models often outperform machine learning methods and that the quality of our predictions for any of the forecasting horizons of 1, 3 or 7 days are comparable as measured in MAE. Our approach is able to predict attendances at these emergency departments one day in advance up to a mean absolute error of ±14 and ±10 patients corresponding to a mean absolute percentage error of 6.8% and 8.6% respectively. CONCLUSIONS: Simple linear methods like generalized linear models are often better or at least as good as ensemble learning methods like the gradient boosting or random forest algorithm. However, though sophisticated machine learning methods are not necessarily better than linear models, they improve the diversity of model predictions so that stacked predictions can be more robust than any single model including the best performing one.

Clinical and laboratory evaluation of SARS-CoV-2 lateral flow assays for use in a national COVID-19 seroprevalence survey.

BACKGROUND: Accurate antibody tests are essential to monitor the SARS-CoV-2 pandemic. Lateral flow immunoassays (LFIAs) can deliver testing at scale. However, reported performance varies, and sensitivity analyses have generally been conducted on serum from hospitalised patients. For use in community testing, evaluation of finger-prick self-tests, in non-hospitalised individuals, is required. METHODS: Sensitivity analysis was conducted on 276 non-hospitalised participants. All had tested positive for SARS-CoV-2 by reverse transcription PCR and were ≥21 days from symptom onset. In phase I, we evaluated five LFIAs in clinic (with finger prick) and laboratory (with blood and sera) in comparison to (1) PCR-confirmed infection and (2) presence of SARS-CoV-2 antibodies on two 'in-house' ELISAs. Specificity analysis was performed on 500 prepandemic sera. In phase II, six additional LFIAs were assessed with serum. FINDINGS: 95% (95% CI 92.2% to 97.3%) of the infected cohort had detectable antibodies on at least one ELISA. LFIA sensitivity was variable, but significantly inferior to ELISA in 8 out of 11 assessed. Of LFIAs assessed in both clinic and laboratory, finger-prick self-test sensitivity varied from 21% to 92% versus PCR-confirmed cases and from 22% to 96% versus composite ELISA positives. Concordance between finger-prick and serum testing was at best moderate (kappa 0.56) and, at worst, slight (kappa 0.13). All LFIAs had high specificity (97.2%-99.8%). INTERPRETATION: LFIA sensitivity and sample concordance is variable, highlighting the importance of evaluations in setting of intended use. This rigorous approach to LFIA evaluation identified a test with high specificity (98.6% (95%CI 97.1% to 99.4%)), moderate sensitivity (84.4% with finger prick (95% CI 70.5% to 93.5%)) and moderate concordance, suitable for seroprevalence surveys.

Quality improvement education for medical students: a near-peer pilot study.

BACKGROUND: Quality improvement (QI) is an essential component of modern clinical practice. Front-line professionals offer valuable perspectives on areas for improvement and are motivated to deliver change. In the UK, all junior doctors are expected to participate in QI in order to advance to the next stage of their training. However, UK undergraduates receive no standardized training in QI methods. This is perpetuated within medical schools by a lack of teaching capacity and competing priorities, and may lead to tokenistic engagement with future QI projects. METHODS: We describe a near-peer teaching programme designed to introduce students to QI methods. This pilot study was conceived and delivered in full by junior doctors and used existing resources to ensure high quality teaching content. 111 fifth-year medical students from the University of Cambridge were taught in interactive, participative workshops that encourage them to develop their own QI change ideas and projects. Core topics included the model for improvement, driver diagrams, stakeholder engagement, measurement for improvement and analysing and presenting data. Students completed surveys before and immediately after this intervention to assess their understanding of and confidence in utilizing QI methods. Questionnaires were also completed by junior doctor tutors. RESULTS: Analysis of questionnaires completed before and immediately after the intervention revealed statistically significant improvements in students' self-reported understanding of QI (p < 0.05) and confidence in applying techniques to their own work (p < 0.05). Students expressed a preference for QI teaching delivered by junior doctors, citing a relaxed learning environment and greater relevance to their stage of training. Tutors reported increased confidence in using QI techniques and a greater willingness to engage with QI in future. CONCLUSIONS: In this single-centre study, near-peer teaching produced significant improvements in students' self-reported understanding of QI and confidence in applying QI methods. Near-peer teaching may constitute a sustainable means of teaching essential QI skills at undergraduate level. Future work must evaluate objective measures of student engagement with and competence in conducting QI.

Understanding healthcare processes: how marginal gains can improve quality and value for children and families.

Understanding how to identify and improve clinical pathways has proven a key tool in quality improvement. These techniques originated beyond healthcare, but are increasingly applied to healthcare. This paper outlines the history of the technique transfer and how to use in clinical practice.

The impact of atypical intrahospital transfers on patient outcomes: a mixed methods study.

The architectural design of hospitals worldwide is centred around individual departments, which require the movement of patients between wards. However, patients do not always take the simplest route from admission to discharge, but can experience convoluted movement patterns, particularly when bed availability is low. Few studies have explored the impact of these rarer, atypical trajectories. Using a mixed-method explanatory sequential study design, we firstly used three continuous years of electronic health record data prior to the Covid-19 pandemic, from 55,152 patients admitted to a London hospital network to define the ward specialities by patient type using the Herfindahl-Hirschman index. We explored the impact of 'regular transfers' between pairs of wards with shared specialities, 'atypical transfers' between pairs of wards with no shared specialities and 'site transfers' between pairs of wards in different hospital site locations, on length of stay, 30-day readmission and mortality. Secondly, to understand the possible reasons behind atypical transfers we conducted three focus groups and three in-depth interviews with site nurse practitioners and bed managers within the same hospital network. We found that at least one atypical transfer was experienced by 12.9% of patients. Each atypical transfer is associated with a larger increase in length of stay, 2.84 days (95% CI 2.56-3.12), compared to regular transfers, 1.92 days (95% CI 1.82-2.03). No association was found between odds of mortality, or 30-day readmission and atypical transfers after adjusting for confounders. Atypical transfers appear to be driven by complex patient conditions, a lack of hospital capacity, the need to reach specific services and facilities, and more exceptionally, rare events such as major incidents. Our work provides an important first step in identifying unusual patient movement and its impacts on key patient outcomes using a system-wide, data-driven approach. The broader impact of moving patients between hospital wards, and possible downstream effects should be considered in hospital policy and service planning.

The use of a consultant-led ward round checklist to improve paediatric prescribing: an interrupted time series study.

UNLABELLED: A Check and Correct checklist has previously been developed to increase feedback on prescribing quality and enhance physicians' focus on patients' drug charts during ward rounds. Our objective was to assess the impact of introducing such a prescribing checklist on the quality and safety of inpatient prescribing in two paediatric wards in a London teaching hospital. Between 15 March 2011 and 15 May 2011 (pre-intervention) and between 23 May 2011 and 23 July 2011 (post-intervention), we recorded rates of both technical prescription writing errors and clinical prescribing errors twice a week. During the pre-intervention period, the overall technical error rate was 10.8 % (95 % confidence interval 10.3 %-11.2 %); the clinical error rate was 4.7 % (3.4 %-6.6 %). The most common errors were absence of prescriber's contact details and dose omissions. After the implementation of Check and Correct, error rates were 7.3 % (6.9 %-7.8 %) and 5.5 % (3.9 %-7.9 %), respectively. Segmented regression analysis revealed a significant decrease of -5.0 % in the technical error rate (-7.1 to -2.9 %; -37.7 % relative decrease; R (2) = 0.604) following the intervention, independent of changes in overall medical records' documentation quality. Regarding clinical errors, no significant impact of the intervention could be detected. CONCLUSION: Implementing a Check and Correct checklist led to an improvement in the quality of prescription writing. Although a change in culture may be needed to maximise its potential, we would recommend its more widespread use and evaluation.

Maximising learning on ward rounds.

This study considers different ways of maximising learning opportunities during ward rounds, with particular emphasis on the strengths and challenges of the paediatric environment. The focus is on the most common types of ward round - in acute units involving predominantly trainees - but we hope there will also be much that will interest those who work in other settings such as community clinics. Alongside a review of the best available evidence from the literature, and underpinned by educational theory, suggestions for maximising learning on ward rounds are presented. Many of these ideas were generated from working in small groups with over 90 experienced paediatricians, each with particular experience and interest in medical education, as part of the Royal College of Paediatrics and Child Health's Paediatric Educators' Programme, the PEP.

Enriching the Value of Patient Experience Feedback: Web-Based Dashboard Development Using Co-design and Heuristic Evaluation.

BACKGROUND: There is an abundance of patient experience data held within health care organizations, but stakeholders and staff are often unable to use the output in a meaningful and timely way to improve care delivery. Dashboards, which use visualized data to summarize key patient experience feedback, have the potential to address these issues. OBJECTIVE: The aim of this study is to develop a patient experience dashboard with an emphasis on Friends and Family Test (FFT) reporting, as per the national policy drive. METHODS: A 2-stage approach was used-participatory co-design involving 20 co-designers to develop a dashboard prototype, followed by iterative dashboard testing. Language analysis was performed on free-text patient experience data from the FFT, and the themes and sentiments generated were used to populate the dashboard with associated FFT metrics. Heuristic evaluation and usability testing were conducted to refine the dashboard and assess user satisfaction using the system usability score. RESULTS: The qualitative analysis from the co-design process informed the development of the dashboard prototype with key dashboard requirements and a significant preference for bubble chart display. The heuristic evaluation revealed that most cumulative scores had no usability problems (18/20, 90%), had cosmetic problems only (7/20, 35%), or had minor usability problems (5/20, 25%). The mean System Usability Scale score was 89.7 (SD 7.9), suggesting an excellent rating. CONCLUSIONS: The growing capacity to collect and process patient experience data suggests that data visualization will be increasingly important in turning feedback into improvements to care. Through heuristic usability, we demonstrated that very large FFT data can be presented in a thematically driven, simple visual display without the loss of the nuances and still allow for the exploration of the original free-text comments. This study establishes guidance for optimizing the design of patient experience dashboards that health care providers find meaningful, which in turn drives patient-centered care.

Defining the Enablers and Barriers to the Implementation of Large-scale, Health Care-Related Mobile Technology: Qualitative Case Study in a Tertiary Hospital Setting.

BACKGROUND: The successful implementation of clinical smartphone apps in hospital settings requires close collaboration with industry partners. A large-scale, hospital-wide implementation of a clinical mobile app for health care professionals developed in partnership with Google Health and academic partners was deployed on a bring-your-own-device basis using mobile device management at our UK academic hospital. As this was the first large-scale implementation of this type of innovation in the UK health system, important insights and lessons learned from the deployment may be useful to other organizations considering implementing similar technology in partnership with commercial companies. OBJECTIVE: The aims of this study are to define the key enablers and barriers and to propose a road map for the implementation of a hospital-wide clinical mobile app developed in collaboration with an industry partner as a data processor and an academic partner for independent evaluation. METHODS: Semistructured interviews were conducted with high-level stakeholders from industry, academia, and health care providers who had instrumental roles in the implementation of the app at our hospital. The interviews explored the participants' views on the enablers and barriers to the implementation process. The interviews were analyzed using a broadly deductive approach to thematic analysis. RESULTS: In total, 14 participants were interviewed. Key enablers identified were the establishment of a steering committee with high-level clinical involvement, well-defined roles and responsibilities between partners, effective communication strategies with end users, safe information governance precautions, and increased patient engagement and transparency. Barriers identified were the lack of dedicated resources for mobile change at our hospital, risk aversion, unclear strategy and regulation, and the implications of bring-your-own-device and mobile device management policies. The key lessons learned from the deployment process were highlighted, and a road map for the implementation of large-scale clinical mobile apps in hospital settings was proposed. CONCLUSIONS: Despite partnering with one of the world's biggest technology companies, the cultural and technological change required for mobile working and implementation in health care was found to be a significant challenge. With an increasing requirement for health care organizations to partner with industry for advanced mobile technologies, the lessons learned from our implementation can influence how other health care organizations undertake a similar mobile change and improve the chances of successful widespread mobile transformation.

Identifying factors that promote and limit the effective use of real-time patient experience feedback: a mixed-methods study in secondary care.

OBJECTIVES: The Friends and Family Test (FFT) is commissioned by the National Health Service (NHS) in England to capture patient experience as a real-time feedback initiative for patient-centred quality improvement (QI). The aim of this study was to create a process map in order to identify the factors that promote and limit the effective use of FFT as a real-time feedback initiative for patient-centred QI. SETTING: This study was conducted at a large London NHS Trust. Services include accident and emergency, inpatient, outpatient and maternity, which routinely collect FFT patient experience data. PARTICIPANTS: Healthcare staff and key stakeholders involved in FFT. INTERVENTIONS: Semi-structured interviews were conducted on 15 participants from a broad range of professional groups to evaluate their engagement with the FFT. Interview data were recorded, transcribed and analysed for using deductive thematic analysis. RESULTS: Concerns related to inefficiency in the flow of FFT data, lack of time to analyse FFT reports (with emphasis on high level reporting rather than QI), insufficient access to FFT reports and limited training provided to understand FFT reports for frontline staff. The sheer volume of data received was not amenable to manual thematic analysis resulting in inability to acquire insight from the free text. This resulted in staff ambivalence towards FFT as a near real-time feedback initiative. CONCLUSIONS: The results state that there is too much FFT free text for meaningful analysis, and the output is limited to the provision of sufficient capacity and resource to analyse the data, without consideration of other options, such as text analytics and amending the data collection tool.

Population child health: understanding and addressing complex health needs.

Advances in paediatric care mean that more children with complex medical problems (heart disease, neurodevelopmental problems and so on) are surviving their early years. This has important implications for the design and delivery of healthcare given their extensive multidisciplinary requirements and susceptibility to poor outcomes when not optimally managed. Importantly, their medical needs must also be understood and addressed within the context of the child and family's life circumstances. There is growing recognition that many other factors contribute to a child's complex health needs (CHNs), for example, family problems, fragmentation of health and care provision, psychological difficulties or social issues.To facilitate proactive care for these patients, we must develop accurate ways to identify them. Whole Systems Integrated Care-an online platform that integrates routinely collected data from primary and secondary care-offers an example of how to do this. An algorithm applied to this data identifies children with CHNs from the entire patient population. When tested in a large inner-city GP practice, this analysis shows good concordance with clinical opinion and identifies complex children in the population to a much higher proportion than expected. Ongoing refinement of these data-driven processes will allow accurate quantification and identification of need in local populations, thus aiding the development of tailored services.

SARS-CoV-2 lateral flow assays for possible use in national covid-19 seroprevalence surveys (React 2): diagnostic accuracy study.

OBJECTIVE: To evaluate the performance of new lateral flow immunoassays (LFIAs) suitable for use in a national coronavirus disease 2019 (covid-19) seroprevalence programme (real time assessment of community transmission 2-React 2). DESIGN: Diagnostic accuracy study. SETTING: Laboratory analyses were performed in the United Kingdom at Imperial College, London and university facilities in London. Research clinics for finger prick sampling were run in two affiliated NHS trusts. PARTICIPANTS: Sensitivity analyses were performed on sera stored from 320 previous participants in the React 2 programme with confirmed previous severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection. Specificity analyses were performed on 1000 prepandemic serum samples. 100 new participants with confirmed previous SARS-CoV-2 infection attended study clinics for finger prick testing. INTERVENTIONS: Laboratory sensitivity and specificity analyses were performed for seven LFIAs on a minimum of 200 serum samples from participants with confirmed SARS-CoV-2 infection and 500 prepandemic serum samples, respectively. Three LFIAs were found to have a laboratory sensitivity superior to the finger prick sensitivity of the LFIA currently used in React 2 seroprevalence studies (84%). These LFIAs were then further evaluated through finger prick testing on participants with confirmed previous SARS-CoV-2 infection: two LFIAs (Surescreen, Panbio) were evaluated in clinics in June-July 2020 and the third LFIA (AbC-19) in September 2020. A spike protein enzyme linked immunoassay and hybrid double antigen binding assay were used as laboratory reference standards. MAIN OUTCOME MEASURES: The accuracy of LFIAs in detecting immunoglobulin G (IgG) antibodies to SARS-CoV-2 compared with two reference standards. RESULTS: The sensitivity and specificity of seven new LFIAs that were analysed using sera varied from 69% to 100%, and from 98.6% to 100%, respectively (compared with the two reference standards). Sensitivity on finger prick testing was 77% (95% confidence interval 61.4% to 88.2%) for Panbio, 86% (72.7% to 94.8%) for Surescreen, and 69% (53.8% to 81.3%) for AbC-19 compared with the reference standards. Sensitivity for sera from matched clinical samples performed on AbC-19 was significantly higher with serum than finger prick at 92% (80.0% to 97.7%, P=0.01). Antibody titres varied considerably among cohorts. The numbers of positive samples identified by finger prick in the lowest antibody titre quarter varied among LFIAs. CONCLUSIONS: One new LFIA was identified with clinical performance suitable for potential inclusion in seroprevalence studies. However, none of the LFIAs tested had clearly superior performance to the LFIA currently used in React 2 seroprevalence surveys, and none showed sufficient sensitivity and specificity to be considered for routine clinical use.

Assessing a novel, lab-free, point-of-care test for SARS-CoV-2 (CovidNudge): a diagnostic accuracy study.

BACKGROUND: Access to rapid diagnosis is key to the control and management of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). Laboratory RT-PCR testing is the current standard of care but usually requires a centralised laboratory and significant infrastructure. We describe our diagnostic accuracy assessment of a novel, rapid point-of-care real time RT-PCR CovidNudge test, which requires no laboratory handling or sample pre-processing. METHODS: Between April and May, 2020, we obtained two nasopharyngeal swab samples from individuals in three hospitals in London and Oxford (UK). Samples were collected from three groups: self-referred health-care workers with suspected COVID-19; patients attending emergency departments with suspected COVID-19; and hospital inpatient admissions with or without suspected COVID-19. For the CovidNudge test, nasopharyngeal swabs were inserted directly into a cartridge which contains all reagents and components required for RT-PCR reactions, including multiple technical replicates of seven SARS-CoV-2 gene targets (rdrp1, rdrp2, e-gene, n-gene, n1, n2 and n3) and human ribonuclease P (RNaseP) as sample adequacy control. Swab samples were tested in parallel using the CovidNudge platform, and with standard laboratory RT-PCR using swabs in viral transport medium for processing in a central laboratory. The primary analysis was to compare the sensitivity and specificity of the point-of-care CovidNudge test with laboratory-based testing. FINDINGS: We obtained 386 paired samples: 280 (73%) from self-referred health-care workers, 15 (4%) from patients in the emergency department, and 91 (23%) hospital inpatient admissions. Of the 386 paired samples, 67 tested positive on the CovidNudge point-of-care platform and 71 with standard laboratory RT-PCR. The overall sensitivity of the point-of-care test compared with laboratory-based testing was 94% (95% CI 86-98) with an overall specificity of 100% (99-100). The sensitivity of the test varied by group (self-referred healthcare workers 94% [95% CI 85-98]; patients in the emergency department 100% [48-100]; and hospital inpatient admissions 100% [29-100]). Specificity was consistent between groups (self-referred health-care workers 100% [95% CI 98-100]; patients in the emergency department 100% [69-100]; and hospital inpatient admissions 100% [96-100]). Point of care testing performance was similar during a period of high background prevalence of laboratory positive tests (25% [95% 20-31] in April, 2020) and low prevalence (3% [95% 1-9] in inpatient screening). Amplification of viral nucleocapsid (n1, n2, and n3) and envelope protein gene (e-gene) were most sensitive for detection of spiked SARS-CoV-2 RNA. INTERPRETATION: The CovidNudge platform was a sensitive, specific, and rapid point of care test for the presence of SARS-CoV-2 without laboratory handling or sample pre-processing. The device, which has been implemented in UK hospitals since May, 2020, could enable rapid decisions for clinical care and testing programmes. FUNDING: National Institute of Health Research (NIHR) Imperial Biomedical Research Centre, NIHR Health Protection Research Unit in Healthcare Associated Infections and Antimicrobial Resistance at Oxford University in partnership with Public Health England, NIHR Biomedical Research Centre Oxford, and DnaNudge.

Understanding case mix across three paediatric services: could integration of primary and secondary general paediatrics alter walk-in emergency attendances?

OBJECTIVE: To understand the case mix of three different paediatric services, reasons for using an acute paediatric service in a region of developing integrated care and where acute attendances could alternatively have been managed. METHODS: Mixed methods service evaluation, including retrospective review of referrals to general paediatric outpatients (n=534) and a virtual integrated service (email advice line) (n=474), as well as a prospective survey of paediatric ambulatory unit (PAU) attendees (n=95) and review by a paediatric consultant/registrar to decide where these cases could alternatively have been managed. RESULTS: The case mix of outpatient referrals and the email advice line was similar, but the case mix for PAU was more acute. The most common parental reasons for attending PAU were referral by a community health professional (27.2%), not being able to get a general practitioner (GP) appointment when desired (21.7%), wanting to avoid accident and emergency (17.4%) and wanting specialist paediatric input (14.1%). More than half of PAU presentations were deemed most appropriate for community management by a GP or midwife. The proportion of cases suitable for community management varied by the reason for attendance, with it highestl for parents reporting not being able to get a GP appointment (85%), and lowest for those referred by community health professionals (29%). CONCLUSIONS: One in two attendances to acute paediatric services could have been managed in the community. Integration of paediatric services could help address parental reasons for attending acute services, as well as facilitating the community management of chronic conditions.

Taking an organisational approach to quality improvement.

The challenge for all healthcare organisations is to develop and implement an approach that will enable improvements to the quality of healthcare to happen. This case study describes some of the thinking, design and learning from Imperial College Healthcare NHS Trust, on the early steps of our journey to create a culture of continuous improvement across the organisation.