Characterising the nationwide burden and predictors of unkept outpatient appointments in the National Health Service in England: A cohort study using a machine learning approach.

BACKGROUND: Unkept outpatient hospital appointments cost the National Health Service £1 billion each year. Given the associated costs and morbidity of unkept appointments, this is an issue requiring urgent attention. We aimed to determine rates of unkept outpatient clinic appointments across hospital trusts in the England. In addition, we aimed to examine the predictors of unkept outpatient clinic appointments across specialties at Imperial College Healthcare NHS Trust (ICHT). Our final aim was to train machine learning models to determine the effectiveness of a potential intervention in reducing unkept appointments. METHODS AND FINDINGS: UK Hospital Episode Statistics outpatient data from 2016 to 2018 were used for this study. Machine learning models were trained to determine predictors of unkept appointments and their relative importance. These models were gradient boosting machines. In 2017-2018 there were approximately 85 million outpatient appointments, with an unkept appointment rate of 5.7%. Within ICHT, there were almost 1 million appointments, with an unkept appointment rate of 11.2%. Hepatology had the highest rate of unkept appointments (17%), and medical oncology had the lowest (6%). The most important predictors of unkept appointments included the recency (25%) and frequency (13%) of previous unkept appointments and age at appointment (10%). A sensitivity of 0.287 was calculated overall for specialties with at least 10,000 appointments in 2016-2017 (after data cleaning). This suggests that 28.7% of patients who do miss their appointment would be successfully targeted if the top 10% least likely to attend received an intervention. As a result, an intervention targeting the top 10% of likely non-attenders, in the full population of patients, would be able to capture 28.7% of unkept appointments if successful. Study limitations include that some unkept appointments may have been missed from the analysis because recording of unkept appointments is not mandatory in England. Furthermore, results here are based on a single trust in England, hence may not be generalisable to other locations. CONCLUSIONS: Unkept appointments remain an ongoing concern for healthcare systems internationally. Using machine learning, we can identify those most likely to miss their appointment and implement more targeted interventions to reduce unkept appointment rates.

Enriching the Value of Patient Experience Feedback: Web-Based Dashboard Development Using Co-design and Heuristic Evaluation.

BACKGROUND: There is an abundance of patient experience data held within health care organizations, but stakeholders and staff are often unable to use the output in a meaningful and timely way to improve care delivery. Dashboards, which use visualized data to summarize key patient experience feedback, have the potential to address these issues. OBJECTIVE: The aim of this study is to develop a patient experience dashboard with an emphasis on Friends and Family Test (FFT) reporting, as per the national policy drive. METHODS: A 2-stage approach was used-participatory co-design involving 20 co-designers to develop a dashboard prototype, followed by iterative dashboard testing. Language analysis was performed on free-text patient experience data from the FFT, and the themes and sentiments generated were used to populate the dashboard with associated FFT metrics. Heuristic evaluation and usability testing were conducted to refine the dashboard and assess user satisfaction using the system usability score. RESULTS: The qualitative analysis from the co-design process informed the development of the dashboard prototype with key dashboard requirements and a significant preference for bubble chart display. The heuristic evaluation revealed that most cumulative scores had no usability problems (18/20, 90%), had cosmetic problems only (7/20, 35%), or had minor usability problems (5/20, 25%). The mean System Usability Scale score was 89.7 (SD 7.9), suggesting an excellent rating. CONCLUSIONS: The growing capacity to collect and process patient experience data suggests that data visualization will be increasingly important in turning feedback into improvements to care. Through heuristic usability, we demonstrated that very large FFT data can be presented in a thematically driven, simple visual display without the loss of the nuances and still allow for the exploration of the original free-text comments. This study establishes guidance for optimizing the design of patient experience dashboards that health care providers find meaningful, which in turn drives patient-centered care.

Using natural language processing to understand, facilitate and maintain continuity in patient experience across transitions of care.

BACKGROUND: Patient centred care necessitates that healthcare experiences and perceived outcomes be considered across all transitions of care. Information encoded within free-text patient experience comments relating to transitions of care are not captured in a systematic way due to the manual resource required. We demonstrate the use of natural language processing (NLP) to extract meaningful information from the Friends and Family Test (FFT). METHODS: Free-text fields identifying favourable service ("What did we do well?") and areas requiring improvement ("What could we do better?") were extracted from 69,285 FFT reports across four care settings at a secondary care National Health Service (NHS) hospital. Sentiment and patient experience themes were coded by three independent coders to produce a training dataset. The textual data was standardised with a series of pre-processing techniques and the performance of six machine learning (ML) models was obtained. The best performing ML model was applied to predict the themes and sentiment from the remaining reports. Comments relating to transitions of care were extracted, categorised by sentiment, and care setting to identify the most frequent words/combinations presented as tri-grams and word clouds. RESULTS: The support vector machine (SVM) ML model produced the highest accuracy in predicting themes and sentiment. The most frequent single words relating to transition and continuity with a negative sentiment were "discharge" in inpatients and Accident and Emergency, "appointment" in outpatients, and "home' in maternity. Tri-grams identified from the negative sentiments such as 'seeing different doctor', 'information aftercare lacking', 'improve discharge process' and 'timing discharge letter' have highlighted some of the problems with care transitions. None of this information was available from the quantitative data. CONCLUSIONS: NLP can be used to identify themes and sentiment from patient experience survey comments relating to transitions of care in all four healthcare settings. With the help of a quality improvement framework, findings from our analysis may be used to guide patient-centred interventions to improve transitional care processes.

Best practices in the real-world data life cycle.

With increasing digitization of healthcare, real-world data (RWD) are available in greater quantity and scope than ever before. Since the 2016 United States 21st Century Cures Act, innovations in the RWD life cycle have taken tremendous strides forward, largely driven by demand for regulatory-grade real-world evidence from the biopharmaceutical sector. However, use cases for RWD continue to grow in number, moving beyond drug development, to population health and direct clinical applications pertinent to payors, providers, and health systems. Effective RWD utilization requires disparate data sources to be turned into high-quality datasets. To harness the potential of RWD for emerging use cases, providers and organizations must accelerate life cycle improvements that support this process. We build on examples obtained from the academic literature and author experience of data curation practices across a diverse range of sectors to describe a standardized RWD life cycle containing key steps in production of useful data for analysis and insights. We delineate best practices that will add value to current data pipelines. Seven themes are highlighted that ensure sustainability and scalability for RWD life cycles: data standards adherence, tailored quality assurance, data entry incentivization, deploying natural language processing, data platform solutions, RWD governance, and ensuring equity and representation in data.

Applying natural language processing and machine learning techniques to patient experience feedback: a systematic review.

OBJECTIVES: Unstructured free-text patient feedback contains rich information, and analysing these data manually would require a lot of personnel resources which are not available in most healthcare organisations.To undertake a systematic review of the literature on the use of natural language processing (NLP) and machine learning (ML) to process and analyse free-text patient experience data. METHODS: Databases were systematically searched to identify articles published between January 2000 and December 2019 examining NLP to analyse free-text patient feedback. Due to the heterogeneous nature of the studies, a narrative synthesis was deemed most appropriate. Data related to the study purpose, corpus, methodology, performance metrics and indicators of quality were recorded. RESULTS: Nineteen articles were included. The majority (80%) of studies applied language analysis techniques on patient feedback from social media sites (unsolicited) followed by structured surveys (solicited). Supervised learning was frequently used (n=9), followed by unsupervised (n=6) and semisupervised (n=3). Comments extracted from social media were analysed using an unsupervised approach, and free-text comments held within structured surveys were analysed using a supervised approach. Reported performance metrics included the precision, recall and F-measure, with support vector machine and Naïve Bayes being the best performing ML classifiers. CONCLUSION: NLP and ML have emerged as an important tool for processing unstructured free text. Both supervised and unsupervised approaches have their role depending on the data source. With the advancement of data analysis tools, these techniques may be useful to healthcare organisations to generate insight from the volumes of unstructured free-text data.

Are disruptive innovations recognised in the healthcare literature? A systematic review.

The study aims to conduct a systematic review to characterise the spread and use of the concept of 'disruptive innovation' within the healthcare sector. We aim to categorise references to the concept over time, across geographical regions and across prespecified healthcare domains. From this, we further aim to critique and challenge the sector-specific use of the concept. PubMed, Medline, Embase, Global Health, PsycINFO, Maternity and Infant Care, and Health Management Information Consortium were searched from inception to August 2019 for references pertaining to disruptive innovations within the healthcare industry. The heterogeneity of the articles precluded a meta-analysis, and neither quality scoring of articles nor risk of bias analyses were required. 245 articles that detailed perceived disruptive innovations within the health sector were identified. The disruptive innovations were categorised into seven domains: basic science (19.2%), device (12.2%), diagnostics (4.9%), digital health (21.6%), education (5.3%), processes (17.6%) and technique (19.2%). The term has been used with increasing frequency annually and is predominantly cited in North American (78.4%) and European (15.2%) articles. The five most cited disruptive innovations in healthcare are 'omics' technologies, mobile health applications, telemedicine, health informatics and retail clinics. The concept 'disruptive innovation' has diffused into the healthcare industry. However, its use remains inconsistent and the recognition of disruption is obscured by other types of innovation. The current definition does not accommodate for prospective scouting of disruptive innovations, a likely hindrance to policy makers. Redefining disruptive innovation within the healthcare sector is therefore crucial for prospectively identifying cost-effective innovations.

From EHR to PHR: let's get the record straight.

This article reflects on the changing nature of health information access and the transition of focus from electronic health records (EHRs) to personal health records (PHRs) along with the challenges and need for alignment of national initiatives for EHR and PHR in the National Health Service (NHS) of the UK. The importance of implementing integrated EHRs as a route to enhance the quality of health delivery has been increasingly understood. EHRs, however, carry several limitations that include major fragmentation through multiple providers and protocols throughout the NHS. Questions over ownership and control of data further complicate the potential for fully utilising records. Analysing the previous initiatives and the current landscape, we identify that adopting a patient health record system can empower patients and allow better harmonisation of clinical data at a national level. We propose regional PHR 'hubs' to provide a universal interface that integrates digital health data at a regional level with further integration at a national level. We propose that these PHR hubs will reduce the complexity of connections, decrease governance challenges and interoperability issues while also providing a safe platform for high-quality scalable and sustainable digital solutions, including artificial intelligence across the UK NHS, serving as an exemplar for other countries which wish to realise the full value of healthcare records.

The effects and preventability of 2627 patient safety incidents related to health information technology failures: a retrospective analysis of 10 years of incident reporting in England and Wales.

BACKGROUND: The use of health information technology (IT) is rapidly increasing to support improvements in the delivery of care. Although health IT is delivering huge benefits, new technology can also introduce unique risks. Despite these risks, evidence on the preventability and effects of health IT failures on patients is scarce. In our study we therefore sought to evaluate the preventability and effects of health IT failures by examining patient safety incidents in England and Wales. METHODS: We designed our study as a retrospective analysis of 10 years of incident reporting in England and Wales. We used text mining with the words "computer", "system", "workstation", and "network" to explore free-text incident descriptors to identify incidents related to health IT failures following a previously described approach. We then applied an n-gram model of searching to identify contiguous sequences of words and provide spatial context. We examined incident details, recorded harm, and preventability. Standard descriptive statistics were applied. Degree of harm was identified according to standardised definitions and preventability was assessed by two independent reviewers. FINDINGS: We identified 2627 incidents related to health IT failures. 2557 (97%) of 2627 incidents were assessed for harm (70 incidents were excluded). 2106 (82%) of 2557 health IT failures caused no harm to patients, 331 (13%) caused low harm, 102 (4%) caused moderate harm, 14 (1%) caused severe harm, and four (<1%) contributed to the death of a patient. 1964 (75%) of 2627 incidents were deemed to be preventable. INTERPRETATION: Health IT is fundamental to the delivery of high-quality care, yet there is a poor understanding of the effects of IT failures on patient safety and whether they can be prevented. Failures are complex and involve interlinked aspects of technology, people, and the environment. Health IT failures are undoubtedly a potential source of substantial harm, but they are likely to be under-reported. Worryingly, three-quarters of IT failures are potentially preventable. There is a need to see health IT as a fundamental tenet of patient safety, develop better methods for capturing the effects of IT failures on patients, and adopt simple measures to reduce their probability and mitigate their risk. FUNDING: The National Institutes of Health Research Imperial Patient Safety Translational Research Centre at Imperial College London.