Undercounting diagnoses in Australian general practice: a data quality study with implications for population health reporting.

BACKGROUND: Diagnosis can often be recorded in electronic medical records (EMRs) as free-text or using a term with a diagnosis code. Researchers, governments, and agencies, including organisations that deliver incentivised primary care quality improvement programs, frequently utilise coded data only and often ignore free-text entries. Diagnosis data are reported for population healthcare planning including resource allocation for patient care. This study sought to determine if diagnosis counts based on coded diagnosis data only, led to under-reporting of disease prevalence and if so, to what extent for six common or important chronic diseases. METHODS: This cross-sectional data quality study used de-identified EMR data from 84 general practices in Victoria, Australia. Data represented 456,125 patients who attended one of the general practices three or more times in two years between January 2021 and December 2022. We reviewed the percentage and proportional difference between patient counts of coded diagnosis entries alone and patient counts of clinically validated free-text entries for asthma, chronic kidney disease, chronic obstructive pulmonary disease, dementia, type 1 diabetes and type 2 diabetes. RESULTS: Undercounts were evident in all six diagnoses when using coded diagnoses alone (2.57-36.72% undercount), of these, five were statistically significant. Overall, 26.4% of all patient diagnoses had not been coded. There was high variation between practices in recording of coded diagnoses, but coding for type 2 diabetes was well captured by most practices. CONCLUSION: In Australia clinical decision support and the reporting of aggregated patient diagnosis data to government that relies on coded diagnoses can lead to significant underreporting of diagnoses compared to counts that also incorporate clinically validated free-text diagnoses. Diagnosis underreporting can impact on population health, healthcare planning, resource allocation, and patient care. We propose the use of phenotypes derived from clinically validated text entries to enhance the accuracy of diagnosis and disease reporting. There are existing technologies and collaborations from which to build trusted mechanisms to provide greater reliability of general practice EMR data used for secondary purposes.

Attention-based multimodal fusion with contrast for robust clinical prediction in the face of missing modalities.

OBJECTIVE: With the increasing amount and growing variety of healthcare data, multimodal machine learning supporting integrated modeling of structured and unstructured data is an increasingly important tool for clinical machine learning tasks. However, it is non-trivial to manage the differences in dimensionality, volume, and temporal characteristics of data modalities in the context of a shared target task. Furthermore, patients can have substantial variations in the availability of data, while existing multimodal modeling methods typically assume data completeness and lack a mechanism to handle missing modalities. METHODS: We propose a Transformer-based fusion model with modality-specific tokens that summarize the corresponding modalities to achieve effective cross-modal interaction accommodating missing modalities in the clinical context. The model is further refined by inter-modal, inter-sample contrastive learning to improve the representations for better predictive performance. We denote the model as Attention-based cRoss-MOdal fUsion with contRast (ARMOUR). We evaluate ARMOUR using two input modalities (structured measurements and unstructured text), six clinical prediction tasks, and two evaluation regimes, either including or excluding samples with missing modalities. RESULTS: Our model shows improved performances over unimodal or multimodal baselines in both evaluation regimes, including or excluding patients with missing modalities in the input. The contrastive learning improves the representation power and is shown to be essential for better results. The simple setup of modality-specific tokens enables ARMOUR to handle patients with missing modalities and allows comparison with existing unimodal benchmark results. CONCLUSION: We propose a multimodal model for robust clinical prediction to achieve improved performance while accommodating patients with missing modalities. This work could inspire future research to study the effective incorporation of multiple, more complex modalities of clinical data into a single model.

Data-driven overdiagnosis definitions: A scoping review.

INTRODUCTION: Adequate methods to promptly translate digital health innovations for improved patient care are essential. Advances in Artificial Intelligence (AI) and Machine Learning (ML) have been sources of digital innovation and hold the promise to revolutionize the way we treat, manage and diagnose patients. Understanding the benefits but also the potential adverse effects of digital health innovations, particularly when these are made available or applied on healthier segments of the population is essential. One of such adverse effects is overdiagnosis. OBJECTIVE: to comprehensively analyze quantification strategies and data-driven definitions for overdiagnosis reported in the literature. METHODS: we conducted a scoping systematic review of manuscripts describing quantitative methods to estimate the proportion of overdiagnosed patients. RESULTS: we identified 46 studies that met our inclusion criteria. They covered a variety of clinical conditions, primarily breast and prostate cancer. Methods to quantify overdiagnosis included both prospective and retrospective methods including randomized clinical trials, and simulations. CONCLUSION: a variety of methods to quantify overdiagnosis have been published, producing widely diverging results. A standard method to quantify overdiagnosis is needed to allow its mitigation during the rapidly increasing development of new digital diagnostic tools.

Are we ready for conformance checking in healthcare? Measuring adherence to clinical guidelines: A scoping systematic literature review.

Clinical guidelines are recommendations of how to diagnose, treat, and manage a patient's medical condition. Health organizations must measure adherence to clinical guidelines to enhance the quality of service, but due to the complexity of the medical environment, there is no simple way of measuring adherence to clinical guidelines. This scoping review will systematically assess the criteria used to measure adherence to clinical guidelines in the past 20 years and explore the suitability of using process mining techniques. We will use a workflow protocol based on declarative and temporal constraints to translate the narrative text rules in the publications into a high-level process model. This approach will enable us to explore the main patterns and gaps identified when measuring adherence to clinical guidelines and how they affect the adoption of process mining techniques. The main contributions of this paper are a) a comprehensive analysis of the criteria used for measuring adherence, considering a diverse set of medical conditions b) a framework that will classify the level of complexity of the rules used to measure adherence based on declarative and temporal constraints c) list of key trends and gaps identified in the literature and how they relate to the use of process mining techniques in healthcare.

"Note Bloat" impacts deep learning-based NLP models for clinical prediction tasks.

One unintended consequence of the Electronic Health Records (EHR) implementation is the overuse of content-importing technology, such as copy-and-paste, that creates "bloated" notes containing large amounts of textual redundancy. Despite the rising interest in applying machine learning models to learn from real-patient data, it is unclear how the phenomenon of note bloat might affect the Natural Language Processing (NLP) models derived from these notes. Therefore, in this work we examine the impact of redundancy on deep learning-based NLP models, considering four clinical prediction tasks using a publicly available EHR database. We applied two deduplication methods to the hospital notes, identifying large quantities of redundancy, and found that removing the redundancy usually has little negative impact on downstream performances, and can in certain circumstances assist models to achieve significantly better results. We also showed it is possible to attack model predictions by simply adding note duplicates, causing changes of correct predictions made by trained models into wrong predictions. In conclusion, we demonstrated that EHR text redundancy substantively affects NLP models for clinical prediction tasks, showing that the awareness of clinical contexts and robust modeling methods are important to create effective and reliable NLP systems in healthcare contexts.

Process mining-driven analysis of COVID-19's impact on vaccination patterns.

Process mining is a discipline sitting between data mining and process science, whose goal is to provide theoretical methods and software tools to analyse process execution data, known as event logs. Although process mining was originally conceived to facilitate business process management activities, research studies have shown the benefit of leveraging process mining in healthcare contexts. However, applying process mining tools to analyse healthcare process execution data is not straightforward. In this paper, we show a methodology to: i) prepare general practice healthcare process data for conducting a process mining analysis; ii) select and apply suitable process mining solutions for successfully executing the analysis; and iii) extract valuable insights from the obtained results, alongside leads for traditional data mining analysis. By doing so, we identified two major challenges when using process mining solutions for analysing healthcare process data, and highlighted benefits and limitations of the state-of-the-art process mining techniques when dealing with highly variable processes and large data-sets. While we provide solutions to the identified challenges, the overarching goal of this study was to detect differences between the patients' health services utilization pattern observed in 2020-during the COVID-19 pandemic and mandatory lock-downs -and the one observed in the prior four years, 2016 to 2019. By using a combination of process mining techniques and traditional data mining, we were able to demonstrate that vaccinations in Victoria did not drop drastically-as other interactions did. On the contrary, we observed a surge of influenza and pneumococcus vaccinations in 2020, as opposed to other research findings of similar studies conducted in different geographical areas.

Process mining for healthcare: Characteristics and challenges.

Process mining techniques can be used to analyse business processes using the data logged during their execution. These techniques are leveraged in a wide range of domains, including healthcare, where it focuses mainly on the analysis of diagnostic, treatment, and organisational processes. Despite the huge amount of data generated in hospitals by staff and machinery involved in healthcare processes, there is no evidence of a systematic uptake of process mining beyond targeted case studies in a research context. When developing and using process mining in healthcare, distinguishing characteristics of healthcare processes such as their variability and patient-centred focus require targeted attention. Against this background, the Process-Oriented Data Science in Healthcare Alliance has been established to propagate the research and application of techniques targeting the data-driven improvement of healthcare processes. This paper, an initiative of the alliance, presents the distinguishing characteristics of the healthcare domain that need to be considered to successfully use process mining, as well as open challenges that need to be addressed by the community in the future.

Epistemonikos: a comprehensive database of systematic reviews for health decision-making.

BACKGROUND: Systematic reviews allow health decisions to be informed by the best available research evidence. However, their number is proliferating quickly, and many skills are required to identify all the relevant reviews for a specific question. METHODS AND FINDINGS: We screen 10 bibliographic databases on a daily or weekly basis, to identify systematic reviews relevant for health decision-making. Using a machine-based approach developed for this project we select reviews, which are then validated by a network of more than 1000 collaborators. After screening over 1,400,000 records we have identified more than 300,000 systematic reviews, which are now stored in a single place and accessible through an easy-to-use search engine. This makes Epistemonikos the largest database of its kind. CONCLUSIONS: Using a systematic approach, recruiting a broad network of collaborators and implementing automated methods, we developed a one-stop shop for systematic reviews relevant for health decision making.

System-Wide Accelerated Implementation of Telemedicine in Response to COVID-19: Mixed Methods Evaluation.

BACKGROUND: As the COVID-19 pandemic disrupted medical practice, telemedicine emerged as an alternative to outpatient visits. However, it is not known how patients and physicians responded to an accelerated implementation of this model of medical care. OBJECTIVE: The aim of this study is to report the system-wide accelerated implementation of telemedicine, compare patient satisfaction between telemedicine and in-person visits, and report provider perceptions. METHODS: This study was conducted at the UC Christus Health Network, a large private academic health network in Santiago, Chile. The satisfaction of patients receiving telemedicine care in March and April 2020 was compared to those receiving in-person care during the same period (concurrent control group) as well as in March and April 2019 (retrospective control group). Patient satisfaction with in-person care was measured using the Net Promoter Score (NPS) survey. Patient satisfaction with telemedicine was assessed with an online survey assessing similar domains. Providers rated their satisfaction and responded to open-ended questions assessing challenges, strategies used to address challenges, the diagnostic process, treatment, and the patient-provider relationship. RESULTS: A total of 3962 patients receiving telemedicine, 1187 patients from the concurrent control group, and 1848 patients from the retrospective control group completed the surveys. Satisfaction was very high with both telemedicine and in-person services. Overall, 263 physicians from over 41 specialties responded to the survey. During telemedicine visits, most providers felt their clinical skills were challenged (61.8%). Female providers felt more challenged than male providers (70.7% versus 50.9%, P=.002). Surgeons, obstetricians, and gynecologists felt their clinical skills were challenged the least, compared to providers from nonsurgical specialties (P<.001). Challenges related to the delivery modality, diagnostic process, and patient-provider relationship differed by provider specialty (P=.046, P<.001, and P=.02, respectively). CONCLUSIONS: Telemedicine implemented in response to the COVID-19 pandemic produced high patient and provider satisfaction. Specialty groups perceived the impact of this new mode of clinical practice differently.

Discovering role interaction models in the Emergency Room using Process Mining.

OBJECTIVES: A coordinated collaboration among different healthcare professionals in Emergency Room (ER) processes is critical to promptly care for patients who arrive at the hospital in a delicate health condition, claiming for an immediate attention. The aims of this study are (i) to discover role interaction models in (ER) processes using process mining techniques; (ii) to understand how healthcare professionals are currently collaborating; and (iii) to provide useful knowledge that can help to improve ER processes. METHODS: A four step method based on process mining techniques is proposed. An ER process of a university hospital was considered as a case study, using 7160 episodes that contains specific ER episode attributes. RESULTS: Insights about how healthcare professionals collaborate in the ER was discovered, including the identification of a prevalent role interaction model along the major triage categories and specific role interaction models for different diagnoses. Also, common and exceptional professional interaction models were discovered at the role level. CONCLUSIONS: This study allows the discovery of role interaction models through the use of real-life clinical data and process mining techniques. Results show a useful way of providing relevant insights about how healthcare professionals collaborate, uncovering opportunities for process improvement.

Multidisciplinary Collaboration in the Treatment of Patients With Type 2 Diabetes in Primary Care: Analysis Using Process Mining.

BACKGROUND: Public health in several countries is characterized by a shortage of professionals and a lack of economic resources. Monitoring and redesigning processes can foster the success of health care institutions, enabling them to provide a quality service while simultaneously reducing costs. Process mining, a discipline that extracts knowledge from information system data to analyze operational processes, affords an opportunity to understand health care processes. OBJECTIVE: Health care processes are highly flexible and multidisciplinary, and health care professionals are able to coordinate in a variety of different ways to treat a diagnosis. The aim of this work was to understand whether the ways in which professionals coordinate their work affect the clinical outcome of patients. METHODS: This paper proposes a method based on the use of process mining to identify patterns of collaboration between physician, nurse, and dietitian in the treatment of patients with type 2 diabetes mellitus and to compare these patterns with the clinical evolution of the patients within the context of primary care. Clustering is used as part of the preprocessing of data to manage the variability, and then process mining is used to identify patterns that may arise. RESULTS: The method is applied in three primary health care centers in Santiago, Chile. A total of seven collaboration patterns were identified, which differed primarily in terms of the number of disciplines present, the participation intensity of each discipline, and the referrals between disciplines. The pattern in which the three disciplines participated in the most equitable and comprehensive manner had a lower proportion of highly decompensated patients compared with those patterns in which the three disciplines participated in an unbalanced manner. CONCLUSIONS: By discovering which collaboration patterns lead to improved outcomes, health care centers can promote the most successful patterns among their professionals so as to improve the treatment of patients. Process mining techniques are useful for discovering those collaborations patterns in flexible and unstructured health care processes.

Development of mobile technologies for the prevention of cervical cancer in Santiago, Chile study protocol: a randomized controlled trial.

BACKGROUND: In Chile, more than 500 women die every year from cervical cancer, and a majority of Chilean women are not up-to-date with their Papanicolau (Pap) test. Mobile health has great potential in many health areas, particularly in health promotion and prevention. There are no randomized controlled trials in Latin America assessing its use in cervical cancer screening. The 'Development of Mobile Technologies for the Prevention of Cervical Cancer in Santiago, Chile' study aims to determine the efficacy of a text-message intervention on Pap test adherence among Chilean women in the metropolitan region of Santiago. METHODS/DESIGN: This study is a parallel randomized-controlled trial of 400 Chilean women aged 25-64 who are non-adherent with current recommendations for Pap test screening. Participants will be randomly assigned to (1) a control arm (usual care) or (2) an intervention arm, where text and voice messages containing information and encouragement to undergo screening will be sent to the women. The primary endpoint is completion of a Pap test within 6 months of baseline assessment, as determined by medical record review at community-based clinics. Medical record reviewers will be blinded to randomization arms. The secondary endpoint is an evaluation of the implementation and usability of the text message intervention as a strategy to improve screening adherence. DISCUSSION: This intervention using mobile technology intends to raise cervical cancer screening adherence and compliance among a Chilean population of low and middle-low socioeconomic status. If successful, this strategy may reduce the incidence of cervical cancer. TRIAL REGISTRATION: Clinicaltrials.gov NCT02376023 Registered 2/17/2015. First participant enrolled Feb 22nd 2016.

Process mining in healthcare: A literature review.

Process Mining focuses on extracting knowledge from data generated and stored in corporate information systems in order to analyze executed processes. In the healthcare domain, process mining has been used in different case studies, with promising results. Accordingly, we have conducted a literature review of the usage of process mining in healthcare. The scope of this review covers 74 papers with associated case studies, all of which were analyzed according to eleven main aspects, including: process and data types; frequently posed questions; process mining techniques, perspectives and tools; methodologies; implementation and analysis strategies; geographical analysis; and medical fields. The most commonly used categories and emerging topics have been identified, as well as future trends, such as enhancing Hospital Information Systems to become process-aware. This review can: (i) provide a useful overview of the current work being undertaken in this field; (ii) help researchers to choose process mining algorithms, techniques, tools, methodologies and approaches for their own applications; and (iii) highlight the use of process mining to improve healthcare processes.

Effectiveness of eHealth interventions and information needs in palliative care: a systematic literature review.

BACKGROUND: One of the key components in palliative care is communication. eHealth technologies can be an effective way to support communications among participants in the process of palliative care. However, it is unclear to what extent information technology has been established in this field. OBJECTIVE: Our goal was to systematically identify studies and analyze the effectiveness of eHealth interventions in palliative care and the information needs of people involved in the palliative care process. METHODS: We conducted a systematic literature search using PubMed, Embase, and LILACS according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. We collected and analyzed quantitative and qualitative data regarding effectiveness of eHealth interventions and users' information needs in palliative care. RESULTS: Our search returned a total of 240 articles, 17 of which met our inclusion criteria. We found no randomized controlled trial studying the effects of eHealth interventions in palliative care. Studies tended to be observational, non-controlled studies, and a few quasi-experimental studies. Overall there was great heterogeneity in the types of interventions and outcome assessments; some studies reported some improvement on quality of care, documentation effort, cost, and communications. The most frequently reported information need concerned pain management. CONCLUSIONS: There is limited evidence around the effectiveness of eHealth interventions for palliative care patients, caregivers, and health care professionals. Focused research on information needs and high-quality clinical trials to assess their effectiveness are needed.

The use of social networking sites for public health practice and research: a systematic review.

BACKGROUND: Social networking sites (SNSs) have the potential to increase the reach and efficiency of essential public health services, such as surveillance, research, and communication. OBJECTIVE: The objective of this study was to conduct a systematic literature review to identify the use of SNSs for public health research and practice and to identify existing knowledge gaps. METHODS: We performed a systematic literature review of articles related to public health and SNSs using PubMed, EMBASE, and CINAHL to search for peer-reviewed publications describing the use of SNSs for public health research and practice. We also conducted manual searches of relevant publications. Each publication was independently reviewed by 2 researchers for inclusion and extracted relevant study data. RESULTS: A total of 73 articles met our inclusion criteria. Most articles (n=50) were published in the final 2 years covered by our search. In all, 58 articles were in the domain of public health research and 15 were in public health practice. Only 1 study was conducted in a low-income country. Most articles (63/73, 86%) described observational studies involving users or usages of SNSs; only 5 studies involved randomized controlled trials. A large proportion (43/73, 59%) of the identified studies included populations considered hard to reach, such as young individuals, adolescents, and individuals at risk of sexually transmitted diseases or alcohol and substance abuse. Few articles (2/73, 3%) described using the multidirectional communication potential of SNSs to engage study populations. CONCLUSIONS: The number of publications about public health uses for SNSs has been steadily increasing in the past 5 years. With few exceptions, the literature largely consists of observational studies describing users and usages of SNSs regarding topics of public health interest. More studies that fully exploit the communication tools embedded in SNSs and study their potential to produce significant effects in the overall population's health are needed.

Clinical Experiences of Working with Patient-Generated Health Data in Primary Care.

This research aims to provide insight into the GP experience with patient-generated health data (PGHD) in a virtual care visit. Despite the prevalence of wearables, including smartwatches, the acceptability of generated data in primary care is understudied. The result of this study from mixed-method analysis showed the basic capabilities of PGHD to enhance clinical decision-making and positive impact on collaboration with the patient. The impact of PGHD on clinician satisfaction was not determined, highlighting the importance of rigorous methodology in future research.

Uncovering Variations in Clinical Notes for NLP Modeling.

Clinical text contains rich patient information and has attracted much research interest in applying Natural Language Processing (NLP) tools to model it. In this study, we quantified and analyzed the textual characteristics of five common clinical note types using multiple measurements, including lexical-level features, semantic content, and grammaticality. We found there exist significant linguistic variations in different clinical note types, while some types tend to be more similar than others.

Retrospective analysis of the impact of electronic medical record alerts on low value care in a pediatric hospital.

OBJECTIVES: Hospital costs continue to rise unsustainably. Up to 20% of care is wasteful including low value care (LVC). This study aimed to understand whether electronic medical record (EMR) alerts are effective at reducing pediatric LVC and measure the impact on hospital costs. MATERIALS AND METHODS: Using EMR data over a 76-month period, we evaluated changes in 4 LVC practices following the implementation of EMR alerts, using time series analysis to control for underlying time-based trends, in a large pediatric hospital in Australia. The main outcome measure was the change in rate of each LVC practice. Balancing measures included the rate of alert adherence as a proxy measure for risk of alert fatigue. Hospital costs were calculated by the volume of LVC avoided multiplied by the unit costs. Costs of the intervention were calculated from clinician and analyst time required. RESULTS: All 4 LVC practices showed a statistically significant reduction following alert implementation. Two LVC practices (blood tests) showed an abrupt change, associated with high rates of alert adherence. The other 2 LVC practices (bronchodilator use in bronchiolitis and electrocardiogram ordering for sleeping bradycardia) showed an accelerated rate of improvement compared to baseline trends with lower rates of alert adherence. Hospital savings were $325 to $180 000 per alert. DISCUSSION AND CONCLUSION: EMR alerts are effective in reducing pediatric LVC practices and offer a cost-saving opportunity to the hospital. Further efforts to leverage EMR alerts in pediatric settings to reduce LVC are likely to support future sustainable healthcare delivery.

Visualising Healthcare Process Variability.

Two similar patients undergoing the same procedure might follow different pathways inside a hospital. Some of this variation is expected, but too much variation is associated with increased adverse events. Currently, there are no standard methods to establish when variability for any given clinical process becomes excessive. In this study we use process mining techniques to describe clinical processes and calculate and visualise clinical variability. We selected a sample of patients undergoing elective coronary bypass surgery from the MIMIC dataset, represented their clinical processes in the form of traces, and calculated variability metrics for each process execution and for the complete set of processes. We then analysed the subset of processes with the highest and lowest relative variability and compared their clinical outcomes. We established that processes with the greatest variability were associated with longer length of stay (LOS) with a dose-response relationship: the higher the variability, the longer the LOS. This study provides an effective way to estimate and visualise clinical variability and to understand its impact on patient relevant outcomes.