Undercounting diagnoses in Australian general practice: a data quality study with implications for population health reporting.

BACKGROUND: Diagnosis can often be recorded in electronic medical records (EMRs) as free-text or using a term with a diagnosis code. Researchers, governments, and agencies, including organisations that deliver incentivised primary care quality improvement programs, frequently utilise coded data only and often ignore free-text entries. Diagnosis data are reported for population healthcare planning including resource allocation for patient care. This study sought to determine if diagnosis counts based on coded diagnosis data only, led to under-reporting of disease prevalence and if so, to what extent for six common or important chronic diseases. METHODS: This cross-sectional data quality study used de-identified EMR data from 84 general practices in Victoria, Australia. Data represented 456,125 patients who attended one of the general practices three or more times in two years between January 2021 and December 2022. We reviewed the percentage and proportional difference between patient counts of coded diagnosis entries alone and patient counts of clinically validated free-text entries for asthma, chronic kidney disease, chronic obstructive pulmonary disease, dementia, type 1 diabetes and type 2 diabetes. RESULTS: Undercounts were evident in all six diagnoses when using coded diagnoses alone (2.57-36.72% undercount), of these, five were statistically significant. Overall, 26.4% of all patient diagnoses had not been coded. There was high variation between practices in recording of coded diagnoses, but coding for type 2 diabetes was well captured by most practices. CONCLUSION: In Australia clinical decision support and the reporting of aggregated patient diagnosis data to government that relies on coded diagnoses can lead to significant underreporting of diagnoses compared to counts that also incorporate clinically validated free-text diagnoses. Diagnosis underreporting can impact on population health, healthcare planning, resource allocation, and patient care. We propose the use of phenotypes derived from clinically validated text entries to enhance the accuracy of diagnosis and disease reporting. There are existing technologies and collaborations from which to build trusted mechanisms to provide greater reliability of general practice EMR data used for secondary purposes.

Seamless EMR data access: Integrated governance, digital health and the OMOP-CDM.

Objectives In this overview, we describe theObservational Medical Outcomes Partnership Common Data Model (OMOP-CDM), the established governance processes employed in EMR data repositories, and demonstrate how OMOP transformed data provides a lever for more efficient and secure access to electronic medical record (EMR) data by health service providers and researchers.Methods Through pseudonymisation and common data quality assessments, the OMOP-CDM provides a robust framework for converting complex EMR data into a standardised format. This allows for the creation of shared end-to-end analysis packages without the need for direct data exchange, thereby enhancing data security and privacy. By securely sharing de-identified and aggregated data and conducting analyses across multiple OMOP-converted databases, patient-level data is securely firewalled within its respective local site.Results By simplifying data management processes and governance, and through the promotion of interoperability, the OMOP-CDM supports a wide range of clinical, epidemiological, and translational research projects, as well as health service operational reporting.Discussion Adoption of the OMOP-CDM internationally and locally enables conversion of vast amounts of complex, and heterogeneous EMR data into a standardised structured data model, simplifies governance processes, and facilitates rapid repeatable cross-institution analysis through shared end-to-end analysis packages, without the sharing of data.Conclusion The adoption of the OMOP-CDM has the potential to transform health data analytics by providing a common platform for analysing EMR data across diverse healthcare settings.

Visualising Healthcare Process Variability.

Two similar patients undergoing the same procedure might follow different pathways inside a hospital. Some of this variation is expected, but too much variation is associated with increased adverse events. Currently, there are no standard methods to establish when variability for any given clinical process becomes excessive. In this study we use process mining techniques to describe clinical processes and calculate and visualise clinical variability. We selected a sample of patients undergoing elective coronary bypass surgery from the MIMIC dataset, represented their clinical processes in the form of traces, and calculated variability metrics for each process execution and for the complete set of processes. We then analysed the subset of processes with the highest and lowest relative variability and compared their clinical outcomes. We established that processes with the greatest variability were associated with longer length of stay (LOS) with a dose-response relationship: the higher the variability, the longer the LOS. This study provides an effective way to estimate and visualise clinical variability and to understand its impact on patient relevant outcomes.

Designing an Interprofessional Online Course to Foster Learning Health Systems.

The Learning Health Systems (LHS) framework demonstrates the potential for iterative interrogation of health data in real time and implementation of insights into practice. Yet, the lack of appropriately skilled workforce results in an inability to leverage existing data to design innovative solutions. We developed a tailored professional development program to foster a skilled workforce. The short course is wholly online, for interdisciplinary professionals working in the digital health arena. To transform healthcare systems, the workforce needs an understanding of LHS principles, data driven approaches, and the need for diversly skilled learning communities that can tackle these complex problems together.

Uncovering Variations in Clinical Notes for NLP Modeling.

Clinical text contains rich patient information and has attracted much research interest in applying Natural Language Processing (NLP) tools to model it. In this study, we quantified and analyzed the textual characteristics of five common clinical note types using multiple measurements, including lexical-level features, semantic content, and grammaticality. We found there exist significant linguistic variations in different clinical note types, while some types tend to be more similar than others.

A Hybrid Physical-Digital Simulation Laboratory to Expedite Context-Aware Design and Usability Testing in Digital Health.

Fit within existing physical and digitalised workflows is a critical aspect of digital health software usability. Early, iterative exploration of contextual usability issues is complicated by barriers of access to healthcare settings. The Validitron SimLab is a new facility for digital health prototyping that augments immersive, realistic physical environments with a digital sandbox allowing new and existing software to be easily set up and tested in the physical space.

Clinical Experiences of Working with Patient-Generated Health Data in Primary Care.

This research aims to provide insight into the GP experience with patient-generated health data (PGHD) in a virtual care visit. Despite the prevalence of wearables, including smartwatches, the acceptability of generated data in primary care is understudied. The result of this study from mixed-method analysis showed the basic capabilities of PGHD to enhance clinical decision-making and positive impact on collaboration with the patient. The impact of PGHD on clinician satisfaction was not determined, highlighting the importance of rigorous methodology in future research.

Adapting an Environmental Scan for 'Insights Reporting': Learnings from an Online Brain Cancer Peer Support Platform.

When developing a digital health solution, product owners, healthcare professionals, researchers, IT teams, and consumers require timely, accurate contextual information to inform solution development. Insights Reporting can rapidly draw together information from literature, end users and existing technology to inform the development process. This was the case when creating an online brain cancer peer support platform where solution development was conducted in parallel with contextual information synthesis. This paper discusses the novel adaptation of an environmental scan methodology using codesign and multiple layers of qualitative rigor, to create Insights Reporting. This seven-step process can be completed in two months and results in salient points of knowledge that can rapidly inform the design of a solution, creating a shared understanding of a digital health phenomenon. Project members noted that Insights Reporting surfaces previously inaccessible knowledge, catalyzes decision-making and allows all stakeholders to influence the report agenda, affirming principles of digital health equity.

Identifying Frailty in Older Adults in Rural Victoria, Australia: A Secondary Analysis of Population Health Data.

Objectives: To determine the prevalence of frailty among community-dwelling older adults in regional Victoria, Australia. Methods: Frailty status of 376 participants from the Crossroads II cross-sectional study was assessed by selected markers of frailty. The selected variables were psychometrically tested. Associations between frailty and socio-demographic, environmental and health factors were analysed using chi-square, ANOVA and binary logistic regression (BLR). Results: Estimated prevalence of frailty was 39.4%. BLR indicated that frailty decreased with higher educational attainment, (OR = .23; 95% CI: .10-.51) increased for divorced/separated participants (OR = 2.68; 95% CI: 1.29-5.56) and when having three (OR = 3.27; 95% CI: 1.07-9.98), four (OR = 7.20; 95% CI: 2.22-23.31) or five or more chronic conditions (OR = 9.18; 95% CI: 2.83-29.72). Discussion: Frailty in this Australian regional community-dwelling sample was higher than other studies conducted in urban areas of Australia. Present results highlight the importance of exploring the multidimensionality of the frailty construct to have a better understanding which factors are associated with the development of this syndrome.

Retrospective analysis of the impact of electronic medical record alerts on low value care in a pediatric hospital.

OBJECTIVES: Hospital costs continue to rise unsustainably. Up to 20% of care is wasteful including low value care (LVC). This study aimed to understand whether electronic medical record (EMR) alerts are effective at reducing pediatric LVC and measure the impact on hospital costs. MATERIALS AND METHODS: Using EMR data over a 76-month period, we evaluated changes in 4 LVC practices following the implementation of EMR alerts, using time series analysis to control for underlying time-based trends, in a large pediatric hospital in Australia. The main outcome measure was the change in rate of each LVC practice. Balancing measures included the rate of alert adherence as a proxy measure for risk of alert fatigue. Hospital costs were calculated by the volume of LVC avoided multiplied by the unit costs. Costs of the intervention were calculated from clinician and analyst time required. RESULTS: All 4 LVC practices showed a statistically significant reduction following alert implementation. Two LVC practices (blood tests) showed an abrupt change, associated with high rates of alert adherence. The other 2 LVC practices (bronchodilator use in bronchiolitis and electrocardiogram ordering for sleeping bradycardia) showed an accelerated rate of improvement compared to baseline trends with lower rates of alert adherence. Hospital savings were $325 to $180 000 per alert. DISCUSSION AND CONCLUSION: EMR alerts are effective in reducing pediatric LVC practices and offer a cost-saving opportunity to the hospital. Further efforts to leverage EMR alerts in pediatric settings to reduce LVC are likely to support future sustainable healthcare delivery.

Data-driven overdiagnosis definitions: A scoping review.

INTRODUCTION: Adequate methods to promptly translate digital health innovations for improved patient care are essential. Advances in Artificial Intelligence (AI) and Machine Learning (ML) have been sources of digital innovation and hold the promise to revolutionize the way we treat, manage and diagnose patients. Understanding the benefits but also the potential adverse effects of digital health innovations, particularly when these are made available or applied on healthier segments of the population is essential. One of such adverse effects is overdiagnosis. OBJECTIVE: to comprehensively analyze quantification strategies and data-driven definitions for overdiagnosis reported in the literature. METHODS: we conducted a scoping systematic review of manuscripts describing quantitative methods to estimate the proportion of overdiagnosed patients. RESULTS: we identified 46 studies that met our inclusion criteria. They covered a variety of clinical conditions, primarily breast and prostate cancer. Methods to quantify overdiagnosis included both prospective and retrospective methods including randomized clinical trials, and simulations. CONCLUSION: a variety of methods to quantify overdiagnosis have been published, producing widely diverging results. A standard method to quantify overdiagnosis is needed to allow its mitigation during the rapidly increasing development of new digital diagnostic tools.

Attention-based multimodal fusion with contrast for robust clinical prediction in the face of missing modalities.

OBJECTIVE: With the increasing amount and growing variety of healthcare data, multimodal machine learning supporting integrated modeling of structured and unstructured data is an increasingly important tool for clinical machine learning tasks. However, it is non-trivial to manage the differences in dimensionality, volume, and temporal characteristics of data modalities in the context of a shared target task. Furthermore, patients can have substantial variations in the availability of data, while existing multimodal modeling methods typically assume data completeness and lack a mechanism to handle missing modalities. METHODS: We propose a Transformer-based fusion model with modality-specific tokens that summarize the corresponding modalities to achieve effective cross-modal interaction accommodating missing modalities in the clinical context. The model is further refined by inter-modal, inter-sample contrastive learning to improve the representations for better predictive performance. We denote the model as Attention-based cRoss-MOdal fUsion with contRast (ARMOUR). We evaluate ARMOUR using two input modalities (structured measurements and unstructured text), six clinical prediction tasks, and two evaluation regimes, either including or excluding samples with missing modalities. RESULTS: Our model shows improved performances over unimodal or multimodal baselines in both evaluation regimes, including or excluding patients with missing modalities in the input. The contrastive learning improves the representation power and is shown to be essential for better results. The simple setup of modality-specific tokens enables ARMOUR to handle patients with missing modalities and allows comparison with existing unimodal benchmark results. CONCLUSION: We propose a multimodal model for robust clinical prediction to achieve improved performance while accommodating patients with missing modalities. This work could inspire future research to study the effective incorporation of multiple, more complex modalities of clinical data into a single model.

Self-Rated Oral Health as a Valid Measure of Oral Health Status in Adults Living in Rural Australia.

It is unclear how well self-rated oral health (SROH) reflects actual oral health status in the rural Australian population. Therefore, this study aimed to compare the clinically assessed oral health status and SROH of adults living in rural Australia. The data were from 574 participants who took part in the Crossroads II cross-sectional study. Three trained and calibrated dentists evaluated the oral health status of participants based on WHO criteria. SROH was assessed with the question 'Overall, how would you rate the health of your teeth and gums?', with a score ranging from excellent = 5 to poor = 1. A logistic regression analysis (LRA) was performed, allowing us to assess factors associated with SROH. The mean age of participants was 59.2 years (SD 16.3), and 55.3% were female. The key results from the LRA show poorer SROH in those with more missing teeth (OR = 1.05; 95% CI; 1.01-1.08), more decayed teeth (OR = 1.28; 95% CI: 1.11-1.46), and more significant clinical attachment loss of periodontal tissue (6mm or more) (OR = 2.63; 95% CI: 1.29-5.38). This study found an association between negative SROH and clinical indicators used to measure poor oral health status, suggesting that self-rated oral health is an indicator of oral health status. When planning dental healthcare programs, self-reported oral health should be considered a proxy measure for oral health status.

"Note Bloat" impacts deep learning-based NLP models for clinical prediction tasks.

One unintended consequence of the Electronic Health Records (EHR) implementation is the overuse of content-importing technology, such as copy-and-paste, that creates "bloated" notes containing large amounts of textual redundancy. Despite the rising interest in applying machine learning models to learn from real-patient data, it is unclear how the phenomenon of note bloat might affect the Natural Language Processing (NLP) models derived from these notes. Therefore, in this work we examine the impact of redundancy on deep learning-based NLP models, considering four clinical prediction tasks using a publicly available EHR database. We applied two deduplication methods to the hospital notes, identifying large quantities of redundancy, and found that removing the redundancy usually has little negative impact on downstream performances, and can in certain circumstances assist models to achieve significantly better results. We also showed it is possible to attack model predictions by simply adding note duplicates, causing changes of correct predictions made by trained models into wrong predictions. In conclusion, we demonstrated that EHR text redundancy substantively affects NLP models for clinical prediction tasks, showing that the awareness of clinical contexts and robust modeling methods are important to create effective and reliable NLP systems in healthcare contexts.

Process mining-driven analysis of COVID-19's impact on vaccination patterns.

Process mining is a discipline sitting between data mining and process science, whose goal is to provide theoretical methods and software tools to analyse process execution data, known as event logs. Although process mining was originally conceived to facilitate business process management activities, research studies have shown the benefit of leveraging process mining in healthcare contexts. However, applying process mining tools to analyse healthcare process execution data is not straightforward. In this paper, we show a methodology to: i) prepare general practice healthcare process data for conducting a process mining analysis; ii) select and apply suitable process mining solutions for successfully executing the analysis; and iii) extract valuable insights from the obtained results, alongside leads for traditional data mining analysis. By doing so, we identified two major challenges when using process mining solutions for analysing healthcare process data, and highlighted benefits and limitations of the state-of-the-art process mining techniques when dealing with highly variable processes and large data-sets. While we provide solutions to the identified challenges, the overarching goal of this study was to detect differences between the patients' health services utilization pattern observed in 2020-during the COVID-19 pandemic and mandatory lock-downs -and the one observed in the prior four years, 2016 to 2019. By using a combination of process mining techniques and traditional data mining, we were able to demonstrate that vaccinations in Victoria did not drop drastically-as other interactions did. On the contrary, we observed a surge of influenza and pneumococcus vaccinations in 2020, as opposed to other research findings of similar studies conducted in different geographical areas.

Are we ready for conformance checking in healthcare? Measuring adherence to clinical guidelines: A scoping systematic literature review.

Clinical guidelines are recommendations of how to diagnose, treat, and manage a patient's medical condition. Health organizations must measure adherence to clinical guidelines to enhance the quality of service, but due to the complexity of the medical environment, there is no simple way of measuring adherence to clinical guidelines. This scoping review will systematically assess the criteria used to measure adherence to clinical guidelines in the past 20 years and explore the suitability of using process mining techniques. We will use a workflow protocol based on declarative and temporal constraints to translate the narrative text rules in the publications into a high-level process model. This approach will enable us to explore the main patterns and gaps identified when measuring adherence to clinical guidelines and how they affect the adoption of process mining techniques. The main contributions of this paper are a) a comprehensive analysis of the criteria used for measuring adherence, considering a diverse set of medical conditions b) a framework that will classify the level of complexity of the rules used to measure adherence based on declarative and temporal constraints c) list of key trends and gaps identified in the literature and how they relate to the use of process mining techniques in healthcare.

Process mining for healthcare: Characteristics and challenges.

Process mining techniques can be used to analyse business processes using the data logged during their execution. These techniques are leveraged in a wide range of domains, including healthcare, where it focuses mainly on the analysis of diagnostic, treatment, and organisational processes. Despite the huge amount of data generated in hospitals by staff and machinery involved in healthcare processes, there is no evidence of a systematic uptake of process mining beyond targeted case studies in a research context. When developing and using process mining in healthcare, distinguishing characteristics of healthcare processes such as their variability and patient-centred focus require targeted attention. Against this background, the Process-Oriented Data Science in Healthcare Alliance has been established to propagate the research and application of techniques targeting the data-driven improvement of healthcare processes. This paper, an initiative of the alliance, presents the distinguishing characteristics of the healthcare domain that need to be considered to successfully use process mining, as well as open challenges that need to be addressed by the community in the future.

Concept Drift Detection to Assess the Diffusion of Process Innovations in Healthcare.

Introduction: Clinical guidelines recommend best care pathways for many clinical conditions. When significant data from clinical trials become available, new clinical guidelines are published, formalizing the advances in the field. When we consider clinical guidelines as processes of care, we can use process mining techniques to discover how innovations diffuse into healthcare and how they modify the execution of clinical processes. Methods: We conducted a study to assess the changes in process execution patterns after a 2013 update of an acute ischemic stroke (AIS) guideline. We used MIMIC-IV as the data source, including patients from 2008 until 2019 hospitalized with an AIS and applied drift detection methods to measure changes in the therapeutic process. We performed statistical tests to determine whether the underlying distribution of events reflects the changes in the guidelines post-2013. Results: Ischemic stroke patients show few significant changes in clinical practices, despite an update in guidelines. The positive control group of aortic valve replacement patients shows a significant change in clinical practices surrounding this procedure. Conclusions: This study demonstrates the use of drift detection methods as a novel method to study the diffusion of innovations in healthcare settings from a process perspective.