Undercounting diagnoses in Australian general practice: a data quality study with implications for population health reporting.

BACKGROUND: Diagnosis can often be recorded in electronic medical records (EMRs) as free-text or using a term with a diagnosis code. Researchers, governments, and agencies, including organisations that deliver incentivised primary care quality improvement programs, frequently utilise coded data only and often ignore free-text entries. Diagnosis data are reported for population healthcare planning including resource allocation for patient care. This study sought to determine if diagnosis counts based on coded diagnosis data only, led to under-reporting of disease prevalence and if so, to what extent for six common or important chronic diseases. METHODS: This cross-sectional data quality study used de-identified EMR data from 84 general practices in Victoria, Australia. Data represented 456,125 patients who attended one of the general practices three or more times in two years between January 2021 and December 2022. We reviewed the percentage and proportional difference between patient counts of coded diagnosis entries alone and patient counts of clinically validated free-text entries for asthma, chronic kidney disease, chronic obstructive pulmonary disease, dementia, type 1 diabetes and type 2 diabetes. RESULTS: Undercounts were evident in all six diagnoses when using coded diagnoses alone (2.57-36.72% undercount), of these, five were statistically significant. Overall, 26.4% of all patient diagnoses had not been coded. There was high variation between practices in recording of coded diagnoses, but coding for type 2 diabetes was well captured by most practices. CONCLUSION: In Australia clinical decision support and the reporting of aggregated patient diagnosis data to government that relies on coded diagnoses can lead to significant underreporting of diagnoses compared to counts that also incorporate clinically validated free-text diagnoses. Diagnosis underreporting can impact on population health, healthcare planning, resource allocation, and patient care. We propose the use of phenotypes derived from clinically validated text entries to enhance the accuracy of diagnosis and disease reporting. There are existing technologies and collaborations from which to build trusted mechanisms to provide greater reliability of general practice EMR data used for secondary purposes.

Pilot implementation of a telehealth course for health professions students.

OBJECTIVES: This study examined the level of technology proficiency amongst healthcare professions students. Additionally, the study provides an evaluation of the pilot implementation, as well as the effect of a 7-module telehealth course on the level of adoption and future use of telehealth amongst future Australian healthcare workforce. METHODS: Students from four health-sciences departments at the University of Melbourne, Australia, participated in this pilot study by completing the course and an online questionnaire, which included both structured and open-ended questions. The questionnaire included: 12-items on socio-demographic and Internet utilization; 34-items about acceptance and use of telehealth adapted from the Unified Theory of Acceptance and Use of Technology (UTAUT2) questionnaire; and 22-items about confidence in using the Internet and ICT, adapted from Technology Proficiency Self-Assessment Questionnaire for 21st Century Learning (TPSA-C-21). RESULTS: The evaluation included 26 students who expressed confidence in their Internet/ICT skills They showed enthusiasm for telehealth and recognized its potential benefits, but also emphasized the value of face-to-face interactions. They requested information on legal and aspects and additional learning. Post-test assessments indicated improvements in overall acceptance and use attitudes towards telehealth and on six dimensions of the UTAUT2 instrument. Participation in the course indicated improvements in students' overall acceptance and use attitudes and on six of the ten dimensions of the UTAUT2 instrument (p < 0.05). CONCLUSION: This preliminary evaluation indicated that the telehealth course was a positive and enjoyable learning experience for students with appropriate structure and information. The course was successful in improving students' acceptance and use of health technology. The study identified areas in which further development might be required. As such, the course represents a helpful approach for telehealth training among health professions students. Further evaluation with larger samples is required.

Data-driven overdiagnosis definitions: A scoping review.

INTRODUCTION: Adequate methods to promptly translate digital health innovations for improved patient care are essential. Advances in Artificial Intelligence (AI) and Machine Learning (ML) have been sources of digital innovation and hold the promise to revolutionize the way we treat, manage and diagnose patients. Understanding the benefits but also the potential adverse effects of digital health innovations, particularly when these are made available or applied on healthier segments of the population is essential. One of such adverse effects is overdiagnosis. OBJECTIVE: to comprehensively analyze quantification strategies and data-driven definitions for overdiagnosis reported in the literature. METHODS: we conducted a scoping systematic review of manuscripts describing quantitative methods to estimate the proportion of overdiagnosed patients. RESULTS: we identified 46 studies that met our inclusion criteria. They covered a variety of clinical conditions, primarily breast and prostate cancer. Methods to quantify overdiagnosis included both prospective and retrospective methods including randomized clinical trials, and simulations. CONCLUSION: a variety of methods to quantify overdiagnosis have been published, producing widely diverging results. A standard method to quantify overdiagnosis is needed to allow its mitigation during the rapidly increasing development of new digital diagnostic tools.

Attention-based multimodal fusion with contrast for robust clinical prediction in the face of missing modalities.

OBJECTIVE: With the increasing amount and growing variety of healthcare data, multimodal machine learning supporting integrated modeling of structured and unstructured data is an increasingly important tool for clinical machine learning tasks. However, it is non-trivial to manage the differences in dimensionality, volume, and temporal characteristics of data modalities in the context of a shared target task. Furthermore, patients can have substantial variations in the availability of data, while existing multimodal modeling methods typically assume data completeness and lack a mechanism to handle missing modalities. METHODS: We propose a Transformer-based fusion model with modality-specific tokens that summarize the corresponding modalities to achieve effective cross-modal interaction accommodating missing modalities in the clinical context. The model is further refined by inter-modal, inter-sample contrastive learning to improve the representations for better predictive performance. We denote the model as Attention-based cRoss-MOdal fUsion with contRast (ARMOUR). We evaluate ARMOUR using two input modalities (structured measurements and unstructured text), six clinical prediction tasks, and two evaluation regimes, either including or excluding samples with missing modalities. RESULTS: Our model shows improved performances over unimodal or multimodal baselines in both evaluation regimes, including or excluding patients with missing modalities in the input. The contrastive learning improves the representation power and is shown to be essential for better results. The simple setup of modality-specific tokens enables ARMOUR to handle patients with missing modalities and allows comparison with existing unimodal benchmark results. CONCLUSION: We propose a multimodal model for robust clinical prediction to achieve improved performance while accommodating patients with missing modalities. This work could inspire future research to study the effective incorporation of multiple, more complex modalities of clinical data into a single model.

Are we ready for conformance checking in healthcare? Measuring adherence to clinical guidelines: A scoping systematic literature review.

Clinical guidelines are recommendations of how to diagnose, treat, and manage a patient's medical condition. Health organizations must measure adherence to clinical guidelines to enhance the quality of service, but due to the complexity of the medical environment, there is no simple way of measuring adherence to clinical guidelines. This scoping review will systematically assess the criteria used to measure adherence to clinical guidelines in the past 20 years and explore the suitability of using process mining techniques. We will use a workflow protocol based on declarative and temporal constraints to translate the narrative text rules in the publications into a high-level process model. This approach will enable us to explore the main patterns and gaps identified when measuring adherence to clinical guidelines and how they affect the adoption of process mining techniques. The main contributions of this paper are a) a comprehensive analysis of the criteria used for measuring adherence, considering a diverse set of medical conditions b) a framework that will classify the level of complexity of the rules used to measure adherence based on declarative and temporal constraints c) list of key trends and gaps identified in the literature and how they relate to the use of process mining techniques in healthcare.

"Note Bloat" impacts deep learning-based NLP models for clinical prediction tasks.

One unintended consequence of the Electronic Health Records (EHR) implementation is the overuse of content-importing technology, such as copy-and-paste, that creates "bloated" notes containing large amounts of textual redundancy. Despite the rising interest in applying machine learning models to learn from real-patient data, it is unclear how the phenomenon of note bloat might affect the Natural Language Processing (NLP) models derived from these notes. Therefore, in this work we examine the impact of redundancy on deep learning-based NLP models, considering four clinical prediction tasks using a publicly available EHR database. We applied two deduplication methods to the hospital notes, identifying large quantities of redundancy, and found that removing the redundancy usually has little negative impact on downstream performances, and can in certain circumstances assist models to achieve significantly better results. We also showed it is possible to attack model predictions by simply adding note duplicates, causing changes of correct predictions made by trained models into wrong predictions. In conclusion, we demonstrated that EHR text redundancy substantively affects NLP models for clinical prediction tasks, showing that the awareness of clinical contexts and robust modeling methods are important to create effective and reliable NLP systems in healthcare contexts.

Process mining-driven analysis of COVID-19's impact on vaccination patterns.

Process mining is a discipline sitting between data mining and process science, whose goal is to provide theoretical methods and software tools to analyse process execution data, known as event logs. Although process mining was originally conceived to facilitate business process management activities, research studies have shown the benefit of leveraging process mining in healthcare contexts. However, applying process mining tools to analyse healthcare process execution data is not straightforward. In this paper, we show a methodology to: i) prepare general practice healthcare process data for conducting a process mining analysis; ii) select and apply suitable process mining solutions for successfully executing the analysis; and iii) extract valuable insights from the obtained results, alongside leads for traditional data mining analysis. By doing so, we identified two major challenges when using process mining solutions for analysing healthcare process data, and highlighted benefits and limitations of the state-of-the-art process mining techniques when dealing with highly variable processes and large data-sets. While we provide solutions to the identified challenges, the overarching goal of this study was to detect differences between the patients' health services utilization pattern observed in 2020-during the COVID-19 pandemic and mandatory lock-downs -and the one observed in the prior four years, 2016 to 2019. By using a combination of process mining techniques and traditional data mining, we were able to demonstrate that vaccinations in Victoria did not drop drastically-as other interactions did. On the contrary, we observed a surge of influenza and pneumococcus vaccinations in 2020, as opposed to other research findings of similar studies conducted in different geographical areas.

Process mining for healthcare: Characteristics and challenges.

Process mining techniques can be used to analyse business processes using the data logged during their execution. These techniques are leveraged in a wide range of domains, including healthcare, where it focuses mainly on the analysis of diagnostic, treatment, and organisational processes. Despite the huge amount of data generated in hospitals by staff and machinery involved in healthcare processes, there is no evidence of a systematic uptake of process mining beyond targeted case studies in a research context. When developing and using process mining in healthcare, distinguishing characteristics of healthcare processes such as their variability and patient-centred focus require targeted attention. Against this background, the Process-Oriented Data Science in Healthcare Alliance has been established to propagate the research and application of techniques targeting the data-driven improvement of healthcare processes. This paper, an initiative of the alliance, presents the distinguishing characteristics of the healthcare domain that need to be considered to successfully use process mining, as well as open challenges that need to be addressed by the community in the future.

Epistemonikos: a comprehensive database of systematic reviews for health decision-making.

BACKGROUND: Systematic reviews allow health decisions to be informed by the best available research evidence. However, their number is proliferating quickly, and many skills are required to identify all the relevant reviews for a specific question. METHODS AND FINDINGS: We screen 10 bibliographic databases on a daily or weekly basis, to identify systematic reviews relevant for health decision-making. Using a machine-based approach developed for this project we select reviews, which are then validated by a network of more than 1000 collaborators. After screening over 1,400,000 records we have identified more than 300,000 systematic reviews, which are now stored in a single place and accessible through an easy-to-use search engine. This makes Epistemonikos the largest database of its kind. CONCLUSIONS: Using a systematic approach, recruiting a broad network of collaborators and implementing automated methods, we developed a one-stop shop for systematic reviews relevant for health decision making.

System-Wide Accelerated Implementation of Telemedicine in Response to COVID-19: Mixed Methods Evaluation.

BACKGROUND: As the COVID-19 pandemic disrupted medical practice, telemedicine emerged as an alternative to outpatient visits. However, it is not known how patients and physicians responded to an accelerated implementation of this model of medical care. OBJECTIVE: The aim of this study is to report the system-wide accelerated implementation of telemedicine, compare patient satisfaction between telemedicine and in-person visits, and report provider perceptions. METHODS: This study was conducted at the UC Christus Health Network, a large private academic health network in Santiago, Chile. The satisfaction of patients receiving telemedicine care in March and April 2020 was compared to those receiving in-person care during the same period (concurrent control group) as well as in March and April 2019 (retrospective control group). Patient satisfaction with in-person care was measured using the Net Promoter Score (NPS) survey. Patient satisfaction with telemedicine was assessed with an online survey assessing similar domains. Providers rated their satisfaction and responded to open-ended questions assessing challenges, strategies used to address challenges, the diagnostic process, treatment, and the patient-provider relationship. RESULTS: A total of 3962 patients receiving telemedicine, 1187 patients from the concurrent control group, and 1848 patients from the retrospective control group completed the surveys. Satisfaction was very high with both telemedicine and in-person services. Overall, 263 physicians from over 41 specialties responded to the survey. During telemedicine visits, most providers felt their clinical skills were challenged (61.8%). Female providers felt more challenged than male providers (70.7% versus 50.9%, P=.002). Surgeons, obstetricians, and gynecologists felt their clinical skills were challenged the least, compared to providers from nonsurgical specialties (P<.001). Challenges related to the delivery modality, diagnostic process, and patient-provider relationship differed by provider specialty (P=.046, P<.001, and P=.02, respectively). CONCLUSIONS: Telemedicine implemented in response to the COVID-19 pandemic produced high patient and provider satisfaction. Specialty groups perceived the impact of this new mode of clinical practice differently.

Discovering role interaction models in the Emergency Room using Process Mining.

OBJECTIVES: A coordinated collaboration among different healthcare professionals in Emergency Room (ER) processes is critical to promptly care for patients who arrive at the hospital in a delicate health condition, claiming for an immediate attention. The aims of this study are (i) to discover role interaction models in (ER) processes using process mining techniques; (ii) to understand how healthcare professionals are currently collaborating; and (iii) to provide useful knowledge that can help to improve ER processes. METHODS: A four step method based on process mining techniques is proposed. An ER process of a university hospital was considered as a case study, using 7160 episodes that contains specific ER episode attributes. RESULTS: Insights about how healthcare professionals collaborate in the ER was discovered, including the identification of a prevalent role interaction model along the major triage categories and specific role interaction models for different diagnoses. Also, common and exceptional professional interaction models were discovered at the role level. CONCLUSIONS: This study allows the discovery of role interaction models through the use of real-life clinical data and process mining techniques. Results show a useful way of providing relevant insights about how healthcare professionals collaborate, uncovering opportunities for process improvement.

Multidisciplinary Collaboration in the Treatment of Patients With Type 2 Diabetes in Primary Care: Analysis Using Process Mining.

BACKGROUND: Public health in several countries is characterized by a shortage of professionals and a lack of economic resources. Monitoring and redesigning processes can foster the success of health care institutions, enabling them to provide a quality service while simultaneously reducing costs. Process mining, a discipline that extracts knowledge from information system data to analyze operational processes, affords an opportunity to understand health care processes. OBJECTIVE: Health care processes are highly flexible and multidisciplinary, and health care professionals are able to coordinate in a variety of different ways to treat a diagnosis. The aim of this work was to understand whether the ways in which professionals coordinate their work affect the clinical outcome of patients. METHODS: This paper proposes a method based on the use of process mining to identify patterns of collaboration between physician, nurse, and dietitian in the treatment of patients with type 2 diabetes mellitus and to compare these patterns with the clinical evolution of the patients within the context of primary care. Clustering is used as part of the preprocessing of data to manage the variability, and then process mining is used to identify patterns that may arise. RESULTS: The method is applied in three primary health care centers in Santiago, Chile. A total of seven collaboration patterns were identified, which differed primarily in terms of the number of disciplines present, the participation intensity of each discipline, and the referrals between disciplines. The pattern in which the three disciplines participated in the most equitable and comprehensive manner had a lower proportion of highly decompensated patients compared with those patterns in which the three disciplines participated in an unbalanced manner. CONCLUSIONS: By discovering which collaboration patterns lead to improved outcomes, health care centers can promote the most successful patterns among their professionals so as to improve the treatment of patients. Process mining techniques are useful for discovering those collaborations patterns in flexible and unstructured health care processes.

Development of mobile technologies for the prevention of cervical cancer in Santiago, Chile study protocol: a randomized controlled trial.

BACKGROUND: In Chile, more than 500 women die every year from cervical cancer, and a majority of Chilean women are not up-to-date with their Papanicolau (Pap) test. Mobile health has great potential in many health areas, particularly in health promotion and prevention. There are no randomized controlled trials in Latin America assessing its use in cervical cancer screening. The 'Development of Mobile Technologies for the Prevention of Cervical Cancer in Santiago, Chile' study aims to determine the efficacy of a text-message intervention on Pap test adherence among Chilean women in the metropolitan region of Santiago. METHODS/DESIGN: This study is a parallel randomized-controlled trial of 400 Chilean women aged 25-64 who are non-adherent with current recommendations for Pap test screening. Participants will be randomly assigned to (1) a control arm (usual care) or (2) an intervention arm, where text and voice messages containing information and encouragement to undergo screening will be sent to the women. The primary endpoint is completion of a Pap test within 6 months of baseline assessment, as determined by medical record review at community-based clinics. Medical record reviewers will be blinded to randomization arms. The secondary endpoint is an evaluation of the implementation and usability of the text message intervention as a strategy to improve screening adherence. DISCUSSION: This intervention using mobile technology intends to raise cervical cancer screening adherence and compliance among a Chilean population of low and middle-low socioeconomic status. If successful, this strategy may reduce the incidence of cervical cancer. TRIAL REGISTRATION: Clinicaltrials.gov NCT02376023 Registered 2/17/2015. First participant enrolled Feb 22nd 2016.

Process mining in healthcare: A literature review.

Process Mining focuses on extracting knowledge from data generated and stored in corporate information systems in order to analyze executed processes. In the healthcare domain, process mining has been used in different case studies, with promising results. Accordingly, we have conducted a literature review of the usage of process mining in healthcare. The scope of this review covers 74 papers with associated case studies, all of which were analyzed according to eleven main aspects, including: process and data types; frequently posed questions; process mining techniques, perspectives and tools; methodologies; implementation and analysis strategies; geographical analysis; and medical fields. The most commonly used categories and emerging topics have been identified, as well as future trends, such as enhancing Hospital Information Systems to become process-aware. This review can: (i) provide a useful overview of the current work being undertaken in this field; (ii) help researchers to choose process mining algorithms, techniques, tools, methodologies and approaches for their own applications; and (iii) highlight the use of process mining to improve healthcare processes.

Effectiveness of eHealth interventions and information needs in palliative care: a systematic literature review.

BACKGROUND: One of the key components in palliative care is communication. eHealth technologies can be an effective way to support communications among participants in the process of palliative care. However, it is unclear to what extent information technology has been established in this field. OBJECTIVE: Our goal was to systematically identify studies and analyze the effectiveness of eHealth interventions in palliative care and the information needs of people involved in the palliative care process. METHODS: We conducted a systematic literature search using PubMed, Embase, and LILACS according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. We collected and analyzed quantitative and qualitative data regarding effectiveness of eHealth interventions and users' information needs in palliative care. RESULTS: Our search returned a total of 240 articles, 17 of which met our inclusion criteria. We found no randomized controlled trial studying the effects of eHealth interventions in palliative care. Studies tended to be observational, non-controlled studies, and a few quasi-experimental studies. Overall there was great heterogeneity in the types of interventions and outcome assessments; some studies reported some improvement on quality of care, documentation effort, cost, and communications. The most frequently reported information need concerned pain management. CONCLUSIONS: There is limited evidence around the effectiveness of eHealth interventions for palliative care patients, caregivers, and health care professionals. Focused research on information needs and high-quality clinical trials to assess their effectiveness are needed.

The use of social networking sites for public health practice and research: a systematic review.

BACKGROUND: Social networking sites (SNSs) have the potential to increase the reach and efficiency of essential public health services, such as surveillance, research, and communication. OBJECTIVE: The objective of this study was to conduct a systematic literature review to identify the use of SNSs for public health research and practice and to identify existing knowledge gaps. METHODS: We performed a systematic literature review of articles related to public health and SNSs using PubMed, EMBASE, and CINAHL to search for peer-reviewed publications describing the use of SNSs for public health research and practice. We also conducted manual searches of relevant publications. Each publication was independently reviewed by 2 researchers for inclusion and extracted relevant study data. RESULTS: A total of 73 articles met our inclusion criteria. Most articles (n=50) were published in the final 2 years covered by our search. In all, 58 articles were in the domain of public health research and 15 were in public health practice. Only 1 study was conducted in a low-income country. Most articles (63/73, 86%) described observational studies involving users or usages of SNSs; only 5 studies involved randomized controlled trials. A large proportion (43/73, 59%) of the identified studies included populations considered hard to reach, such as young individuals, adolescents, and individuals at risk of sexually transmitted diseases or alcohol and substance abuse. Few articles (2/73, 3%) described using the multidirectional communication potential of SNSs to engage study populations. CONCLUSIONS: The number of publications about public health uses for SNSs has been steadily increasing in the past 5 years. With few exceptions, the literature largely consists of observational studies describing users and usages of SNSs regarding topics of public health interest. More studies that fully exploit the communication tools embedded in SNSs and study their potential to produce significant effects in the overall population's health are needed.

Clinical Experiences of Working with Patient-Generated Health Data in Primary Care.

This research aims to provide insight into the GP experience with patient-generated health data (PGHD) in a virtual care visit. Despite the prevalence of wearables, including smartwatches, the acceptability of generated data in primary care is understudied. The result of this study from mixed-method analysis showed the basic capabilities of PGHD to enhance clinical decision-making and positive impact on collaboration with the patient. The impact of PGHD on clinician satisfaction was not determined, highlighting the importance of rigorous methodology in future research.

Uncovering Variations in Clinical Notes for NLP Modeling.

Clinical text contains rich patient information and has attracted much research interest in applying Natural Language Processing (NLP) tools to model it. In this study, we quantified and analyzed the textual characteristics of five common clinical note types using multiple measurements, including lexical-level features, semantic content, and grammaticality. We found there exist significant linguistic variations in different clinical note types, while some types tend to be more similar than others.

Modelling patient trajectories in emergency department simulations using retrospective patient cohorts.

Computer simulations of emergency departments (EDs) are tools that can support managing and optimising ED operations. A core component of ED simulation models is patient trajectories, defined as the series of activities patients undergo in the ED from arrival to discharge. The combined duration of these activities, and waiting times between them, determines a patient's length of stay (LOS). Patient trajectories are often calibrated and validated solely based on the estimated acuity of patients assigned upon arrival. However, acuity is a prospective patient indicator that inconsistently reflects patients' actual urgency and resource usage as seen retrospectively upon discharge. Here, we propose a data-driven ED simulation model in which patient trajectories are modelled based on both acuity and retrospective patient indicators. We show that including retrospective patient indicators recovers the observed LOS distributions more accurately than when using acuity alone. We also demonstrate how the use of retrospective patient indicators leads to more plausible estimates of the impact of increased stress in the ED on patients' LOS. Our work exemplifies how we can better utilise ED data to make the development and evaluation of ED simulation models more accurate and robust, enabling them to provide more reliable and useful operational insights.