Real-life implementation and evaluation of the e-referral system SIPILINK.

INTRODUCTION: General Practitioners (GPs) play a key role of gatekeeper, as they coordinate patients' care. However, most of them reported having difficulty to refer patients to hospital, especially in semi-urgent context. To facilitate the referral of semi-urgent patients, we implemented an e-referral platform, named SIPILINK, within 4 wards from a large public French hospital (internal medicine, diabetology, gynaecological surgery and oncology wards). Here, we aimed to evaluate the SIPILINK e-referral platform after 2 years of implementation. METHODS: The evaluation included a multidimensional assessment based on the RE-AIM framework with the analysis of implementation, requests, health professionals' satisfaction, and estimated hospital payment. RESULTS: Over 2 years of implementation, GPs sent 113 requests to hospital. Hospital respected the time of response requested by GPs in 93 % of cases and proposed a consultation or hospitalization in respectively 40.7 % and 10.6 % of cases. 100 % of GPs and 78 % of Hospital Practitioners (HPs) were satisfied with the quality of exchanges. 77 % of HPs and 100 % of Care Pathway Managers (CPMs) found that patient care pathways were improved. Nearly all practitioners would recommend this platform for patient referrals. DISCUSSION: SIPILINK shows promise in streamlining the referral process, enhancing communication, and improving patient care pathways. Further studies including the impact on the quality of care, are needed to assess its effectiveness and sustainability in healthcare settings.

An Integrated Pipeline for Phenotypic Characterization, Clustering and Visualization of Patient Cohorts in a Rare Disease-Oriented Clinical Data Warehouse.

Rare diseases pose significant challenges due to their heterogeneity and lack of knowledge. This study develops a comprehensive pipeline interoperable with a document-oriented clinical data warehouse, integrating cohort characterization, patient clustering and interpretation. Leveraging NLP, semantic similarity, machine learning and visualization, the pipeline enables the identification of prevalent phenotype patterns and patient stratification. To enhance interpretability, discriminant phenotypes characterizing each cluster are provided. Users can visually test hypotheses by marking patients exhibiting specific keywords in the EHR like genes, drugs and procedures. Implemented through a web interface, the pipeline enables clinicians to navigate through different modules, discover intricate patterns and generate interpretable insights that may advance rare diseases understanding, guide decision-making, and ultimately improve patient outcomes.

Evaluation of BERT-Based Models on Patient Data from French Social Media.

With the objective of extracting new knowledge about rare diseases from social media messages, we evaluated three models on a Named Entity Recognition (NER) task, consisting of extracting phenotypes and treatments from social media messages. We trained the three models on a dataset with social media messages about Developmental and Epileptic Encephalopathies and more common diseases. This preliminary study revealed that CamemBERT and CamemBERT-bio exhibit similar performance on social media testimonials, slightly outperforming DrBERT. It also highlighted that their performance was lower on this type of data than on structured health datasets. Limitations, including a narrow focus on NER performance and dataset-specific evaluation, call for further research to fully assess model capabilities on larger and more diverse datasets.

Building ontology-based temporal databases for data reuse: An applied example on hospital organizational structures.

Keeping track of data semantics and data changes in the databases is essential to support retrospective studies and the reproducibility of longitudinal clinical analysis by preventing false conclusions from being drawn from outdated data. A knowledge model combined with a temporal model plays an essential role in organizing the data and improving query expressiveness across time and multiple institutions. This paper presents a modelling framework for temporal relational databases using an ontology to derive a shareable and interoperable data model. The framework is based on: OntoRela an ontology-driven database modelling approach and Unified Historicization Framework a temporal database modelling approach. The method was applied to hospital organizational structures to show the impact of tracking organizational changes on data quality assessment, healthcare activities and data access rights. The paper demonstrated the usefulness of an ontology to provide a formal, interoperable, and reusable definition of entities and their relationships, as well as the adequacy of the temporal database to store, trace, and query data over time.

Objectivizing issues in the diagnosis of complex rare diseases: lessons learned from testing existing diagnosis support systems on ciliopathies.

BACKGROUND: There are approximately 8,000 different rare diseases that affect roughly 400 million people worldwide. Many of them suffer from delayed diagnosis. Ciliopathies are rare monogenic disorders characterized by a significant phenotypic and genetic heterogeneity that raises an important challenge for clinical diagnosis. Diagnosis support systems (DSS) applied to electronic health record (EHR) data may help identify undiagnosed patients, which is of paramount importance to improve patients' care. Our objective was to evaluate three online-accessible rare disease DSSs using phenotypes derived from EHRs for the diagnosis of ciliopathies. METHODS: Two datasets of ciliopathy cases, either proven or suspected, and two datasets of controls were used to evaluate the DSSs. Patient phenotypes were automatically extracted from their EHRs and converted to Human Phenotype Ontology terms. We tested the ability of the DSSs to diagnose cases in contrast to controls based on Orphanet ontology. RESULTS: A total of 79 cases and 38 controls were selected. Performances of the DSSs on ciliopathy real world data (best DSS with area under the ROC curve = 0.72) were not as good as published performances on the test set used in the DSS development phase. None of these systems obtained results which could be described as "expert-level". Patients with multisystemic symptoms were generally easier to diagnose than patients with isolated symptoms. Diseases easily confused with ciliopathy generally affected multiple organs and had overlapping phenotypes. Four challenges need to be considered to improve the performances: to make the DSSs interoperable with EHR systems, to validate the performances in real-life settings, to deal with data quality, and to leverage methods and resources for rare and complex diseases. CONCLUSION: Our study provides insights into the complexities of diagnosing highly heterogenous rare diseases and offers lessons derived from evaluation existing DSSs in real-world settings. These insights are not only beneficial for ciliopathy diagnosis but also hold relevance for the enhancement of DSS for various complex rare disorders, by guiding the development of more clinically relevant rare disease DSSs, that could support early diagnosis and finally make more patients eligible for treatment.

Performance and clinical utility of a new supervised machine-learning pipeline in detecting rare ciliopathy patients based on deep phenotyping from electronic health records and semantic similarity.

BACKGROUND: Rare diseases affect approximately 400 million people worldwide. Many of them suffer from delayed diagnosis. Among them, NPHP1-related renal ciliopathies need to be diagnosed as early as possible as potential treatments have been recently investigated with promising results. Our objective was to develop a supervised machine learning pipeline for the detection of NPHP1 ciliopathy patients from a large number of nephrology patients using electronic health records (EHRs). METHODS AND RESULTS: We designed a pipeline combining a phenotyping module re-using unstructured EHR data, a semantic similarity module to address the phenotype dependence, a feature selection step to deal with high dimensionality, an undersampling step to address the class imbalance, and a classification step with multiple train-test split for the small number of rare cases. The pipeline was applied to thirty NPHP1 patients and 7231 controls and achieved good performances (sensitivity 86% with specificity 90%). A qualitative review of the EHRs of 40 misclassified controls showed that 25% had phenotypes belonging to the ciliopathy spectrum, which demonstrates the ability of our system to detect patients with similar conditions. CONCLUSIONS: Our pipeline reached very encouraging performance scores for pre-diagnosing ciliopathy patients. The identified patients could then undergo genetic testing. The same data-driven approach can be adapted to other rare diseases facing underdiagnosis challenges.

Clinical decision support system in emergency telephone triage: A scoping review of technical design, implementation and evaluation.

OBJECTIVES: Emergency department overcrowding could be improved by upstream telephone triage. Emergency telephone triage aims at managing and orientating adequately patients as early as possible and distributing limited supply of staff and materials. This complex task could be improved with the use of Clinical decision support systems (CDSS). The aim of this scoping review was to identify literature gaps for the future development and evaluation of CDSS for Emergency telephone triage. MATERIALS AND METHODS: We present here a scoping review of CDSS designed for emergency telephone triage, and compared them in terms of functional characteristics, technical design, health care implementation and methodologies used for evaluation, following the PRISMA-ScR guidelines. RESULTS: Regarding design, 19 CDSS were retrieved: 12 were knowledge based CDSS (decisional algorithms built according to guidelines or clinical expertise) and 7 were data driven (statistical, machine learning, or deep learning models). Most of them aimed at assisting nurses or non-medical staff by providing patient orientation and/or severity/priority assessment. Eleven were implemented in real life, and only three were connected to the Electronic Health Record. Regarding evaluation, CDSS were assessed through various aspects: intrinsic characteristics, impact on clinical practice or user apprehension. Only one pragmatic trial and one randomized controlled trial were conducted. CONCLUSION: This review highlights the potential of a hybrid system, user tailored, flexible, connected to the electronic health record, which could work with oral, video and digital data; and the need to evaluate CDSS on intrinsic characteristics and impact on clinical practice, iteratively at each distinct stage of the IT lifecycle.

A Radiomics Approach to Identify Immunologically Active Tumor in Patients with Head and Neck Squamous Cell Carcinomas.

BACKGROUND: We recently developed a gene-expression-based HOT score to identify the hot/cold phenotype of head and neck squamous cell carcinomas (HNSCCs), which is associated with the response to immunotherapy. Our goal was to determine whether radiomic profiling from computed tomography (CT) scans can distinguish hot and cold HNSCC. METHOD: We included 113 patients from The Cancer Genome Atlas (TCGA) and 20 patients from the Groupe Hospitalier Pitié-Salpêtrière (GHPS) with HNSCC, all with available pre-treatment CT scans. The hot/cold phenotype was computed for all patients using the HOT score. The IBEX software (version 4.11.9, accessed on 30 march 2020) was used to extract radiomic features from the delineated tumor region in both datasets, and the intraclass correlation coefficient (ICC) was computed to select robust features. Machine learning classifier models were trained and tested in the TCGA dataset and validated using the area under the receiver operator characteristic curve (AUC) in the GHPS cohort. RESULTS: A total of 144 radiomic features with an ICC >0.9 was selected. An XGBoost model including these selected features showed the best performance prediction of the hot/cold phenotype with AUC = 0.86 in the GHPS validation dataset. CONCLUSIONS AND RELEVANCE: We identified a relevant radiomic model to capture the overall hot/cold phenotype of HNSCC. This non-invasive approach could help with the identification of patients with HNSCC who may benefit from immunotherapy.

Assessment of the Early Detection of Anosmia and Ageusia Symptoms in COVID-19 on Twitter: Retrospective Study.

BACKGROUND: During the unprecedented COVID-19 pandemic, social media has been extensively used to amplify the spread of information and to express personal health-related experiences regarding symptoms, including anosmia and ageusia, 2 symptoms that have been reported later than other symptoms. OBJECTIVE: Our objective is to investigate to what extent Twitter users reported anosmia and ageusia symptoms in their tweets and if they connected them to COVID-19, to evaluate whether these symptoms could have been identified as COVID-19 symptoms earlier using Twitter rather than the official notice. METHODS: We collected French tweets posted between January 1, 2020, and March 31, 2020, containing anosmia- or ageusia-related keywords. Symptoms were detected using fuzzy matching. The analysis consisted of 3 parts. First, we compared the coverage of anosmia and ageusia symptoms in Twitter and in traditional media to determine if the association between COVID-19 and anosmia or ageusia could have been identified earlier through Twitter. Second, we conducted a manual analysis of anosmia- and ageusia-related tweets to obtain quantitative and qualitative insights regarding their nature and to assess when the first associations between COVID-19 and these symptoms were established. We randomly annotated tweets from 2 periods: the early stage and the rapid spread stage of the epidemic. For each tweet, each symptom was annotated regarding 3 modalities: symptom (yes or no), associated with COVID-19 (yes, no, or unknown), and whether it was experienced by someone (yes, no, or unknown). Third, to evaluate if there was a global increase of tweets mentioning anosmia or ageusia in early 2020, corresponding to the beginning of the COVID-19 epidemic, we compared the tweets reporting experienced anosmia or ageusia between the first periods of 2019 and 2020. RESULTS: In total, 832 (respectively 12,544) tweets containing anosmia (respectively ageusia) related keywords were extracted over the analysis period in 2020. The comparison to traditional media showed a strong correlation without any lag, which suggests an important reactivity of Twitter but no earlier detection on Twitter. The annotation of tweets from 2020 showed that tweets correlating anosmia or ageusia with COVID-19 could be found a few days before the official announcement. However, no association could be found during the first stage of the pandemic. Information about the temporality of symptoms and the psychological impact of these symptoms could be found in the tweets. The comparison between early 2020 and early 2019 showed no difference regarding the volumes of tweets. CONCLUSIONS: Based on our analysis of French tweets, associations between COVID-19 and anosmia or ageusia by web users could have been found on Twitter just a few days before the official announcement but not during the early stage of the pandemic. Patients share qualitative information on Twitter regarding anosmia or ageusia symptoms that could be of interest for future analyses.

Improving Patient Similarity Using Different Modalities of Phenotypes Extracted from Clinical Narratives.

In the context of medical concept extraction, it is critical to determine if clinical signs or symptoms mentioned in the text were present or absent, experienced by the patient or their relatives. Previous studies have focused on the NLP aspect but not on how to leverage this supplemental information for clinical applications. In this paper, we aim to use the patient similarity networks framework to aggregate different phenotyping modalities. NLP techniques were applied to extract phenotypes and predict their modalities from 5470 narrative reports of 148 patients with ciliopathies (a group of rare diseases). Patient similarities were computed using each modality separately for aggregation and clustering. We found that aggregating negated phenotypes improved patient similarity, but further aggregating relatives' phenotypes worsened the result. We suggest that different modalities of phenotypes can contribute to patient similarity, but they should be aggregated carefully and with appropriate similarity metrics and aggregation models.

The Smart Data Extractor, a Clinician Friendly Solution to Accelerate and Improve the Data Collection During Clinical Trials.

In medical research, the traditional way to collect data, i.e. browsing patient files, has been proven to induce bias, errors, human labor and costs. We propose a semi-automated system able to extract every type of data, including notes. The Smart Data Extractor pre-populates clinic research forms by following rules. We performed a cross-testing experiment to compare semi-automated to manual data collection. 20 target items had to be collected for 79 patients. The average time to complete one form was 6'81" for manual data collection and 3'22" with the Smart Data Extractor. There were also more mistakes during manual data collection (163 for the whole cohort) than with the Smart Data Extractor (46 for the whole cohort). We present an easy to use, understandable and agile solution to fill out clinical research forms. It reduces human effort and provides higher quality data, avoiding data re-entry and fatigue induced errors.

The genetic landscape and clinical spectrum of nephronophthisis and related ciliopathies.

Nephronophthisis (NPH) is an autosomal-recessive ciliopathy representing one of the most frequent causes of kidney failure in childhood characterized by a broad clinical and genetic heterogeneity. Applied to one of the worldwide largest cohorts of patients with NPH, genetic analysis encompassing targeted and whole exome sequencing identified disease-causing variants in 600 patients from 496 families with a detection rate of 71%. Of 788 pathogenic variants, 40 known ciliopathy genes were identified. However, the majority of patients (53%) bore biallelic pathogenic variants in NPHP1. NPH-causing gene alterations affected all ciliary modules defined by structural and/or functional subdomains. Seventy six percent of these patients had progressed to kidney failure, of which 18% had an infantile form (under five years) and harbored variants affecting the Inversin compartment or intraflagellar transport complex A. Forty eight percent of patients showed a juvenile (5-15 years) and 34% a late-onset disease (over 15 years), the latter mostly carrying variants belonging to the Transition Zone module. Furthermore, while more than 85% of patients with an infantile form presented with extra-kidney manifestations, it only concerned half of juvenile and late onset cases. Eye involvement represented a predominant feature, followed by cerebellar hypoplasia and other brain abnormalities, liver and skeletal defects. The phenotypic variability was in a large part associated with mutation types, genes and corresponding ciliary modules with hypomorphic variants in ciliary genes playing a role in early steps of ciliogenesis associated with juvenile-to-late onset NPH forms. Thus, our data confirm a considerable proportion of late-onset NPH suggesting an underdiagnosis in adult chronic kidney disease.

Putting undergraduate medical students in AI-CDSS designers' shoes: An innovative teaching method to develop digital health critical thinking.

INTRODUCTION: Digital health programs are urgently needed to accelerate the adoption of Artificial Intelligence and Clinical Decision Support Systems (AI-CDSS) in clinical settings. However, such programs are still lacking for undergraduate medical students, and new approaches are required to prepare them for the arrival of new and unknown technologies. At University Paris Cité medical school, we designed an innovative program to develop the digital health critical thinking of undergraduate medical students that consisted of putting medical students in AI-CDSS designers' shoes. METHODS: We followed the six steps of Kern's approach for curriculum development: identification of needs, definition of objectives, design of an educational strategy, implementation, development of an assessment and design of program evaluation. RESULTS: A stand-alone and elective AI-CDSS program was implemented for fourth-year medical students. Each session was designed from an AI-CDSS designer viewpoint, with theoretical and practical teaching and brainstorming time on a project that consisted of designing an AI-CDSS in small groups. From 2021 to 2022, 15 students were enrolled: they rated the program 4.4/5, and 80% recommended it. Seventy-four percent considered that they had acquired new skills useful for clinical practice, and 66% felt more confident with technologies. The AI-CDSS program aroused great enthusiasm and strong engagement of students: 8 designed an AI-CDSS and wrote two scientific 5-page articles presented at the Medical Informatics Europe conference; 4 students were involved in a CDSS research project; 2 students asked for a hospital internship in digital health; and 1 decided to pursue PhD training. DISCUSSION: Putting students in AI-CDSS designers' shoes seemed to be a fruitful and innovative strategy to develop digital health skills and critical thinking toward AI technologies. We expect that such programs could help future doctors work in rapidly evolving digitalized environments and position themselves as key leaders in digital health.

Prevalence of Contraindications to Nirmatrelvir-Ritonavir Among Hospitalized Patients With COVID-19 at Risk for Progression to Severe Disease.

This cohort study examines the prevalence of contraindications to nirmatrelvir-ritonavir in patients hospitalized with COVID-19.

Ethical Aspects of Artificial Intelligence in Radiation Oncology.

Radiation oncology is a field that heavily relies on new technology. Data science and artificial intelligence will have an important role in the entire radiotherapy workflow. A new paradigm of routine healthcare data reuse to automate treatments and provide decision support is emerging. This review will discuss the ethical aspects of the use of artificial intelligence (AI) in radiation oncology. More specifically, the review will discuss the evolution of work through the ages, as well as the impact AI will have on it. We will then explain why AI opens a new technical era for the field and we will conclude on the challenges in the years to come.

Determining the Set of Items to Include in Breast Operative Reports, Using Clustering Algorithms on Retrospective Data Extracted from Clinical DataWarehouse.

Medical reports are key elements to guarantee the quality, and continuity of care but their quality remains an issue. Standardization and structuration of reports can increase their quality, but are usually based on expert opinions. Here, we hypothesize that a structured model of medical reports could be learnt using machine learning on retrospective medical reports extracted from clinical data warehouses (CDW). To investigate our hypothesis, we extracted breast cancer operative reports from our CDW. Each document was preprocessed and split into sentences. Clustering was performed using TFIDF, Paraphrase or Universal Sentence Encoder along with K-Means, DBSCAN, or Hierarchical clustering. The best couple was TFIDF/K-Means, providing a sentence coverage of 89 % on our dataset; and allowing to identify 7 main categories of items to include in breast cancer operative reports. These results are encouraging for a document preset creation task and should then be validated and implemented in real life.

Using an Ontology to Derive a Sharable and Interoperable Relational Data Model for Heterogeneous Healthcare Data and Various Applications.

BACKGROUND: A large volume of heavily fragmented data is generated daily in different healthcare contexts and is stored using various structures with different semantics. This fragmentation and heterogeneity make secondary use of data a challenge. Data integration approaches that derive a common data model from sources or requirements have some advantages. However, these approaches are often built for a specific application where the research questions are known. Thus, the semantic and structural reconciliation is often not reusable nor reproducible. A recent integration approach using knowledge models has been developed with ontologies that provide a strong semantic foundation. Nonetheless, deriving a data model that captures the richness of the ontology to store data with their full semantic remains a challenging task. OBJECTIVES: This article addresses the following question: How to design a sharable and interoperable data model for storing heterogeneous healthcare data and their semantic to support various applications? METHOD: This article describes a method using an ontological knowledge model to automatically generate a data model for a domain of interest. The model can then be implemented in a relational database which efficiently enables the collection, storage, and retrieval of data while keeping semantic ontological annotations so that the same data can be extracted for various applications for further processing. RESULTS: This article (1) presents a comparison of existing methods for generating a relational data model from an ontology using 23 criteria, (2) describes standard conversion rules, and (3) presents O n t o R e l a , a prototype developed to demonstrate the conversion rules. CONCLUSION: This work is a first step toward automating and refining the generation of sharable and interoperable relational data models using ontologies with a freely available tool. The remaining challenges to cover all the ontology richness in the relational model are pointed out.

Design of an Ontology-Based Triage System for Patients with Chronic Pain.

OBJECTIVE: Waiting time for a consultation for chronic pain is a widespread health problem. This paper presents the design of an ontology use to assess patients referred to a consultation for chronic pain. METHODS: We designed OntoDol, an ontology of pain domain for patient triage based on priority degrees. Terms were extracted from clinical practice guidelines and mapped to SNOMED-CT concepts through the Python module Owlready2. Selected SNOMED-CT concepts, relationships, and the TIME ontology, were implemented in the ontology using Protégé. Decision rules were implemented with SWRL. We evaluated OntoDol on 5 virtual cases. RESULTS: OntoDol contains 762 classes, 92 object properties and 18 SWRL rules to assign patients to 4 categories of priority. OntoDol was able to assert every case and classify them in the right category of priority. CONCLUSION: Further works will extend OntoDol to other diseases and assess OntoDol with real world data from the hospital.

Using Deep Learning to Improve Phenotyping from Clinical Reports.

With the development of clinical databases and the ubiquity of EHRs, physicians and researchers alike have access to an unprecedented amount of data. Complexity of the available data has also increased since clinical reports are also included and require frameworks with natural language processing capabilities in order to process them and extract information not found in other types of documents. In the following work we implement a data processing pipeline performing phenotyping, disambiguation, negation and subject prediction on such reports. We compare it to an existing solution routinely used in a children's hospital with special focus on genetic diseases. We show that by replacing components based on rules and pattern matching with components leveraging deep learning models and fine-tuned word embeddings we obtain performance improvements of 7%, 10% and 27% in terms of F1 measure for each task. The solution we devised will help build more reliable decision support systems.

Towards a Clinical Decision Support System for Helping Medical Students in Emergency Call Centers.

In critical situations such as pandemic, medical students are often called to help in emergency call centers. However, they may encounter difficulties in phone triage because of a lack of medical skills. Here, we aim at developing a Clinical Decision Support System for helping medical students in phone call triage of pediatric patients. The system is based on the PAT (Pediatric Assessment Triangle) and local guidelines. It is composed of two interfaces. The first allows a quick assessment of severity signs, and the second provides recommendations and additional elements such as "elements to keep in mind" or "medical advice to give to patient". The system was evaluated by 20 medical students, with two fictive clinical cases. 75% of them found the content useful and clear, and the navigation easy. 65% would feel more reassured to have this system in emergency call centers. Further works are planned to improve the system before implementation in real-life.