Digital Emergency Management for a Complex One Health Landscape: the Need for Standardization, Integration, and Interoperability.

OBJECTIVE: Planning reliable long-term planning actions to handle disruptive events requires a timely development of technological infrastructures, as well as the set-up of focused strategies for emergency management. The paper aims to highlight the needs for standardization, integration, and interoperability between Accident & Emergency Informatics (A&EI) and One Digital Health (ODH), as fields capable of dealing with peculiar dynamics for a technology-boosted management of emergencies under an overarching One Health panorama. METHODS: An integrative analysis of the literature was conducted to draw attention to specific foci on the correlation between ODH and A&EI, in particular: (i) the management of disruptive events from private smart spaces to diseases spreading, and (ii) the concepts of (health-related) quality of life and well-being. RESULTS: A digitally-focused management of emergency events that tackles the inextricable interconnectedness between humans, animals, and surrounding environment, demands standardization, integration, and systems interoperability. A consistent and finalized process of adoption and implementation of methods and tools from the International Standard Accident Number (ISAN), via findability, accessibility, interoperability, and reusability (FAIR) data principles, to Medical Informatics and Digital Health Multilingual Ontology (MIMO) - capable of looking at different approaches to encourage the integration between the ODH framework and the A&EI vision, provides a first answer to these needs. CONCLUSIONS: ODH and A&EI look at different scales but with similar goals for converging health and environmental-related data management standards to enable multi-sources, interdisciplinary, and real-time data integration and interoperability. This allows holistic digital health both in routine and emergency events.

Trends in computerized provider order entry: 20-year bibliometric overview.

Background: Drug-related problems (DRPs) can lead to serious health issues and have significant economic impacts on healthcare systems. One solution to address this issue is the use of computerized physician order entry systems (CPOE), which can help prevent DRPs by reducing the risk of medication errors. Objective: The purpose of this study is to provide an analysis on scientific production of the past 20 years in order to describe trends in academic publishing on CPOE and to identify the major topics as well as the predominant actors (journals, countries) involved in this field. Methods: A PubMed search was carried out to extract articles related to computerized provider order entry during the period January 1st 2003- December 31st 2022 using a specific query. Data were downloaded from PubMed in Extensible Markup Language (XML) and were processed through a dedicated parser. Results: A total of 2,946 articles were retrieved among 623 journals. One third of these articles were published in eight journals. Publications grew strongly from 2002 to 2006, with a dip in 2008 followed by an increase again in 2009. After 2009, there follows a decreasing until 2022.The most producing countries are the USA with 51.39% of the publication over the period by France (3.80%), and Canada (3.77%). About disciplines, the top 3 is: "medical informatics" (21.62% of articles), "pharmacy" (19.04%), and "pediatrics" (6.56%). Discussion: This study provides an overview of publication trends related to CPOE, which exhibited a significant increase in the first decade of the 21st century followed by a decline after 2009. Possible reasons for this decline include the emergence of digital health tools beyond CPOE, as well as healthcare professionals experiencing alert fatigue of the current system. Conclusion: Future research should focus on analyzing publication trends in the field of medical informatics and decision-making tools to identify other areas of interest that may have surpassed the development of CPOE.

Integrating a new knowledge organisation system for monoclonal antibodies for therapeutic use authorised in Europe into HeTOP terminology-ontology server.

Monoclonal antibodies (MAs) are increasingly used in the therapeutic arsenal. Clinical Data Warehouses (CDWs) offer unprecedented opportunities for research on real-word data. The objective of this work is to develop a knowledge organization system on MAs for therapeutic use (MATUs) applicable in Europe to query CDWs from a multi-terminology server (HeTOP). After expert consensus, three main health thesauri were selected: the MeSH thesaurus, the National Cancer Institute thesaurus (NCIt) and the SNOMED CT. These thesauri contain 1,723 MAs concepts, but only 99 (5.7 %) are identified as MATUs. The knowledge organisation system proposed in this article is a six-level hierarchical system according to their main therapeutic target. It includes 193 different concepts organised in a cross lingual terminology server, which will allow the inclusion of semantic extensions. Ninety nine (51.3 %) MATUs concepts and 94 (48.7 %) hierarchical concepts composed the knowledge organisation system. Two separates groups (an expert group and a validation group) carried out the selection, creation and validation processes. Queries identify, for unstructured data, 83 out of 99 (83.8 %) MATUs corresponding to 45,262 patients, 347,035 hospital stays and 427,544 health documents, and for structured data, 61 out of 99 (61.6 %) MATUs corresponding to 9,218 patients, 59,643 hospital stays and 104,737 hospital prescriptions. The volume of data in the CDW demonstrated the potential for using these data in clinical research, although not all MATUs are present in the CDW (16 missing for unstructured data and 38 for structured data). The knowledge organisation system proposed here improves the understanding of MATUs, the quality of queries and helps clinical researchers retrieve relevant medical information. The use of this model in CDW allows for the rapid identification of a large number of patients and health documents, either directly by a MATU of interest (e.g. Rituximab) but also by searching for parent concepts (e.g. Anti-CD20 Monoclonal Antibody).

Modeling and integrating interactions involving the CYP450 enzyme system in a multi-terminology server: Contribution to information extraction from a clinical data warehouse.

INTRODUCTION: The cytochrome P450 (CYP450) enzyme system is involved in the metabolism of certain drugs and is responsible for most drug interactions. These interactions result in either an enzymatic inhibition or an enzymatic induction mechanism that has an impact on the therapeutic management of patients. Detecting these drug interactions will allow for better predictability in therapeutic response. Therefore, computerized solutions can represent a valuable help for clinicians in their tasks of detection. OBJECTIVE: The objective of this study is to provide a structured data-source of interactions involving the CYP450 enzyme system. These interactions are aimed to be integrated in the cross-lingual multi-terminology server HeTOP (Health Terminologies and Ontologies Portal), to support the query processing of the clinical data warehouse (CDW) EDSaN (Entrepôt de Données de Santé Normand). MATERIAL AND METHODS: A selection and curation of drug components (DCs) that share a relationship with the CYP450 system was performed from several international data sources. The DCs were linked according to the type of relationship which can be substrate, inhibitor, or inducer. These relationships were then integrated into the HeTOP server. To validate the CYP450 relationships, a semantic query was performed on the CDW, whose search engine is founded on HeTOP data (concepts, terms, and relations). RESULTS: A total of 776 DCs are associated by a new interaction relationship, integrated in HeTOP, by 14 enzymes. These are CYP450 1A2, 2A6, 2B6, 2C8, 2C9, 2C18, 2C19, 2D6, 2E1, 3A4, 3A7, 11B1,11B2 mitochondrial and P-glycoprotein, constituting a total of 2,088 relationships. A general modelling of cytochromic interactions was performed. From this model, 233,006 queries were processed in less than two hours, demonstrating the usefulness and performance of our CDW implementation. Moreover, they showed that in our university hospital, the concurrent prescription that could cause a cytochromic interaction is Bisoprolol with Amiodarone by enzymatic inhibition for 2,493 patients. DISCUSSION: The queries submitted to the CDW EDSaN allowed to highlight the most prescribed molecules simultaneously and potentially responsible for cytochromic interactions. In a second step, it would be interesting to evaluate the real clinical impact by looking for possible adverse effects of these interactions in the patients' files. Other computational solutions for cytochromic interactions exist. The impact of CYP450 is particularly important for drugs with narrow therapeutic window (NTW) as they can lead to increased toxicity or therapeutic failure. It is also important to define which drug component is a pro-drug and to considerate the many genetic polymorphisms of patients. CONCLUSION: The HeTOP server contains a non-negligible number of relationships between drug components and CYP450 from multiple reference sources. These data allow us to query our Clinical Data Warehouse to highlight these cytochromic interactions. It would be interesting in the future to assess the actual clinical impact in hospital reports.

Medical informatics and digital health multilingual ontology (MIMO): A tool to improve international collaborations.

BACKGROUND: Even if English is the leading language for international communication, it is essential to keep in mind that research runs at the local level by local teams generally communicating in their local/national language, especially in Europe among European projects. OBJECTIVE: Therefore, the European Federation for Medical Informatics - Working Group on Health Informatics for Inter-regional Cooperation" has one objective: To develop a multilingual ontology focusing on Health Informatics and Digital Health as a collaboration tool that improves international and, in particular, European collaborations. RESULTS: We have developed the Medical Informatics and Digital Health Multilingual Ontology (MIMO). Hosted on the Health Terminology/Ontology Portal (HeTOP), MIMO contains around 1,000 concepts, 460 MeSH Descriptors, 220 MeSH Concepts, and more than 300 newly created concepts. MIMO is continuously updated to comprise as recent as possible concepts and their translations in more than 30 languages. Moreover, the MIMO's development team constantly improves MIMO content and supporting information. Thus, during workshop discussions and one-on-one exchanges, the MIMO team has collected domain experts' opinions about the community's interests and suggestions for future enhancements. Moreover, MIMO will be integrated to support the annotation and categorization of research products into the HosmartAI European project involving more than 20 countries around Europe and worldwide. CONCLUSION: MIMO is hosted by HeTOP (Health Terminology/Ontology Portal), which integrates 100 terminologies and ontologies in 55 languages. MIMO is freely available online. MIMO is portable to other knowledge platforms as part of MIMO's main aims to facilitate communication between medical librarians, translators, and researchers as well as to support students' self-learning.

Reliability of citations of medRxiv preprints in articles published on COVID-19 in the world leading medical journals.

INTRODUCTION: Preprints have been widely cited during the COVID-19 pandemics, even in the major medical journals. However, since subsequent publication of preprint is not always mentioned in preprint repositories, some may be inappropriately cited or quoted. Our objectives were to assess the reliability of preprint citations in articles on COVID-19, to the rate of publication of preprints cited in these articles and to compare, if relevant, the content of the preprints to their published version. METHODS: Articles published on COVID in 2020 in the BMJ, The Lancet, the JAMA and the NEJM were manually screened to identify all articles citing at least one preprint from medRxiv. We searched PubMed, Google and Google Scholar to assess if the preprint had been published in a peer-reviewed journal, and when. Published articles were screened to assess if the title, data or conclusions were identical to the preprint version. RESULTS: Among the 205 research articles on COVID published by the four major medical journals in 2020, 60 (29.3%) cited at least one medRxiv preprint. Among the 182 preprints cited, 124 were published in a peer-reviewed journal, with 51 (41.1%) before the citing article was published online and 73 (58.9%) later. There were differences in the title, the data or the conclusion between the preprint cited and the published version for nearly half of them. MedRxiv did not mentioned the publication for 53 (42.7%) of preprints. CONCLUSIONS: More than a quarter of preprints citations were inappropriate since preprints were in fact already published at the time of publication of the citing article, often with a different content. Authors and editors should check the accuracy of the citations and of the quotations of preprints before publishing manuscripts that cite them.

Using Clinical Data Warehouse to Optimize the Vaccination Strategy Against COVID-19: A Use Case in France.

Clinical Data Warehouses (CDW) are gold mines and may be useful to manage the COVID-19 outbreak. This article details the use of CDW in order to retrieve patients for vaccination purposes. A list of 34 diseases (or conditions) was published by French Health Authorities to target individuals at a high risk of developing a severe form of COVID. Using a multilevel search engine, 23 queries were built based on structured or unstructured data using natural language processing features. The Diagnosis Related Group coding system was used alone in three queries (13.0%), coupled with unstructured data in four queries (17.4%), and unstructured data were used alone in 16 queries (69.6%). Eleven diseases (conditions) were too broad to be translated into queries. Finally, 6,006 unique re-identified patients were retrieved. This use case demonstrates the usefulness of the Rouen University Hospital CDW in retrieving patients for other purposes than translational research.

Qualitative Evaluation of a Drug Terminology Server.

BACKGROUND: Although the drug is finished, identifiable, there is no universally accepted standard for naming them. The objective of this work is to evaluate qualitatively the HeTOP drug terminology server by two categories of students: (a) pharmacy students and (b) a control group. METHODS: A formal evaluation was built to measure the perception of users about the HeTOP drug server, using the three mains questions about "teaching interest", "skill interest" (or competence) and "ergonomics". RESULTS: The three pharmacy student subgroups gave the best and the worst score to the same categories. CONCLUSION: All three criteria are rated above 6.5 out of 10. The HeTOP drug terminology server is freely available to "non drug" specialists (URL: www.hetop.eu/hetop/drugs/).

OntoBioStat: Supporting Causal Diagram Design and Analysis.

Suitable causal inference in biostatistics can be best achieved by knowledge representation thanks to causal diagrams or directed acyclic graphs. However, necessary and sufficient causes are not easily represented. Since existing ontologies do not fill this gap, we designed OntoBioStat in order to enable covariate selection support based on causal relation representations. OntoBioStat automatic ontological causal diagram construction and inferences are detailed in this study. OntoBioStat inferences are allowed by Semantic Web Rule Language rules and axioms. First, statements made by the users include outcome, exposure, covariate, and causal relation specification. Then, reasoning enable automatic construction using generic instances of Meta_Variable and Necessary_Variable classes. Finally, inferred classes highlighted potential bias such as confounder-like. Ontological causal diagram built with OntoBioStat was compared to a standard causal diagram (without OntoBioStat) in a theoretical study. It was found that confounding and bias were not completely identified by the standard causal diagram, and erroneous covariate sets were provided. Further research is needed in order to make OntoBioStat more usable.

Evaluation of Doc'EDS: a French semantic search tool to query health documents from a clinical data warehouse.

BACKGROUND: Unstructured data from electronic health records represent a wealth of information. Doc'EDS is a pre-screening tool based on textual and semantic analysis. The Doc'EDS system provides a graphic user interface to search documents in French. The aim of this study was to present the Doc'EDS tool and to provide a formal evaluation of its semantic features. METHODS: Doc'EDS is a search tool built on top of the clinical data warehouse developed at Rouen University Hospital. This tool is a multilevel search engine combining structured and unstructured data. It also provides basic analytical features and semantic utilities. A formal evaluation was conducted to measure the impact of Natural Language Processing algorithms. RESULTS: Approximately 18.1 million narrative documents are stored in Doc'EDS. The formal evaluation was conducted in 5000 clinical concepts that were manually collected. The F-measures of negative concepts and hypothetical concepts were respectively 0.89 and 0.57. CONCLUSION: In this formal evaluation, we have shown that Doc'EDS is able to deal with language subtleties to enhance an advanced full text search in French health documents. The Doc'EDS tool is currently used on a daily basis to help researchers to identify patient cohorts thanks to unstructured data.

Assisting Data Retrieval with a Drug Knowledge Graph.

The Normandy health data warehouse EDSaN integrates the medication orders from the University Hospital of Rouen (France). This study aims at describing the design and the evaluation of an information retrieval system founded on a complex and semantically augmented knowledge graph dedicated to EDSaN drugs' prescriptions. The system is intended to help the selection of drugs in the search process by health professionals. The manual evaluation of the relevance of the returned drugs showed encouraging results as expected. A deeper analysis in order to improve the ranking method is needed and will be performed in a future work.

Patient and Graph Embeddings for Predictive Diagnosis of Drug Iatrogenesis.

In the context of the IA.TROMED project we intend to develop and evaluate original algorithmic methods that will rely on semantic enrichment of embeddings by combining new deep learning algorithms, such as models founded on transformers, and symbolic artificial intelligence. The documents' embeddings, the graphs' embeddings of biomedical concepts, and patients' embeddings, all of them semantically enriched with aligned formal ontologies and semantic networks, will constitute a layer that will play the role of a queryable and searchable knowledge base that will supply the IA.TROMED's clinical, predictive, and iatrogenic diagnosis support module.

Ontological Models Supporting Covariates Selection in Observational Studies.

In the context of causal inference, biostatisticians use causal diagrams to select covariates in order to build multivariate models. These diagrams represent datasets variables and their relations but have some limitations (representing interactions, bidirectional causal relations). The MetBrAYN project aims at building an ontological-based process to tackle these issues. The knowledge acquired by the biostatistician during a methodological consultation for a research question will be represented in a general ontology. In order to aggregate various forms of knowledge the ontology will act as a wrapper. Ontology-based causal diagrams will be semi-automatically built. Founded on inference rules, the global system will help biostatisticians to curate it and to visualize recommended covariates for their research question.

Performance evaluation of three semantic expansions to query PubMed.

BACKGROUND: PubMed is one of the most important basic tools to access medical literature. Semantic query expansion using synonyms can improve retrieval efficacy. OBJECTIVE: The objective was to evaluate the performance of three semantic query expansion strategies. METHODS: Queries were built for forty MeSH descriptors using three semantic expansion strategies (MeSH synonyms, UMLS mappings, and mappings created by the CISMeF team), then sent to PubMed. To evaluate expansion performances for each query, the first twenty citations were selected, and their relevance were judged by three independent evaluators based on the title and abstract. RESULTS: Queries built with the UMLS expansion provided new citations with a slightly higher mean precision (74.19%) than with the CISMeF expansion (70.28%), although the difference was not significant. Inter-rater agreement was 0.28. Results varied greatly depending on the descriptor selected. DISCUSSION: The number of citations retrieved by the three strategies and their precision varied greatly according to the descriptor. This heterogeneity could be explained by the quality of the synonyms. Optimal use of these different expansions would be through various combinations of UMLS and CISMeF intersections or unions. CONCLUSION: Information retrieval tools should propose different semantic expansions depending on the descriptor and the search objectives.

Manual Evaluation of the Automatic Mapping of International Classification of Diseases (ICD)-11 (in French).

A lexical method was used to map ICD-11 to the terminologies included in the HeTOP server. About half of ICD-11 codes (47.76%) were mapped to at least one concept. The developed tool reached a global precision of 0.98 and a recall of 0.66. Lexical methods are powerful methods to map health terminologies. Supervised and manual mapping is still necessary to complete the mapping.

Identification of the Best Semantic Expansion to Query PubMed Through Automatic Performance Assessment of Four Search Strategies on All Medical Subject Heading Descriptors: Comparative Study.

BACKGROUND: With the continuous expansion of available biomedical data, efficient and effective information retrieval has become of utmost importance. Semantic expansion of queries using synonyms may improve information retrieval. OBJECTIVE: The aim of this study was to automatically construct and evaluate expanded PubMed queries of the form "preferred term"[MH] OR "preferred term"[TIAB] OR "synonym 1"[TIAB] OR "synonym 2"[TIAB] OR …, for each of the 28,313 Medical Subject Heading (MeSH) descriptors, by using different semantic expansion strategies. We sought to propose an innovative method that could automatically evaluate these strategies, based on the three main metrics used in information science (precision, recall, and F-measure). METHODS: Three semantic expansion strategies were assessed. They differed by the synonyms used to build the queries as follows: MeSH synonyms, Unified Medical Language System (UMLS) mappings, and custom mappings (Catalogue et Index des Sites Médicaux de langue Française [CISMeF]). The precision, recall, and F-measure metrics were automatically computed for the three strategies and for the standard automatic term mapping (ATM) of PubMed. The method to automatically compute the metrics involved computing the number of all relevant citations (A), using National Library of Medicine indexing as the gold standard ("preferred term"[MH]), the number of citations retrieved by the added terms ("synonym 1"[TIAB] OR "synonym 2"[TIAB] OR …) (B), and the number of relevant citations retrieved by the added terms (combining the previous two queries with an "AND" operator) (C). It was possible to programmatically compute the metrics for each strategy using each of the 28,313 MeSH descriptors as a "preferred term," corresponding to 239,724 different queries built and sent to the PubMed application program interface. The four search strategies were ranked and compared for each metric. RESULTS: ATM had the worst performance for all three metrics among the four strategies. The MeSH strategy had the best mean precision (51%, SD 23%). The UMLS strategy had the best recall and F-measure (41%, SD 31% and 36%, SD 24%, respectively). CISMeF had the second best recall and F-measure (40%, SD 31% and 35%, SD 24%, respectively). However, considering a cutoff of 5%, CISMeF had better precision than UMLS for 1180 descriptors, better recall for 793 descriptors, and better F-measure for 678 descriptors. CONCLUSIONS: This study highlights the importance of using semantic expansion strategies to improve information retrieval. However, the performances of a given strategy, relatively to another, varied greatly depending on the MeSH descriptor. These results confirm there is no ideal search strategy for all descriptors. Different semantic expansions should be used depending on the descriptor and the user's objectives. Thus, we developed an interface that allows users to input a descriptor and then proposes the best semantic expansion to maximize the three main metrics (precision, recall, and F-measure).

Building a Semantic Health Data Warehouse in the Context of Clinical Trials: Development and Usability Study.

BACKGROUND: The huge amount of clinical, administrative, and demographic data recorded and maintained by hospitals can be consistently aggregated into health data warehouses with a uniform data model. In 2017, Rouen University Hospital (RUH) initiated the design of a semantic health data warehouse enabling both semantic description and retrieval of health information. OBJECTIVE: This study aimed to present a proof of concept of this semantic health data warehouse, based on the data of 250,000 patients from RUH, and to assess its ability to assist health professionals in prescreening eligible patients in a clinical trials context. METHODS: The semantic health data warehouse relies on 3 distinct semantic layers: (1) a terminology and ontology portal, (2) a semantic annotator, and (3) a semantic search engine and NoSQL (not only structured query language) layer to enhance data access performances. The system adopts an entity-centered vision that provides generic search capabilities able to express data requirements in terms of the whole set of interconnected conceptual entities that compose health information. RESULTS: We assessed the ability of the system to assist the search for 95 inclusion and exclusion criteria originating from 5 randomly chosen clinical trials from RUH. The system succeeded in fully automating 39% (29/74) of the criteria and was efficiently used as a prescreening tool for 73% (54/74) of them. Furthermore, the targeted sources of information and the search engine-related or data-related limitations that could explain the results for each criterion were also observed. CONCLUSIONS: The entity-centered vision contrasts with the usual patient-centered vision adopted by existing systems. It enables more genericity in the information retrieval process. It also allows to fully exploit the semantic description of health information. Despite their semantic annotation, searching within clinical narratives remained the major challenge of the system. A finer annotation of the clinical texts and the addition of specific functionalities would significantly improve the results. The semantic aspect of the system combined with its generic entity-centered vision enables the processing of a large range of clinical questions. However, an important part of health information remains in clinical narratives, and we are currently investigating novel approaches (deep learning) to enhance the semantic annotation of those unstructured data.

Word Embedding for the French Natural Language in Health Care: Comparative Study.

BACKGROUND: Word embedding technologies, a set of language modeling and feature learning techniques in natural language processing (NLP), are now used in a wide range of applications. However, no formal evaluation and comparison have been made on the ability of each of the 3 current most famous unsupervised implementations (Word2Vec, GloVe, and FastText) to keep track of the semantic similarities existing between words, when trained on the same dataset. OBJECTIVE: The aim of this study was to compare embedding methods trained on a corpus of French health-related documents produced in a professional context. The best method will then help us develop a new semantic annotator. METHODS: Unsupervised embedding models have been trained on 641,279 documents originating from the Rouen University Hospital. These data are not structured and cover a wide range of documents produced in a clinical setting (discharge summary, procedure reports, and prescriptions). In total, 4 rated evaluation tasks were defined (cosine similarity, odd one, analogy-based operations, and human formal evaluation) and applied on each model, as well as embedding visualization. RESULTS: Word2Vec had the highest score on 3 out of 4 rated tasks (analogy-based operations, odd one similarity, and human validation), particularly regarding the skip-gram architecture. CONCLUSIONS: Although this implementation had the best rate for semantic properties conservation, each model has its own qualities and defects, such as the training time, which is very short for GloVe, or morphological similarity conservation observed with FastText. Models and test sets produced by this study will be the first to be publicly available through a graphical interface to help advance the French biomedical research.

Impact on medical practice of accessing pharmaceutical records.

INTRODUCTION: The pharmaceutical record system (PRS) is a French nationwide centralized electronic database shared among all community pharmacists listing all drugs dispensed by community pharmacists in the last four months. The objective of this study, the Medication Assessment Through Real time Information eXchange - Distributed Pharmaceutical Record System (MATRIX - DPRS) study, was to assess the clinical impact of the PRS upon granting access to physicians in three hospital specialties: anesthesiology, emergency medicine and geriatrics. MATERIAL AND METHODS: A multicenter prospective study was conducted in six hospital departments, two per specialty. Participating physicians noted medication information found exclusively in the pharmaceutical record (PR) of each patient unavailable elsewhere and any diagnostic or therapeutic management changes resulting from the PR information. The primary objective was to assess the proportion of diagnostic or therapeutic management changes attributable to the PR among patients who had an accessible PR. RESULTS: The inclusion level ranged from 1.1 to 30% in the six departments. The rate of diagnostic or therapeutic management changes was highest in geriatrics (n = 31/67; 46.3% 95% Confidence IntervaI (CI): 34.0-58.9%) and lowest in anesthesiology (n = 36/227; 15.9% 95% CI: 11.4-21.3%). Emergency medicine was intermediate (n = 5/22; 22.7% 95% CI: 7.8-45.4%). CONCLUSION: Although the inclusion rate and statistical precision were low, these findings suggest that the information contained in the PRS is useful and may result in modifying patient management in a sizeable proportion of patients. This opens the prospect of evaluating other hospital specialties, as well as primary and secondary care settings.