A 12-gene pharmacogenetic panel to prevent adverse drug reactions: an open-label, multicentre, controlled, cluster-randomised crossover implementation study.

BACKGROUND: The benefit of pharmacogenetic testing before starting drug therapy has been well documented for several single gene-drug combinations. However, the clinical utility of a pre-emptive genotyping strategy using a pharmacogenetic panel has not been rigorously assessed. METHODS: We conducted an open-label, multicentre, controlled, cluster-randomised, crossover implementation study of a 12-gene pharmacogenetic panel in 18 hospitals, nine community health centres, and 28 community pharmacies in seven European countries (Austria, Greece, Italy, the Netherlands, Slovenia, Spain, and the UK). Patients aged 18 years or older receiving a first prescription for a drug clinically recommended in the guidelines of the Dutch Pharmacogenetics Working Group (ie, the index drug) as part of routine care were eligible for inclusion. Exclusion criteria included previous genetic testing for a gene relevant to the index drug, a planned duration of treatment of less than 7 consecutive days, and severe renal or liver insufficiency. All patients gave written informed consent before taking part in the study. Participants were genotyped for 50 germline variants in 12 genes, and those with an actionable variant (ie, a drug-gene interaction test result for which the Dutch Pharmacogenetics Working Group [DPWG] recommended a change to standard-of-care drug treatment) were treated according to DPWG recommendations. Patients in the control group received standard treatment. To prepare clinicians for pre-emptive pharmacogenetic testing, local teams were educated during a site-initiation visit and online educational material was made available. The primary outcome was the occurrence of clinically relevant adverse drug reactions within the 12-week follow-up period. Analyses were irrespective of patient adherence to the DPWG guidelines. The primary analysis was done using a gatekeeping analysis, in which outcomes in people with an actionable drug-gene interaction in the study group versus the control group were compared, and only if the difference was statistically significant was an analysis done that included all of the patients in the study. Outcomes were compared between the study and control groups, both for patients with an actionable drug-gene interaction test result (ie, a result for which the DPWG recommended a change to standard-of-care drug treatment) and for all patients who received at least one dose of index drug. The safety analysis included all participants who received at least one dose of a study drug. This study is registered with ClinicalTrials.gov, NCT03093818 and is closed to new participants. FINDINGS: Between March 7, 2017, and June 30, 2020, 41 696 patients were assessed for eligibility and 6944 (51·4 % female, 48·6% male; 97·7% self-reported European, Mediterranean, or Middle Eastern ethnicity) were enrolled and assigned to receive genotype-guided drug treatment (n=3342) or standard care (n=3602). 99 patients (52 [1·6%] of the study group and 47 [1·3%] of the control group) withdrew consent after group assignment. 652 participants (367 [11·0%] in the study group and 285 [7·9%] in the control group) were lost to follow-up. In patients with an actionable test result for the index drug (n=1558), a clinically relevant adverse drug reaction occurred in 152 (21·0%) of 725 patients in the study group and 231 (27·7%) of 833 patients in the control group (odds ratio [OR] 0·70 [95% CI 0·54-0·91]; p=0·0075), whereas for all patients, the incidence was 628 (21·5%) of 2923 patients in the study group and 934 (28·6%) of 3270 patients in the control group (OR 0·70 [95% CI 0·61-0·79]; p <0·0001). INTERPRETATION: Genotype-guided treatment using a 12-gene pharmacogenetic panel significantly reduced the incidence of clinically relevant adverse drug reactions and was feasible across diverse European health-care system organisations and settings. Large-scale implementation could help to make drug therapy increasingly safe. FUNDING: European Union Horizon 2020.

Benchmark datasets driving artificial intelligence development fail to capture the needs of medical professionals.

Publicly accessible benchmarks that allow for assessing and comparing model performances are important drivers of progress in artificial intelligence (AI). While recent advances in AI capabilities hold the potential to transform medical practice by assisting and augmenting the cognitive processes of healthcare professionals, the coverage of clinically relevant tasks by AI benchmarks is largely unclear. Furthermore, there is a lack of systematized meta-information that allows clinical AI researchers to quickly determine accessibility, scope, content and other characteristics of datasets and benchmark datasets relevant to the clinical domain. To address these issues, we curated and released a comprehensive catalogue of datasets and benchmarks pertaining to the broad domain of clinical and biomedical natural language processing (NLP), based on a systematic review of literature and. A total of 450 NLP datasets were manually systematized and annotated with rich metadata, such as targeted tasks, clinical applicability, data types, performance metrics, accessibility and licensing information, and availability of data splits. We then compared tasks covered by AI benchmark datasets with relevant tasks that medical practitioners reported as highly desirable targets for automation in a previous empirical study. Our analysis indicates that AI benchmarks of direct clinical relevance are scarce and fail to cover most work activities that clinicians want to see addressed. In particular, tasks associated with routine documentation and patient data administration workflows are not represented despite significant associated workloads. Thus, currently available AI benchmarks are improperly aligned with desired targets for AI automation in clinical settings, and novel benchmarks should be created to fill these gaps.

Neural sentence embedding models for semantic similarity estimation in the biomedical domain.

BACKGROUND: Neural network based embedding models are receiving significant attention in the field of natural language processing due to their capability to effectively capture semantic information representing words, sentences or even larger text elements in low-dimensional vector space. While current state-of-the-art models for assessing the semantic similarity of textual statements from biomedical publications depend on the availability of laboriously curated ontologies, unsupervised neural embedding models only require large text corpora as input and do not need manual curation. In this study, we investigated the efficacy of current state-of-the-art neural sentence embedding models for semantic similarity estimation of sentences from biomedical literature. We trained different neural embedding models on 1.7 million articles from the PubMed Open Access dataset, and evaluated them based on a biomedical benchmark set containing 100 sentence pairs annotated by human experts and a smaller contradiction subset derived from the original benchmark set. RESULTS: Experimental results showed that, with a Pearson correlation of 0.819, our best unsupervised model based on the Paragraph Vector Distributed Memory algorithm outperforms previous state-of-the-art results achieved on the BIOSSES biomedical benchmark set. Moreover, our proposed supervised model that combines different string-based similarity metrics with a neural embedding model surpasses previous ontology-dependent supervised state-of-the-art approaches in terms of Pearson's r (r = 0.871) on the biomedical benchmark set. In contrast to the promising results for the original benchmark, we found our best models' performance on the smaller contradiction subset to be poor. CONCLUSIONS: In this study, we have highlighted the value of neural network-based models for semantic similarity estimation in the biomedical domain by showing that they can keep up with and even surpass previous state-of-the-art approaches for semantic similarity estimation that depend on the availability of laboriously curated ontologies, when evaluated on a biomedical benchmark set. Capturing contradictions and negations in biomedical sentences, however, emerged as an essential area for further work.

Fast and scalable neural embedding models for biomedical sentence classification.

BACKGROUND: Biomedical literature is expanding rapidly, and tools that help locate information of interest are needed. To this end, a multitude of different approaches for classifying sentences in biomedical publications according to their coarse semantic and rhetoric categories (e.g., Background, Methods, Results, Conclusions) have been devised, with recent state-of-the-art results reported for a complex deep learning model. Recent evidence showed that shallow and wide neural models such as fastText can provide results that are competitive or superior to complex deep learning models while requiring drastically lower training times and having better scalability. We analyze the efficacy of the fastText model in the classification of biomedical sentences in the PubMed 200k RCT benchmark, and introduce a simple pre-processing step that enables the application of fastText on sentence sequences. Furthermore, we explore the utility of two unsupervised pre-training approaches in scenarios where labeled training data are limited. RESULTS: Our fastText-based methodology yields a state-of-the-art F1 score of.917 on the PubMed 200k benchmark when sentence ordering is taken into account, with a training time of only 73 s on standard hardware. Applying fastText on single sentences, without taking sentence ordering into account, yielded an F1 score of.852 (training time 13 s). Unsupervised pre-training of N-gram vectors greatly improved the results for small training set sizes, with an increase of F1 score of.21 to.74 when trained on only 1000 randomly picked sentences without taking sentence ordering into account. CONCLUSIONS: Because of it's ease of use and performance, fastText should be among the first choices of tools when tackling biomedical text classification problems with large corpora. Unsupervised pre-training of N-gram vectors on domain-specific corpora also makes it possible to apply fastText when labeled training data are limited.

Digital health understanding and preparedness of medical students: a cross-sectional study.

Digitalisation is changing all areas of our daily life. This changing environment requires new competences from physicians in all specialities. This study systematically surveyed the knowledge, attitude, and interests of medical students. These results will help further develop the medical curriculum, as well as increase our understanding of future physicians by other healthcare market players. A web-based survey consisting of four sections was developed: Section one queried demographic data, section two assessed the current digital health knowledge of medical students, section three queried their attitudes about the future impact of digital health in medicine and section four assessed the recommendations medical students have for the medical curriculum in terms of digital health. This survey was distributed to all (11,978) student at all public Austrian medical schools. A total of 8.4% of the medical student population started the survey. At the knowledge self-assessment section, the medical students reached mean of 11.74 points (SD 4.42) out of a possible maximum of 32 (female mean 10.66/ SD 3.87, male mean 13.34/SD 4.50). The attitude section showed that students see digitalisation as a threat, especially with respect to the patient-physician relationship. The curriculum recommendation section showed a high interest for topics related to AI, a per study year increasing interest in impact of digital health in communication, as well as a decreasing interest in robotic related topics. The attitude towards digital health can be described as sceptical. To ensure that future physicians keep pace with this development and fulfil their responsibility towards the society, medical schools need to be more proactive to foster the understanding of medical students that digital health will persistently alter the medical practice.

A curated, ontology-based, large-scale knowledge graph of artificial intelligence tasks and benchmarks.

Research in artificial intelligence (AI) is addressing a growing number of tasks through a rapidly growing number of models and methodologies. This makes it difficult to keep track of where novel AI methods are successfully - or still unsuccessfully - applied, how progress is measured, how different advances might synergize with each other, and how future research should be prioritized. To help address these issues, we created the Intelligence Task Ontology and Knowledge Graph (ITO), a comprehensive, richly structured and manually curated resource on artificial intelligence tasks, benchmark results and performance metrics. The current version of ITO contains 685,560 edges, 1,100 classes representing AI processes and 1,995 properties representing performance metrics. The primary goal of ITO is to enable analyses of the global landscape of AI tasks and capabilities. ITO is based on technologies that allow for easy integration and enrichment with external data, automated inference and continuous, collaborative expert curation of underlying ontological models. We make the ITO dataset and a collection of Jupyter notebooks utilizing ITO openly available.

Mapping global dynamics of benchmark creation and saturation in artificial intelligence.

Benchmarks are crucial to measuring and steering progress in artificial intelligence (AI). However, recent studies raised concerns over the state of AI benchmarking, reporting issues such as benchmark overfitting, benchmark saturation and increasing centralization of benchmark dataset creation. To facilitate monitoring of the health of the AI benchmarking ecosystem, we introduce methodologies for creating condensed maps of the global dynamics of benchmark creation and saturation. We curate data for 3765 benchmarks covering the entire domains of computer vision and natural language processing, and show that a large fraction of benchmarks quickly trends towards near-saturation, that many benchmarks fail to find widespread utilization, and that benchmark performance gains for different AI tasks are prone to unforeseen bursts. We analyze attributes associated with benchmark popularity, and conclude that future benchmarks should emphasize versatility, breadth and real-world utility.

Pharmacogenomics decision support in the U-PGx project: Results and advice from clinical implementation across seven European countries.

BACKGROUND: The clinical implementation of pharmacogenomics (PGx) could be one of the first milestones towards realizing personalized medicine in routine care. However, its widespread adoption requires the availability of suitable clinical decision support (CDS) systems, which is often impeded by the fragmentation or absence of adequate health IT infrastructures. We report results of CDS implementation in the large-scale European research project Ubiquitous Pharmacogenomics (U-PGx), in which PGx CDS was rolled out and evaluated across more than 15 clinical sites in the Netherlands, Spain, Slovenia, Italy, Greece, United Kingdom and Austria, covering a wide variety of healthcare settings. METHODS: We evaluated the CDS implementation process through qualitative and quantitative process indicators. Quantitative indicators included statistics on generated PGx reports, median time from sampled upload until report delivery and statistics on report retrievals via the mobile-based CDS tool. Adoption of different CDS tools, uptake and usability were further investigated through a user survey among healthcare providers. Results of a risk assessment conducted prior to the implementation process were retrospectively analyzed and compared to actual encountered difficulties and their impact. RESULTS: As of March 2021, personalized PGx reports were produced from 6884 genotyped samples with a median delivery time of twenty minutes. Out of 131 invited healthcare providers, 65 completed the questionnaire (response rate: 49.6%). Overall satisfaction rates with the different CDS tools varied between 63.6% and 85.2% per tool. Delays in implementation were caused by challenges including institutional factors and complexities in the development of required tools and reference data resources, such as genotype-phenotype mappings. CONCLUSIONS: We demonstrated the feasibility of implementing a standardized PGx decision support solution in a multinational, multi-language and multi-center setting. Remaining challenges for future wide-scale roll-out include the harmonization of existing PGx information in guidelines and drug labels, the need for strategies to lower the barrier of PGx CDS adoption for healthcare institutions and providers, and easier compliance with regulatory and legal frameworks.

Effects of Medical Device Regulations on the Development of Stand-Alone Medical Software: A Pilot Study.

BACKGROUND: Medical device regulations which aim to ensure safety standards do not only apply to hardware devices but also to standalone medical software, e.g. mobile apps. OBJECTIVES: To explore the effects of these regulations on the development and distribution of medical standalone software. METHODS: We invited a convenience sample of 130 domain experts to participate in an online survey about the impact of current regulations on the development and distribution of medical standalone software. RESULTS: 21 respondents completed the questionnaire. Participants reported slight positive effects on usability, reliability, and data security of their products, whereas the ability to modify already deployed software and customization by end users were negatively impacted. The additional time and costs needed to go through the regulatory process were perceived as the greatest obstacles in developing and distributing medical software. CONCLUSION: Further research is needed to compare positive effects on software quality with negative impacts on market access and innovation. Strategies for avoiding over-regulation while still ensuring safety standards need to be devised.

Implementing pharmacogenomics decision support across seven European countries: The Ubiquitous Pharmacogenomics (U-PGx) project.

Clinical pharmacogenomics (PGx) has the potential to make pharmacotherapy safer and more effective by utilizing genetic patient data for drug dosing and selection. However, widespread adoption of PGx depends on its successful integration into routine clinical care through clinical decision support tools, which is often hampered by insufficient or fragmented infrastructures. This paper describes the setup and implementation of a unique multimodal, multilingual clinical decision support intervention consisting of digital, paper-, and mobile-based tools that are deployed across implementation sites in seven European countries participating in the Ubiquitous PGx (U-PGx) project.

The Importance of Gene-Drug-Drug-Interactions in Pharmacogenomics Decision Support: An Analysis Based on Austrian Claims Data.

While pharmacogenomic testing combined with clinical decision support has the potential to increase the safety and efficacy of medical treatments, the intake of multiple prescription drugs can - if not sufficiently addressed by decision support solutions - impair the effectiveness of such interventions by modulating the capacity of precisely those enzymes whose function pharmacogenomic tests try to predict. We quantified the potential extent of such drug-mediated mismatches between genotype-derived phenotypes and real phenotypes, commonly called "phenoconversion", by screening claims data from 1,587,829 Austrian health insurance holders of the years 2006 and 2007 for concomitant prescriptions of drugs that can be dosed based on pharmacogenomics, and drugs that modulate enzyme activity. In total, 232,398 such prescription overlaps were detected, of which more than half (54.6%) could be attributed to co-prescriptions of moderate or strong modulators. Our results indicate that prescription drug-mediated phenoconversion is not uncommon, and should therefore be adequately reflected in decision support solutions by integrating algorithms to detect potential gene-drug-drug interactions.

Examining perceptions of the usefulness and usability of a mobile-based system for pharmacogenomics clinical decision support: a mixed methods study.

Background. Pharmacogenomic testing has the potential to improve the safety and efficacy of pharmacotherapy, but clinical application of pharmacogenetic knowledge has remained uncommon. Clinical Decision Support (CDS) systems could help overcome some of the barriers to clinical implementation. The aim of this study was to evaluate the perception and usability of a web- and mobile-enabled CDS system for pharmacogenetics-guided drug therapy-the Medication Safety Code (MSC) system-among potential users (i.e., physicians and pharmacists). Furthermore, this study sought to collect data on the practicability and comprehensibility of potential layouts of a proposed personalized pocket card that is intended to not only contain the machine-readable data for use with the MSC system but also human-readable data on the patient's pharmacogenomic profile. Methods. We deployed an emergent mixed methods design encompassing (1) qualitative interviews with pharmacists and pharmacy students, (2) a survey among pharmacogenomics experts that included both qualitative and quantitative elements and (3) a quantitative survey among physicians and pharmacists. The interviews followed a semi-structured guide including a hypothetical patient scenario that had to be solved by using the MSC system. The survey among pharmacogenomics experts focused on what information should be printed on the card and how this information should be arranged. Furthermore, the MSC system was evaluated based on two hypothetical patient scenarios and four follow-up questions on the perceived usability. The second survey assessed physicians' and pharmacists' attitude towards the MSC system. Results. In total, 101 physicians, pharmacists and PGx experts coming from various relevant fields evaluated the MSC system. Overall, the reaction to the MSC system was positive across all investigated parameters and among all user groups. The majority of participants were able to solve the patient scenarios based on the recommendations displayed on the MSC interface. A frequent request among participants was to provide specific listings of alternative drugs and concrete dosage instructions. Negligence of other patient-specific factors for choosing the right treatment such as renal function and co-medication was a common concern related to the MSC system, while data privacy and cost-benefit considerations emerged as the participants' major concerns regarding pharmacogenetic testing in general. The results of the card layout evaluation indicate that a gene-centered and tabulated presentation of the patient's pharmacogenomic profile is helpful and well-accepted. Conclusions. We found that the MSC system was well-received among the physicians and pharmacists included in this study. A personalized pocket card that lists a patient's metabolizer status along with critically affected drugs can alert physicians and pharmacists to the availability of essential therapy modifications.

Incidence of Exposure of Patients in the United States to Multiple Drugs for Which Pharmacogenomic Guidelines Are Available.

Pre-emptive pharmacogenomic (PGx) testing of a panel of genes may be easier to implement and more cost-effective than reactive pharmacogenomic testing if a sufficient number of medications are covered by a single test and future medication exposure can be anticipated. We analysed the incidence of exposure of individual patients in the United States to multiple drugs for which pharmacogenomic guidelines are available (PGx drugs) within a selected four-year period (2009-2012) in order to identify and quantify the incidence of pharmacotherapy in a nation-wide patient population that could be impacted by pre-emptive PGx testing based on currently available clinical guidelines. In total, 73 024 095 patient records from private insurance, Medicare Supplemental and Medicaid were included. Patients enrolled in Medicare Supplemental age > = 65 or Medicaid age 40-64 had the highest incidence of PGx drug use, with approximately half of the patients receiving at least one PGx drug during the 4 year period and one fourth to one third of patients receiving two or more PGx drugs. These data suggest that exposure to multiple PGx drugs is common and that it may be beneficial to implement wide-scale pre-emptive genomic testing. Future work should therefore concentrate on investigating the cost-effectiveness of multiplexed pre-emptive testing strategies.

Analyzing the potential for incorrect haplotype calls with different pharmacogenomic assays in different populations: a simulation based on 1000 Genomes data.

AIM: Many currently available pharmacogenomic assays and algorithms interrogate a set of 'tag' polymorphisms for inferring haplotypes. We wanted to test the accuracy of such haplotype inferences across different populations. MATERIALS & METHODS: We simulated haplotype inferences made by existing pharmacogenomic assays for seven important pharmacogenes based on full genome data of 2504 persons in the 1000 Genomes dataset. RESULTS: A sizable fraction of samples did not match any of the haplotypes in the star allele nomenclature systems. We found no clear population bias in the accuracy of results of simulated assays. CONCLUSION: Haplotype nomenclatures and inference algorithms need to be improved to adequately capture pharmacogenomic diversity in human populations.

Towards a global IT system for personalized medicine: the Medicine Safety Code initiative.

The availability of pharmacogenomic data of individual patients can significantly improve physicians' prescribing behavior, lead to a reduced incidence of adverse drug events and an improvement of effectiveness of treatment. The Medicine Safety Code (MSC) initiative is an effort to improve the ability of clinicians and patients to share pharmacogenomic data and to use it at the point of care. The MSC is a standardized two-dimensional barcode that captures individual pharmacogenomic data. The system is backed by a web service that allows the decoding and interpretation of anonymous MSCs without requiring the installation of dedicated software. The system is based on a curated, ontology-based knowledge base representing pharmacogenomic definitions and clinical guidelines. The MSC system performed well in preliminary tests. To evaluate the system in realistic health care settings and to translate it into practical applications, the future participation of stakeholders in clinical institutions, medical researchers, pharmaceutical companies, genetic testing providers, health IT companies and health insurance organizations will be essential.

Towards a global IT system for personalized medicine: the Medicine Safety Code initiative.

The availability of pharmacogenomic data of individual patients can significantly improve physicians' prescribing behavior, lead to a reduced incidence of adverse drug events and an improvement of effectiveness of treatment. The Medicine Safety Code (MSC) initiative is an effort to improve the ability of clinicians and patients to share pharmacogenomic data and to use it at the point of care. The MSC is a standardized two-dimensional barcode that captures individual pharmacogenomic data. The system is backed by a web service that allows the decoding and interpretation of anonymous MSCs without requiring the installation of dedicated software. The system is based on a curated, ontology-based knowledge base representing pharmacogenomic definitions and clinical guidelines. The MSC system performed well in preliminary tests. To evaluate the system in realistic health care settings and to translate it into practical applications, the future participation of stakeholders in clinical institutions, researchers, pharmaceutical companies, genetic testing providers, health IT companies and health insurance organizations will be essential.

An ontology-based, mobile-optimized system for pharmacogenomic decision support at the point-of-care.

BACKGROUND: The development of genotyping and genetic sequencing techniques and their evolution towards low costs and quick turnaround have encouraged a wide range of applications. One of the most promising applications is pharmacogenomics, where genetic profiles are used to predict the most suitable drugs and drug dosages for the individual patient. This approach aims to ensure appropriate medical treatment and avoid, or properly manage, undesired side effects. RESULTS: We developed the Medicine Safety Code (MSC) service, a novel pharmacogenomics decision support system, to provide physicians and patients with the ability to represent pharmacogenomic data in computable form and to provide pharmacogenomic guidance at the point-of-care. Pharmacogenomic data of individual patients are encoded as Quick Response (QR) codes and can be decoded and interpreted with common mobile devices without requiring a centralized repository for storing genetic patient data. In this paper, we present the first fully functional release of this system and describe its architecture, which utilizes Web Ontology Language 2 (OWL 2) ontologies to formalize pharmacogenomic knowledge and to provide clinical decision support functionalities. CONCLUSIONS: The MSC system provides a novel approach for enabling the implementation of personalized medicine in clinical routine.