Large Language Models for Synthetic Tabular Health Data: A Benchmark Study.

Synthetic tabular health data plays a crucial role in healthcare research, addressing privacy regulations and the scarcity of publicly available datasets. This is essential for diagnostic and treatment advancements. Among the most promising models are transformer-based Large Language Models (LLMs) and Generative Adversarial Networks (GANs). In this paper, we compare LLM models of the Pythia LLM Scaling Suite with varying model sizes ranging from 14M to 1B, against a reference GAN model (CTGAN). The generated synthetic data are used to train random forest estimators for classification tasks to make predictions on the real-world data. Our findings indicate that as the number of parameters increases, LLM models outperform the reference GAN model. Even the smallest 14M parameter models perform comparably to GANs. Moreover, we observe a positive correlation between the size of the training dataset and model performance. We discuss implications, challenges, and considerations for the real-world usage of LLM models for synthetic tabular data generation.

Modelling Events in Biomedical Decision Support Systems Using Ontologies.

Biomedical decision support systems play a crucial role in modern healthcare by assisting clinicians in making informed decisions. Events, such as physiological changes or drug reactions, are integral components of these systems, influencing patient outcomes and treatment strategies. However, effectively modeling events within these systems presents significant challenges due to the complexity and dynamic nature of medical data. Especially the differentiation between events and processes as well as the nature of events is often unclear. This paper explores approaches to modeling events in biomedical decision support systems, considering factors such as ontology-based representation. By addressing these challenges, we strive to provide the means for enhancing the functionality and interpretability of biomedical decision support systems concerning events.

What Kind of Transformer Models to Use for the ICD-10 Codes Classification Task.

Coding according to the International Classification of Diseases (ICD)-10 and its clinical modifications (CM) is inherently complex and expensive. Natural Language Processing (NLP) assists by simplifying the analysis of unstructured data from electronic health records, thereby facilitating diagnosis coding. This study investigates the suitability of transformer models for ICD-10 classification, considering both encoder and encoder-decoder architectures. The analysis is performed on clinical discharge summaries from the Medical Information Mart for Intensive Care (MIMIC)-IV dataset, which contains an extensive collection of electronic health records. Pre-trained models such as BioBERT, ClinicalBERT, ClinicalLongformer, and ClinicalBigBird are adapted for the coding task, incorporating specific preprocessing techniques to enhance performance. The findings indicate that increasing context length improves accuracy, and that the difference in accuracy between encoder and encoder-decoder models is negligible.

An ontology-based tool for modeling and documenting events in neurosurgery.

BACKGROUND: Intraoperative neurophysiological monitoring (IOM) plays a pivotal role in enhancing patient safety during neurosurgical procedures. This vital technique involves the continuous measurement of evoked potentials to provide early warnings and ensure the preservation of critical neural structures. One of the primary challenges has been the effective documentation of IOM events with semantically enriched characterizations. This study aimed to address this challenge by developing an ontology-based tool. METHODS: We structured the development of the IOM Documentation Ontology (IOMDO) and the associated tool into three distinct phases. The initial phase focused on the ontology's creation, drawing from the OBO (Open Biological and Biomedical Ontology) principles. The subsequent phase involved agile software development, a flexible approach to encapsulate the diverse requirements and swiftly produce a prototype. The last phase entailed practical evaluation within real-world documentation settings. This crucial stage enabled us to gather firsthand insights, assessing the tool's functionality and efficacy. The observations made during this phase formed the basis for essential adjustments to ensure the tool's productive utilization. RESULTS: The core entities of the ontology revolve around central aspects of IOM, including measurements characterized by timestamp, type, values, and location. Concepts and terms of several ontologies were integrated into IOMDO, e.g., the Foundation Model of Anatomy (FMA), the Human Phenotype Ontology (HPO) and the ontology for surgical process models (OntoSPM) related to general surgical terms. The software tool developed for extending the ontology and the associated knowledge base was built with JavaFX for the user-friendly frontend and Apache Jena for the robust backend. The tool's evaluation involved test users who unanimously found the interface accessible and usable, even for those without extensive technical expertise. CONCLUSIONS: Through the establishment of a structured and standardized framework for characterizing IOM events, our ontology-based tool holds the potential to enhance the quality of documentation, benefiting patient care by improving the foundation for informed decision-making. Furthermore, researchers can leverage the semantically enriched data to identify trends, patterns, and areas for surgical practice enhancement. To optimize documentation through ontology-based approaches, it's crucial to address potential modeling issues that are associated with the Ontology of Adverse Events.

Using the PharmCAT tool for Pharmacogenetic clinical decision support.

Here, we will provide our insights into the usage of PharmCAT as part of a pharmacogenetic clinical decision support pipeline, which addresses the challenges in mapping clinical dosing guidelines to variants to be extracted from genetic datasets. After a general outline of pharmacogenetics, we describe some features of PharmCAT and how we integrated it into a pharmacogenetic clinical decision support system within a clinical information system. We conclude with promising developments regarding future PharmCAT releases.

Opportunities and challenges of supervised machine learning for the classification of motor evoked potentials according to muscles.

BACKGROUND: Even for an experienced neurophysiologist, it is challenging to look at a single graph of an unlabeled motor evoked potential (MEP) and identify the corresponding muscle. We demonstrate that supervised machine learning (ML) can successfully perform this task. METHODS: Intraoperative MEP data from supratentorial surgery on 36 patients was included for the classification task with 4 muscles: Extensor digitorum (EXT), abductor pollicis brevis (APB), tibialis anterior (TA) and abductor hallucis (AH). Three different supervised ML classifiers (random forest (RF), k-nearest neighbors (kNN) and logistic regression (LogReg)) were trained and tested on either raw or compressed data. Patient data was classified considering either all 4 muscles simultaneously, 2 muscles within the same extremity (EXT versus APB), or 2 muscles from different extremities (EXT versus TA). RESULTS: In all cases, RF classifiers performed best and kNN second best. The highest performances were achieved on raw data (4 muscles 83%, EXT versus APB 89%, EXT versus TA 97% accuracy). CONCLUSIONS: Standard ML methods show surprisingly high performance on a classification task with intraoperative MEP signals. This study illustrates the power and challenges of standard ML algorithms when handling intraoperative signals and may lead to intraoperative safety improvements.

Nurse Scheduling with State-of-the Art Open-Source Tools.

Nurse scheduling is still an unsolved issue, as it is NP-hard and highly context-dependent. Despite this fact, the practice needs guidance on how to tackle this problem without using costly commercial tools. Concretely, we have the following use case: a Swiss hospital is planning a new station designed for nurse training. The capacity planning is finished, and the hospital wants to assess whether shift planning with known constraints leads to valid solutions. Here, a mathematical model is combined with a genetic algorithm. We trust the solution of the mathematical model more, but if it does not provide a valid solution, we try out an alternative. Our solutions indicate that actual capacity planning together with the hard constraints cannot lead to valid staff schedules. The central conclusion is that more degrees of freedom are necessary and that open-source tools OMPR and DEAP are valuable alternatives to commercial products such as Wrike or Shiftboard, in which the degree of freedom of customization is reduced in favor of easiness of use.

Is Medical Informatics a Scientific Discipline or Just Applied Computer Science?

The aim of this paper is to investigate whether and how medical informatics can claim to have a sound scientific basis. Why is such clarification fruitful? First, it provides a common ground for the core principles, theories and methods used to gain knowledge and to guide the practice. Without such a ground, medical informatics might be subsumed to medical engineering at one institution and to life sciences at another institution or might be just regarded as an application domain within computer science. We will provide a succinct outline of the philosophy of science, after which we provide an application of the related notions in order to decide the scientific status of medical informatics. We justify viewing medical informatics as an interdisciplinary field with a paradigm that can be formulated as "user-centered process-orientation in the healthcare setting". Even if MI is not merely applied computer science, it still remains uncertain whether it will attain the status of a mature science, especially without comprehensive theories.

Implementing Informative-Based Active Learning in Biomedical Record Linkage for the Splink Package in Python.

In biomedical record linkage, efficient determination of a threshold to decide at which level of similarity two records should be classified as belonging to the same patient is frequently still an open issue. Here, we describe how to implement an efficient active learning strategy that puts into practice a measure of usefulness of training sets for such a task. Our results show that active learning should always be considered when training data is to be produced via manual labeling. In addition to that, active learning gives a quick indication how complex a problem is by looking into the label frequencies: If the most difficult entities are always stemming from the same class, then the classifier will probably have less problems in distinguishing the classes. In big data applications, these two properties are essential, as the problems of under- and overfitting are exacerbated in such contexts.

What Kind of Ontologies Do We Need in the Biomedical Domain?

We tackle the question as to what sort of ontologies we primarily need in the biomedical domain. For this purpose, we will first provide a simple categorization of ontologies and describe an important use case related to modeling and documenting events. Then, the impact of using upper-level ontologies as a basis to address our use case will be shown in order to derive an answer to our research question. Although formal ontologies can serve as a starting point to understand conceptualization in a domain and facilitate interesting inferences, it is even more important to account for the dynamic and changing nature of knowledge. Being unconstrained by pre-defined categories and relationships can facilitate timely enrichment of a conceptual scheme and provide links and dependency structures in an informal manner. Semantic enrichment can be achieved by other mechanisms such as tagging or the creation of synsets as, for example, provided in WordNet.

Explaining Contextualized Word Embeddings in Biomedical Research - A Qualitative Investigation.

Contextualized word embeddings proved to be highly successful quantitative representations of words that allow to efficiently solve various tasks such as clinical entity normalization in unstructured texts. In this paper, we investigate how the Saussurean sign theory can be used as a qualitative explainable AI method for word embeddings. Our assumption is that the main goal of XAI is to produce confidence and/or trust, which can be gained through quantitative as well as quantitative approaches. One important result is related to the fact that the differential structure of language as explained by Saussure corresponds to the possibility of adding and subtracting word embeddings. On the other hand, these mathematical structures provide insights into the inner workings of natural language.

The Sortal Concept in the Context of Biomedical Record Linkage.

Biomedical Record Linkage is especially designed for linking data of patients in different data repositories. An important question in this context is whether singling-out is sufficient for identifying a patient, and if not, what is in general required for identification. To provide hints for an answer, we will extend previous works on the concept of identity and extend the sortal concept, stemming from analytical philosophy and upper-level ontologies. A sortal is a concept that is associated with an identity criterion. For example, the concept "set" has the identity criterion "having the same members". Based on a description of a record linkage setting, we operationalize the sortal concept by providing a distinction between the digital representation of a person (d-sortal) and the person in flesh (b-sortal).

Mapping SNOMED CT Codes to Semi-Structured Texts via an NLP Pipeline.

In the project presented here, we used NLP tools for annotating German medical trainings documents with SNOMED CT codes. Following research question was addressed: Is it possible to automate the annotation of training documents with an NLP pipeline especially designed for this task but requiring translation into English? The goal of our stakeholder, an institution responsible for the continuing education of physicians, was to facilitate the switch between different medical trainings programs by coding the same requirement with the same SNOMED CT code, even if the wording is different. We first describe how we chose the concrete NLP tools, after which the concrete steps for implementing our prototype are outlined: the NLP pipeline construction, the implementation, and the validation. We infer three important lessons from our results: (i) self-supervision is no free lunch and should be based on a sophisticated task, (ii) the translation via DeepL can be too context-dependent for a peculiar use case, and (iii) ontology extraction can increase efficiency as well as accuracy.

A Consent Tool for Secondary Use of Biomedical Data.

To pursue scientific goals with patient data usually requires informed consent from the data subjects. Such a consent constitutes a contract between the research institute and the patient. Several issues must be included in the consent to be valid, for example, how the data is processed and stored as well as specifics of the research questions for which the data is going to be used. Here, we describe the development and the implementation of a user-friendly IT solution that supports the process-oriented obtainment of consents. Current solutions often focus only on the benefits for the researcher. Our solution intends to add value to all participants and to reduce paperwork to a minimum. The consent Tool was evaluated by a usability test using the UEQ Method (User Experience Questionnaire) and received positive feedback - both efficiency and originality were rated above the average UEQ-Benchmark. Nevertheless, the lack of compatibility with the technical infrastructure of the hospital was a significant shortcoming. Hence, although there is a general interest in digitized solutions in the healthcare sector, there are still many hurdles to implement them and roll them out.

Medical Informatics in a Tension Between Black-Box AI and Trust.

For medical informaticians, it became more and more crucial to assess the benefits and disadvantages of AI-based solutions as promising alternatives for many traditional tools. Besides quantitative criteria such as accuracy and processing time, healthcare providers are often interested in qualitative explanations of the solutions. Explainable AI provides methods and tools, which are interpretable enough that it affords different stakeholders a qualitative understanding of its solutions. Its main purpose is to provide insights into the black-box mechanism of machine learning programs. Our goal here is to advance the problem of qualitatively assessing AI from the perspective of medical informaticians by providing insights into the central notions, namely: explainability, interpretability, understanding, trust, and confidence.

Pro-Active Detection of Potentially Wrong Diagnoses Due to Substantial Changes of Laboratory Measurements.

For guiding decisions on medical diagnoses and diagnoses, it is crucial to receive valid laboratory test results. However, such results can be implausible for the physician, even if the measurements are within the range of known reference values. There are technical sources of implausible results that are related to the laboratory environment, which are frequently not detected through usual measures for ensuring technical validity. Here, we describe the development of a quality assurance tool that tackles this problem and replaces the current manual statistical analyses at the Center for Laboratory Medicine in St Gallen (ZLM). Further analysis of the factors responsible for shifts in laboratory test results requires to collect and analyze data related to reagents as well as calibration or reference probes. Due to a lack of standard operating procedures in many laboratories with respect to these processes, this remains one of the big challenges.

Developing an Ontology for Documenting Adverse Events While Avoiding Pitfalls.

Ontologies promise more benefits than terminologies in terms of data annotation and computer-assisted reasoning, by defining a hierarchy of terms and their relations within a domain. Here, we present central insights related to the development of an ontology for documenting events during interoperative neuromonitoring (IOM), for which we used the Basic Formal Ontology (BFO) as an upper-level ontology. This work has the following two goals: to describe the development of the IOM ontology and to guide the practice with respect to documenting of biomedical events, as available ontologies pose difficulties on certain issues. We address the following issues: (i) differentiate between the sets documentation, identification, continuant and explanation, understanding, occurrent as we had problems in applying the available ontology of adverse events, (ii) covering diseases and injuries in a consistent way, and (iii) deciding on which level to define relations.

Embedding Risk-Based Anonymization into Data Access Control for Providing Individual-Level Health Data in a Secure Way.

Especially in biomedical research, individual-level data must be protected due to the sensitivity of the data that is associated with patients. The broad goal of scientific data re-use is to allow many researchers to derive new hypotheses and insights from the data while preserving privacy. Data usage control (DUC) as an attribute-based access mechanism promises to overcome the limitations of traditional access control models achieving that goal. Park and Sandhu provided the usage control (UCON) model as an instance of DUC, which defines policies that evaluate certain attributes. Here, we present an UCON-based architecture, which is augmented with risk-based anonymization as provided by the R package sdcMicro and an extensible Access Control Markup Language (XACML) environment with a core policy decision point as implemented by authzforce.

On the Concepts of Identity and Similarity in the Context of Biomedical Record Linkage.

Record linkage refers to a range of methods for merging and consolidating data in a manner such that duplicates are detected and false links are avoided. It is crucial for such a task to discern between similarity and identity of entities. This paper explores the implications of the ontological concepts of identity for record linkage (RL) on biomedical data sets. In order to draw substantial conclusions, we use the differentiation between numerical identity, qualitative identity and relational identity. We will discuss the problems of using similarity measures for record pairs and quality identity for ascertaining the real status of these pairs. We conclude that relational identity should be operationalized for RL.

Decentralized Digital Health Services Caught Between the Pressure for Innovation and the Burden of Regulations.

Multiple challenges await third-party digital health services when trying to enter the health market. Prominent examples of such services are clinical decision support systems provided as external software. Uncertainty about their challenges, technical as well as legal, pose serious hurdles for many innovations to be adopted early on. There are many options and trade-offs to provide digital healthcare solutions as a third-party service. This paper discusses them by referring to a pharmacogenetic decision support service. By providing best-practices, scenario descriptions and templates designed for third-party services with respect to legal and technical issues, obstacles and uncertainties can be reduced, which will have an impact on better diagnoses and treatments in the healthcare system.