Requirements analysis for an AI-based clinical decision support system for general practitioners: a user-centered design process.

BACKGROUND: As the first point of contact for patients with health issues, general practitioners (GPs) are frequently confronted with patients presenting with non-specific symptoms of unclear origin. This can result in delayed, prolonged or false diagnoses. To accelerate and improve the diagnosis of diseases, clinical decision support systems would appear to be an appropriate tool. The objective of the project 'Smart physician portal for patients with unclear disease' (SATURN) is to employ a user-centered design process based on the requirements analysis presented in this paper to develop an artificial Intelligence (AI)-based diagnosis support system that specifically addresses the needs of German GPs. METHODS: Requirements analysis for a GP-specific diagnosis support system was conducted in an iterative process with five GPs. First, interviews were conducted to analyze current workflows and the use of digital applications in cases of diagnostic uncertainty (as-is situation). Second, we focused on collecting and prioritizing tasks to be performed by an ideal smart physician portal (to-be situation) in a workshop. We then developed a task model with corresponding user requirements. RESULTS: Numerous GP-specific user requirements were identified concerning the tasks and subtasks: performing data entry (open system, enter patient data), reviewing results (receiving and evaluating results), discussing results (with patients and colleagues), scheduling further diagnostic procedures, referring to specialists (select, contact, make appointments), and case closure. Suggested features particularly concerned the process of screening and assessing results: e.g., the system should focus more on atypical patterns of common diseases than on rare diseases only, display probabilities of differential diagnoses, ensure sources and results are transparent, and mark diagnoses that have already been ruled out. Moreover, establishing a means of using the platform to communicate with colleagues and transferring patient data directly from electronic patient records to the system was strongly recommended. CONCLUSIONS: Essential user requirements to be considered in the development and design of a diagnosis system for primary care could be derived from the analysis. They form the basis for mockup-development and system engineering.

[The medical informatics initiative and rare diseases: next-generation routine data for diagnosis, therapy selection and research]. / Die Medizininformatik-Initiative und Seltene Erkrankungen: Routinedaten der nächsten Generation für Diagnose, Therapiewahl und Forschung.

People with rare diseases (RDs) have particular potential to benefit from digitisation in the healthcare system. The National Action Alliance for People with Rare Diseases (NAMSE) has campaigned for SE to be specifically taken into account in the digitisation of the healthcare system in Germany. The topic was addressed within the Medical Informatics Initiative (MII) of the Federal Ministry of Education and Research (BMBF). Here, starting with university hospitals, a digital infrastructure is currently being established for the data protection-compliant multiple use of standardised care and research data. Since 2020, part of the initiative has been the CORD-MI project (Collaboration on Rare Diseases) in which university hospitals and other partners throughout Germany have joined forces to improve patient care and research in the field of rare diseases.This article highlights how the MII takes into account the concerns of SE and what opportunities the "new routine data" obtained offer. A SE module was included in the "MII core data set" - an information model based on the data standard fast healthcare interoperability resources (FHIR). Data collected in the context of care and research routines can thus be exchanged between the participating institutions in the future and support, for example, diagnosis, therapy selection and research projects in the field of SE. The CORD-MI project has set itself the goal of obtaining insights into the care situation of people with SE with the help of exemplary questions and then drawing conclusions for further necessary steps in the area of digitalisation.

Evaluation of a clinical decision support system for rare diseases: a qualitative study.

BACKGROUND: Rare Diseases (RDs) are difficult to diagnose. Clinical Decision Support Systems (CDSS) could support the diagnosis for RDs. The Medical Informatics in Research and Medicine (MIRACUM) consortium developed a CDSS for RDs based on distributed clinical data from eight German university hospitals. To support the diagnosis for difficult patient cases, the CDSS uses data from the different hospitals to perform a patient similarity analysis to obtain an indication of a diagnosis. To optimize our CDSS, we conducted a qualitative study to investigate usability and functionality of our designed CDSS. METHODS: We performed a Thinking Aloud Test (TA-Test) with RDs experts working in Rare Diseases Centers (RDCs) at MIRACUM locations which are specialized in diagnosis and treatment of RDs. An instruction sheet with tasks was prepared that the participants should perform with the CDSS during the study. The TA-Test was recorded on audio and video, whereas the resulting transcripts were analysed with a qualitative content analysis, as a ruled-guided fixed procedure to analyse text-based data. Furthermore, a questionnaire was handed out at the end of the study including the System Usability Scale (SUS). RESULTS: A total of eight experts from eight MIRACUM locations with an established RDC were included in the study. Results indicate that more detailed information about patients, such as descriptive attributes or findings, can help the system perform better. The system was rated positively in terms of functionality, such as functions that enable the user to obtain an overview of similar patients or medical history of a patient. However, there is a lack of transparency in the results of the CDSS patient similarity analysis. The study participants often stated that the system should present the user with an overview of exact symptoms, diagnosis, and other characteristics that define two patients as similar. In the usability section, the CDSS received a score of 73.21 points, which is ranked as good usability. CONCLUSIONS: This qualitative study investigated the usability and functionality of a CDSS of RDs. Despite positive feedback about functionality of system, the CDSS still requires some revisions and improvement in transparency of the patient similarity analysis.

[se-atlas.de-Medical care atlas for people with rare diseases]. / se-atlas.de ­ Versorgungsatlas für Menschen mit Seltenen Erkrankungen.

In the European Union a disease is classified as rare if it affects no more than 5 out of 10,000 people. Currently, there are more than 6000 rare diseases, consisting of a large and heterogeneous number of different diseases that are complex in their symptomatology, multidimensional and therefore difficult to classify in everyday medical practice. This complicates the diagnosis and treatment as well as finding a suitable contact person, as there are only a few experts for each individual rare disease. The medical care atlas for rare diseases www.se-atlas.de enables the search for care facilities and patient organizations for specific rare diseases by disease name and presents the search results geographically. It also provides an overview of all German centers for rare diseases, which are a contact point for patients with an unclear diagnosis. The se-atlas serves as a compass through the heterogeneous amount of information on care facilities for rare diseases and provides low-threshold information for a broad user group, from affected persons to members of the medical care team.

Interviews with experts in rare diseases for the development of clinical decision support system software - a qualitative study.

BACKGROUND: Patients with rare diseases (RDs) are often diagnosed too late or not at all. Clinical decision support systems (CDSSs) could support the diagnosis in RDs. The MIRACUM (Medical Informatics in Research and Medicine) consortium, which is one of four funded consortia in the German Medical Informatics Initiative, will develop a CDSS for RDs based on distributed clinical data from ten university hospitals. This qualitative study aims to investigate (1) the relevant organizational conditions for the operation of a CDSS for RDs when diagnose patients (e.g. the diagnosis workflow), (2) which data is necessary for decision support, and (3) the appropriate user group for such a CDSS. METHODS: Interviews were carried out with RDs experts. Participants were recruited from staff physicians at the Rare Disease Centers (RDCs) at the MIRACUM locations, which offer diagnosis and treatment of RDs. An interview guide was developed with a category-guided deductive approach. The interviews were recorded on an audio device and then transcribed into written form. We continued data collection until all interviews were completed. Afterwards, data analysis was performed using Mayring's qualitative content analysis approach. RESULTS: A total of seven experts were included in the study. The results show that medical center guides and physicians from RDC B-centers (with a focus on different RDs) are involved in the diagnostic process. Furthermore, interdisciplinary case discussions between physicians are conducted. The experts explained that RDs exist which cannot be fully differentiated, but rather described only by their overall symptoms or findings: diagnosis is dependent on the disease or disease group. At the end of the diagnostic process, most centers prepare a summary of the patient case. Furthermore, the experts considered both physicians and experts from the B-centers to be potential users of a CDSS. The experts also have different experiences with CDSS for RDs. CONCLUSIONS: This qualitative study is a first step towards establishing the requirements for the development of a CDSS for RDs. Further research is necessary to create solutions by also including the experts on RDs.

[Registries for rare diseases : OSSE - An open-source framework for technical implementation]. / Register für seltene Erkrankungen : OSSE ­ ein Open-Source-Framework für die technische Umsetzung.

Meager amounts of data stored locally, a small number of experts, and a broad spectrum of technological solutions incompatible with each other characterize the landscape of registries for rare diseases in Germany. Hence, the free software Open Source Registry for Rare Diseases (OSSE) was created to unify and streamline the process of establishing specific rare disease patient registries. The data to be collected is specified based on metadata descriptions within the registry framework's so-called metadata repository (MDR), which was developed according to the ISO/IEC 11179 standard. The use of a central MDR allows for sharing the same data elements across any number of registries, thus providing a technical prerequisite for making data comparable and mergeable between registries and promoting interoperability.With OSSE, the foundation is laid to operate linked patient registries while respecting strong data protection regulations. Using the federated search feature, data for clinical studies can be identified across registries. Data integrity, however, remains intact since no actual data leaves the premises without the owner's consent. Additionally, registry solutions other than OSSE can participate via the OSSE bridgehead, which acts as a translator between OSSE registry networks and non-OSSE registries. The pseudonymization service Mainzelliste adds further data protection.Currently, more than 10 installations are under construction in clinical environments (including university hospitals in Frankfurt, Hamburg, Freiburg and Münster). The feedback given by the users will influence further development of OSSE. As an example, the installation process of the registry for undiagnosed patients at University Hospital Frankfurt is described in more detail.

Rare Diseases in Citizen Science - Preliminary Experiences in Developing a Personal Health App.

SelEe is a German citizen science project aiming to develop a smartphone app for a patient-managed record. The goal is to study rare diseases with the support of interested citizens and people affected by rare diseases. We established a core research team, including professional researchers (leading the project) and citizens. Citizens have the opportunity to discuss the progress, make suggestions regarding the app's design and data entry and contribute to the dissemination of the project. To gather feedback and experiences from the core research team, we performed an online questionnaire regarding the topics "influence and communication", "improvements and learning effect", and "satisfaction". Finally, 9 citizens of the core research team participated. The results show that the citizens are very satisfied with the design of the app, their participation opportunities and the communication in the project.

Editing Physicians' Responses Using GPT-4 for Academic Research.

The integration of Artificial Intelligence (AI) into digital healthcare, particularly in the anonymisation and processing of health information, holds considerable potential. OBJECTIVES: To develop a methodology using Generative Pre-trained Transformer (GPT) models to preserve the essence of medical advice in doctors' responses, while editing them for use in scientific studies. METHODS: German and English responses from EXABO, a rare respiratory disease platform, were processed using iterative refinement and other prompt engineering techniques, with a focus on removing identifiable and irrelevant content. RESULTS: Of 40 responses tested, 31 were accurately modified according to the developed guidelines. Challenges included misclassification and incomplete removal, with incremental prompting proving more accurate than combined prompting. CONCLUSION: GPT-4 models show promise in medical response editing, but face challenges in accuracy and consistency. Precision in prompt engineering is essential in medical contexts to minimise bias and retain relevant information.

Translation of Ontological Concepts from English into German Using Commercial Translation Software and Expert Evaluation.

Medical ontologies are mostly available in English. This presents a language barrier that is a limitation in research and automated processing of patient data. The manual translation of ontologies is complex and time-consuming. However, there are commercial translation tools that have shown promising results in the field of medical terminology translation. The aim of this study is to translate selected terms of the Human Phenotype Ontology (HPO) from English into German using commercial translators. Six medical experts evaluated the translation candidates in an iterative process. The results show commercial translators, with DeepL in the lead, provide translations that are positively evaluated by experts. With a broader study scope and additional optimization techniques, commercial translators could support and facilitate the process of translating medical ontologies.

Initial User-Centred Design of an AI-Based Clinical Decision Support System for Primary Care.

A clinical decision support system based on different methods of artificial intelligence (AI) can support the diagnosis of patients with unclear diseases by providing tentative diagnoses as well as proposals for further steps. In a user-centred-design process, we aim to find out how general practitioners envision the user interface of an AI-based clinical decision support system for primary care. A first user-interface prototype was developed using the task model based on user requirements from preliminary work. Five general practitioners evaluated the prototype in two workshops. The discussion of the prototype resulted in categorized suggestions with key messages for further development of the AI-based clinical decision support system, such as the integration of intelligent parameter requests. The early inclusion of different user feedback facilitated the implementation of a user interface for a user-friendly decision support system.

Exploring patient-centered design solutions of a telehealth app for HIV - A qualitative study.

BACKGROUND: The Communication and Tracing App HIV (COMTRAC-HIV) project is developing a mobile health (mHealth) app for integrated care of HIV patients in Germany. The complexity of HIV treatment and continuous care necessitates the need for tailored mHealth solutions. This qualitative study explores design solutions and a prototype to enhance the app's functionality and effectiveness. METHODS: A total of eight HIV patients and pre-exposure prophylaxis (PrEP) users, recruited at the HIV Center Frankfurt, participated in focus groups and thinking-aloud tests (TA test). In the focus groups, design solutions were discussed for user-interface clarity, leading to the development of an interactive prototype, the usability of which was evaluated with a TA test. Data collection involved video/audio recordings. Qualitative analysis was conducted using a deductive category system, and focused on app design and usage in focus groups, and layout, navigation, interaction, terminology, comprehension, feedback, and level of satisfaction in TA tests. RESULTS: The app was commended for its simple, clear design, especially its medication reminders and health tracking features. Opinions on the symptom diary varied however, respondents noting it more suitable for HIV users than PrEP users. Privacy concerns suggest avoiding display of HIV-specific information. Suggested improvements include e.g. image uploads, drug interaction checks and prescription tracking. A total of 25 usability issues were identified in the TA test, with most found in the layout (n = 6), navigation (n = 5), interaction (n = 5), and terminology (n = 5) categories. Two examples are non-intuitive controls and illogical button placement. Despite these disadvantages, participants noted positive impressions (n = 5) in the satisfaction category. CONCLUSION: The study emphasizes the need for patient-centered design in mobile HIV care solutions, highlighting to the app's user-friendliness and potential to enhance care. Further research is necessary to refine the app's functionality and to align it with clinical and patients' privacy needs.

Diagnosis Support for Rare Diseases Using Phenotypic Profiles.

The common occurrence of characteristic symptoms can be used to infer diagnoses. The aim of this study is to show how syndrome similarity analysis using given phenotypic profiles can help in the diagnosis of rare diseases. HPO was used to map syndromes and phenotypic profiles. The system architecture described is planned to be implemented in a clinical decision support system for unclear diseases.

Machine translation of standardised medical terminology using natural language processing: A scoping review.

Standardised medical terminologies are used to ensure accurate and consistent communication of information and to facilitate data exchange. Currently, many terminologies are only available in English, which hinders international research and automated processing of medical data. Natural language processing (NLP) and Machine Translation (MT) methods can be used to automatically translate these terms. This scoping review examines the research on automated translation of standardised medical terminology. A search was performed in PubMed and Web of Science and results were screened for eligibility by title and abstract as well as full text screening. In addition to bibliographic data, the following data items were considered: 'terminology considered', 'terms considered', 'source language', 'target language', 'translation type', 'NLP technique', 'NLP system', 'machine translation system', 'data source' and 'translation quality'. The results showed that the most frequently translated terminology is SNOMED CT (39.1%), followed by MeSH (13%), ICD (13%) and UMLS (8.7%). The most common source language is English (55.9%), and the most common target language is German (41.2%). Translation methods are often based on Statistical Machine Translation (SMT) (41.7%) and, more recently, Neural Machine Translation (NMT) (30.6%), but can also be combined with various MT methods. Commercial translators such as Google Translate (36.4%) and automatic validation methods such as BLEU (22.2%) are frequently used tools for translation and subsequent validation.

Methods Used in the Development of Common Data Models for Health Data: Scoping Review.

BACKGROUND: Common data models (CDMs) are essential tools for data harmonization, which can lead to significant improvements in the health domain. CDMs unite data from disparate sources and ease collaborations across institutions, resulting in the generation of large standardized data repositories across different entities. An overview of existing CDMs and methods used to develop these data sets may assist in the development process of future models for the health domain, such as for decision support systems. OBJECTIVE: This scoping review investigates methods used in the development of CDMs for health data. We aim to provide a broad overview of approaches and guidelines that are used in the development of CDMs (ie, common data elements or common data sets) for different health domains on an international level. METHODS: This scoping review followed the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) checklist. We conducted the literature search in prominent databases, namely, PubMed, Web of Science, Science Direct, and Scopus, starting from January 2000 until March 2022. We identified and screened 1309 articles. The included articles were evaluated based on the type of adopted method, which was used in the conception, users' needs collection, implementation, and evaluation phases of CDMs, and whether stakeholders (such as medical experts, patients' representatives, and IT staff) were involved during the process. Moreover, the models were grouped into iterative or linear types based on the imperativeness of the stages during development. RESULTS: We finally identified 59 articles that fit our eligibility criteria. Of these articles, 45 specifically focused on common medical conditions, 10 focused on rare medical conditions, and the remaining 4 focused on both conditions. The development process usually involved stakeholders but in different ways (eg, working group meetings, Delphi approaches, interviews, and questionnaires). Twenty-two models followed an iterative process. CONCLUSIONS: The included articles showed the diversity of methods used to develop a CDM in different domains of health. We highlight the need for more specialized CDM development methods in the health domain and propose a suggestive development process that might ease the development of CDMs in the health domain in the future.

Interviews with HIV Experts for Development of a Mobile Health Application in HIV Care-A Qualitative Study.

The Communication and Tracing App HIV (COMTRAC-HIV) project aims to develop a mobile health application for integrated care of HIV patients due to the low availability of those apps in Germany. This study addressed organizational conditions and necessary app functionalities, especially for the care of late diagnosed individuals (late presenters) and those using pre-exposure prophylaxis. We followed a human-centered design approach and interviewed HIV experts in Germany to describe the context of use of the app. The interviews were paraphrased and analyzed with a qualitative content analysis. To define the context of use, user group profiles were defined and tasks derived, which will represent the functionalities of the app. A total of eight experts were included in the study. The results show that the app should include a symptom diary for entering symptoms, side effects, and their intensity. It offers chat/video call functionality for communication with an HIV expert, appointment organization, and sharing findings. The app should also provide medication overview and reminders for medications and appointments. This qualitative study is a first step towards the development of an app for HIV individuals in Germany. Further research includes involving patients in the initial app design and test design usability.

Enhancing HIV Patient Support Through Telehealth: Exploring Design Solutions.

In recent years, telemedicine has advanced significantly, offering new possibilities for improving healthcare and patient outcomes. This paper presents a telemedicine app for HIV patients, developed using a human-centered design approach. Designed to meet the diverse and specific needs of Pre-Exposure Prophylaxis (PrEP) users and Late Presenters (LP), the app is part of the COMTRAC-HIV Project at the University Hospital Frankfurt. Through interviews with HIV experts and healthcare professionals, initial design solutions were derived. The paper explores the app's design process, core functionalities, and future directions, aiming to provide comprehensive support for individuals living with HIV.

User-Centred Development of a Diagnosis Support System for Rare Diseases.

The diagnosis of rare diseases is often challenging for physicians, but can be supported by Clinical Decision Support Systems. The MIRACUM consortia, which includes ten university hospitals in Germany, develops a Clinical Decision Support System to support the diagnosis of patients with rare diseases. The users are involved in different phases using a user-centred design process. This publication has the objective to summarize the results of all studies performed in context of the requirements elicitation and to derive concrete requirements for the development of the system. Several studies were performed for requirements elicitation: a cross-sectional survey, expert interviews and a focus group. Participants were experts in rare diseases of the MIRACUM locations. 32 requirements were derived and implemented in a prototype. The prototype allows similarity analyses as a decision support functionality by comparing patients without a diagnosis to patients with a rare disease. In the final evaluation, the prototype was rated with a good usability. Since the system is limited in its functionality, further work and improvements are necessary to make it ready for clinical usage.

The ERN-LUNG Population Registry: Aims, Software-Implementation and First Results.

The ERN-LUNG Population Registry is a new European-wide collection of patients with rare lung diseases, allowing patients to register online in the registry. Medical experts can recruit patients in the registry for disease-specific registries and care options. The Population Registry was implemented on the basis of the open source software OSSE and extended by functions for the self-registration of patients. Patients were invited through patient organizations between May and November 2022. 115 patients registered online in the registry, whereas 60 of them provided full data in the registry form. After first months of usage, further dissemination of the registry is necessary to reach more patients, e.g. by recruiting them via medical centres directly. Improvements of the registry should be conducted to achieve a higher number of fully completed forms.

Conception, Development and Validation of Classification Methods for Coding Support of Rare Diseases Using Artificial Intelligence.

Automated coding of diseases can support hospitals in the billing of inpatient cases with the health insurance funds. This paper describes the implementation and evaluation of classification methods for two selected Rare Diseases. Different classifiers of an off-the-shelf system and an own application are applied in a supervised learning process and comparatively examined for their suitability and reliability. Using Natural Language Processing and Machine Learning, disease entities are recognized from unstructured historical patient records and new billing cases are coded automatically. The results of the performed classifications show that even with small datasets (≤ 200), high correctness (F1 score ∼0.8) can be achieved in predicting new cases.

Identifying project topics and requirements in a citizen science project in rare diseases: a participative study.

BACKGROUND: Due to their low prevalence (< 5 in 10,000), rare diseases are an important area of research, with the active participation of those affected being a key factor. In the Citizen Science project "SelEe" (Researching rare diseases in a citizen science approach), citizens collaborate with researchers using a digital application, developed as part of the project together with those affected, to answer research questions on rare diseases. The aim of this study was to define the rare diseases to be considered, the project topics and the initial requirements for the implementation in a digital application. METHODS: To address our research questions, we took several steps to engage citizens, especially those affected by rare diseases. This approach included the following methods: pre- and post-survey (questionnaire), two workshops with focus group discussion and a requirements analysis workshop (with user stories). RESULTS: In the pre-survey, citizens suggested 45 different rare diseases and many different disease groups to be considered in the project. Two main project topics (A) "Patient-guided documentation and data collection" (20 votes) and (B) "Exchange of experience and networking" (13 votes) were identified as priorities in the workshops and ranked in the post-survey. The requirements workshop resulted in ten user stories and six initial requirements to be implemented in the digital application. CONCLUSION: Qualitative, citizen science research can be used to collectively identify stakeholder needs, project topics and requirements for a digital application in specific areas, such as rare diseases.