Development of the CDISC Pediatrics User Guide: a CDISC and conect4children collaboration.

Introduction: The conect4children (c4c) project aims to facilitate efficient planning and delivery of paediatric clinical trials. One objective of c4c is data standardization and reuse. Interoperability and reusability of paediatric clinical trial data is challenging due to a lack of standardization. The Clinical Data Interchange Standards Consortium (CDISC) standards that are required or recommended for regulatory submissions in several countries lack paediatric specificity with limited awareness within academic institutions. To address this, c4c and CDISC collaborated to develop the Pediatrics User Guide (PUG) consisting of cross-cutting data items that are routinely collected in paediatric clinical trials, factoring in all paediatric age ranges. Methods and Results: The development of the PUG consisted of six stages. During the scoping phase, subtopics (each containing several clinically relevant concepts) were suggested and debated for inclusion in the PUG. Ninety concepts were selected for the modelling phase. Concept maps describing the Research Topic and representation procedure were developed for the 19 concepts that had no (or partial) previous modelling in CDISC. Next, metadata and implementation examples were developed for concepts. This was followed by a CDISC internal review and a public review. For both these review stages, the feedback comments were either implemented or rejected based on budget, timelines, expert review, and scope. The PUG was published on the CDISC website on February 23, 2023. Discussion: The PUG is a first step in bridging the lack of child specific CDISC standards, particularly within academia. Several academic and industrial partners were involved in the development of the PUG, and c4c has undertaken multiple steps to publicize the PUG within its academic partner organizations - in particular, the European Reference Networks (ERNs) that are developing registries and dictionaries in 24 disease areas. In the long term, continued use of the PUG in paediatric clinical trials will enable the pooling of data from multiple trials, which is particularly important for medical domains with small populations.

Assessing the use of HL7 FHIR for implementing the FAIR guiding principles: a case study of the MIMIC-IV Emergency Department module.

Objectives: To provide a real-world example on how and to what extent Health Level Seven Fast Healthcare Interoperability Resources (FHIR) implements the Findable, Accessible, Interoperable, and Reusable (FAIR) guiding principles for scientific data. Additionally, presents a list of FAIR implementation choices for supporting future FAIR implementations that use FHIR. Materials and methods: A case study was conducted on the Medical Information Mart for Intensive Care-IV Emergency Department (MIMIC-ED) dataset, a deidentified clinical dataset converted into FHIR. The FAIRness of this dataset was assessed using a set of common FAIR assessment indicators. Results: The FHIR distribution of MIMIC-ED, comprising an implementation guide and demo data, was more FAIR compared to the non-FHIR distribution. The FAIRness score increased from 60 to 82 out of 95 points, a relative improvement of 37%. The most notable improvements were observed in interoperability, with a score increase from 5 to 19 out of 19 points, and reusability, with a score increase from 8 to 14 out of 24 points. A total of 14 FAIR implementation choices were identified. Discussion: Our work examined how and to what extent the FHIR standard contributes to FAIR data. Challenges arose from interpreting the FAIR assessment indicators. This study stands out for providing a real-world example of a dataset that was made more FAIR using FHIR. Conclusion: To the best of our knowledge, this is the first study that formally assessed the conformance of a FHIR dataset to the FAIR principles. FHIR improved the accessibility, interoperability, and reusability of MIMIC-ED. Future research should focus on implementing FHIR in research data infrastructures.

Assessing resolvability, parsability, and consistency of RDF resources: a use case in rare diseases.

INTRODUCTION: Healthcare data and the knowledge gleaned from it play a key role in improving the health of current and future patients. These knowledge sources are regularly represented as 'linked' resources based on the Resource Description Framework (RDF). Making resources 'linkable' to facilitate their interoperability is especially important in the rare-disease domain, where health resources are scattered and scarce. However, to benefit from using RDF, resources need to be of good quality. Based on existing metrics, we aim to assess the quality of RDF resources related to rare diseases and provide recommendations for their improvement. METHODS: Sixteen resources of relevance for the rare-disease domain were selected: two schemas, three metadatasets, and eleven ontologies. These resources were tested on six objective metrics regarding resolvability, parsability, and consistency. Any URI that failed the test based on any of the six metrics was recorded as an error. The error count and percentage of each tested resource were recorded. The assessment results were represented in RDF, using the Data Quality Vocabulary schema. RESULTS: For three out of the six metrics, the assessment revealed quality issues. Eleven resources have non-resolvable URIs with proportion to all URIs ranging from 0.1% (6/6,712) in the Anatomical Therapeutic Chemical Classification to 13.7% (17/124) in the WikiPathways Ontology; seven resources have undefined URIs; and two resources have incorrectly used properties of the 'owl:ObjectProperty' type. Individual errors were examined to generate suggestions for the development of high-quality RDF resources, including the tested resources. CONCLUSION: We assessed the resolvability, parsability, and consistency of RDF resources in the rare-disease domain, and determined the extent of these types of errors that potentially affect interoperability. The qualitative investigation on these errors reveals how they can be avoided. All findings serve as valuable input for the development of a guideline for creating high-quality RDF resources, thereby enhancing the interoperability of biomedical resources.

Correctly structured problem lists lead to better and faster clinical decision-making in electronic health records compared to non-curated problem lists: A single-blinded crossover randomized controlled trial.

BACKGROUND: Correctly structured problem lists in electronic health records (EHRs) offer major benefits to patient care. Without structured lists, diagnosis information is often scatteredly documented in free text, which may contribute to errors and inefficient information retrieval. This study aims to assess whether EHRs with correctly structured problem lists result in better and faster clinical decision-making compared to non-curated problem lists. METHODS: Two versions of two patient records (A and B) were created in an EHR training environment: one version included diagnosis information structured and coded on the problem list ("correctly structured problem list"), the other version had missing problem list diagnoses and diagnosis information partly documented in free text ("non-curated problem list"). In this single-blinded crossover randomized controlled trial, healthcare providers, who can prescribe medications, from two Dutch university medical center locations first evaluated a randomized version of patient A, then B. Participants were asked to motivate their answer to two medication prescription questions. One (test) question required information similarly presented in both record versions. The second (comparison) question required information documented on problem lists and/or in notes. The primary outcome measure was the correctness of the motivated answer to the comparison question. Secondary outcome measure was the time to answer and motivate both questions correctly. RESULTS: As planned, 160 participants enrolled. Two were excluded for not meeting inclusion criteria. Correctly structured problem lists increased providers' ability to answer the comparison question correctly (56.3 % versus 33.5 %, McNemar odds ratio 2.80 (1.65-4.93) 95 %-CI). Median time to answer both questions correctly was significantly lower for EHRs with correctly structured problem lists (Wilcoxon-signed-rank test p = 0.00002, with incorrect answers coded equally at slowest time). CONCLUSIONS: Correctly structured problem lists lead to better and faster clinical decision-making. Increased structured problem lists usage may be warranted for which implementation policies should be developed.

An eHealth App (CAPABLE) Providing Symptom Monitoring, Well-Being Interventions, and Educational Material for Patients With Melanoma Treated With Immune Checkpoint Inhibitors: Protocol for an Exploratory Intervention Trial.

BACKGROUND: Since treatment with immune checkpoint inhibitors (ICIs) is becoming standard therapy for patients with high-risk and advanced melanoma, an increasing number of patients experience treatment-related adverse events such as fatigue. Until now, studies have demonstrated the benefits of using eHealth tools to provide either symptom monitoring or interventions to reduce treatment-related symptoms such as fatigue. However, an eHealth tool that facilitates the combination of both symptom monitoring and symptom management in patients with melanoma treated with ICIs is still needed. OBJECTIVE: In this pilot study, we will explore the use of the CAPABLE (Cancer Patients Better Life Experience) app in providing symptom monitoring, education, and well-being interventions on health-related quality of life (HRQoL) outcomes such as fatigue and physical functioning, as well as patients' acceptance and usability of using CAPABLE. METHODS: This prospective, exploratory pilot study will examine changes in fatigue over time in 36 patients with stage III or IV melanoma during treatment with ICI using CAPABLE (a smartphone app and multisensory smartwatch). This cohort will be compared to a prospectively collected cohort of patients with melanoma treated with standard ICI therapy. CAPABLE will be used for a minimum of 3 and a maximum of 6 months. The primary endpoint in this study is the change in fatigue between baseline and 3 and 6 months after the start of treatment. Secondary end points include HRQoL outcomes, usability, and feasibility parameters. RESULTS: Study inclusion started in April 2023 and is currently ongoing. CONCLUSIONS: This pilot study will explore the effect, usability, and feasibility of CAPABLE in patients with melanoma during treatment with ICI. Adding the CAPABLE system to active treatment is hypothesized to decrease fatigue in patients with high-risk and advanced melanoma during treatment with ICIs compared to a control group receiving standard care. The Medical Ethics Committee NedMec (Amsterdam, The Netherlands) granted ethical approval for this study (reference number 22-981/NL81970.000.22). TRIAL REGISTRATION: ClinicalTrials.gov NCT05827289; https://clinicaltrials.gov/study/NCT05827289. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/49252.

The effects of COVID-19 on child mental health: Biannual assessments up to April 2022 in a clinical and two general population samples.

Background: The COVID-19 pandemic has had an acute impact on child mental and social health, but long-term effects are still unclear. We examined how child mental health has developed since the start of the COVID-19 pandemic up to 2 years into the pandemic (April 2022). Methods: We included children (age 8-18) from two general population samples (N = 222-1333 per measurement and N = 2401-13,362 for pre-covid data) and one clinical sample receiving psychiatric care (N = 334-748). Behavioral questionnaire data were assessed five times from April 2020 till April 2022 and pre-pandemic data were available for both general population samples. We collected parent-reported data on internalizing and externalizing problems with the Brief Problem Monitor and self-reported data on Anxiety, Depressive symptoms, Sleep-related impairments, Anger, Global health, and Peer relations with the Patient-Reported Outcomes Measurement Information System (PROMIS®). Results: In all samples, parents reported overall increased internalizing problems, but no increases in externalizing problems, in their children. Children from the general population self-reported increased mental health problems from before to during the pandemic on all six PROMIS domains, with generally worst scores in April 2021, and scores improving toward April 2022 but not to pre-pandemic norms. Children from the clinical sample reported increased mental health problems throughout the pandemic, with generally worst scores in April 2021 or April 2022 and no improvement. We found evidence of minor age effects and no sex effects. Conclusions: Child mental health in the general population has deteriorated during the first phase of the COVID-19 pandemic, has improved since April 2021, but has not yet returned to pre-pandemic levels. Children in psychiatric care show worsening of mental health problems during the pandemic, which has not improved since. Changes in child mental health should be monitored comprehensively to inform health care and policy.

Determining and assessing characteristics of data element names impacting the performance of annotation using Usagi.

INTRODUCTION: Hospitals generate large amounts of data and this data is generally modeled and labeled in a proprietary way, hampering its exchange and integration. Manually annotating data element names to internationally standardized data element identifiers is a time-consuming effort. Tools can support performing this task automatically. This study aimed to determine what factors influence the quality of automatic annotations. METHODS: Data element names were used from the Dutch COVID-19 ICU Data Warehouse containing data on intensive care patients with COVID-19 from 25 hospitals in the Netherlands. In this data warehouse, the data had been merged using a proprietary terminology system while also storing the original hospital labels (synonymous names). Usagi, an OHDSI annotation tool, was used to perform the annotation for the data. A gold standard was used to determine if Usagi made correct annotations. Logistic regression was used to determine if the number of characters, number of words, match score (Usagi's certainty) and hospital label origin influenced Usagi's performance to annotate correctly. RESULTS: Usagi automatically annotated 30.5% of the data element names correctly and 5.5% of the synonymous names. The match score is the best predictor for Usagi finding the correct annotation. It was determined that the AUC of data element names was 0.651 and 0.752 for the synonymous names respectively. The AUC for the individual hospital label origins varied between 0.460 to 0.905. DISCUSSION: The results show that Usagi performed better to annotate the data element names than the synonymous names. The hospital origin in the synonymous names dataset was associated with the amount of correctly annotated concepts. Hospitals that performed better had shorter synonymous names and fewer words. Using shorter data element names or synonymous names should be considered to optimize the automatic annotating process. Overall, the performance of Usagi is too poor to completely rely on for automatic annotation.

The Case Manager: An Agent Controlling the Activation of Knowledge Sources in a FHIR-Based Distributed Reasoning Environment.

BACKGROUND: Within the CAPABLE project the authors developed a multi-agent system that relies on a distributed architecture. The system provides cancer patients with coaching advice and supports their clinicians with suitable decisions based on clinical guidelines. OBJECTIVES: As in many multi-agent systems we needed to coordinate the activities of all agents involved. Moreover, since the agents share a common blackboard where all patients' data are stored, we also needed to implement a mechanism for the prompt notification of each agent upon addition of new information potentially triggering its activation. METHODS: The communication needs have been investigated and modeled using the HL7-FHIR (Health Level 7-Fast Healthcare Interoperability Resources) standard to ensure proper semantic interoperability among agents. Then a syntax rooted in the FHIR search framework has been defined for representing the conditions to be monitored on the system blackboard for activating each agent. RESULTS: The Case Manager (CM) has been implemented as a dedicated component playing the role of an orchestrator directing the behavior of all agents involved. Agents dynamically inform the CM about the conditions to be monitored on the blackboard, using the syntax we developed. The CM then notifies each agent whenever any condition of interest occurs. The functionalities of the CM and other actors have been validated using simulated scenarios mimicking the ones that will be faced during pilot studies and in production. CONCLUSION: The CM proved to be a key facilitator for properly achieving the required behavior of our multi-agent system. The proposed architecture may also be leveraged in many clinical contexts for integrating separate legacy services, turning them into a consistent telemedicine framework and enabling application reusability.

Assessing the FAIRness of databases on the EHDEN portal: A case study on two Dutch ICU databases.

OBJECTIVE: To address the growing need for effective data reuse in health research, healthcare institutions need to make their data Findable, Accessible, Interoperable, and Reusable (FAIR). A prevailing method to model databases for interoperability is the Observational Medical Outcomes Partnership (OMOP) Common Data Model (CDM), developed by the Observational Health Data Sciences and Informatics (OHDSI) initiative. A European repository for OMOP CDM-converted databases called the "European Health Data & Evidence Network (EHDEN) portal" was developed, aiming to make these databases Findable and Accessible. This paper aims to assess the FAIRness of databases on the EHDEN portal. MATERIALS AND METHODS: Two researchers involved in the OMOP CDM conversion of separate Dutch Intensive Care Unit (ICU) research databases each manually assessed their own database using seventeen metrics. These were defined by the FAIRsFAIR project as a list of minimum requirements for a database to be FAIR. Each metric is given a score from zero to four based on how well the database adheres to the metric. The maximum score for each metric varies from one to four based on the importance of the metric. RESULTS: Fourteen out of the seventeen metrics were unanimously rated: seven were rated the highest score, one was rated half of the highest score, and five were rated the lowest score. The remaining three metrics were assessed differently for the two use cases. The total scores achieved were 15.5 and 12 out of a maximum of 25. CONCLUSION: The main omissions in supporting FAIRness were the lack of globally unique identifiers such as Uniform Resource Identifiers (URIs) in the OMOP CDM and the lack of metadata standardization and linkage in the EHDEN portal. By implementing these in future updates, the EHDEN portal can be more FAIR.

Automated approach for quality assessment of RDF resources.

INTRODUCTION: The Semantic Web community provides a common Resource Description Framework (RDF) that allows representation of resources such that they can be linked. To maximize the potential of linked data - machine-actionable interlinked resources on the Web - a certain level of quality of RDF resources should be established, particularly in the biomedical domain in which concepts are complex and high-quality biomedical ontologies are in high demand. However, it is unclear which quality metrics for RDF resources exist that can be automated, which is required given the multitude of RDF resources. Therefore, we aim to determine these metrics and demonstrate an automated approach to assess such metrics of RDF resources. METHODS: An initial set of metrics are identified through literature, standards, and existing tooling. Of these, metrics are selected that fulfil these criteria: (1) objective; (2) automatable; and (3) foundational. Selected metrics are represented in RDF and semantically aligned to existing standards. These metrics are then implemented in an open-source tool. To demonstrate the tool, eight commonly used RDF resources were assessed, including data models in the healthcare domain (HL7 RIM, HL7 FHIR, CDISC CDASH), ontologies (DCT, SIO, FOAF, ORDO), and a metadata profile (GRDDL). RESULTS: Six objective metrics are identified in 3 categories: Resolvability (1), Parsability (1), and Consistency (4), and represented in RDF. The tool demonstrates that these metrics can be automated, and application in the healthcare domain shows non-resolvable URIs (ranging from 0.3% to 97%) among all eight resources and undefined URIs in HL7 RIM, and FHIR. In the tested resources no errors were found for parsability and the other three consistency metrics for correct usage of classes and properties. CONCLUSION: We extracted six objective and automatable metrics from literature, as the foundational quality requirements of RDF resources to maximize the potential of linked data. Automated tooling to assess resources has shown to be effective to identify quality issues that must be avoided. This approach can be expanded to incorporate more automatable metrics so as to reflect additional quality dimensions with the assessment tool implementing more metrics.

Evaluation of Patient-Friendly Diagnosis Clarifications in a Hospital Patient Portal.

BACKGROUND: Medical data can be difficult to comprehend for patients, but only a limited number of patient-friendly terms and definitions are available to clarify medical concepts. Therefore, we developed an algorithm that generalizes diagnoses to more general concepts that do have patient-friendly terms and definitions in SNOMED CT. We implemented the generalizations, and diagnosis clarifications with synonyms and definitions that were already available, in the problem list of a hospital patient portal. OBJECTIVE: We aimed to assess the extent to which the clarifications cover the diagnoses in the problem list, the extent to which clarifications are used and appreciated by patient portal users, and to explore differences in viewing problems and clarifications between subgroups of users and diagnoses. METHODS: We measured the coverage of diagnoses by clarifications, usage of the problem list and the clarifications, and user, patient and diagnosis characteristics with aggregated, routinely available electronic health record and log file data. Additionally, patient portal users provided quantitative and qualitative feedback about the clarification quality. RESULTS: Of all patient portal users who viewed diagnoses on their problem list (n = 2,660), 89% had one or more diagnoses with clarifications. In addition, 55% of patient portal users viewed the clarifications. Users who rated the clarifications (n = 108) considered the clarifications to be of good quality on average, with a median rating per patient of 6 (interquartile range: 4-7; from 1 very bad to 7 very good). Users commented that they found clarifications to be clear and recognized the clarifications from their own experience, but sometimes also found the clarifications incomplete or disagreed with the diagnosis itself. CONCLUSION: This study shows that the clarifications are used and appreciated by patient portal users. Further research and development will be dedicated to the maintenance and further quality improvement of the clarifications.

Internalizing problems before and during the COVID-19 pandemic in independent samples of Dutch children and adolescents with and without pre-existing mental health problems.

The aim of the study was to assess internalizing problems before and during the pandemic with data from Dutch consortium Child and adolescent mental health and wellbeing in times of the COVID-19 pandemic, consisting of two Dutch general population samples (GS) and two clinical samples (CS) referred to youth/psychiatric care. Measures of internalizing problems were obtained from ongoing data collections pre-pandemic (NGS = 35,357; NCS = 4487) and twice during the pandemic, in Apr-May 2020 (NGS = 3938; clinical: NCS = 1008) and in Nov-Dec 2020 (NGS = 1489; NCS = 1536), in children and adolescents (8-18 years) with parent (Brief Problem Monitor) and/or child reports (Patient-Reported Outcomes Measurement Information System®). Results show that, in the general population, internalizing problems were higher during the first peak of the pandemic compared to pre-pandemic based on both child and parent reports. Yet, over the course of the pandemic, on both child and parent reports, similar or lower levels of internalizing problems were observed. Children in the clinical population reported more internalizing symptoms over the course of the pandemic while parents did not report differences in internalizing symptoms from pre-pandemic to the first peak of the pandemic nor over the course of the pandemic. Overall, the findings indicate that children and adolescents of both the general and clinical population were affected negatively by the pandemic in terms of their internalizing problems. Attention is therefore warranted to investigate long-term effects and to monitor if internalizing problems return to pre-pandemic levels or if they remain elevated post-pandemic.

Use of an open-source electronic health record to establish a "virtual hospital": A tale of two curricula.

BACKGROUND: The electronic health record (EHR) is central to medical informatics. Its use is also recognized as an important skill for future clinicians. Typically, medical students' first exposure to an EHR is when they start their clinical internships, and medical informatics students may or may not get experience with an EHR before graduation. We describe the process of implementing an open-source EHR in two curricula: Medicine and Medical informatics. For medical students, the primary goals were to allow students to practice analyzing information from the EHR, creating therapeutic plans, and communicating with their colleagues via the EHR before they start their first clinical rotations. For medical informatics students, the primary goal was to give students hands-on experience with creating decision support in an EHR. APPROACH: We used the OpenMRS electronic health record with a custom decision support module based on Arden Syntax. Medical students needed a secure, stable environment to practice medical reasoning. Medical informatics students needed a more isolated system to experiment with the EHR's internal configuration. Both student groups needed synthetic patient cases that were realistic, but in different aspects. For medical students, it is essential that these cases are clinically consistent, and events unfold in a logical order. By contrast, synthetic data for medical informatics students should mimic the data quality problems found in real patient data. OUTCOMES: Medical informatics students show more mature reasoning about data quality issues and workflow integration than prior to using the EHR. Comments on both course evaluations have been positive, including comments on how working with a real-world EHR provides a realistic experience. CONCLUSION: The open-source EHR OpenMRS has proven to be a valuable addition to both the medicine and medical informatics curriculum. Both sets of students experience use of the EHR as giving them valuable, realistic learning experiences.

Towards FAIRification of sensitive and fragmented rare disease patient data: challenges and solutions in European reference network registries.

INTRODUCTION: Rare disease patient data are typically sensitive, present in multiple registries controlled by different custodians, and non-interoperable. Making these data Findable, Accessible, Interoperable, and Reusable (FAIR) for humans and machines at source enables federated discovery and analysis across data custodians. This facilitates accurate diagnosis, optimal clinical management, and personalised treatments. In Europe, twenty-four European Reference Networks (ERNs) work on rare disease registries in different clinical domains. The process and the implementation choices for making data FAIR ('FAIRification') differ among ERN registries. For example, registries use different software systems and are subject to different legal regulations. To support the ERNs in making informed decisions and to harmonise FAIRification, the FAIRification steward team was established to work as liaisons between ERNs and researchers from the European Joint Programme on Rare Diseases. RESULTS: The FAIRification steward team inventoried the FAIRification challenges of the ERN registries and proposed solutions collectively with involved stakeholders to address them. Ninety-eight FAIRification challenges from 24 ERNs' registries were collected and categorised into "training" (31), "community" (9), "modelling" (12), "implementation" (26), and "legal" (20). After curating and aggregating highly similar challenges, 41 unique FAIRification challenges remained. The two categories with the most challenges were "training" (15) and "implementation" (9), followed by "community" (7), and then "modelling" (5) and "legal" (5). To address all challenges, eleven types of solutions were proposed. Among them, the provision of guidelines and the organisation of training activities resolved the "training" challenges, which ranged from less-technical "coffee-rounds" to technical workshops, from informal FAIR Games to formal hackathons. Obtaining implementation support from technical experts was the solution type for tackling the "implementation" challenges. CONCLUSION: This work shows that a dedicated team of FAIR data stewards is an asset for harmonising the various processes of making data FAIR in a large organisation with multiple stakeholders. Additionally, multi-levelled training activities are required to accommodate the diverse needs of the ERNs. Finally, the lessons learned from the experience of the FAIRification steward team described in this paper may help to increase FAIR awareness and provide insights into FAIRification challenges and solutions of rare disease registries.

Cancer as a death sentence: developing an initial program theory for an IVR intervention.

To address current trends in poor health-seeking behaviour and late cancer diagnosis in many low- and middle-income countries, like Uganda, it is important to explore innovative awareness building interventions. One possible intervention is a common digital format, an interactive voice response (IVR) system, which is suitable for individuals with low technological and reading literacy. It is increasingly acknowledged that developing digital interventions requires co-creation with relevant stakeholders and explication of program developers' assumptions, to make them effective, sustainable, and scalable. To this end, we sought to develop an initial program theory for a co-created IVR system for cancer awareness in Uganda. Utilising principles of the realist approach, a qualitative exploratory study was conducted through seven focus group discussions (FGDs) with people living with cancer (PLWC), health workers, and policy makers. Thematic analysis of the transcripts resulted in the emergence of four major themes. Through all themes the most consistent finding was that myths, misconceptions, and misinformation about cancer were related to every aspect of the cancer journey and influenced the experiences and lives of PLWC and their caregivers. Participants were positive about the potential of an IVR system but also had reservations about the design and reach of the system. The resulting initial program theory proposes that a context-specific IVR system has the potential to improve awareness on cancer, provided attention is given to aspects such as language, message framing, and accuracy.

Performance assessment of ontology matching systems for FAIR data.

BACKGROUND: Ontology matching should contribute to the interoperability aspect of FAIR data (Findable, Accessible, Interoperable, and Reusable). Multiple data sources can use different ontologies for annotating their data and, thus, creating the need for dynamic ontology matching services. In this experimental study, we assessed the performance of ontology matching systems in the context of a real-life application from the rare disease domain. Additionally, we present a method for analyzing top-level classes to improve precision. RESULTS: We included three ontologies (NCIt, SNOMED CT, ORDO) and three matching systems (AgreementMakerLight 2.0, FCA-Map, LogMap 2.0). We evaluated the performance of the matching systems against reference alignments from BioPortal and the Unified Medical Language System Metathesaurus (UMLS). Then, we analyzed the top-level ancestors of matched classes, to detect incorrect mappings without consulting a reference alignment. To detect such incorrect mappings, we manually matched semantically equivalent top-level classes of ontology pairs. AgreementMakerLight 2.0, FCA-Map, and LogMap 2.0 had F1-scores of 0.55, 0.46, 0.55 for BioPortal and 0.66, 0.53, 0.58 for the UMLS respectively. Using vote-based consensus alignments increased performance across the board. Evaluation with manually created top-level hierarchy mappings revealed that on average 90% of the mappings' classes belonged to top-level classes that matched. CONCLUSIONS: Our findings show that the included ontology matching systems automatically produced mappings that were modestly accurate according to our evaluation. The hierarchical analysis of mappings seems promising when no reference alignments are available. All in all, the systems show potential to be implemented as part of an ontology matching service for querying FAIR data. Future research should focus on developing methods for the evaluation of mappings used in such mapping services, leading to their implementation in a FAIR data ecosystem.

Inaccurate recording of routinely collected data items influences identification of COVID-19 patients.

BACKGROUND: During the Coronavirus disease 2019 (COVID-19) pandemic it became apparent that it is difficult to extract standardized Electronic Health Record (EHR) data for secondary purposes like public health decision-making. Accurate recording of, for example, standardized diagnosis codes and test results is required to identify all COVID-19 patients. This study aimed to investigate if specific combinations of routinely collected data items for COVID-19 can be used to identify an accurate set of intensive care unit (ICU)-admitted COVID-19 patients. METHODS: The following routinely collected EHR data items to identify COVID-19 patients were evaluated: positive reverse transcription polymerase chain reaction (RT-PCR) test results; problem list codes for COVID-19 registered by healthcare professionals and COVID-19 infection labels. COVID-19 codes registered by clinical coders retrospectively after discharge were also evaluated. A gold standard dataset was created by evaluating two datasets of suspected and confirmed COVID-19-patients admitted to the ICU at a Dutch university hospital between February 2020 and December 2020, of which one set was manually maintained by intensivists and one set was extracted from the EHR by a research data management department. Patients were labeled 'COVID-19' if their EHR record showed diagnosing COVID-19 during or right before an ICU-admission. Patients were labeled 'non-COVID-19' if the record indicated no COVID-19, exclusion or only suspicion during or right before an ICU-admission or if COVID-19 was diagnosed and cured during non-ICU episodes of the hospitalization in which an ICU-admission took place. Performance was determined for 37 queries including real-time and retrospective data items. We used the F1 score, which is the harmonic mean between precision and recall. The gold standard dataset was split into one subset including admissions between February and April and one subset including admissions between May and December to determine accuracy differences. RESULTS: The total dataset consisted of 402 patients: 196 'COVID-19' and 206 'non-COVID-19' patients. F1 scores of search queries including EHR data items that can be extracted real-time ranged between 0.68 and 0.97 and for search queries including the data item that was retrospectively registered by clinical coders F1 scores ranged between 0.73 and 0.99. F1 scores showed no clear pattern in variability between the two time periods. CONCLUSIONS: Our study showed that one cannot rely on individual routinely collected data items such as coded COVID-19 on problem lists to identify all COVID-19 patients. If information is not required real-time, medical coding from clinical coders is most reliable. Researchers should be transparent about their methods used to extract data. To maximize the ability to completely identify all COVID-19 cases alerts for inconsistent data and policies for standardized data capture could enable reliable data reuse.

Exploring supportive care and information needs through a proposed eHealth application among melanoma patients undergoing systemic therapy: a qualitative study.

BACKGROUND: During and after systemic therapy, patients with high risk and advanced melanoma experience challenges regarding cancer-related symptoms, treatment-related adverse events, and an impact of these symptoms on their physical and psychosocial well-being. Few studies have investigated the specific needs of these patients and the potential role of eHealth applications in meeting those needs. OBJECTIVE: To explore the supportive care and information needs of high risk and advanced melanoma patients, and how these needs can be supported by eHealth applications. METHODS: In this qualitative study, semi-structured interviews with high risk and advanced melanoma patients during or after systemic treatment were conducted to understand their needs and requirements as possible end-users of mobile eHealth applications. Interview transcripts were independently coded and thematically analyzed. RESULTS: Thirteen participants consented to be interviewed, aged 31 to 71 years. Nearly all patients (n = 12, 92%) experienced unmet information and supportive care needs during and after active treatment. Patients expected to value eHealth applications that facilitate information gathering, wellbeing interventions, and symptom management. The majority of patients (n = 10, 77%) anticipated various advantages from using an eHealth application, including increased autonomy, higher quality of life, and improved disease self-management. DISCUSSION: High risk and advanced melanoma patients have unmet supportive care and information needs during and after systemic treatment. The use of eHealth applications might be an effective way to meet these unmet needs. Patients anticipate a variety of advantages from using these applications, including deriving various benefits from the use of these applications, such as enhanced autonomy.

Perceptions and behavior of clinical researchers and research support staff regarding data FAIRification.

The FAIR Data Principles are being rapidly adopted by many research institutes and funders worldwide. This study aimed to assess the awareness and attitudes of clinical researchers and research support staff regarding data FAIRification. A questionnaire was distributed to researchers and support staff in six Dutch University Medical Centers and Electronic Data Capture platform users. 164 researchers and 21 support staff members completed the questionnaire. 62.8% of the researchers and 81.0% of the support staff are currently undertaking at least some effort to achieve any aspect of FAIR, 11.0% and 23.8%, respectively, address all aspects. Only 46.6% of the researchers add metadata to their datasets, 39.7% add metadata to data elements, and 35.9% deposit their data in a repository. 94.7% of the researchers are aware of the usefulness of their data being FAIR for others and 89.3% are, given the right resources and support, willing to FAIRify their data. Institutions and funders should, therefore, develop FAIRification training and tools and should (financially) support researchers and staff throughout the process.

FAIRifying a Quality Registry Using OMOP CDM: Challenges and Solutions.

The need for health data to be internationally Findable, Accessible, Interoperable and Reusable (FAIR) and thereby support integrative analysis with other datasets has become crystal clear in the ongoing pandemic. The Dutch National Intensive Care Evaluation (NICE) quality registry adopted the Observational Medical Outcomes Partnership Common Database Model (OMOP CDM) to achieve a FAIR database. In the process of adopting the OMOP CDM, many modeling, technical, and communication challenges needed to be solved. Through communication with the OMOP CDM implementation community, previously done research and trial-and-error we found solutions that we believe can help other healthcare institutions, especially ICU quality registries, FAIRify their databases.