How can we reward you? A compliance and reward ontology (CaRO) for eliciting quantitative reward rules for engagement in mHealth app and healthy behaviors.

OBJECTIVE: When developing mHealth apps with point reward systems, knowledge engineers and domain experts should define app requirements capturing quantitative reward patterns that reflect patient compliance with health behaviors. This is a difficult task, and they could be aided by an ontology that defines systematically quantitative behavior goals that address more than merely the recommended behavior but also rewards for partial compliance or practicing the behavior more than recommended. No ontology and algorithm exist for defining point rewards systematically. METHODS: We developed an OWL ontology for point rewards that leverages the Basic Formal Ontology, the Behaviour Change Intervention Ontology and the Gamification Domain Ontology. This Compliance and Reward Ontology (CaRO) allows defining temporal elementary reward patterns for single and multiple sessions of practicing a behavior. These could be assembled to define more complex temporal patterns for persistence behavior over longer time intervals as well as logical combinations of simpler reward patterns. We also developed an algorithm for calculating the points that should be rewarded to users, given data regarding their actual performance. A natural language generation algorithm generates from ontology instances app requirements in the form of user stories. To assess the usefulness of the ontology and algorithms, information system students who are trained to be system analysts/knowledge engineers evaluated whether the ontology and algorithms can improve the requirement elicitation of point rewards for compliance patterns more completely and correctly. RESULTS: For single-session rewards, the ontology improved formulation of two of the six requirements as well as the total time for specifying them. For multi-session rewards, the ontology improved formulation of five of the 11 requirements. CONCLUSION: CaRO is a first attempt of its kind, and it covers all of the cases of compliance and reward pattern definitions that were needed for a full-scale system that was developed as part of a large European project. The ontology and algorithm are available at https://github.com/capable-project/rewards.

Leveraging generative AI for clinical evidence synthesis needs to ensure trustworthiness.

Evidence-based medicine promises to improve the quality of healthcare by empowering medical decisions and practices with the best available evidence. The rapid growth of medical evidence, which can be obtained from various sources, poses a challenge in collecting, appraising, and synthesizing the evidential information. Recent advancements in generative AI, exemplified by large language models, hold promise in facilitating the arduous task. However, developing accountable, fair, and inclusive models remains a complicated undertaking. In this perspective, we discuss the trustworthiness of generative AI in the context of automated summarization of medical evidence.

An eHealth App (CAPABLE) Providing Symptom Monitoring, Well-Being Interventions, and Educational Material for Patients With Melanoma Treated With Immune Checkpoint Inhibitors: Protocol for an Exploratory Intervention Trial.

BACKGROUND: Since treatment with immune checkpoint inhibitors (ICIs) is becoming standard therapy for patients with high-risk and advanced melanoma, an increasing number of patients experience treatment-related adverse events such as fatigue. Until now, studies have demonstrated the benefits of using eHealth tools to provide either symptom monitoring or interventions to reduce treatment-related symptoms such as fatigue. However, an eHealth tool that facilitates the combination of both symptom monitoring and symptom management in patients with melanoma treated with ICIs is still needed. OBJECTIVE: In this pilot study, we will explore the use of the CAPABLE (Cancer Patients Better Life Experience) app in providing symptom monitoring, education, and well-being interventions on health-related quality of life (HRQoL) outcomes such as fatigue and physical functioning, as well as patients' acceptance and usability of using CAPABLE. METHODS: This prospective, exploratory pilot study will examine changes in fatigue over time in 36 patients with stage III or IV melanoma during treatment with ICI using CAPABLE (a smartphone app and multisensory smartwatch). This cohort will be compared to a prospectively collected cohort of patients with melanoma treated with standard ICI therapy. CAPABLE will be used for a minimum of 3 and a maximum of 6 months. The primary endpoint in this study is the change in fatigue between baseline and 3 and 6 months after the start of treatment. Secondary end points include HRQoL outcomes, usability, and feasibility parameters. RESULTS: Study inclusion started in April 2023 and is currently ongoing. CONCLUSIONS: This pilot study will explore the effect, usability, and feasibility of CAPABLE in patients with melanoma during treatment with ICI. Adding the CAPABLE system to active treatment is hypothesized to decrease fatigue in patients with high-risk and advanced melanoma during treatment with ICIs compared to a control group receiving standard care. The Medical Ethics Committee NedMec (Amsterdam, The Netherlands) granted ethical approval for this study (reference number 22-981/NL81970.000.22). TRIAL REGISTRATION: ClinicalTrials.gov NCT05827289; https://clinicaltrials.gov/study/NCT05827289. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/49252.

Identifying underlying patterns in Alzheimer's disease trajectory: a deep learning approach and Mendelian randomization analysis.

Background: Alzheimer's disease (AD) is a heterogeneously progressive neurodegeneration disorder with varied rates of deterioration, either between subjects or within different stages of a certain subject. Estimating the course of AD at early stages has treatment implications. We aimed to analyze disease progression to identify distinct patterns in AD trajectory. Methods: We proposed a deep learning model to identify underlying patterns in the trajectory from cognitively normal (CN) to a state of mild cognitive impairment (MCI) to AD dementia, by jointly predicting time-to-conversion and clustering out distinct subgroups characterized by comprehensive features as well as varied progression rates. We designed and validated our model on the ADNI dataset (1370 participants). Prediction of time-to-conversion in AD trajectory was used to validate the expression of the identified patterns. Causality between patterns and time-to-conversion was further inferred using Mendelian randomization (MR) analysis. External validation was performed on the AIBL dataset (233 participants). Findings: The proposed model clustered out patterns characterized by significantly different biomarkers and varied progression rates. The discovered patterns also showed a strong prediction ability, as indicated by hazard ratio (CN→MCI, HR = 3.51, p < 0.001; MCI→AD, HR = 8.11, p < 0.001), C-Index (CN→MCI, 0.618; MCI→AD, 0.718), and AUC (CN→MCI, 3 years 0.802, 5 years 0.876; MCI→AD, 3 years 0.914, 5 years 0.957). In the external validation cohort, our model demonstrated competitive performance on conversion time prediction (CN→MCI, C-Index = 0.693; MCI→AD, C-Index = 0.752). Moreover, suggestive associations between CN→MCI/MCI→AD patterns with four/three SNPs were mediated and MR analysis indicated a causal link between MCI→AD patterns and time-to-conversion in the first three years. Interpretation: Our proposed model identifies biologically and clinically meaningful patterns from real-world data and provides promising performance on time-to-conversion prediction in AD trajectory, which could promote the understanding of disease progression, facilitate clinical trial design, and provide potential for decision-making. Funding: The National Key Research and Development Program of China, the Key R&D Program of Zhejiang, and the National Nature Science Foundation of China.

Achieving trust in health-behavior-change artificial intelligence apps (HBC-AIApp) development: A multi-perspective guide.

OBJECTIVE: Trust determines the success of Health-Behavior-Change Artificial Intelligence Apps (HBC-AIApp). Developers of such apps need theory-based practical methods that can guide them in achieving such trust. Our study aimed to develop a comprehensive conceptual model and development process that can guide developers how to build HBC-AIApp in order to support trust creation among the app's users. METHODS: We apply a multi-disciplinary approach where medical informatics, human-centered design, and holistic health methods are integrated to address the trust challenge in HBC-AIApps. The integration extends a conceptual model of trust in AI developed by Jermutus et al., whose properties guide the extension of the IDEAS (integrate, design, assess, and share) HBC-App development process. RESULTS: The HBC-AIApp framework consists of three main blocks: (1) system development methods that study the users' complex reality, hence, their perceptions, needs, goals and environment; (2) mediators and other stakeholders who are important for developing and operating the HBC-AIApp, boundary objects that examine users' activities via the HBC-AIApp; and (3) the HBC-AIApp's structural components, AI logic, and physical implementation. These blocks come together to provide the extended conceptual model of trust in HBC-AIApps and the extended IDEAS process. DISCUSSION: The developed HBC-AIApp framework drew from our own experience in developing trust in HBC-AIApp. Further research will focus on studying the application of the proposed comprehensive HBC-AIApp development framework and whether applying it supports trust creation in such apps.

The IHI Rochester Report 2022 on Healthcare Informatics Research: Resuming After the CoViD-19.

In 2020, the CoViD-19 pandemic spread worldwide in an unexpected way and suddenly modified many life issues, including social habits, social relationships, teaching modalities, and more. Such changes were also observable in many different healthcare and medical contexts. Moreover, the CoViD-19 pandemic acted as a stress test for many research endeavors, and revealed some limitations, especially in contexts where research results had an immediate impact on the social and healthcare habits of millions of people. As a result, the research community is called to perform a deep analysis of the steps already taken, and to re-think steps for the near and far future to capitalize on the lessons learned due to the pandemic. In this direction, on June 09th-11th, 2022, a group of twelve healthcare informatics researchers met in Rochester, MN, USA. This meeting was initiated by the Institute for Healthcare Informatics-IHI, and hosted by the Mayo Clinic. The goal of the meeting was to discuss and propose a research agenda for biomedical and health informatics for the next decade, in light of the changes and the lessons learned from the CoViD-19 pandemic. This article reports the main topics discussed and the conclusions reached. The intended readers of this paper, besides the biomedical and health informatics research community, are all those stakeholders in academia, industry, and government, who could benefit from the new research findings in biomedical and health informatics research. Indeed, research directions and social and policy implications are the main focus of the research agenda we propose, according to three levels: the care of individuals, the healthcare system view, and the population view.

A community-of-practice-based evaluation methodology for knowledge intensive computational methods and its application to multimorbidity decision support.

OBJECTIVE: The study has dual objectives. Our first objective (1) is to develop a community-of-practice-based evaluation methodology for knowledge-intensive computational methods. We target a whitebox analysis of the computational methods to gain insight on their functional features and inner workings. In more detail, we aim to answer evaluation questions on (i) support offered by computational methods for functional features within the application domain; and (ii) in-depth characterizations of the underlying computational processes, models, data and knowledge of the computational methods. Our second objective (2) involves applying the evaluation methodology to answer questions (i) and (ii) for knowledge-intensive clinical decision support (CDS) methods, which operationalize clinical knowledge as computer interpretable guidelines (CIG); we focus on multimorbidity CIG-based clinical decision support (MGCDS) methods that target multimorbidity treatment plans. MATERIALS AND METHODS: Our methodology directly involves the research community of practice in (a) identifying functional features within the application domain; (b) defining exemplar case studies covering these features; and (c) solving the case studies using their developed computational methods-research groups detail their solutions and functional feature support in solution reports. Next, the study authors (d) perform a qualitative analysis of the solution reports, identifying and characterizing common themes (or dimensions) among the computational methods. This methodology is well suited to perform whitebox analysis, as it directly involves the respective developers in studying inner workings and feature support of computational methods. Moreover, the established evaluation parameters (e.g., features, case studies, themes) constitute a re-usable benchmark framework, which can be used to evaluate new computational methods as they are developed. We applied our community-of-practice-based evaluation methodology on MGCDS methods. RESULTS: Six research groups submitted comprehensive solution reports for the exemplar case studies. Solutions for two of these case studies were reported by all groups. We identified four evaluation dimensions: detection of adverse interactions, management strategy representation, implementation paradigms, and human-in-the-loop support. Based on our whitebox analysis, we present answers to the evaluation questions (i) and (ii) for MGCDS methods. DISCUSSION: The proposed evaluation methodology includes features of illuminative and comparison-based approaches; focusing on understanding rather than judging/scoring or identifying gaps in current methods. It involves answering evaluation questions with direct involvement of the research community of practice, who participate in setting up evaluation parameters and solving exemplar case studies. Our methodology was successfully applied to evaluate six MGCDS knowledge-intensive computational methods. We established that, while the evaluated methods provide a multifaceted set of solutions with different benefits and drawbacks, no single MGCDS method currently provides a comprehensive solution for MGCDS. CONCLUSION: We posit that our evaluation methodology, applied here to gain new insights into MGCDS, can be used to assess other types of knowledge-intensive computational methods and answer other types of evaluation questions. Our case studies can be accessed at our GitHub repository (https://github.com/william-vw/MGCDS).

SATO (IDEAS expAnded wiTh BCIO): Workflow for designers of patient-centered mobile health behaviour change intervention applications.

Designing effective theory-driven digital behaviour change interventions (DBCI) is a challenging task. To ease the design process, and assist with knowledge sharing and evaluation of the DBCI, we propose the SATO (IDEAS expAnded wiTh BCIO) design workflow based on the IDEAS (Integrate, Design, Assess, and Share) framework and aligned with the Behaviour Change Intervention Ontology (BCIO). BCIO is a structural representation of the knowledge in behaviour change domain supporting evaluation of behaviour change interventions (BCIs) but it is not straightforward to utilise it during DBCI design. IDEAS (Integrate, Design, Assess, and Share) framework guides multi-disciplinary teams through the mobile health (mHealth) application development life-cycle but it is not aligned with BCIO entities. SATO couples BCIO entities with workflow steps and extends IDEAS Integrate stage with consideration of customisation and personalisation. We provide a checklist of the activities that should be performed during intervention planning with concrete examples and a tutorial accompanied with case studies from the Cancer Better Life Experience (CAPABLE) European project. In the process of creating this workflow, we found the necessity to extend the BCIO to support the scenarios of multiple clinical goals in the same application. To ensure the SATO steps are easy to follow for the incomers to the field, we performed a preliminary evaluation of the workflow with two knowledge engineers, working on novel mHealth app design tasks.

Participatory Design of a Mobile App to Safeguard Mental Resilience in the Context of Drug Use in Young Adults: Multi-Method Study.

BACKGROUND: Existing mental health apps are largely not aimed at generally healthy young people who may be experimenting with addictive substances and mind-altering experiences. OBJECTIVE: The aim of this study is to examine the interest and expectations of young people regarding a proposed smartphone app designed to help protect and promote mental health and resilience in the face of risks associated with substance use. METHODS: The study was based on agile system development and had 3 empirical substudies. Our feasibility study (study 1) included an anonymous questionnaire that examined the potential interest of young people in this type of app. It was answered by 339 Israelis aged 18-30 years. The second part of the feasibility study was a pilot study with 1.2% (4/339) of the people who answered the questionnaire and expressed interest in participating in a focus group. They tested and refined the elements planned for the focus groups. Study 2 was a participatory design study involving 7 focus groups of 5 to 7 participants each (young people aged 18-35 years, n=38). Persona development, open discussion, and a Technology Acceptance Model questionnaire were used to elicit user expectations and requirements for the app and to understand the perceived usefulness and usability of the proposed features. Study 3 comprised in-depth interviews with experts in the field of youth mental health and drug use to enlist their professional opinion regarding the value of such an app and recommendations about the features it should include. RESULTS: The mock-up for the proposed app had five key features: personalized assessment of risk for a drug-associated mental crisis, support for self-monitoring, useful information (eg, warning signs and first-aid guidelines), resilience-building exercises, and a support center. Participants rated highly the usefulness of all 5 main features and 96% (24/25) of the specific features we proposed within those main categories. The participants also suggested additional features as well as a new user persona we had not considered: the parents or family members of the young person. The focus groups rated highly the perceived usability of the app. Most of the experts saw value in all the main features and suggested specific knowledge sources for the app's content. Finally, participants of both the feasibility study and the participatory design study expressed moderate to high interest in using the app for self-help and high interest in using the app to help friends. CONCLUSIONS: The findings provide preliminary encouraging support for the 5 main features suggested by the research team and reinforce recommendations for mobile health apps found in the literature. The findings emphasize the insight that this kind of app should be designed primarily for use by individuals seeking to help others.

Safety of Endoscopic Sinus Surgery in the Elderly-Are Octogenarian Patients at a Higher Risk?

BACKGROUND: Surgeons are often faced with concerns regarding the risks versus benefits of endoscopic sinus surgery (ESS) in elderly patients. OBJECTIVE: To analyze the risk for complications of ESS in the elderly (age ≥70 years) compared to younger patients, with emphasis on octogenarians. METHODS: Retrospective review of medical charts of adult patients who underwent ESS at a tertiary referral center during the years 2014 to 2018. RESULTS: We compared 128 elderly patients with 276 matched younger patients. In the elderly group mean age was 76 years (range, 70-91 years ). Thirty-one elderly patients were 80 years or older. Surgical complications in the elderly patients were 3.9%. Minor complications were 2.3% and major complications were 1.7%. The surgical complications rate was similar in the younger group (8%, P value: .127). Medical complications were observed in 2.3% comparing to 0.7% in younger patients. Interestingly, age, revision surgery, extent and duration of surgery, and modality of anesthesia were not identified as risk factors. Only ischemic heart disease (IHD) was identified as a risk factor for complications in a multivariate analysis in elderly patients. Comparison of elderly patients younger than 80 years with octogenarians revealed no difference in complication rate between these groups. CONCLUSIONS: Overall, ESS was found to be a safe procedure in elderly patients compared to younger patients. Octogenarian patients should not be denied upfront surgery. IHD is a risk factor for complications in elderly patients.

A Health eLearning Ontology and Procedural Reasoning Approach for Developing Personalized Courses to Teach Patients about Their Medical Condition and Treatment.

We propose a methodological framework to support the development of personalized courses that improve patients' understanding of their condition and prescribed treatment. Inspired by Intelligent Tutoring Systems (ITSs), the framework uses an eLearning ontology to express domain and learner models and to create a course. We combine the ontology with a procedural reasoning approach and precompiled plans to operationalize a design across disease conditions. The resulting courses generated by the framework are personalized across four patient axes-condition and treatment, comprehension level, learning style based on the VARK (Visual, Aural, Read/write, Kinesthetic) presentation model, and the level of understanding of specific course content according to Bloom's taxonomy. Customizing educational materials along these learning axes stimulates and sustains patients' attention when learning about their conditions or treatment options. Our proposed framework creates a personalized course that prepares patients for their meetings with specialists and educates them about their prescribed treatment. We posit that the improvement in patients' understanding of prescribed care will result in better outcomes and we validate that the constructs of our framework are appropriate for representing content and deriving personalized courses for two use cases: anticoagulation treatment of an atrial fibrillation patient and lower back pain management to treat a lumbar degenerative disc condition. We conduct a mostly qualitative study supported by a quantitative questionnaire to investigate the acceptability of the framework among the target patient population and medical practitioners.

Enhancing the IDEAS Framework with Ontology: Designing Digital Interventions for Improving Cancer Patients' Wellbeing.

Developing effective digital interventions to help patients form healthy habits is a challenging goal. IDEAS is a step-by-step framework that allows developers to draw ideas from intended users and behavioral theories, and ideate implementation strategies for them, followed by rapid prototype development. Based on our long experience with developing generic knowledge-based clinical decision support systems (CDSS) and integrating them with electronic health records (EHR) to deliver patient-specific advice, we observed a challenge that IDEAS is not addressing: the semantic detailing of the clinical knowledge behind the digital intervention and relevant patient data that could be used to personalize the digital intervention. To close the gap, we augmented two steps of IDEAS with an ontology that structures the target behavior as classes, derived from HL7 Fast Healthcare Interoperability Resources standard. We exemplify the augmented IDEAS with a case study taken from the Horizon 2020 CAPABLE project, that uses Fogg's Tiny Habits behavioral model to improve the sleep of cancer patients via Tai Chi.

Towards a framework for comparing functionalities of multimorbidity clinical decision support: A literature-based feature set and benchmark cases.

Multimorbidity, the coexistence of two or more health conditions, has become more prevalent as mortality rates in many countries have declined and their populations have aged. Multimorbidity presents significant difficulties for Clinical Decision Support Systems (CDSS), particularly in cases where recommendations from relevant clinical guidelines offer conflicting advice. A number of research groups are developing computer-interpretable guideline (CIG) modeling formalisms that integrate recommendations from multiple Clinical Practice Guidelines (CPGs) for knowledge-based multimorbidity decision support. In this paper we describe work towards the development of a framework for comparing the different approaches to multimorbidity CIG-based clinical decision support (MGCDS). We present (1) a set of features for MGCDS, which were derived using a literature review and evaluated by physicians using a survey, and (2) a set of benchmarking case studies, which illustrate the clinical application of these features. This work represents the first necessary step in a broader research program aimed at the development of a benchmark framework that allows for standardized and comparable MGCDS evaluations, which will facilitate the assessment of functionalities of MGCDS, as well as highlight important gaps in the state-of-the-art. We also outline our future work on developing the framework, specifically, (3) a standard for reporting MGCDS solutions for the benchmark case studies, and (4) criteria for evaluating these MGCDS solutions. We plan to conduct a large-scale comparison study of existing MGCDS based on the comparative framework.

Towards a goal-oriented methodology for clinical-guideline-based management recommendations for patients with multimorbidity: GoCom and its preliminary evaluation.

Patients with chronic multimorbidity are becoming more common as life expectancy increases, making it necessary for physicians to develop complex management plans. We are looking at the patient management process as a goal-attainment problem. Hence, our aim is to develop a goal-oriented methodology for providing decision support for managing patients with multimorbidity continuously, as the patient's health state is progressing and new goals arise (e.g., treat ulcer, prevent osteoporosis). Our methodology allows us to detect and mitigate inconsistencies among guideline recommendations stemming from multiple clinical guidelines, while consulting medical ontologies and terminologies and relying on patient information standards. This methodology and its implementation as a decision-support system, called GoCom, starts with computer-interpretable clinical guidelines (CIGs) for single problems that are formalized using the PROforma CIG language. We previously published the architecture of the system as well as a CIG elicitation guide for enriching PROforma tasks with properties referring to vocabulary codes of goals and physiological effects of management plans. In this paper, we provide a formalization of the conceptual model of GoCom that generates, for each morbidity of the patient, a patient-specific goal tree that results from the PROforma engine's enactment of the CIG with the patient's data. We also present the "Controller" algorithm that drives the GoCom system. Given a new problem that a patient develops, the Controller detects inconsistencies among goals pertaining to different comorbid problems and consults the CIGs to generate alternative non-conflicted and goal-oriented management plans that address the multiple goals simultaneously. In this stage of our research, the inconsistencies that can be detected are of two types - starting vs. stopping medications that belong to the same medication class hierarchy, and detecting opposing physiological effect goals that are specified in concurrent CIGs (e.g., decreased blood pressure vs. increased blood pressure). However, the design of GoCom is modular and generic and allows the future introduction of additional interaction detection and mitigation strategies. Moreover, GoCom generates explanations of the alternative non-conflicted management plans, based on recommendations stemming from the clinical guidelines and reasoning patterns. GoCom's functionality was evaluated using three cases of multimorbidity interactions that were checked by our three clinicians. Usefulness was evaluated with two studies. The first evaluation was a pilot study with ten 6th year medical students and the second evaluation was done with 27 6th medical students and interns. The participants solved complex realistic cases of multimorbidity patients: with and without decision-support, two cases in the first evaluation and 6 cases in the second evaluation. Use of GoCom increased completeness of the patient management plans produced by the medical students from 0.44 to 0.71 (P-value of 0.0005) in the first evaluation, and from 0.31 to 0.78 (P-value < 0.0001) in the second evaluation. Correctness in the first evaluation was very high with (0.98) or without the system (0.91), with non-significant difference (P-value ≥ 0.17). In the second evaluation, use of GoCom increased correctness from 0.68 to 0.83 (P-value of 0.001). In addition, GoCom's explanation and visualization were perceived as useful by the vast majority of participants. While GoCom's detection of goal interactions is currently limited to detection of starting vs. stopping the same medication or medication subclasses and detecting conflicting physiological effects of concurrent medications, the evaluation demonstrated potential of the system for improving clinical decision-making for multimorbidity patients.

A layered computer-interpretable guideline model for easing the update of locally adapted clinical guidelines.

Maintenance of computer-interpretable guidelines is complicated by evolving medical knowledge and by the requirement to customize content to local practice settings. We developed a framework to support knowledge engineers in customization and maintenance of computer-interpretable guidelines specified in the PROforma formalism. In our layered approach, the computer-interpretable guidelines containing the original clinical guideline serves as the primary layer and local customizations form secondary layers that adhere to its schema while augmenting it. Java code unifies the layers into a single enactable computer-interpretable guidelines. We performed a pilot experiment to verify the effectiveness of a layered framework. In this first attempt, we evaluated the hypothesis that the layered computer-interpretable guidelines framework supports knowledge engineers in maintenance of customized computer-interpretable guidelines. Participants who used the layered framework completed an update process of the primary knowledge in less time and made fewer errors as compared to those using the single-layer framework.