How can we reward you? A compliance and reward ontology (CaRO) for eliciting quantitative reward rules for engagement in mHealth app and healthy behaviors.

OBJECTIVE: When developing mHealth apps with point reward systems, knowledge engineers and domain experts should define app requirements capturing quantitative reward patterns that reflect patient compliance with health behaviors. This is a difficult task, and they could be aided by an ontology that defines systematically quantitative behavior goals that address more than merely the recommended behavior but also rewards for partial compliance or practicing the behavior more than recommended. No ontology and algorithm exist for defining point rewards systematically. METHODS: We developed an OWL ontology for point rewards that leverages the Basic Formal Ontology, the Behaviour Change Intervention Ontology and the Gamification Domain Ontology. This Compliance and Reward Ontology (CaRO) allows defining temporal elementary reward patterns for single and multiple sessions of practicing a behavior. These could be assembled to define more complex temporal patterns for persistence behavior over longer time intervals as well as logical combinations of simpler reward patterns. We also developed an algorithm for calculating the points that should be rewarded to users, given data regarding their actual performance. A natural language generation algorithm generates from ontology instances app requirements in the form of user stories. To assess the usefulness of the ontology and algorithms, information system students who are trained to be system analysts/knowledge engineers evaluated whether the ontology and algorithms can improve the requirement elicitation of point rewards for compliance patterns more completely and correctly. RESULTS: For single-session rewards, the ontology improved formulation of two of the six requirements as well as the total time for specifying them. For multi-session rewards, the ontology improved formulation of five of the 11 requirements. CONCLUSION: CaRO is a first attempt of its kind, and it covers all of the cases of compliance and reward pattern definitions that were needed for a full-scale system that was developed as part of a large European project. The ontology and algorithm are available at https://github.com/capable-project/rewards.

Provision and evaluation of explanations within an automated planning-based approach to solving the multimorbidity problem.

The multimorbidity problem involves the identification and mitigation of adverse interactions that occur when multiple computer interpretable guidelines are applied concurrently to develop a treatment plan for a patient diagnosed with multiple diseases. Solving this problem requires decision support approaches which are difficult to comprehend for physicians. As such, the rationale for treatment plans generated by these approaches needs to be provided. OBJECTIVE: To develop an explainability component for an automated planning-based approach to the multimorbidity problem, and to assess the fidelity and interpretability of generated explanations using a clinical case study. METHODS: The explainability component leverages the task-network model for representing computer interpretable guidelines. It generates post-hoc explanations composed of three aspects that answer why specific clinical actions are in a treatment plan, why specific revisions were applied, and how factors like medication cost, patient's adherence, etc. influence the selection of specific actions. The explainability component is implemented as part of MitPlan, where we revised our planning-based approach to support explainability. We developed an evaluation instrument based on the system causability scale and other vetted surveys to evaluate the fidelity and interpretability of its explanations using a two dimensional comparison study design. RESULTS: The explainability component was implemented for MitPlan and tested in the context of a clinical case study. The fidelity and interpretability of the generated explanations were assessed using a physician-focused evaluation study involving 21 participants from two different specialties and two levels of experience. Results show that explanations provided by the explainability component in MitPlan are of acceptable fidelity and interpretability, and that the clinical justification of the actions in a treatment plan is important to physicians. CONCLUSION: We created an explainability component that enriches an automated planning-based approach to solving the multimorbidity problem with meaningful explanations for actions in a treatment plan. This component relies on the task-network model to represent computer interpretable guidelines and as such can be ported to other approaches that also use the task-network model representation. Our evaluation study demonstrated that explanations that support a physician's understanding of the clinical reasons for the actions in a treatment plan are useful and important.

SATO (IDEAS expAnded wiTh BCIO): Workflow for designers of patient-centered mobile health behaviour change intervention applications.

Designing effective theory-driven digital behaviour change interventions (DBCI) is a challenging task. To ease the design process, and assist with knowledge sharing and evaluation of the DBCI, we propose the SATO (IDEAS expAnded wiTh BCIO) design workflow based on the IDEAS (Integrate, Design, Assess, and Share) framework and aligned with the Behaviour Change Intervention Ontology (BCIO). BCIO is a structural representation of the knowledge in behaviour change domain supporting evaluation of behaviour change interventions (BCIs) but it is not straightforward to utilise it during DBCI design. IDEAS (Integrate, Design, Assess, and Share) framework guides multi-disciplinary teams through the mobile health (mHealth) application development life-cycle but it is not aligned with BCIO entities. SATO couples BCIO entities with workflow steps and extends IDEAS Integrate stage with consideration of customisation and personalisation. We provide a checklist of the activities that should be performed during intervention planning with concrete examples and a tutorial accompanied with case studies from the Cancer Better Life Experience (CAPABLE) European project. In the process of creating this workflow, we found the necessity to extend the BCIO to support the scenarios of multiple clinical goals in the same application. To ensure the SATO steps are easy to follow for the incomers to the field, we performed a preliminary evaluation of the workflow with two knowledge engineers, working on novel mHealth app design tasks.

A community-of-practice-based evaluation methodology for knowledge intensive computational methods and its application to multimorbidity decision support.

OBJECTIVE: The study has dual objectives. Our first objective (1) is to develop a community-of-practice-based evaluation methodology for knowledge-intensive computational methods. We target a whitebox analysis of the computational methods to gain insight on their functional features and inner workings. In more detail, we aim to answer evaluation questions on (i) support offered by computational methods for functional features within the application domain; and (ii) in-depth characterizations of the underlying computational processes, models, data and knowledge of the computational methods. Our second objective (2) involves applying the evaluation methodology to answer questions (i) and (ii) for knowledge-intensive clinical decision support (CDS) methods, which operationalize clinical knowledge as computer interpretable guidelines (CIG); we focus on multimorbidity CIG-based clinical decision support (MGCDS) methods that target multimorbidity treatment plans. MATERIALS AND METHODS: Our methodology directly involves the research community of practice in (a) identifying functional features within the application domain; (b) defining exemplar case studies covering these features; and (c) solving the case studies using their developed computational methods-research groups detail their solutions and functional feature support in solution reports. Next, the study authors (d) perform a qualitative analysis of the solution reports, identifying and characterizing common themes (or dimensions) among the computational methods. This methodology is well suited to perform whitebox analysis, as it directly involves the respective developers in studying inner workings and feature support of computational methods. Moreover, the established evaluation parameters (e.g., features, case studies, themes) constitute a re-usable benchmark framework, which can be used to evaluate new computational methods as they are developed. We applied our community-of-practice-based evaluation methodology on MGCDS methods. RESULTS: Six research groups submitted comprehensive solution reports for the exemplar case studies. Solutions for two of these case studies were reported by all groups. We identified four evaluation dimensions: detection of adverse interactions, management strategy representation, implementation paradigms, and human-in-the-loop support. Based on our whitebox analysis, we present answers to the evaluation questions (i) and (ii) for MGCDS methods. DISCUSSION: The proposed evaluation methodology includes features of illuminative and comparison-based approaches; focusing on understanding rather than judging/scoring or identifying gaps in current methods. It involves answering evaluation questions with direct involvement of the research community of practice, who participate in setting up evaluation parameters and solving exemplar case studies. Our methodology was successfully applied to evaluate six MGCDS knowledge-intensive computational methods. We established that, while the evaluated methods provide a multifaceted set of solutions with different benefits and drawbacks, no single MGCDS method currently provides a comprehensive solution for MGCDS. CONCLUSION: We posit that our evaluation methodology, applied here to gain new insights into MGCDS, can be used to assess other types of knowledge-intensive computational methods and answer other types of evaluation questions. Our case studies can be accessed at our GitHub repository (https://github.com/william-vw/MGCDS).

Ideating Mobile Health Behavioral Support for Compliance to Therapy for Patients with Chronic Disease: A Case Study of Atrial Fibrillation Management.

Poor patient compliance to therapy results in a worsening condition that often increases healthcare costs. In the MobiGuide project, we developed an evidence-based clinical decision-support system that delivered personalized reminders and recommendations to patients, helping to achieve higher therapy compliance. Yet compliance could still be improved and therefore building on the MobiGuide project experience, we designed a new component called the Motivational Patient Assistant (MPA) that is integrated within the MobiGuide architecture to further improve compliance. This component draws from psychological theories to provide behavioral support to improve patient engagement and thereby increasing patients' compliance. Behavior modification interventions are delivered via mobile technology at patients' home environments. Our approach was inspired by the IDEAS (Integrate, Design, Assess, and Share) framework for developing effective digital interventions to change health behavior; it goes beyond this approach by extending the Ideation phase' concepts into concrete backend architectural components and graphical user-interface designs that implement behavioral interventions. We describe in detail our ideation approach and how it was applied to design the user interface of MPA for anticoagulation therapy for the atrial fibrillation patients. We report results of a preliminary evaluation involving patients and care providers that shows the potential usefulness of the MPA for improving compliance to anticoagulation therapy.

Comprehensive mitigation framework for concurrent application of multiple clinical practice guidelines.

In this work we propose a comprehensive framework based on first-order logic (FOL) for mitigating (identifying and addressing) interactions between multiple clinical practice guidelines (CPGs) applied to a multi-morbid patient while also considering patient preferences related to the prescribed treatment. With this framework we respond to two fundamental challenges associated with clinical decision support: (1) concurrent application of multiple CPGs and (2) incorporation of patient preferences into the decision making process. We significantly expand our earlier research by (1) proposing a revised and improved mitigation-oriented representation of CPGs and secondary medical knowledge for addressing adverse interactions and incorporating patient preferences and (2) introducing a new mitigation algorithm. Specifically, actionable graphs representing CPGs allow for parallel and temporal activities (decisions and actions). Revision operators representing secondary medical knowledge support temporal interactions and complex revisions across multiple actionable graphs. The mitigation algorithm uses the actionable graphs, revision operators and available (and possibly incomplete) patient information represented in FOL. It relies on a depth-first search strategy to find a valid sequence of revisions and uses theorem proving and model finding techniques to identify applicable revision operators and to establish a management scenario for a given patient if one exists. The management scenario defines a safe (interaction-free) and preferred set of activities together with possible patient states. We illustrate the use of our framework with a clinical case study describing two patients who suffer from chronic kidney disease, hypertension, and atrial fibrillation, and who are managed according to CPGs for these diseases. While in this paper we are primarily concerned with the methodological aspects of mitigation, we also briefly discuss a high-level proof of concept implementation of the proposed framework in the form of a clinical decision support system (CDSS). The proposed mitigation CDSS "insulates" clinicians from the complexities of the FOL representations and provides semantically meaningful summaries of mitigation results. Ultimately we plan to implement the mitigation CDSS within our MET (Mobile Emergency Triage) decision support environment.

Using Semantic Components to Represent Dynamics of an Interdisciplinary Healthcare Team in a Multi-Agent Decision Support System.

In healthcare organizations, clinical workflows are executed by interdisciplinary healthcare teams (IHTs) that operate in ways that are difficult to manage. Responding to a need to support such teams, we designed and developed the MET4 multi-agent system that allows IHTs to manage patients according to presentation-specific clinical workflows. In this paper, we describe a significant extension of the MET4 system that allows for supporting rich team dynamics (understood as team formation, management and task-practitioner allocation), including selection and maintenance of the most responsible physician and more complex rules of selecting practitioners for the workflow tasks. In order to develop this extension, we introduced three semantic components: (1) a revised ontology describing concepts and relations pertinent to IHTs, workflows, and managed patients, (2) a set of behavioral rules describing the team dynamics, and (3) an instance base that stores facts corresponding to instances of concepts from the ontology and to relations between these instances. The semantic components are represented in first-order logic and they can be automatically processed using theorem proving and model finding techniques. We employ these techniques to find models that correspond to specific decisions controlling the dynamics of IHT. In the paper, we present the design of extended MET4 with a special focus on the new semantic components. We then describe its proof-of-concept implementation using the WADE multi-agent platform and the Z3 solver (theorem prover/model finder). We illustrate the main ideas discussed in the paper with a clinical scenario of an IHT managing a patient with chronic kidney disease.

Mitigation of adverse interactions in pairs of clinical practice guidelines using constraint logic programming.

We propose a new method to mitigate (identify and address) adverse interactions (drug-drug or drug-disease) that occur when a patient with comorbid diseases is managed according to two concurrently applied clinical practice guidelines (CPGs). A lack of methods to facilitate the concurrent application of CPGs severely limits their use in clinical practice and the development of such methods is one of the grand challenges for clinical decision support. The proposed method responds to this challenge. We introduce and formally define logical models of CPGs and other related concepts, and develop the mitigation algorithm that operates on these concepts. In the algorithm we combine domain knowledge encoded as interaction and revision operators using the constraint logic programming (CLP) paradigm. The operators characterize adverse interactions and describe revisions to logical models required to address these interactions, while CLP allows us to efficiently solve the logical models - a solution represents a feasible therapy that may be safely applied to a patient. The mitigation algorithm accepts two CPGs and available (likely incomplete) patient information. It reports whether mitigation has been successful or not, and on success it gives a feasible therapy and points at identified interactions (if any) together with the revisions that address them. Thus, we consider the mitigation algorithm as an alerting tool to support a physician in the concurrent application of CPGs that can be implemented as a component of a clinical decision support system. We illustrate our method in the context of two clinical scenarios involving a patient with duodenal ulcer who experiences an episode of transient ischemic attack.

Using graph rewriting to operationalize medical knowledge for the revision of concurrently applied clinical practice guidelines.

Clinical practice guidelines (CPGs) are patient management tools that synthesize medical knowledge into an actionable format. CPGs are disease specific with limited applicability to the management of complex patients suffering from multimorbidity. For the management of these patients, CPGs need to be augmented with secondary medical knowledge coming from a variety of knowledge repositories. The operationalization of this knowledge is key to increasing CPGs' uptake in clinical practice. In this work, we propose an approach to operationalizing secondary medical knowledge inspired by graph rewriting. We assume that the CPGs can be represented as task network models, and provide an approach for representing and applying codified medical knowledge to a specific patient encounter. We formally define revisions that model and mitigate adverse interactions between CPGs and we use a vocabulary of terms to instantiate these revisions. We demonstrate the application of our approach using synthetic and clinical examples. We conclude by identifying areas for future work with the vision of developing a theory of mitigation that will facilitate the development of comprehensive decision support for the management of multimorbid patients.

Why did AI get this one wrong? - Tree-based explanations of machine learning model predictions.

Increasingly complex learning methods such as boosting, bagging and deep learning have made ML models more accurate, but harder to interpret and explain, culminating in black-box machine learning models. Model developers and users alike are often presented with a trade-off between performance and intelligibility, especially in high-stakes applications like medicine. In the present article we propose a novel methodological approach for generating explanations for the predictions of a generic machine learning model, given a specific instance for which the prediction has been made. The method, named AraucanaXAI, is based on surrogate, locally-fitted classification and regression trees that are used to provide post-hoc explanations of the prediction of a generic machine learning model. Advantages of the proposed XAI approach include superior fidelity to the original model, ability to deal with non-linear decision boundaries, and native support to both classification and regression problems. We provide a packaged, open-source implementation of the AraucanaXAI method and evaluate its behaviour in a number of different settings that are commonly encountered in medical applications of AI. These include potential disagreement between the model prediction and physician's expert opinion and low reliability of the prediction due to data scarcity.

The Case Manager: An Agent Controlling the Activation of Knowledge Sources in a FHIR-Based Distributed Reasoning Environment.

BACKGROUND: Within the CAPABLE project the authors developed a multi-agent system that relies on a distributed architecture. The system provides cancer patients with coaching advice and supports their clinicians with suitable decisions based on clinical guidelines. OBJECTIVES: As in many multi-agent systems we needed to coordinate the activities of all agents involved. Moreover, since the agents share a common blackboard where all patients' data are stored, we also needed to implement a mechanism for the prompt notification of each agent upon addition of new information potentially triggering its activation. METHODS: The communication needs have been investigated and modeled using the HL7-FHIR (Health Level 7-Fast Healthcare Interoperability Resources) standard to ensure proper semantic interoperability among agents. Then a syntax rooted in the FHIR search framework has been defined for representing the conditions to be monitored on the system blackboard for activating each agent. RESULTS: The Case Manager (CM) has been implemented as a dedicated component playing the role of an orchestrator directing the behavior of all agents involved. Agents dynamically inform the CM about the conditions to be monitored on the blackboard, using the syntax we developed. The CM then notifies each agent whenever any condition of interest occurs. The functionalities of the CM and other actors have been validated using simulated scenarios mimicking the ones that will be faced during pilot studies and in production. CONCLUSION: The CM proved to be a key facilitator for properly achieving the required behavior of our multi-agent system. The proposed architecture may also be leveraged in many clinical contexts for integrating separate legacy services, turning them into a consistent telemedicine framework and enabling application reusability.

An eHealth App (CAPABLE) Providing Symptom Monitoring, Well-Being Interventions, and Educational Material for Patients With Melanoma Treated With Immune Checkpoint Inhibitors: Protocol for an Exploratory Intervention Trial.

BACKGROUND: Since treatment with immune checkpoint inhibitors (ICIs) is becoming standard therapy for patients with high-risk and advanced melanoma, an increasing number of patients experience treatment-related adverse events such as fatigue. Until now, studies have demonstrated the benefits of using eHealth tools to provide either symptom monitoring or interventions to reduce treatment-related symptoms such as fatigue. However, an eHealth tool that facilitates the combination of both symptom monitoring and symptom management in patients with melanoma treated with ICIs is still needed. OBJECTIVE: In this pilot study, we will explore the use of the CAPABLE (Cancer Patients Better Life Experience) app in providing symptom monitoring, education, and well-being interventions on health-related quality of life (HRQoL) outcomes such as fatigue and physical functioning, as well as patients' acceptance and usability of using CAPABLE. METHODS: This prospective, exploratory pilot study will examine changes in fatigue over time in 36 patients with stage III or IV melanoma during treatment with ICI using CAPABLE (a smartphone app and multisensory smartwatch). This cohort will be compared to a prospectively collected cohort of patients with melanoma treated with standard ICI therapy. CAPABLE will be used for a minimum of 3 and a maximum of 6 months. The primary endpoint in this study is the change in fatigue between baseline and 3 and 6 months after the start of treatment. Secondary end points include HRQoL outcomes, usability, and feasibility parameters. RESULTS: Study inclusion started in April 2023 and is currently ongoing. CONCLUSIONS: This pilot study will explore the effect, usability, and feasibility of CAPABLE in patients with melanoma during treatment with ICI. Adding the CAPABLE system to active treatment is hypothesized to decrease fatigue in patients with high-risk and advanced melanoma during treatment with ICIs compared to a control group receiving standard care. The Medical Ethics Committee NedMec (Amsterdam, The Netherlands) granted ethical approval for this study (reference number 22-981/NL81970.000.22). TRIAL REGISTRATION: ClinicalTrials.gov NCT05827289; https://clinicaltrials.gov/study/NCT05827289. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/49252.

MitPlan: A planning approach to mitigating concurrently applied clinical practice guidelines.

As the population ages, patients' complexity and the scope of their care is increasing. Over 60% of the population is 65 years of age or older and suffers from multi-morbidity, which is associated with two times as many patient-physician encounters. Yet clinical practice guidelines (CPGs) are developed to treat a single disease. To reconcile these two competing issues, previously we developed a framework for mitigation, i.e., identifying and addressing adverse interactions in multi-morbid patients managed according to multiple CPGs. That framework relies on first-order logic (FOL) to represent CPGs and secondary medical knowledge and FOL theorem proving to establish valid patient management plans. In the work presented here, we leverage our earlier research and simplify the mitigation process by representing it as a planning problem using the Planning Domain Definition Language (PDDL). This new framework, called MitPlan, identifies and addresses adverse interactions using durative planning actions that embody clinical actions (including medication administration and patient testing), supports a physician-defined length of planning horizons, and optimizes plans based on patient preferences and action costs. It supports a variety of criteria when developing management plans, including the total cost of prescribed treatment and the cost of the revisions to be introduced. The solution to MitPlan's planning problem is a sequence of timed actions that are easy to interpret when creating a management plan. We demonstrate MitPlan's capabilities using illustrative and clinical case studies.

A Health eLearning Ontology and Procedural Reasoning Approach for Developing Personalized Courses to Teach Patients about Their Medical Condition and Treatment.

We propose a methodological framework to support the development of personalized courses that improve patients' understanding of their condition and prescribed treatment. Inspired by Intelligent Tutoring Systems (ITSs), the framework uses an eLearning ontology to express domain and learner models and to create a course. We combine the ontology with a procedural reasoning approach and precompiled plans to operationalize a design across disease conditions. The resulting courses generated by the framework are personalized across four patient axes-condition and treatment, comprehension level, learning style based on the VARK (Visual, Aural, Read/write, Kinesthetic) presentation model, and the level of understanding of specific course content according to Bloom's taxonomy. Customizing educational materials along these learning axes stimulates and sustains patients' attention when learning about their conditions or treatment options. Our proposed framework creates a personalized course that prepares patients for their meetings with specialists and educates them about their prescribed treatment. We posit that the improvement in patients' understanding of prescribed care will result in better outcomes and we validate that the constructs of our framework are appropriate for representing content and deriving personalized courses for two use cases: anticoagulation treatment of an atrial fibrillation patient and lower back pain management to treat a lumbar degenerative disc condition. We conduct a mostly qualitative study supported by a quantitative questionnaire to investigate the acceptability of the framework among the target patient population and medical practitioners.

The Case Manager: Driving Medical Reasoning in a Distributed Environment for Home Patient Monitoring.

The CAPABLE project has been funded by the EU Horizon 2020 Programme over the years 2020-24 to support home care. A system is being designed and implemented supporting remote monitoring and virtual coaching for cancer patients. The system is based on a distributed modular architecture involving many components encapsulating various knowledge. The Case Manager has been designed as a separate component with the aim of coordinating the problem solving strategies. A first version of the Case Manager has been released and used by the components in a prototypical scenario shown at the first project review.

Towards a framework for comparing functionalities of multimorbidity clinical decision support: A literature-based feature set and benchmark cases.

Multimorbidity, the coexistence of two or more health conditions, has become more prevalent as mortality rates in many countries have declined and their populations have aged. Multimorbidity presents significant difficulties for Clinical Decision Support Systems (CDSS), particularly in cases where recommendations from relevant clinical guidelines offer conflicting advice. A number of research groups are developing computer-interpretable guideline (CIG) modeling formalisms that integrate recommendations from multiple Clinical Practice Guidelines (CPGs) for knowledge-based multimorbidity decision support. In this paper we describe work towards the development of a framework for comparing the different approaches to multimorbidity CIG-based clinical decision support (MGCDS). We present (1) a set of features for MGCDS, which were derived using a literature review and evaluated by physicians using a survey, and (2) a set of benchmarking case studies, which illustrate the clinical application of these features. This work represents the first necessary step in a broader research program aimed at the development of a benchmark framework that allows for standardized and comparable MGCDS evaluations, which will facilitate the assessment of functionalities of MGCDS, as well as highlight important gaps in the state-of-the-art. We also outline our future work on developing the framework, specifically, (3) a standard for reporting MGCDS solutions for the benchmark case studies, and (4) criteria for evaluating these MGCDS solutions. We plan to conduct a large-scale comparison study of existing MGCDS based on the comparative framework.

Automatic indexing and retrieval of encounter-specific evidence for point-of-care support.

Evidence-based medicine relies on repositories of empirical research evidence that can be used to support clinical decision making for improved patient care. However, retrieving evidence from such repositories at local sites presents many challenges. This paper describes a methodological framework for automatically indexing and retrieving empirical research evidence in the form of the systematic reviews and associated studies from The Cochrane Library, where retrieved documents are specific to a patient-physician encounter and thus can be used to support evidence-based decision making at the point of care. Such an encounter is defined by three pertinent groups of concepts - diagnosis, treatment, and patient, and the framework relies on these three groups to steer indexing and retrieval of reviews and associated studies. An evaluation of the indexing and retrieval components of the proposed framework was performed using documents relevant for the pediatric asthma domain. Precision and recall values for automatic indexing of systematic reviews and associated studies were 0.93 and 0.87, and 0.81 and 0.56, respectively. Moreover, precision and recall for the retrieval of relevant systematic reviews and associated studies were 0.89 and 0.81, and 0.92 and 0.89, respectively. With minor modifications, the proposed methodological framework can be customized for other evidence repositories.

MET3-AE system to support management of pediatric asthma exacerbation in the emergency department.

A decision making process behind the management of pediatric patients with asthma exacerbations in the Emergency Department includes three stages: data collection, diagnosis formulation and treatment planning. These stages are associated with activities involving different types of clinical knowledge: factual, conceptual and procedural. Effective decision support should span over the entire decision making process and facilitate the use of diversified clinical knowledge. In this paper we present MET3-AE - a point of care decision support system that satisfies this requirement. The system helps emergency physician collect data, evaluate exacerbation severity, plan corresponding treatment and retrieve clinical evidence associated with a given treatment plan. It was developed using ontology-driven and multi-agent methodologies and implemented with open source software. The system is accessible on tablet and desktop computers and smartphones, and it interacts with other hospital information systems. It was successfully verified in a simulated clinical setting and now it is undergoing testing in a teaching hospital.