BACKGROUND: The field of eHealth is growing rapidly and chaotically. Health care professionals need guidance on reviewing and assessing health-related smartphone apps to propose appropriate ones to their patients. However, to date, no framework or evaluation tool fulfills this purpose. OBJECTIVE: Before developing a tool to help health care professionals assess and recommend apps to their patients, we aimed to create an overview of published criteria to describe and evaluate health apps. METHODS: We conducted a systematic review to identify existing criteria for eHealth smartphone app evaluation. Relevant databases and trial registers were queried for articles. Articles were included that (1) described tools, guidelines, dimensions, or criteria to evaluate apps, (2) were available in full text, and (3) were written in English, French, German, Italian, Portuguese, or Spanish. We proposed a conceptual framework for app evaluation based on the dimensions reported in the selected articles. This was revised iteratively in discussion rounds with international stakeholders. The conceptual framework was used to synthesize the reported evaluation criteria. The list of criteria was discussed and refined by the research team. RESULTS: Screening of 1258 articles yielded 128 (10.17%) that met the inclusion criteria. Of these 128 articles, 30 (23.4%) reported the use of self-developed criteria and described their development processes incompletely. Although 43 evaluation instruments were used only once, 6 were used in multiple studies. Most articles (83/128, 64.8%) did not report following theoretical guidelines; those that did noted 37 theoretical frameworks. On the basis of the selected articles, we proposed a conceptual framework to explore 6 app evaluation dimensions: context, stakeholder involvement, features and requirements, development processes, implementation, and evaluation. After standardizing the definitions, we identified 205 distinct criteria. Through consensus, the research team relabeled 12 of these and added 11 more-mainly related to ethical, legal, and social aspects-resulting in 216 evaluation criteria. No criteria had to be moved between dimensions. CONCLUSIONS: This study provides a comprehensive overview of criteria currently used in clinical practice to describe and evaluate apps. This is necessary as no reviewed criteria sets were inclusive, and none included consistent definitions and terminology. Although the resulting overview is impractical for use in clinical practice in its current form, it confirms the need to craft it into a purpose-built, theory-driven tool. Therefore, in a subsequent step, based on our current criteria set, we plan to construct an app evaluation tool with 2 parts: a short section (including 1-3 questions/dimension) to quickly disqualify clearly unsuitable apps and a longer one to investigate more likely candidates in closer detail. We will use a Delphi consensus-building process and develop a user manual to prepare for this undertaking. TRIAL REGISTRATION: PROSPERO International Prospective Register of Systematic Reviews CRD42021227064; https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42021227064.
Mobile Applications , Telemedicine , Humans , Consensus , Databases, Factual
BACKGROUND: While effectiveness outcomes of eHealth-facilitated integrated care models (eICMs) in transplant and oncological populations are promising, implementing and sustaining them in real-world settings remain challenging. Allogeneic stem cell transplant (alloSCT) patients could benefit from an eICM to enhance health outcomes. To combat health deterioration, integrating chronic illness management, including continuous symptom and health behaviour monitoring, can shorten reaction times. We will test the 1st-year post-alloSCT effectiveness and evaluate bundled implementation strategies to support the implementation of a newly developed and adapted eICM in allogeneic stem cell transplantation facilitated by eHealth (SMILe-ICM). SMILe-ICM has been designed by combining implementation, behavioural, and computer science methods. Adaptions were guided by FRAME and FRAME-IS. It consists of four modules: 1) monitoring & follow-up; 2) infection prevention; 3) physical activity; and 4) medication adherence, delivered via eHealth and a care coordinator (an Advanced Practice Nurse). The implementation was supported by contextually adapted implementation strategies (e.g., creating new clinical teams, informing local opinion leaders). METHODS: Using a hybrid effectiveness-implementation randomised controlled trial, we will include a consecutive sample of 80 adult alloSCT patients who were transplanted and followed by University Hospital Basel (Switzerland). Inclusion criteria are basic German proficiency; elementary computer literacy; internet access; and written informed consent. Patients will be excluded if their condition prevents the use of technology, or if they are followed up only at external centres. Patient-level (1:1) stratified randomisation into a usual care group and a SMILe-ICM group will take place 10 days pre-transplantation. To gauge the SMILe-ICM's effectiveness primary outcome (re-hospitalisation rate), secondary outcomes (healthcare utilization costs; length of inpatient re-hospitalizations, medication adherence; treatment and self-management burden; HRQoL; Graft-versus-Host Disease rate; survival; overall survival rate) and implementation outcomes (acceptability, appropriateness, feasibility, fidelity), we will use multi-method, multi-informant assessment (via questionnaires, interviews, electronic health record data, cost capture methods). DISCUSSION: The SMILe-ICM has major innovative potential for reengineering alloSCT follow-up care, particularly regarding short- and medium-term outcomes. Our dual focus on implementation and effectiveness will both inform optimization of the SMILe-ICM and provide insights regarding implementation strategies and pathway, understudied in eHealth-facilitated ICMs in chronically ill populations. TRIAL REGISTRATION: ClinicalTrials.gov. Identifier: NCT04789863 . Registered April 01, 2021.
Delivery of Health Care, Integrated , Hematopoietic Stem Cell Transplantation , Self-Management , Telemedicine , Adult , Chronic Disease , Health Behavior , Humans , Randomized Controlled Trials as Topic
INTRODUCTION: An online interactive repository of available medication adherence technologies may facilitate their selection and adoption by different stakeholders. Developing a repository is among the main objectives of the European Network to Advance Best practices and technoLogy on medication adherencE (ENABLE) COST Action (CA19132). However, meeting the needs of diverse stakeholders requires careful consideration of the repository structure. METHODS AND ANALYSIS: A real-time online Delphi study by stakeholders from 39 countries with research, practice, policy, patient representation and technology development backgrounds will be conducted. Eleven ENABLE members from 9 European countries formed an interdisciplinary steering committee to develop the repository structure, prepare study protocol and perform it. Definitions of medication adherence technologies and their attributes were developed iteratively through literature review, discussions within the steering committee and ENABLE Action members, following ontology development recommendations. Three domains (product and provider information (D1), medication adherence descriptors (D2) and evaluation and implementation (D3)) branching in 13 attribute groups are proposed: product and provider information, target use scenarios, target health conditions, medication regimen, medication adherence management components, monitoring/measurement methods and targets, intervention modes of delivery, target behaviour determinants, behaviour change techniques, intervention providers, intervention settings, quality indicators and implementation indicators. Stakeholders will evaluate the proposed definition and attributes' relevance, clarity and completeness and have multiple opportunities to reconsider their evaluations based on aggregated feedback in real-time. Data collection will stop when the predetermined response rate will be achieved. We will quantify agreement and perform analyses of process indicators on the whole sample and per stakeholder group. ETHICS AND DISSEMINATION: Ethical approval for the COST ENABLE activities was granted by the Malaga Regional Research Ethics Committee. The Delphi protocol was considered compliant regarding data protection and security by the Data Protection Officer from University of Basel. Findings from the Delphi study will form the basis for the ENABLE repository structure and related activities.
Counseling , Medication Adherence , Delphi Technique , Europe , Humans , Technology
PURPOSE: Little is known about allogeneic stem cell transplant (alloSCT) patients' medication adherence strategies. Acceptability and preferences regarding electronic monitoring (EM) systems to assess all three phases of medication adherence (ie, initiation, implementation, persistence) are crucial to allow their successful implementation in clinical or research settings but have not yet been evaluated. We therefore aimed to explore: 1) alloSCT patients' medication adherence and self-management strategies; and 2) their acceptability and preferences of three different EM systems (MEMS Cap, Helping Hand, Button) as part of the Swiss SMILe study. PATIENTS AND METHODS: Respecting anti-pandemic measures, we used a purposive sample of six adult alloSCT patients from the University Hospital Basel, Switzerland (USB)-6 weeks to 2 years post-alloSCT-to conduct three focus group sessions with two patients each. Using a semi-structured outline, we explored 1) patients' medication adherence strategies and medication self-management; and 2) their acceptance and preferences regarding EM use. The three tested EM systems were available for testing during each session. Discussions were audio-recorded, visualized using mind-mapping and analyzed using Mayring's qualitative content analysis. RESULTS: Patients (33% females; mean age 54.6±16.3 years; 10.4±8.4 months post-alloSCT) used medication adherence enhancing strategies (eg, preparing pillbox, linking intake to a habit). Still, they indicated that post-alloSCT medication management was challenging (eg, frequent schedule changes). All participants preferred the MEMS Button. Participants said its small size and the possibility to combine it with existing pillboxes (eg, putting it into/next to them) made them more confident about implementing it in their daily lives. CONCLUSION: Regarding EM systems for medication adherence, end-user preferences and acceptability influence adoption and fidelity. Of the three systems tested, our sample found the MEMS Button most acceptable and most preferable. Therefore, we will use it for our USB SMILe study.
Background: Contextually adapting complex interventions and tailoring their implementation strategies is key to a successful and sustainable implementation. While reporting guidelines for adaptations and tailoring exist, less is known about how to conduct context-specific adaptations of complex health care interventions. Aims: To describe in methodological terms how the merging of contextual analysis results (step 1) with stakeholder involvement, and considering overarching regulations (step 2) informed our adaptation of an Integrated Care Model (ICM) for SteM cell transplantatIon faciLitated by eHealth (SMILe) and the tailoring of its implementation strategies (step 3). Methods: Step 1: We used a mixed-methods design at University Hospital Basel, guided by the Basel Approach for coNtextual ANAlysis (BANANA). Step 2: Adaptations of the SMILe-ICM and tailoring of implementation strategies were discussed with an interdisciplinary team (n = 28) by considering setting specific and higher-level regulatory scenarios. Usability tests were conducted with patients (n = 5) and clinicians (n = 4). Step 3: Adaptations were conducted by merging our results from steps 1 and 2 using the Framework for Reporting Adaptations and Modifications-Enhanced (FRAME). We tailored implementation strategies according to the Expert Recommendations for Implementing Change (ERIC) compilation. Results: Step 1: Current clinical practice was mostly acute-care-driven. Patients and clinicians valued eHealth-facilitated ICMs to support trustful patient-clinician relationships and the fitting of eHealth components to context-specific needs. Step 2: Based on information from project group meetings, adaptations were necessary on the organizational level (e.g., delivery of self-management information). Regulations informed the tailoring of SMILe-ICM`s visit timepoints and content; data protection management was adapted following Swiss regulations; and steering group meetings supported infrastructure access. The usability tests informed further adaptation of technology components. Step 3: Following FRAME and ERIC, SMILe-ICM and its implementation strategies were contextually adapted and tailored to setting-specific needs. Discussion: This study provides a context-driven methodological approach on how to conduct intervention adaptation including the tailoring of its implementation strategies. The revealed meso-, and macro-level differences of the contextual analysis suggest a more targeted approach to enable an in-depth adaptation process. A theory-guided adaptation phase is an important first step and should be sufficiently incorporated and budgeted in implementation science projects.
BACKGROUND: Growing numbers of randomized controlled trials (RCTs) are showing the effectiveness of interventions to improve medication adherence in transplantation recipients. However, real-world implementation is still a major challenge. This systematic review assesses the range of information available in RCTs supporting these interventions' clinical adoption in adult transplant populations. METHODS: We included RCTs of interventions that a) targeted any phase of medication adherence in solid organ or allogeneic stem cell transplantation recipients and b) were published between January 2015 and November 2020. We excluded study protocols, conference abstracts and studies focusing only on pediatric populations. We identified relevant database and trial registries as well as traced references backward and citations forward. Implementation-relevant information was evaluated using adapted versions of Peters' ten criteria: 1. healthcare/organizational context; 2. social/economic/policy context; 3. patient involvement; 4. other stakeholder involvement; 5. sample representativeness; 6. trial conducted in a real-world-setting; 7. presence of feasibility study; 8. implementation strategy; 9. process evaluation; 10. implementation outcomes, using a stoplight color-rating system. RESULTS: Screening 17'004 titles/abstracts resulted in 23 eligible RCTs, including 2'339 patients (n = 19-209/study). All included studies focused on the implementation phase of medication adherence. The best-reported criteria were feasibility study (43%), representative sample (17%) and conducted in a real-world-setting (17%). Least reported were context (9%), implementation strategies (4%), process evaluation (4%). CONCLUSIONS: RCTs testing medication adherence interventions tend to report limited implementation-relevant information. This hinders their translation to real-world transplant settings. Integrating implementation science principles early in the conceptualization of RCTs would fuel real-world-translation, reducing research waste.
Medication Adherence , Humans , Randomized Controlled Trials as Topic
PURPOSE: Allogeneic stem cell transplantation would benefit from re-engineering care towards an integrated eHealth-facilitated care model. With this paper we aim to: (1) describe the development of an integrated care model (ICM) in allogeneic SteM-cell-transplantatIon faciLitated by eHealth (SMILe) by combining implementation, behavioral, and computer science methods (e.g., contextual analysis, Behavior Change Wheel, and user-centered design combined with agile software development); and (2) describe that model's characteristics and its application in clinical practice. METHODS: The SMILe intervention's development consisted of four steps, with implementation science methods informing each: (1) planning its set-up within a theoretical foundation; (2) using behavioral science methods to develop the content; (3) choosing and developing its delivery method (human/technology) using behavioral and computer science methods; and (4) describing its characteristics and application in clinical practice. RESULTS: The SMILe intervention is embedded within the eHealth enhanced Chronic Care Model, entailing four self-management intervention modules, targeting monitoring and follow-up of important medical and symptom-related parameters, infection prevention, medication adherence, and physical activity. Interventions are delivered partly face-to-face by a care coordinator embedded within the transplant team, and partly via the SMILeApp that connects patients to the transplant team, who can monitor and rapidly respond to any relevant changes within 1 year post-transplant. CONCLUSION: This paper provides stepwise guidance on how implementation, behavioral, and computer science methods can be used to develop interventions aiming to improve care for stem cell transplant patients in real-world clinical settings. This new care model is currently being tested in a hybrid I effectiveness-implementation trial.
Hematopoietic Stem Cell Transplantation , Self-Management , Telemedicine , Humans , Implementation Science , Medication Adherence
BACKGROUND: Medication adherence to immunosuppressants in allogeneic stem cell transplantation (alloSCT) is essential to achieve favorable clinical outcomes (e.g. control of Graft-versus-Host Disease). Over 600 apps supporting medication adherence exist, yet they lack successful implementation and sustainable use likely because of lack of end-user involvement and theoretical underpinnings in their development and insufficient attention to implementation methods to support their use in real-life settings. Medication adherence has three phases: initiation, implementation and persistence. We report the theory-driven development of an intervention module to support medication adherence (implementation and persistence phase) in alloSCT outpatients as a first step for future digitization and implementation in clinical setting within the SMILe project (Development, implementation and testing of an integrated care model in allogeneic SteM cell transplantatIon faciLitated by eHealth). METHODS: We applied Michie's Behavior Change Wheel (BCW) and the Capability-Opportunity-Motivation and Behavior (COM-B) model using three suggested stages followed by one stage added by our team regarding preparation for digitization of the intervention: (I) Defining the problem in behavioral terms; (II) Identifying intervention options; (III) Identifying content and implementation options; (IV) SMILe Care Model Prototype Development. Scientific evidence, data from a contextual analysis and patients'/caregivers' and clinical experts' inputs were compiled to work through these steps. RESULTS: (I) Correct immunosuppressant taking and timing were defined as target behaviors. The intervention's focus was determined within the COM-B dimensions Capability (lack of knowledge, lack of routine), Opportunity (lack of cues, interruptions in daily routine) and Motivation (lack of problem solving, trivialization). (II) Five intervention functions were chosen, i.e. education, training, modelling, persuasion and enablement. (III) Twenty-four behavior change techniques were selected, e.g., goal setting, action planning and problem solving. (IV) Finally, seventeen user stories were developed to guide the SMILeApp's software development process. CONCLUSION: Our example on the theory-driven development of an intervention module in alloSCT delivered by eHealth and transplant team using a rigorous 3 + 1-stage approach based on BCW, COM-B and agile software development techniques, can be used as methodological guidance for other eHealth intervention developers. Our approach has the potential to enhance successful implementation and sustained use of eHealth solutions in real-life settings.
Hematopoietic Stem Cell Transplantation , Implementation Science , Medication Adherence , Telemedicine/methods , Transplantation Conditioning , Humans , Immunosuppressive Agents/therapeutic use
