Psychometric properties of two implementation measures: Normalization MeAsure Development questionnaire (NoMAD) and organizational readiness for implementing change (ORIC).

Background: Effective interventions need to be implemented successfully to achieve impact. Two theory-based measures exist for measuring the effectiveness of implementation strategies and monitor implementation progress. The Normalization MeAsure Development questionnaire (NoMAD) explores the four core concepts (Coherence, Cognitive Participation, Collective Action, Reflexive Monitoring) of the Normalization Process Theory. The Organizational Readiness for Implementing Change (ORIC) is based on the theory of Organizational Readiness for Change, measuring organization members' psychological and behavioral preparedness for implementing a change. We examined the measurement properties of the NoMAD and ORIC in a multi-national implementation effectiveness study. Method: Twelve mental health organizations in nine countries implemented Internet-based cognitive behavioral therapy (iCBT) for common mental disorders. Staff involved in iCBT service delivery (n = 318) participated in the study. Both measures were translated into eight languages using a standardized forward-backward translation procedure. Correlations between measures and subscales were estimated to examine convergent validity. The theoretical factor structures of the scales were tested using confirmatory factor analysis (CFA). Test-retest reliability was based on the correlation between scores at two time points 3 months apart. Internal consistency was assessed using Cronbach's alpha. Floor and ceiling effects were quantified using the proportion of zero and maximum scores. Results: NoMAD and ORIC measure related but distinct latent constructs. The CFA showed that the use of a total score for each measure is appropriate. The theoretical subscales of the NoMAD had adequate internal consistency. The total scale had high internal consistency. The total ORIC scale and subscales demonstrated high internal consistency. Test-retest reliability was suboptimal for both measures and floor and ceiling effects were absent. Conclusions: This study confirmed the psychometric properties of the NoMAD and ORIC in multi-national mental health care settings. While measuring on different but related aspects of implementation processes, the NoMAD and ORIC prove to be valid and reliable across different language settings.

Why was the study done?: Effective interventions need to be implemented successfully to achieve impact. Reliable measurement instruments are needed to determine if an implementation was successful or not. Two theory-based instruments exist for measuring the effectiveness of implementation strategies and monitor progress. The NoMAD measures aspects of normalization related to sense-making, willingness to implement, the work people do, and reflection. The Organizational Readiness for Implementing Change (ORIC) measures organization members' preparedness for implementing a change. What did we do?: This study examined whether the NoMAD and ORIC measure what they are supposed to measure. We translated the instruments from English to eight languages (Albanian, Danish, Dutch, French, German, Italian, and Spanish/Catalan) We applied various statistical methods to confirm the measurement properties, including correlations of scales, factor structures, test­retest reliability, consistency and floor and ceiling effects. 318 mental health professionals from nine countries participated in the study. What did we find?: For both instruments, total scores can be used as well as the subscale scores. Internal consistency for ORIC was high and for NoMAD adequate. Test­retest reliability was demonstrated, and floor and ceiling effects were rare. What does this mean?: NoMAD and ORIC are reliable instruments for measuring implementation processes and outcomes across mental health care settings in different countries and languages. They measure related but different aspects of implementation processes and outcomes. The measures are brief, and theory supported. However, more work is to be done on interpreting scores in relation to implementation success and regarding changes over time.

Towards an Implementation-STakeholder Engagement Model (I-STEM) for improving health and social care services.

BACKGROUND: The implementation science literature acknowledges a need for engagement of key stakeholders when designing, delivering and evaluating implementation work. To date, the literature reports minimal or focused stakeholder engagement, where stakeholders are engaged in either barrier identification and/or barrier prioritisation. This paper begins to answer calls from the literature for the development of tools and guidance to support comprehensive stakeholder engagement in implementation research and practice. The paper describes the systematic development of the Implementation-STakeholder Engagement Model (I-STEM) in the context of an international, large-scale empirical implementation study (ImpleMentAll) aimed at evaluating the effectiveness of a tailored implementation toolkit. The I-STEM is a sensitising tool that defines key considerations and activities for undertaking stakeholder engagement activities across an implementation process. METHODS: In-depth, semistructured interviews and observations were conducted with implementers who were tailoring implementation strategies to integrate and embed internet-based cognitive behavioural therapy (iCBT) services in 12 routine mental health care organisations in nine countries in Europe and Australia. The analytical process was informed by principles of first- and third-generation Grounded Theory, including constant comparative method. RESULTS: We conducted 55 interviews and observed 19 implementation-related activities (e.g., team meetings and technical support calls). The final outcome of our analysis is expressed in an initial version of the I-STEM, consisting of five interrelated concepts: engagement objectives, stakeholder mapping, engagement approaches, engagement qualities and engagement outcomes. Engagement objectives are goals that implementers plan to achieve by working with stakeholders in the implementation process. Stakeholder mapping involves identifying a range of organisations, groups or people who may be instrumental in achieving the engagement objectives. Engagement approaches define the type of work that is undertaken with stakeholders to achieve the engagement objectives. Engagement qualities define the logistics of the engagement approach. Lastly, every engagement activity may result in a range of engagement outcomes. CONCLUSION: The I-STEM represents potential avenues for substantial stakeholder engagement activity across key phases of an implementation process. It provides a conceptual model for the planning, delivery, evaluation and reporting of stakeholder engagement activities. The I-STEM is nonprescriptive and highlights the importance of a flexible, iterative approach to stakeholder engagement. It is developmental and will require application and validation across a range of implementation activities. PATIENT OR PUBLIC CONTRIBUTION: Patient contribution to ImpleMentAll trial was facilitated by GAMIAN-Europe at all stages-from grant development to dissemination. GAMIAN-Europe brings together a wide variety of patient representation organisations (local, regional and national) from almost all European countries. GAMIAN-Europe was involved in pilot testing the ItFits-toolkit and provided their views on the various aspects, including stakeholder engagement. Patients were also represented in the external advisory board providing support and advice on the design, conduct and interpretation of the wider project, including the development of the ItFits-toolkit. TRIAL REGISTRATION: ClinicalTrials.gov NCT03652883. Retrospectively registered on 29 August 2018.

Effectiveness of Self-guided Tailored Implementation Strategies in Integrating and Embedding Internet-Based Cognitive Behavioral Therapy in Routine Mental Health Care: Results of a Multicenter Stepped-Wedge Cluster Randomized Trial.

BACKGROUND: Internet-based cognitive behavioral therapy (iCBT) services for common mental health disorders have been found to be effective. There is a need for strategies that improve implementation in routine practice. One-size-fits-all strategies are likely to be ineffective. Tailored implementation is considered as a promising approach. The self-guided integrated theory-based Framework for intervention tailoring strategies toolkit (ItFits-toolkit) supports local implementers in developing tailored implementation strategies. Tailoring involves identifying local barriers; matching selected barriers to implementation strategies; developing an actionable work plan; and applying, monitoring, and adapting where necessary. OBJECTIVE: This study aimed to compare the effectiveness of the ItFits-toolkit with implementation-as-usual (IAU) in implementing iCBT services in 12 routine mental health care organizations in 9 countries in Europe and Australia. METHODS: A stepped-wedge cluster randomized trial design with repeated measures was applied. The trial period lasted 30 months. The primary outcome was the normalization of iCBT delivery by service providers (therapists, referrers, IT developers, and administrators), which was measured with the Normalization Measure Development as a proxy for implementation success. A 3-level linear mixed-effects modeling was applied to estimate the effects. iCBT service uptake (referral and treatment completion rates) and implementation effort (hours) were used as secondary outcomes. The perceived satisfaction (Client Satisfaction Questionnaire), usability (System Usability Scale), and impact of the ItFits-toolkit by implementers were used to assess the acceptability of the ItFits-toolkit. RESULTS: In total, 456 mental health service providers were included in this study. Compared with IAU, the ItFits-toolkit had a small positive statistically significant effect on normalization levels in service providers (mean 0.09, SD 0.04; P=.02; Cohen d=0.12). The uptake of iCBT by patients was similar to that of IAU. Implementers did not spend more time on implementation work when using the ItFits-toolkit and generally regarded the ItFits-toolkit as usable and were satisfied with it. CONCLUSIONS: The ItFits-toolkit performed better than the usual implementation activities in implementing iCBT services in routine practice. There is practical utility in the ItFits-toolkit for supporting implementers in developing and applying effective tailored implementation strategies. However, the effect on normalization levels among mental health service providers was small. These findings warrant modesty regarding the effectiveness of self-guided tailored implementation of iCBT services in routine practice. TRIAL REGISTRATION: ClinicalTrials.gov NCT03652883; https://clinicaltrials.gov/ct2/show/NCT03652883. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): RR2-10.1186/s13063-020-04686-4.

Feasibility of Digital Cognitive Behavioral Therapy for Depressed Older Adults With the Moodbuster Platform: Protocol for 2 Pilot Feasibility Studies.

BACKGROUND: Internet-based interventions can be effective in the treatment of depression. However, internet-based interventions for older adults with depression are scarce, and little is known about their feasibility and effectiveness. OBJECTIVE: To present the design of 2 studies aiming to assess the feasibility of internet-based cognitive behavioral treatment for older adults with depression. We will assess the feasibility of an online, guided version of the Moodbuster platform among depressed older adults from the general population as well as the feasibility of a blended format (combining integrated face-to-face sessions and internet-based modules) in a specialized mental health care outpatient clinic. METHODS: A single-group, pretest-posttest design will be applied in both settings. The primary outcome of the studies will be feasibility in terms of (1) acceptance and satisfaction (measured with the Client Satisfaction Questionnaire-8), (2) usability (measured with the System Usability Scale), and (3) engagement (measured with the Twente Engagement with eHealth Technologies Scale). Secondary outcomes include (1) the severity of depressive symptoms (measured with the 8-item Patient Health Questionnaire depression scale), (2) participant and therapist experience with the digital technology (measured with qualitative interviews), (3) the working alliance between patients and practitioners (from both perspectives; measured with the Working Alliance Inventory-Short Revised questionnaire), (4) the technical alliance between patients and the platform (measured with the Working Alliance Inventory for Online Interventions-Short Form questionnaire), and (5) uptake, in terms of attempted and completed modules. A total of 30 older adults with mild to moderate depressive symptoms (Geriatric Depression Scale 15 score between 5 and 11) will be recruited from the general population. A total of 15 older adults with moderate to severe depressive symptoms (Geriatric Depression Scale 15 score between 8 and 15) will be recruited from a specialized mental health care outpatient clinic. A mixed methods approach combining quantitative and qualitative analyses will be adopted. Both the primary and secondary outcomes will be further explored with individual semistructured interviews and synthesized descriptively. Descriptive statistics (reported as means and SDs) will be used to examine the primary and secondary outcome measures. Within-group depression severity will be analyzed using a 2-tailed, paired-sample t test to investigate differences between time points. The interviews will be recorded and analyzed using thematic analysis. RESULTS: The studies were funded in October 2019. Recruitment started in September 2022. CONCLUSIONS: The results of these pilot studies will show whether this platform is feasible for use by the older adult population in a blended, guided format in the 2 settings and will represent the first exploration of the size of the effect of Moodbuster in terms of decreased depressive symptoms. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): PRR1-10.2196/41445.

A Data-Driven Clustering Method for Discovering Profiles in the Dynamics of Major Depressive Disorder Using a Smartphone-Based Ecological Momentary Assessment of Mood.

Background: Although major depressive disorder (MDD) is characterized by a pervasive negative mood, research indicates that the mood of depressed patients is rarely entirely stagnant. It is often dynamic, distinguished by highs and lows, and it is highly responsive to external and internal regulatory processes. Mood dynamics can be defined as a combination of mood variability (the magnitude of the mood changes) and emotional inertia (the speed of mood shifts). The purpose of this study is to explore various distinctive profiles in real-time monitored mood dynamics among MDD patients in routine mental healthcare. Methods: Ecological momentary assessment (EMA) data were collected as part of the cross-European E-COMPARED trial, in which approximately half of the patients were randomly assigned to receive the blended Cognitive Behavioral Therapy (bCBT). In this study a subsample of the bCBT group was included (n = 287). As part of bCBT, patients were prompted to rate their current mood (on a 1-10 scale) using a smartphone-based EMA application. During the first week of treatment, the patients were prompted to rate their mood on three separate occasions during the day. Latent profile analyses were subsequently applied to identify distinct profiles based on average mood, mood variability, and emotional inertia across the monitoring period. Results: Overall, four profiles were identified, which we labeled as: (1) "very negative and least variable mood" (n = 14) (2) "negative and moderate variable mood" (n = 204), (3) "positive and moderate variable mood" (n = 41), and (4) "negative and highest variable mood" (n = 28). The degree of emotional inertia was virtually identical across the profiles. Conclusions: The real-time monitoring conducted in the present study provides some preliminary indications of different patterns of both average mood and mood variability among MDD patients in treatment in mental health settings. Such varying patterns were not found for emotional inertia.

Examining the Theoretical Framework of Behavioral Activation for Major Depressive Disorder: Smartphone-Based Ecological Momentary Assessment Study.

BACKGROUND: Behavioral activation (BA), either as a stand-alone treatment or as part of cognitive behavioral therapy, has been shown to be effective for treating depression. The theoretical underpinnings of BA derive from Lewinsohn et al's theory of depression. The central premise of BA is that having patients engage in more pleasant activities leads to them experiencing more pleasure and elevates their mood, which, in turn, leads to further (behavioral) activation. However, there is a dearth of empirical evidence about the theoretical framework of BA. OBJECTIVE: This study aims to examine the assumed (temporal) associations of the 3 constructs in the theoretical framework of BA. METHODS: Data were collected as part of the "European Comparative Effectiveness Research on Internet-based Depression Treatment versus treatment-as-usual" trial among patients who were randomly assigned to receive blended cognitive behavioral therapy (bCBT). As part of bCBT, patients completed weekly assessments of their level of engagement in pleasant activities, the pleasure they experienced as a result of these activities, and their mood over the course of the treatment using a smartphone-based ecological momentary assessment (EMA) application. Longitudinal cross-lagged and cross-sectional associations of 240 patients were examined using random intercept cross-lagged panel models. RESULTS: The analyses did not reveal any statistically significant cross-lagged coefficients (all P>.05). Statistically significant cross-sectional positive associations between activities, pleasure, and mood levels were identified. Moreover, the levels of engagement in activities, pleasure, and mood slightly increased over the duration of the treatment. In addition, mood seemed to carry over, over time, while both levels of engagement in activities and pleasurable experiences did not. CONCLUSIONS: The results were partially in accordance with the theoretical framework of BA, insofar as the analyses revealed cross-sectional relationships between levels of engagement in activities, pleasurable experiences deriving from these activities, and enhanced mood. However, given that no statistically significant temporal relationships were revealed, no conclusions could be drawn about potential causality. A shorter measurement interval (eg, daily rather than weekly EMA reports) might be more attuned to detecting potential underlying temporal pathways. Future research should use an EMA methodology to further investigate temporal associations, based on theory and how treatments are presented to patients. TRIAL REGISTRATION: ClinicalTrials.gov, NCT02542891, https://clinicaltrials.gov/ct2/show/NCT02542891; German Clinical Trials Register, DRKS00006866, https://tinyurl.com/ybja3xz7; Netherlands Trials Register, NTR4962, https://www.trialregister.nl/trial/4838; ClinicalTrials.Gov, NCT02389660, https://clinicaltrials.gov/ct2/show/NCT02389660; ClinicalTrials.gov, NCT02361684, https://clinicaltrials.gov/ct2/show/NCT02361684; ClinicalTrials.gov, NCT02449447, https://clinicaltrials.gov/ct2/show/NCT02449447; ClinicalTrials.gov, NCT02410616, https://clinicaltrials.gov/ct2/show/NCT02410616; ISRCTN registry, ISRCTN12388725, https://www.isrctn.com/ISRCTN12388725.

Discovering different profiles in the dynamics of depression based on real-time monitoring of mood: a first exploration.

BACKGROUND: Although depression is typically characterized by a persistent depressed mood, mood dynamics do seem to vary across a depressed population. Heterogeneity of mood variability (magnitude of changes) and emotional inertia (speed at which mood shifts) is seen in clinical practice. However, studies investigating the heterogeneity of these mood dynamics are still scarce. The aim of the present study is to explore different distinctive profiles in real-time monitored mood dynamics among depressed persons. METHODS: After completing baseline measures, mildly-to-moderately depressed persons (n = 37) were prompted to rate their current mood (1-10 scale) on their smartphones, 3 times a day for 7 consecutive days. Latent profile analyses were applied to identify profiles based on average mood, variability of mood and emotional inertia as reported by the participants. RESULTS: Two profiles were identified in this sample. The overwhelming majority of the sample belonged to profile 1 (n = 31). Persons in profile 1 were characterized by a mood just above the cutoff for positive mood (M = 6.27), with smaller mood shifts (lower variability [SD = 1.05]) than those in profile 2 (n = 6), who displayed an overall negative mood (M = 4.72) and larger mood shifts (higher variability [SD = 1.95]) but at similar speed (emotional inertia) (AC = 0.19, AC = 0.26, respectively). CONCLUSIONS: The present study provides preliminary indications for patterns of average mood and mood variability, but not emotional inertia, among mildly-to-moderately depressed persons.

Tailored implementation of internet-based cognitive behavioural therapy in the multinational context of the ImpleMentAll project: a study protocol for a stepped wedge cluster randomized trial.

BACKGROUND: Internet-based Cognitive Behavioural Therapy (iCBT) is found effective in treating common mental disorders. However, the use of these interventions in routine care is limited. The international ImpleMentAll study is funded by the European Union's Horizon 2020 programme. It is concerned with studying and improving methods for implementing evidence-based iCBT services for common mental disorders in routine mental health care. A digitally accessible implementation toolkit (ItFits-toolkit) will be introduced to mental health care organizations with the aim to facilitate the ongoing implementation of iCBT services within local contexts. This study investigates the effectiveness of the ItFits-toolkit by comparing it to implementation-as-usual activities. METHODS: A stepped wedge cluster randomized controlled trial (SWT) design will be applied. Over a trial period of 30 months, the ItFits-toolkit will be introduced sequentially in twelve routine mental health care organizations in primary and specialist care across nine countries in Europe and Australia. Repeated measures are applied to assess change over time in the outcome variables. The effectiveness of the ItFits-toolkit will be assessed in terms of the degree of normalization of the use of the iCBT services. Several exploratory outcomes including uptake of the iCBT services will be measured to feed the interpretation of the primary outcome. Data will be collected via a centralized data collection system and analysed using generalized linear mixed modelling. A qualitative process evaluation of routine implementation activities and the use of the ItFits-toolkit will be conducted within this study. DISCUSSION: The ImpleMentAll study is a large-scale international research project designed to study the effectiveness of tailored implementation. Using a SWT design that allows to examine change over time, this study will investigate the effect of tailored implementation on the normalization of the use of iCBT services and their uptake. It will provide a better understanding of the process and methods of tailoring implementation strategies. If found effective, the ItFits-toolkit will be made accessible for mental health care service providers, to help them overcome their context-specific implementation challenges. TRIAL REGISTRATION: ClinicalTrials.gov NCT03652883 . Retrospectively registered on 29 August 2018.

Experienced Burden of and Adherence to Smartphone-Based Ecological Momentary Assessment in Persons with Affective Disorders.

(1) Background: The use of smartphone-based ecological momentary assessment (EMA) questionnaires in affective disorder research has rapidly increased. Though, a thorough understanding of experienced burden of and adherence to EMA is crucial in determining the usefulness of EMA. (2) Methods: Persons with current affective disorders (n = 100), remitted persons (n = 190), and healthy controls (n = 94) participated in a smartphone-based EMA two-week monitoring period. Our primary outcomes were (momentary) perceived burden of and adherence to EMA. (3) Results: In the whole sample, lower positive and higher negative affect were associated with slightly higher levels of perceived momentary burden (B = -0.23 [95%CI = -0.27-0.19], B = 0.30 [95%CI = 0.24-0.37], respectively). The persons with current affective disorders reported slightly higher levels of experienced momentary burden (Mdn = 1.98 [IQR = 1.28-2.57]), than the remitted persons (Mdn = 1.64 [IQR = 1.11-2.24]) and healthy controls (Mdn = 1.28 [IQR = 1.04-1.92]). Nevertheless, the persons with current affective disorders still showed very high adherence rates (Mdn = 94.3% [IQR = 87.9-97.1]), at rates on a par with the remitted persons (Mdn = 94.3% [IQR = 90.0-97.1]) and healthy controls (Mdn = 94.3% [IQR = 90.0-98.6]). (4) Discussion: Frequent momentary questionnaires of mental well-being are slightly more burdensome to the persons with current affective disorders, but this does not seem to have a negative impact on adherence. Their high rate of adherence to EMA-which was similar to that in remitted persons and healthy controls -suggests that it is feasible to apply (short-duration) EMA.

Why Uptake of Blended Internet-Based Interventions for Depression Is Challenging: A Qualitative Study on Therapists' Perspectives.

(1) Background: Blended cognitive behavioral therapy (bCBT; online and face-to-face sessions) seems a promising alternative alongside regular face-to-face CBT depression treatment in specialized mental health care organizations. Therapists are key in the uptake of bCBT. This study focuses on therapists' perspectives on usability, satisfaction, and factors that promote or hinder the use of bCBT in routine practice; (2) Methods: Three focus groups (n = 8, n = 7, n = 6) and semi-structured in-depth interviews (n = 15) were held throughout the Netherlands. Beforehand, the participating therapists (n = 36) completed online questionnaires on usability and satisfaction. Interviews were analyzed by thematic analysis; (3) Results: Therapists found the usability sufficient and were generally satisfied with providing bCBT. The thematic analysis showed three main themes on promoting and hindering factors: (1) therapists' needs regarding bCBT uptake, (2) therapists' role in motivating patients for bCBT, and (3) therapists' experiences with bCBT; (4) Conclusions: Overall, therapists were positive; bCBT can be offered by all CBT-trained therapists and future higher uptake is expected. Especially the pre-set structure of bCBT was found beneficial for both therapists and patients. Nevertheless, therapists did not experience promised time-savings-rather, the opposite. Besides, there are still teething problems and therapeutic shortcomings that need improvement in order to motivate therapists to use bCBT.