Evaluating use of web-based interventions: an example of a Dutch sexual health intervention.

With the current increase in web-based interventions, the question of how to measure, and consequently improve engagement in such interventions is gaining more importance. Modern day web analytics tools make it easy to monitor use of web-based interventions. However, in this article, we propose that it would be more meaningful to first examine how the developers envisioned the use of the intervention to establish behavior change (i.e. intended use), before looking into how the intervention is ultimately used with web analytics (i.e. actual use). Such an approach responds to the regularly expressed concern that behavioral interventions are often poorly described, leading to less meaningful evaluations as it is not clear what exactly is being evaluated. Using a page on chlamydia prevention (104 557 pageviews in 2020) from a Dutch sexual health intervention (Sense), we demonstrate the value of acyclic behavior change diagrams (ABCDs) as a method to visualize intended use of an intervention. ABCDs show at a glance how behavior change principles are applied in an intervention and target determinants of behavior. Based on this ABCD, we investigate actual use of the intervention, using web analytics tool Matomo. Despite being intended to stimulate STI-testing, only 14% of the 35 347 transfers from this page led to the STI-testing page and a high bounce rate (79%) and relatively high exit rate were reported (69%). Recommendations to further interpret the data are given. This real-life example demonstrates the potential of combining ABCDs and Matomo as methods to gain insight into use of web-based interventions.

Acyclic behavior change diagrams: a tool to report and analyze interventions.

Background: Behavior change interventions have a vital role in enhancing human health and well-being. Nevertheless, concerns have been raised about suboptimal reporting of behavior change interventions, making analyses, replications, and intervention re-use hard or impossible to conduct. Objective: This paper introduces acyclic behavior change diagrams (ABCDs) to achieve more transparent development, evaluation, and reporting of behavior change interventions. ABCDs are a visual representation of the assumptions regarding causal-structural chains that underlie putative active ingredients of behavior change interventions. These causal-structural chains link the behavior change principles that are applied in an intervention to the (determinants of) behavior targeted in that intervention. Conclusions: ABCDs are helpful in making implicit assumptions explicit and help communicate assumptions with team members and other stakeholders. Moreover, we believe they make evaluation easier, and their machine-readability allows for ABCDs to be imported directly into (systematic review) databases with negligible costs while disclosing complete and accurate data. Finally, the ABCD approach fits well with other initiatives to gain a deeper understanding and synthesis of the literature on active intervention elements.