More than just tracking time: Complex measures of user engagement with an internet-based health promotion intervention.

BACKGROUND: There has been a rise in internet-based health interventions without a concomitant focus on new methods to measure user engagement and its effect on outcomes. We describe current user tracking methods for internet-based health interventions and offer suggestions for improvement based on the design and pilot testing of healthMpowerment.org (HMP). METHODS: HMP is a multi-component online intervention for young Black men and transgender women who have sex with men (YBMSM/TW) to reduce risky sexual behaviors, promote healthy living and build social support. The intervention is non-directive, incorporates interactive features, and utilizes a point-based reward system. Fifteen YBMSM/TW (age 20-30) participated in a one-month pilot study to test the usability and efficacy of HMP. Engagement with the intervention was tracked using a customized data capture system and validated with Google Analytics. Usage was measured in time spent (total and across sections) and points earned. RESULTS: Average total time spent on HMP was five hours per person (range 0-13). Total time spent was correlated with total points earned and overall site satisfaction. CONCLUSION: Measuring engagement in internet-based interventions is crucial to determining efficacy. Multiple methods of tracking helped derive more comprehensive user profiles. Results highlighted the limitations of measures to capture user activity and the elusiveness of the concept of engagement.

Achieving HIV risk reduction through HealthMpowerment.org, a user-driven eHealth intervention for young Black men who have sex with men and transgender women who have sex with men.

Young, Black men who have sex with men and transgender women who have sex with men (YBMSM/TW) are at disproportionate risk for HIV and other sexually transmitted infections (HIV/STI). HealthMpowerment.org (HMP) is a mobile phone optimised online intervention that utilises behaviour change and gaming theories to reduce risky sexual behaviours and build community among HIV-positive and negative YBMSM/TW. The intervention is user-driven, provides social support, and utilises a point reward system. A four-week pilot trial was conducted with a diverse group of 15 YBMSM/TW. During exit interviews, participants described how HMP components led to behaviour changes such as asking partners' sexual history, increased condom use, and HIV/STI testing. The user-driven structure, interactivity, and rewards appeared to facilitate sustained user engagement and the mobile platform provided relevant information in real-time. Participants described the reward elements of exceeding their previous scores and earning points toward prizes as highly motivating. HMP showed promise for being able to deliver a sufficient intervention dose and we found a trend toward higher dose received and more advanced stages of behaviour change. In this pilot trial, HMP was well accepted and demonstrates promise for translating virtual intervention engagement into actual behaviour change to reduce HIV risk behaviours.