Engagement and retention: measuring breadth and depth of participant use of an online intervention.

BACKGROUND: The Internet provides us with tools (user metrics or paradata) to evaluate how users interact with online interventions. Analysis of these paradata can lead to design improvements. OBJECTIVE: The objective was to explore the qualities of online participant engagement in an online intervention. We analyzed the paradata in a randomized controlled trial of alternative versions of an online intervention designed to promote consumption of fruit and vegetables. METHODS: Volunteers were randomized to 1 of 3 study arms involving several online sessions. We created 2 indirect measures of breadth and depth to measure different dimensions and dynamics of program engagement based on factor analysis of paradata measures of Web pages visited and time spent online with the intervention materials. Multiple regression was used to assess influence of engagement on retention and change in dietary intake. RESULTS: Baseline surveys were completed by 2513 enrolled participants. Of these, 86.3% (n = 2168) completed the follow-up surveys at 3 months, 79.6% (n = 2027) at 6 months, and 79.4% (n = 1995) at 12 months. The 2 tailored intervention arms exhibited significantly more engagement than the untailored arm (P < .01). Breadth and depth measures of engagement were significantly associated with completion of follow-up surveys (odds ratios [OR] = 4.11 and 2.12, respectively, both P values < .001). The breadth measure of engagement was also significantly positively associated with a key study outcome, the mean increase in fruit and vegetable consumption (P < .001). CONCLUSIONS: By exploring participants' exposures to online interventions, paradata are valuable in explaining the effects of tailoring in increasing participant engagement in the intervention. Controlling for intervention arm, greater engagement is also associated with retention of participants and positive change in a key outcome of the intervention, dietary change. This paper demonstrates the utility of paradata capture and analysis for evaluating online health interventions. TRIAL REGISTRATION: NCT00169312; http://clinicaltrials.gov/ct2/show/NCT00169312 (Archived by WebCite at http://www.webcitation.org/5u8sSr0Ty).

Identifying and oversampling Hispanics by the Passel-Word surname list for enrollment in a web-based nutritional intervention.

OBJECTIVE: To describe the enrollment rates and characteristics of Hispanics and non-Hispanics from Kaiser Permanente Colorado invited to participate in a web-based intervention promoting increased fruit and vegetable consumption. DESIGN: Hispanics were identified by the Passel-Word Spanish surname list. Characteristics associated with the likelihood of enrollment overall and by ethnicity were examined by logistic regression. RESULTS: A total of 174 (6.1%) probable Hispanics and 340 probable non-Hispanics (11.8%) enrolled. Hispanics were 48% less likely to enroll than non-Hispanics, females were almost four times as likely to enroll as males, and those living in a census tract associated with higher income levels were 41% more likely to enroll than other income groups. Among Hispanics, females were 87% more likely to enroll than males and those living in a census tract associated with higher income levels were 62% more likely to enroll than other income groups. Among non-Hispanics, the odds for enrolling increased 14% for each decade increase of age, females were 43% more likely to enroll than males and those living in a census tract associated with higher income levels were 68% more likely to enroll than those in other income groups. CONCLUSION: Identifying Hispanics through surname for oversampling can be successful in terms of sampling yield and accuracy. However, our results suggest that Hispanics are less likely to enroll in a web-based nutritional intervention. Additional research is needed to identify methods of attracting more Hispanic subjects to these kinds of interventions.

A randomized clinical trial evaluating online interventions to improve fruit and vegetable consumption.

OBJECTIVES: We assessed change in fruit and vegetable intake in a population-based sample, comparing an online untailored program (arm 1) with a tailored behavioral intervention (arm 2) and with a tailored behavioral intervention plus motivational interviewing-based counseling via e-mail (arm 3). METHODS: We conducted a randomized controlled intervention trial, enrolling members aged 21 to 65 years from 5 health plans in Seattle, Washington; Denver, Colorado; Minneapolis, Minnesota; Detroit, Michigan; and Atlanta, Georgia. Participants reported fruit and vegetable intake at baseline and at 3, 6, and 12 months. We assessed mean change in fruit and vegetable servings per day at 12 months after baseline, using a validated self-report fruit and vegetable food frequency questionnaire. RESULTS: Of 2540 trial participants, 80% were followed up at 12 months. Overall baseline mean fruit and vegetable intake was 4.4 servings per day. Average servings increased by more than 2 servings across all study arms (P<.001), with the greatest increase (+2.8 servings) among participants of arm 3 (P=.05, compared with control). Overall program satisfaction was high. CONCLUSIONS: This online nutritional intervention was well received, convenient, easy to disseminate, and associated with sustained dietary change. Such programs have promise as population-based dietary interventions.

Recruitment to a randomized web-based nutritional intervention trial: characteristics of participants compared to non-participants.

BACKGROUND: Web-based behavioral programs efficiently disseminate health information to a broad population, and online tailoring may increase their effectiveness. While the number of Internet-based behavioral interventions has grown in the last several years, additional information is needed to understand the characteristics of subjects who enroll in these interventions, relative to those subjects who are invited to enroll. OBJECTIVE: The aim of the study was to compare the characteristics of participants who enrolled in an online dietary intervention trial (MENU) with those who were invited but chose not to participate, in order to better understand how these groups differ. METHODS: The MENU trial was conducted among five health plans participating in the HMO Cancer Research Network in collaboration with the University of Michigan Center for Health Communication Research. Approximately 6000 health plan members per site, between the ages of 21 and 65, and stratified by gender with oversampling of minority populations, were randomly selected for recruitment and were mailed an invitation letter containing website information and a US$2 bill with the promise of US$20 for completing follow-up surveys. Administrative and area-based data using geocoding along with baseline survey data were used to compare invitees (HMO members sent the introductory letter), responders (those who entered a study ID on the website), and enrollees (those who completed the enrollment process). Generalized estimating equation multivariate and logistic regression models were used to assess predictors of response and enrollment. RESULTS: Of 28,460 members invited to participate, 4270 (15.0%) accessed the website. Of the eligible responders, 2540 (8.9%) completed the consent form and baseline survey and were enrolled and randomized. The odds of responding were 10% lower for every decade of increased age (P < .001), while the likelihood of enrolling was 10% higher for every decade increase in age (P < .001). Women were more likely to respond and to enroll (P < .001). Those living in a census tract associated with higher education levels were more likely to respond and enroll, as well as those residing in tracts with higher income (P < .001). With a 22% (n = 566) enrollment rate for African Americans and 8% (n = 192) for Hispanics, the enrolled sample was more racially and ethnically diverse than the background sampling frame. CONCLUSIONS: Relative to members invited to participate in the Internet-based intervention, those who enrolled were more likely to be older and live in census tracts associated with higher socioeconomic status. While oversampling of minority health plan members generated an enrolled sample that was more racially and ethnically diverse than the overall health plan population, additional research is needed to better understand methods that will expand the penetration of Internet interventions into more socioeconomically diverse populations. TRIAL REGISTRATION: Clinicaltrials.gov NCT00169312; http://clinicaltrials.gov/ct2/show/NCT00169312 (Archived by WebCite at http://www.webcitation.org/5jB50xSfU).

A comparison of smokers' and nonsmokers' fruit and vegetable intake and relevant psychosocial factors.

The authors examined the relation between smoking status and fruit and vegetable (FV) consumption among a population-based sample and examined differences in psychosocial factors that may influence diet and inform intervention efforts. The authors recruited adults (N = 2,540) from 5 US health plans to participate in a Web-based dietary intervention trial. At baseline, smokers ate fewer FV servings per day (p < .001) and were less likely to meet the 5 A Day goal (p < .001). Smokers reported lower self-efficacy, overall motivation, and intrinsic motivation for meeting daily FV recommendations. Fewer smokers expected that eating 5 FV servings a day would reduce their risk for diabetes (p = .02) or obesity (p = .008). Smokers are an important target group for dietary intervention. Intervention efforts should attempt to increase smokers' motivation and confidence in their abilities to change their eating patterns and educate them about the health benefits of eating FV.

Effect of incentives and mailing features on online health program enrollment.

BACKGROUND: With the growing use of Internet-based interventions, strategies are needed to encourage broader participation. This study examined the effects of combinations of monetary incentives and mailing characteristics on enrollment, retention, and cost effectiveness for an online health program. METHODS: In 2004, a recruitment letter was mailed to randomly selected Midwestern integrated health system members aged 21-65 and stratified by gender and race/ethnicity; recipients were randomly pre-assigned to one of 24 combinations of incentives and various mailing characteristics. Enrollment and 3-month retention rates were measured by completion of online surveys. Analysis, completed in 2005, compared enrollment and retention factors using t tests and chi-square tests. Multivariate logistic regression modeling assessed the probability of enrollment and retention. RESULTS: Of 12,289 subjects, 531 (4.3%) enrolled online, ranging from 1% to 11% by incentive combination. Highest enrollment occurred with unconditional incentives, and responses varied by gender. Retention rates ranged from 0% to 100%, with highest retention linked to higher-value incentives. The combination of a $2 bill prepaid incentive and the promise of $20 for retention (10% enrollment and 71% retention) was optimal, considering per-subject recruitment costs ($32 enrollment, $70 retention) and equivalent enrollment by gender and race/ethnicity. CONCLUSIONS: Cash incentives improved enrollment in an online health program. Men and women responded differently to mailing characteristics and incentives. Including a small prepaid monetary incentive ($2 or $5) and revealing the higher promised-retention incentive was cost effective and boosted enrollment.