The Implementation of Recommender Systems for Mental Health Recovery Narratives: Evaluation of Use and Performance.

BACKGROUND: Recommender systems help narrow down a large range of items to a smaller, personalized set. NarraGive is a first-in-field hybrid recommender system for mental health recovery narratives, recommending narratives based on their content and narrator characteristics (using content-based filtering) and on narratives beneficially impacting other similar users (using collaborative filtering). NarraGive is integrated into the Narrative Experiences Online (NEON) intervention, a web application providing access to the NEON Collection of recovery narratives. OBJECTIVE: This study aims to analyze the 3 recommender system algorithms used in NarraGive to inform future interventions using recommender systems for lived experience narratives. METHODS: Using a recently published framework for evaluating recommender systems to structure the analysis, we compared the content-based filtering algorithm and collaborative filtering algorithms by evaluating the accuracy (how close the predicted ratings are to the true ratings), precision (the proportion of the recommended narratives that are relevant), diversity (how diverse the recommended narratives are), coverage (the proportion of all available narratives that can be recommended), and unfairness (whether the algorithms produce less accurate predictions for disadvantaged participants) across gender and ethnicity. We used data from all participants in 2 parallel-group, waitlist control clinical trials of the NEON intervention (NEON trial: N=739; NEON for other [eg, nonpsychosis] mental health problems [NEON-O] trial: N=1023). Both trials included people with self-reported mental health problems who had and had not used statutory mental health services. In addition, NEON trial participants had experienced self-reported psychosis in the previous 5 years. Our evaluation used a database of Likert-scale narrative ratings provided by trial participants in response to validated narrative feedback questions. RESULTS: Participants from the NEON and NEON-O trials provided 2288 and 1896 narrative ratings, respectively. Each rated narrative had a median of 3 ratings and 2 ratings, respectively. For the NEON trial, the content-based filtering algorithm performed better for coverage; the collaborative filtering algorithms performed better for accuracy, diversity, and unfairness across both gender and ethnicity; and neither algorithm performed better for precision. For the NEON-O trial, the content-based filtering algorithm did not perform better on any metric; the collaborative filtering algorithms performed better on accuracy and unfairness across both gender and ethnicity; and neither algorithm performed better for precision, diversity, or coverage. CONCLUSIONS: Clinical population may be associated with recommender system performance. Recommender systems are susceptible to a wide range of undesirable biases. Approaches to mitigating these include providing enough initial data for the recommender system (to prevent overfitting), ensuring that items can be accessed outside the recommender system (to prevent a feedback loop between accessed items and recommended items), and encouraging participants to provide feedback on every narrative they interact with (to prevent participants from only providing feedback when they have strong opinions).

Effectiveness and cost-effectiveness of online recorded recovery narratives in improving quality of life for people with non-psychotic mental health problems: a pragmatic randomized controlled trial.

Narratives describing first-hand experiences of recovery from mental health problems are widely available. Emerging evidence suggests that engaging with mental health recovery narratives can benefit people experiencing mental health problems, but no randomized controlled trial has been conducted as yet. We developed the Narrative Experiences Online (NEON) Intervention, a web application providing self-guided and recommender systems access to a collection of recorded mental health recovery narratives (n=659). We investigated whether NEON Intervention access benefited adults experiencing non-psychotic mental health problems by conducting a pragmatic parallel-group randomized trial, with usual care as control condition. The primary endpoint was quality of life at week 52 assessed by the Manchester Short Assessment (MANSA). Secondary outcomes were psychological distress, hope, self-efficacy, and meaning in life at week 52. Between March 9, 2020 and March 26, 2021, we recruited 1,023 participants from across England (the target based on power analysis was 994), of whom 827 (80.8%) identified as White British, 811 (79.3%) were female, 586 (57.3%) were employed, and 272 (26.6%) were unemployed. Their mean age was 38.4±13.6 years. Mood and/or anxiety disorders (N=626, 61.2%) and stress-related disorders (N=152, 14.9%) were the most common mental health problems. At week 52, our intention-to-treat analysis found a significant baseline-adjusted difference of 0.13 (95% CI: 0.01-0.26, p=0.041) in the MANSA score between the intervention and control groups, corresponding to a mean change of 1.56 scale points per participant, which indicates that the intervention increased quality of life. We also detected a significant baseline-adjusted difference of 0.22 (95% CI: 0.05-0.40, p=0.014) between the groups in the score on the "presence of meaning" subscale of the Meaning in Life Questionnaire, corresponding to a mean change of 1.1 scale points per participant. We found an incremental gain of 0.0142 quality-adjusted life years (QALYs) (95% credible interval: 0.0059 to 0.0226) and a £178 incremental increase in cost (95% credible interval: -£154 to £455) per participant, generating an incremental cost-effectiveness ratio of £12,526 per QALY compared with usual care. This was lower than the £20,000 per QALY threshold used by the National Health Service in England, indicating that the intervention would be a cost-effective use of health service resources. In the subgroup analysis including participants who had used specialist mental health services at baseline, the intervention both reduced cost (-£98, 95% credible interval: -£606 to £309) and improved QALYs (0.0165, 95% credible interval: 0.0057 to 0.0273) per participant as compared to usual care. We conclude that the NEON Intervention is an effective and cost-effective new intervention for people experiencing non-psychotic mental health problems.

Harvest for health gardening intervention feasibility study in cancer survivors.

BACKGROUND: Cancer survivors are at increased risk for second malignancies, cardiovascular disease, diabetes, and functional decline. Evidence suggests that a healthful diet and physical activity may reduce the risk of chronic disease and improve health in this population. METHODS: We conducted a feasibility study to evaluate a vegetable gardening intervention that paired 12 adult and child cancer survivors with Master Gardeners to explore effects on fruit and vegetable intake, physical activity, quality-of-life, and physical function. Throughout the year-long study period, the survivor-Master Gardener dyads worked together to plan/plant three gardens, harvest/rotate plantings, and troubleshoot/correct problems. Data on diet, physical activity, and quality-of-life were collected via surveys; anthropometrics and physical function were objectively measured. Acceptability of the intervention was assessed with a structured debriefing survey. RESULTS: The gardening intervention was feasible (robust enrollment; minimal attrition) and well-received by cancer survivors and Master Gardeners. Improvement in three of four objective measures of strength, agility, and endurance was observed in 90% of survivors, with the following change scores [median (interquartile range)] noted between baseline and one-year follow-up: hand grip test [+ 4.8 (3.0, 6.7) kg], 2.44 meter Get-Up-and-Go [+ 1.0 (+ 1.8, + 0.2) seconds], 30-second chair stand [+ 3.0 (+ 1.0, 5.0) stands], and six-minute walk [+ 11.6 (6.1, 48.8) meters]. Increases of ≥ 1 fruit and vegetable serving/day and ≥ 30 minutes/week of physical activity were observed in 40% and 60%, respectively. CONCLUSION: These preliminary results support the feasibility and acceptability of a mentored gardening intervention and suggest that it may offer a novel and promising strategy to improve fruit and vegetable consumption, physical activity, and physical function in cancer survivors. A larger randomized controlled trial is needed to confirm our results.

Detailed methods of two home-based vegetable gardening intervention trials to improve diet, physical activity, and quality of life in two different populations of cancer survivors.

BACKGROUND: Cancer survivors suffer from long-term adverse effects that reduce health-related quality of life (QOL) and physical functioning, creating an urgent need to develop effective, durable, and disseminable interventions. Harvest for Health, a home-based vegetable gardening intervention, holds promise for these domains. METHODS: This report describes the methods and recruitment experiences from two randomized controlled feasibility trials that employ a waitlist-controlled design. Delivered in partnership with Cooperative Extension Master Gardeners, this intervention provides one-on-one mentorship of cancer survivors in planning and maintaining three seasonal vegetable gardens over 12months. The primary aim is to determine intervention feasibility and acceptability; secondary aims are to explore effects on objective and subjective measures of diet, physical activity and function, and QOL and examine participant factors associated with potential effects. One trial is conducted exclusively among 82 female breast cancer survivors residing in the Birmingham, AL metropolitan area (BBCS); another broadly throughout Alabama among 46 older cancer survivors aged >60 (ASCS). RESULTS: Response rates were 32.6% (BBCS) and 52.3% (ASCS). Both trials exceeded 80% of their accrual target. Leading reasons for ineligibility were removal of >10 lymph nodes (lymphedema risk factor), lack of physician approval, and unwillingness to be randomized to the waitlist. CONCLUSION: To date, recruitment and implementation of Harvest for Health appears feasible. DISCUSSION: Although both studies encountered recruitment challenges, lessons learned can inform future larger-scale studies. Vegetable gardening interventions are of interest to cancer survivors and may provide opportunities to gain life skills leading to improvements in overall health and QOL.