A mixed methods analysis of existing assessment and evaluation tools (AETs) for mental health applications.

Introduction: Mental health Applications (MH Apps) can potentially improve access to high-quality mental health care. However, the recent rapid expansion of MH Apps has created growing concern regarding their safety and effectiveness, leading to the development of AETs (Assessment and Evaluation Tools) to help guide users. This article provides a critical, mixed methods analysis of existing AETs for MH Apps by reviewing the criteria used to evaluate MH Apps and assessing their effectiveness as evaluation tools. Methods: To identify relevant AETs, gray and scholarly literature were located through stakeholder consultation, Internet searching via Google and a literature search of bibliographic databases Medline, APA PsycInfo, and LISTA. Materials in English that provided a tool or method to evaluate MH Apps and were published from January 1, 2000, to January 26, 2021 were considered for inclusion. Results: Thirteen relevant AETs targeted for MH Apps met the inclusion criteria. The qualitative analysis of AETs and their evaluation criteria revealed that despite purporting to focus on MH Apps, the included AETs did not contain criteria that made them more specific to MH Apps than general health applications. There appeared to be very little agreed-upon terminology in this field, and the focus of selection criteria in AETs is often IT-related, with a lesser focus on clinical issues, equity, and scientific evidence. The quality of AETs was quantitatively assessed using the AGREE II, a standardized tool for evaluating assessment guidelines. Three out of 13 AETs were deemed 'recommended' using the AGREE II. Discussion: There is a need for further improvements to existing AETs. To realize the full potential of MH Apps and reduce stakeholders' concerns, AETs must be developed within the current laws and governmental health policies, be specific to mental health, be feasible to implement and be supported by rigorous research methodology, medical education, and public awareness.

Stakeholders' views and opinions on existing guidelines on "How to Choose Mental Health Apps".

Background: Mental health Applications (Mhealth Apps) can change how healthcare is delivered. However, very little is known about the efficacy of Mhealth Apps. Currently, only minimum guidance is available in Assessment and Evaluation Tools (AETs). Therefore, this project aims to understand AET developers' perspectives and end users' experiences and opinions on "how to choose a Mhealth App". Objective: The primary objectives were: (1) obtaining stakeholder's opinions and experiences of development and use of AETs for Mhealth Apps, their weaknesses and strengths, and barriers in their implementation of Mhealth Apps; (2) the experiences of App users, their analyzation and, obstacles in the use of apps; and (3) to quantify themes related to choosing a Mhealth App. Methods: This qualitative study, used a sampling method to recruit six stakeholders (one App developer, two AET developers, an individual with lived experience of mental health illness, and two physicians) who were interviewed using a topic guide. These were examined by researchers (CT, WK, & FN) using thematic content analysis. Additionally, an anonymous online survey of 107 individuals was conducted. Findings: Our analyses revealed six main themes: (a) needs and opportunities; (b) views on Mhealth apps; (c) views & opinions on AETs; (d) implementation barriers; (e) system of evaluation and; (f) future directions. The first key concept was, all stakeholders agreed that Apps could significantly impact mental health and that end-users were unaware of mental health AETs and Apps. Secondly, due to commercial interests, end-users reliability of App evaluations requires clear conflict-free guidelines. Thirdly, AETs should be evaluated and developed through a rigorous methodology. Finally, stakeholders shared insights into future developments for AETs and Mhealth Apps. Additionally, online survey respondents chose a "health professional" as their preferred source of guidance in selecting a Mhealth app (84%) and best suited to develop guidelines (70%). Conclusion: The interviews and survey highlight the need for Mhealth Apps to be regulated and the importance of health professionals' engagement in the implementation process. Similarly, without well-defined roles for App evaluations within the health care system, it is unlikely that AETs will have wider spread use and impact without risk.