The Priorities of End Users of Emergency Department Electronic Health Records: Modified Delphi Study.

BACKGROUND: The needs of the emergency department (ED) pose unique challenges to modern electronic health record (EHR) systems. A diverse case load of high-acuity, high-complexity presentations, and ambulatory patients, all requiring multiple transitions of care, creates a rich environment through which to critically examine EHRs. OBJECTIVE: This investigation aims to capture and analyze the perspective of end users of EHR about the strengths, limitations, and future priorities for EHR in the setting of the ED. METHODS: In the first phase of this investigation, a literature search was conducted to identify 5 key usage categories of ED EHRs. Using key usage categories in the first phase, a modified Delphi study was conducted with a group of 12 panelists with expertise in both emergency medicine and health informatics. Across 3 rounds of surveys, panelists generated and refined a list of strengths, limitations, and key priorities. RESULTS: The findings from this investigation highlighted the preference of panelists for features maximizing functionality of basic clinical features relative to features of disruptive innovation. CONCLUSIONS: By capturing the perspectives of end users in the ED, this investigation highlights areas for the improvement or development of future EHRs in acute care settings.

A qualitative assessment of barriers to healthcare and HIV prevention services among men who have sex with men in non-metropolitan areas of the south.

HIV cases are increasing in the rural Southern United States, especially among men who have sex with men (MSM). To facilitate healthcare access and encourage HIV prevention for non-metropolitan MSM, it is essential to examine their barriers to care. This qualitative study conducted semi-structured interviews with 20 MSM living in non-metropolitan areas of the South. Analysis revealed that MSM experience multiple barriers accessing healthcare in non-metropolitan areas such as finding knowledgeable and affirming providers with desired characteristics and beliefs and communicating with providers about sexual health and HIV prevention. To aid in identification, many respondents expressed a desire for providers to publicly signal that they provide care for sexual and gender minority patients and are an inclusive clinical space. Overall, results suggest that MSM face unique healthcare-related challenges, beyond those typically experienced by the broader population in non-metropolitan areas, because of tailored identity-based needs. To better support MSM in non-metropolitan areas, especially in the South where increased experiences of stigma are found, providers should seek further training regarding sexual health communication and HIV prevention, indicate on websites and in offices that they support sexual and gender minority patients, and provide telehealth services to MSM living in more geographically isolated areas.

Interoperability: A National Person-Centred View.

We respond to Falk's (2022) recent recommendations around interoperability. Although we applaud the notion that the health data of Canadians be usable - that is, machine interpretable - and widely clinically available, Falk's (2022) recommendations must be extended. Specifically, interoperability must centre on the person and people's rights to hold and control a usable copy of their health information for their own purposes. Also, we must acknowledge that Canada is a small global market and avoid internal fragmentation with competing jurisdictional standards. Our national strategy must align with major trading partners. To do otherwise will disenfranchise Canadians and our digital health innovators.

Virtual Specialist Care During the COVID-19 Pandemic: Multimethod Patient Experience Study.

BACKGROUND: Transitioning nonemergency, ambulatory medical care to virtual visits in light of the COVID-19 global pandemic has been a massive shift in philosophy and practice that naturally came with a steep learning curve for patients, physicians, and clinic administrators. OBJECTIVE: We undertook a multimethod study to understand the key factors associated with successful and less successful experiences of virtual specialist care, particularly as they relate to the patient experience of care. METHODS: This study was designed as a multimethod patient experience study using survey methods, descriptive qualitative interview methodology, and administrative virtual care data collected by the hospital decision support team. Six specialty departments participated in the study (endoscopy, orthopedics, neurology, hematology, rheumatology, and gastroenterology). All patients who could speak and read English and attended a virtual specialist appointment in a participating clinic at St. Michael's Hospital (Toronto, Ontario, Canada) between October 1, 2020, and January 30, 2021, were eligible to participate. RESULTS: During the study period, 51,702 virtual specialist visits were conducted in the departments that participated in the study. Of those, 96% were conducted by telephone and 4% by video. In both the survey and interview data, there was an overall consensus that virtual care is a satisfying alternative to in-person care, with benefits such as reduced travel, cost, time, and SARS-CoV-2 exposure, and increased convenience. Our analysis further revealed that the specific reason for the visit and the nature and status of the medical condition are important considerations in terms of guidance on where virtual care is most effective. Technology issues were not reported as a major challenge in our data, given that the majority of "virtual" visits reported by our participants were conducted by telephone, which is an important distinction. Despite the positive value of virtual care discussed by the majority of interview participants, 50% of the survey respondents still indicated they would prefer to see their physician in person. CONCLUSIONS: Patient experience data collected in this study indicate a high level of satisfaction with virtual specialty care, but also signal that there are nuances to be considered to ensure it is an appropriate and sustainable part of the standard of care.

Understanding Engagement and the Potential Impact of an Electronic Drug Repository: Multi-Methods Study.

BACKGROUND: Centralized drug repositories can reduce adverse events and inappropriate prescriptions by enabling access to dispensed medication data at the point of care; however, how they achieve this goal is largely unknown. OBJECTIVE: This study aims to understand the perceived clinical value; the barriers to and enablers of adoption; and the clinician groups for which a provincial, centralized drug repository may provide the most benefit. METHODS: A mixed methods approach, including a web-based survey and semistructured interviews, was used. Participants were clinicians (eg, nurses, physicians, and pharmacists) in Ontario who were eligible to use the digital health drug repository (DHDR), irrespective of actual use. Survey data were ranked on a 7-point adjectival scale and analyzed using descriptive statistics, and interviews were analyzed using qualitative descriptions. RESULTS: Of the 161 survey respondents, only 40 (24.8%) actively used the DHDR. Perceptions of the utility of the DHDR were neutral (mean scores ranged from 4.11 to 4.76). Of the 75.2% (121/161) who did not use the DHDR, 97.5% (118/121) rated access to medication information (eg, dose, strength, and frequency) as important. Reasons for not using the DHDR included the cumbersome access process and the perception that available data were incomplete or inaccurate. Of the 33 interviews completed, 26 (79%) were active DHDR users. The DHDR was a satisfactory source of secondary information; however, the absence of medication instructions and prescribed medications (which were not dispensed) limited its ability to provide a comprehensive profile to meaningfully support clinical decision-making. CONCLUSIONS: Digital drug repositories must be adjusted to align with the clinician's needs to provide value. Ensuring integration with point-of-care systems, comprehensive clinical data, and streamlined onboarding processes would optimize clinically meaningful use. The electronic provision of accessible drug information to providers across health care settings has the potential to improve efficiency and reduce medication errors.

High proportions of rectal and pharyngeal chlamydia and gonorrhoea cases among cisgender men are missed using current CDC screening recommendations.

OBJECTIVES: Pharyngeal and rectal Chlamydia trachomatis (CT) and Neisseria gonorrhoeae (NG) infections are often undiagnosed due to their asymptomatic nature. This study aims to determine (1) the prevalence of CT/NG infections by anatomical site among cisgender men; (2) the proportion of missed CT/NG rectal/pharyngeal infections if urogenital testing alone was performed or screening depended on self-reported behaviour alone; and (3) the predictive probability of self-reported behaviours for rectal CT/NG. METHODS: This cross-sectional study used electronic health records collected at a sexual health clinic in Los Angeles from 18 November 2018 until 28 February 2020. The included patients were ≥18 years of age cisgender men who received CT/NG testing at least once during the study period. We calculated the proportion of missed pharyngeal/rectal CT/NG infections if only urogenital testing had been done and if testing was based only on self-reported anal sex. Separately, we ran logistic regressions for predictive probability of self-reported anal sex on CT/NG rectal infections. RESULTS: Overall, there were 13 476 unique patients with 26 579 visits. The prevalence of any extragenital CT/NG infection was 37.28%. Over 80% rectal/pharyngeal CT cases and over 65% rectal/pharyngeal NG cases would be missed if urogenital testing alone was performed. Likewise, over 35% rectal CT/NG cases would be missed had testing relied on self-reported sexual behaviours alone. CONCLUSIONS: The proportion of missed rectal and pharyngeal CT/NG infections is high. Our data from a sexual health clinic lend support to three-site opt-out testing for cisgender men attending a sexual health/Lesbian, Gay, Bisexual, Transgender, Queer/Questioning (LGBTQ+) specialty clinic regardless of their sexual orientation or reported sexual behaviours.

UpToDate versus DynaMed: a cross-sectional study comparing the speed and accuracy of two point-of-care information tools.

OBJECTIVE: To compare the accuracy, time to answer, user confidence, and user satisfaction between UpToDate and DynaMed (formerly DynaMed Plus), which are two popular point-of-care information tools. METHODS: A crossover study was conducted with medical residents in obstetrics and gynecology and family medicine at the University of Toronto in order to compare the speed and accuracy with which they retrieved answers to clinical questions using UpToDate and DynaMed. Experiments took place between February 2017 and December 2019. Following a short tutorial on how to use each tool and completion of a background survey, participants attempted to find answers to two clinical questions in each tool. Time to answer each question, the chosen answer, confidence score, and satisfaction score were recorded for each clinical question. RESULTS: A total of 57 residents took part in the experiment, including 32 from family medicine and 25 from obstetrics and gynecology. Accuracy in clinical answers was equal between UpToDate (average 1.35 out of 2) and DynaMed (average 1.36 out of 2). However, time to answer was 2.5 minutes faster in UpToDate compared to DynaMed. Participants were also more confident and satisfied with their answers in UpToDate compared to DynaMed. CONCLUSIONS: Despite a preference for UpToDate and a higher confidence in responses, the accuracy of clinical answers in UpToDate was equal to those in DynaMed. Previous exposure to UpToDate likely played a major role in participants' preferences. More research in this area is recommended.

Implementation Strategies to Improve Engagement With a Multi-Institutional Patient Portal: Multimethod Study.

BACKGROUND: Comprehensive multi-institutional patient portals that provide patients with web-based access to their data from across the health system have been shown to improve the provision of patient-centered and integrated care. However, several factors hinder the implementation of these portals. Although barriers and facilitators to patient portal adoption are well documented, there is a dearth of evidence examining how to effectively implement multi-institutional patient portals that transcend traditional boundaries and disparate systems. OBJECTIVE: This study aims to explore how the implementation approach of a multi-institutional patient portal impacted the adoption and use of the technology and to identify the lessons learned to guide the implementation of similar patient portal models. METHODS: This multimethod study included an analysis of quantitative and qualitative data collected during an evaluation of the multi-institutional MyChart patient portal that was deployed in Southwestern Ontario, Canada. Descriptive statistics were performed to understand the use patterns during the first 15 months of implementation (between August 2018 and October 2019). In addition, 42 qualitative semistructured interviews were conducted with 18 administrative stakeholders, 16 patients, 7 health care providers, and 1 informal caregiver to understand how the implementation approach influenced user experiences and to identify strategies for improvement. Qualitative data were analyzed using an inductive thematic analysis approach. RESULTS: Between August 2018 and October 2019, 15,271 registration emails were sent, with 67.01% (10,233/15,271) registered for an account across 38 health care sites. The median number of patients registered per site was 19, with considerable variation (range 1-2114). Of the total number of sites, 55% (21/38) had ≤30 registered patients, whereas only 2 sites had over 1000 registered patients. Interview participants perceived that the patient experience of the portal would have been improved by enhancing the data comprehensiveness of the technology. They also attributed the lack of enrollment to the absence of a broad rollout and marketing strategy across sites. Participants emphasized that provider engagement, change management support, and senior leadership endorsement were central to fostering uptake. Finally, many stated that regional alignment and policy support should have been sought to streamline implementation efforts across participating sites. CONCLUSIONS: Without proper management and planning, multi-institutional portals can suffer from minimal adoption. Data comprehensiveness is the foundational component of these portals and requires aligned policies and a key base of technology infrastructure across all participating sites. It is important to look beyond the category of the technology (ie, patient portal) and consider its functionality (eg, data aggregation, appointment scheduling, messaging) to ensure that it aligns with the underlying strategic priorities of the deployment. It is also critical to establish a clear vision and ensure buy-ins from organizational leadership and health care providers to support a cultural shift that will enable a meaningful and widespread engagement.

A Clinical Communication Tool (Loop) for Team-Based Care in Pediatric and Adult Care Settings: Hybrid Mixed Methods Implementation Study.

BACKGROUND: Communication within the circle of care is central to coordinated, safe, and effective care; yet patients, caregivers, and health care providers often experience poor communication and fragmented care. Through a sequential program of research, the Loop Research Collaborative developed a web-based, asynchronous clinical communication system for team-based care. Loop assembles the circle of care centered on a patient, in private networking spaces called Patient Loops. The patient, their caregiver, or both are part of the Patient Loop. The communication is threaded, it can be filtered and sorted in multiple ways, it is securely stored, and can be exported for upload to a medical record. OBJECTIVE: The objective of this study was to implement and evaluate Loop. The study reporting adheres to the Standards for Reporting Implementation Research. METHODS: The study was a hybrid type II mixed methods design to simultaneously evaluate Loop's clinical and implementation effectiveness, and implementation barriers and facilitators in 6 health care sites. Data included monthly user check-in interviews and bimonthly surveys to capture patient or caregiver experience of continuity of care, in-depth interviews to explore barriers and facilitators based on the Consolidated Framework for Implementation Research (CFIR), and Loop usage extracted directly from the Loop system. RESULTS: We recruited 25 initiating health care providers across 6 sites who then identified patients or caregivers for recruitment. Of 147 patient or caregiver participants who were assessed and met screening criteria, 57 consented and 52 were enrolled on Loop, creating 52 Patient Loops. Across all Patient Loops, 96 additional health care providers consented to join the Loop teams. Loop usage was followed for up to 8 months. The median number of messages exchanged per team was 1 (range 0-28). The monthly check-in and CFIR interviews showed that although participants acknowledged that Loop could potentially fill a gap, existing modes of communication, workflows, incentives, and the lack of integration with the hospital electronic medical records and patient portals were barriers to its adoption. While participants acknowledged Loop's potential value for engaging the patient and caregiver, and for improving communication within the patient's circle of care, Loop's relative advantage was not realized during the study and there was insufficient tension for change. Missing data limited the analysis of continuity of care. CONCLUSIONS: Fundamental structural and implementation challenges persist toward realizing Loop's potential as a shared system of asynchronous communication. Barriers include health information system integration; system, organizational, and individual tension for change; and a fee structure for health care provider compensation for asynchronous communication.

Evidence supporting the standardisation of extragenital gonorrhoea and chlamydia screenings for women.

OBJECTIVE: Current guidelines for women do not include extragenital screening for Chlamydia trachomatis (CT) and Neisseria gonorrhoeae (NG) and do not mention anal sex behaviour. The objective of this cross-sectional study was to determine the number of potentially missed CT and NG cases by relying on urogenital screening and self-reported anal sex behaviour among women. METHODS: Demographic and clinical data of 4658 women attending a community health centre in Los Angeles, California, USA from 2015 to 2018 were examined. CT and NG were detected using nucleic acid amplification test (APTIMA Combo 2, Hologic Gen-Probe, San Diego, California). Demographic and behavioural factors were also examined to assess potentially missed NG/CT cases. Multivariable regression analyses were used to determine whether reported anal sex behaviour predicts NG/CT rectal infection. RESULTS: A total of 193 NG cases and 552 CT cases were identified; however, 53.9% of NG cases and 25.5% of CT cases were identified exclusively through extragenital screening. Of all positive cases of rectal CT, 87.0% did not report anal sex without a condom and 91.3% did not report any anal sex with their last sexual partner. Of all positive cases of rectal NG, 78.9% did not report anal sex without a condom and 76.3% did not report any anal sex with their last sexual partner. Anal sex with last partner was not predictive of NG/CT rectal infection. CONCLUSIONS: Relying solely on urogenital screening and reported behaviour misses NG/CT cases. Extragenital NG/CT screening should be conducted in all women regardless of reported anal sex behaviour.

Catalyzing Digital Health Innovation in Ontario: The Role of an Academic Medical Centre.

Overcoming barriers to health system innovation is an ongoing challenge in Canada. A total of 51 participants attended a digital health symposium in October 2017 to discuss the role of an academic medical centre (AMC) in advancing innovation. The conversation centred around (i) the current state of innovation in healthcare; (ii) the need for an innovation catalyst; and (iii) the roadmap for an AMC to drive change. AMCs can address the barriers to digital health innovation in Canada by providing a centralized network and infrastructure that supports innovation throughout its journey from "bench to bedside" as well as supporting educational reform.

Barriers to Creating Scalable Business Models for Digital Health Innovation in Public Systems: Qualitative Case Study.

BACKGROUND: Health systems are increasingly looking toward the private sector to provide digital solutions to address health care demands. Innovation in digital health is largely driven by small- and medium-sized enterprises (SMEs), yet these companies experience significant barriers to entry, especially in public health systems. Complex and fragmented care models, alongside a myriad of relevant stakeholders (eg, purchasers, providers, and producers of health care products), make developing value propositions for digital solutions highly challenging. OBJECTIVE: This study aims to identify areas for health system improvement to promote the integration of innovative digital health technologies developed by SMEs. METHODS: This paper qualitatively analyzes a series of case studies to identify health system barriers faced by SMEs developing digital health technologies in Canada and proposed solutions to encourage a more innovative ecosystem. The Women's College Hospital Institute for Health System Solutions and Virtual Care established a consultation program for SMEs to help them increase their innovation capacity and take their ideas to market. The consultation involved the SME filling out an onboarding form and review of this information by an expert advisory committee using guided considerations, leading to a recommendation report provided to the SME. This paper reports on the characteristics of 25 SMEs who completed the program and qualitatively analyzed their recommendation reports to identify common barriers to digital health innovation. RESULTS: A total of 2 central themes were identified, each with 3 subthemes. First, a common barrier to system integration was the lack of formal evaluation, with SMEs having limited resources and opportunities to conduct such an evaluation. Second, the health system's current structure does not create incentives for clinicians to use digital technologies, which threatens the sustainability of SMEs' business models. SMEs faced significant challenges in engaging users and payers from the public system due to perverse economic incentives. Physicians are compensated by in-person visits, which actively works against the goals of many digital health solutions of keeping patients out of clinics and hospitals. CONCLUSIONS: There is a significant disconnect between the economic incentives that drive clinical behaviors and the use of digital technologies that would benefit patients' well-being. To encourage the use of digital health technologies, publicly funded health systems need to dedicate funding for the evaluation of digital solutions and streamlined pathways for clinical integration.

Utility and Perceived Value of a Provincial Digital Diagnostic Imaging Repository: Multimethod Study.

BACKGROUND: Timely and comprehensive diagnostic image sharing across institutional and regional boundaries can produce multiple benefits while supporting integrated models of care. In Ontario, Canada, the Diagnostic Imaging Common Service (DICS) was created as a centralized imaging repository to enable the sharing and viewing of diagnostic images and associated reports across hospital-based and community-based clinicians throughout the province. OBJECTIVE: The aims of this study were as follows: (1) to explore real-world utilization and perceived clinical value of the DICS following the provision of system-wide access and (2) to identify strategies to optimize the technology platform functionality and encourage adoption. METHODS: This multimethod study included semistructured interviews with physicians and administrative stakeholders and descriptive analysis of the current DICS usage data. RESULTS: In this study, 41 participants were interviewed, that is, 34 physicians and 7 administrative stakeholders. The following 4 key themes emerged: (1) utilization of the DICS depended on the awareness of the technology and the preferred channels for accessing images, which varied widely, (2) clinical responsibilities and available institutional resources were the drivers of utilization (or lack thereof), (3) centralized image repositories were perceived to offer value at the patient, clinician, and health care system levels, and (4) the enabling factors to realize value included aspects of technology infrastructure (ie, available functionality) alongside policy supports. High-volume DICS usage was not evenly distributed throughout the province. CONCLUSIONS: Suboptimal adoption of the DICS was driven by poor awareness and variations in the clinical workflow. Alignment with physician workflow, policy supports, and investment in key technological features and infrastructure would improve functionality and data comprehensiveness, thereby optimizing health system performance, patient and provider experience, population health, and health care costs.

Quality Gaps of Electronic Health Records in Diabetes Care.

OBJECTIVES: Electronic health records (EHRs) allow standardized data capture for robust quality and performance assessments, but data quality issues may exist. This study compared extracted structured EHR data with chart review from endocrinologists' health-care records at a large, academic ambulatory hospital in Toronto, Ontario. METHODS: Consecutive chart review for the first 10 patients with either type 1 or type 2 diabetes seen between January 1, 2015 and March 1, 2016 was sampled for each of the 10 endocrinologists within the diabetes program, and electronic data extraction was also completed for a core set of structured diabetes care elements. Fisher exact test was used to determine differences in data availability between extracted EHR data and chart review. RESULTS: Out of 100 charts, there was significant under-representation of care elements using EHR extraction compared with chart review for glycated hemoglobin (45% vs 100%; p<0.0001), low-density lipoprotein (36% vs 89%; p<0.0001), urine albumin to creatinine ratio (38% vs 86%; p<0.0001), insulin use (20% vs 72%; p<0.0001), glomerular filtration rate (42% vs 87%, p<0.0001) and use of oral hypoglycemic agent (14% vs 64%; p<0.0001), respectively. EHR data extraction could not be obtained for driving status, hypoglycemia and driving counselling, smoking cessation counselling, family planning counselling and eye examination documentation. CONCLUSIONS: There is poor accuracy of using EHR data extraction during the early adoption phase to understand diabetes care, limiting EHR-based quality assessments.

Artificial Intelligence and the Implementation Challenge.

BACKGROUND: Applications of artificial intelligence (AI) in health care have garnered much attention in recent years, but the implementation issues posed by AI have not been substantially addressed. OBJECTIVE: In this paper, we have focused on machine learning (ML) as a form of AI and have provided a framework for thinking about use cases of ML in health care. We have structured our discussion of challenges in the implementation of ML in comparison with other technologies using the framework of Nonadoption, Abandonment, and Challenges to the Scale-Up, Spread, and Sustainability of Health and Care Technologies (NASSS). METHODS: After providing an overview of AI technology, we describe use cases of ML as falling into the categories of decision support and automation. We suggest these use cases apply to clinical, operational, and epidemiological tasks and that the primary function of ML in health care in the near term will be decision support. We then outline unique implementation issues posed by ML initiatives in the categories addressed by the NASSS framework, specifically including meaningful decision support, explainability, privacy, consent, algorithmic bias, security, scalability, the role of corporations, and the changing nature of health care work. RESULTS: Ultimately, we suggest that the future of ML in health care remains positive but uncertain, as support from patients, the public, and a wide range of health care stakeholders is necessary to enable its meaningful implementation. CONCLUSIONS: If the implementation science community is to facilitate the adoption of ML in ways that stand to generate widespread benefits, the issues raised in this paper will require substantial attention in the coming years.

Linking Quality Improvement and Health Information Technology through the QI-HIT Figure 8.

The implementation of health information technology (HIT) is complex. A method for mitigating complexity is incrementalism. Incrementalism forms the foundation of both incremental software development models, like agile, and the Plan-Do-Study-Act cycles (PDSAs) of quality improvement (QI), yet we often fail to be incremental at the union of the disciplines. We propose a new model for HIT implementation that explicitly links incremental software development cycles with PDSAs, the QI-HIT Figure 8 (QIHIT-F8). We then detail a subsequent local HIT implementation where we demonstrated its use. The QIHIT-F8 requires a reprioritization of project management activities around tests of change, strong QI principles to detect these changes, and the presence of both baseline and prospective data about the chosen indicators. These conditions are most likely to be present when applied to indicators of high strategic importance to an organization.

A Web-Based Mental Health Platform for Individuals Seeking Specialized Mental Health Care Services: Multicenter Pragmatic Randomized Controlled Trial.

BACKGROUND: Web-based self-directed mental health applications are rapidly emerging to address health service gaps and unmet needs for information and support. OBJECTIVE: The aim of this study was to determine if a multicomponent, moderated Web-based mental health application could benefit individuals with mental health symptoms severe enough to warrant specialized mental health care. METHODS: A multicenter, pragmatic randomized controlled trial was conducted across several outpatient mental health programs affiliated with 3 hospital programs in Ontario, Canada. Individuals referred to or receiving treatment, aged 16 years or older, with access to the internet and an email address, and having the ability to navigate a Web-based mental health application were eligible. A total of 812 participants were randomized 2:1 to receive immediate (immediate treatment group, ITG) or delayed (delayed treatment group, DTG) access for 3 months to the Big White Wall (BWW), a multicomponent Web-based mental health intervention based in the United Kingdom and New Zealand. The primary outcome was the total score on the Recovery Assessment Scale, revised (RAS-r) which measures mental health recovery. Secondary outcomes were total scores on the Patient Health Questionnaire-9 item (PHQ-9), the Generalized Anxiety Disorder Questionnaire-7 item (GAD-7), the EuroQOL 5-dimension quality of life questionnaire (EQ-5D-5L), and the Community Integration Questionnaire. An exploratory analysis examined the association between actual BWW use (categorized into quartiles) and outcomes among study completers. RESULTS: Intervention participants achieved small, statistically significant increases in adjusted RAS-r score (4.97 points, 95% CI 2.90 to 7.05), and decreases in PHQ-9 score (-1.83 points, 95% CI -2.85 to -0.82) and GAD-7 score (-1.55 points, 95% CI -2.42 to -0.70). Follow-up was achieved for 55% (446/812) at 3 months, 48% (260/542) of ITG participants and 69% (186/270) of DTG participants. Only 58% (312/542) of ITG participants logged on more than once. Some higher BWW user groups had significantly greater improvements in PHQ-9 and GAD-7 relative to the lowest use group. CONCLUSIONS: The Web-based application may be beneficial; however, many participants did not engage in an ongoing way. This has implications for patient selection and engagement as well as delivery and funding structures for similar Web-based interventions. TRIAL REGISTRATION: ClinicalTrials.gov NCT02896894; https://clinicaltrials.gov/ct2/show/NCT02896894 (Archived by WebCite at http://www.webcitation.org/78LIpnuRO).