Impact of a Web-Based Electronic Health Record on Behavioral Health Service Delivery for Children and Adolescents: Randomized Controlled Trial.

BACKGROUND: Electronic health records (EHRs) have been widely proposed as a mechanism for improving health care quality. However, rigorous research on the impact of EHR systems on behavioral health service delivery is scant, especially for children and adolescents. OBJECTIVE: The current study evaluated the usability of an EHR developed to support the implementation of the Wraparound care coordination model for children and youth with complex behavioral health needs, and impact of the EHR on service processes, fidelity, and proximal outcomes. METHODS: Thirty-four Wraparound facilitators working in two programs in two states were randomized to either use the new EHR (19/34, 56%) or to continue to implement Wraparound services as usual (SAU) using paper-based documentation (15/34, 44%). Key functions of the EHR included standard fields such as youth and family information, diagnoses, assessment data, and progress notes. In addition, there was the maintenance of a coordinated plan of care, progress measurement on strategies and services, communication among team members, and reporting on services, expenditures, and outcomes. All children and youth referred to services for eight months (N=211) were eligible for the study. After excluding those who were ineligible (69/211, 33%) and who declined to participate (59/211, 28%), a total of 83/211 (39%) children and youth were enrolled in the study with 49/211 (23%) in the EHR condition and 34/211 (16%) in the SAU condition. Facilitators serving these youth and families and their supervisors completed measures of EHR usability and appropriateness, supervision processes and activities, work satisfaction, and use of and attitudes toward standardized assessments. Data from facilitators were collected by web survey and, where necessary, by phone interviews. Parents and caregivers completed measures via phone interviews. Related to fidelity and quality of behavioral health care, including Wraparound team climate, working alliance with providers, fidelity to the Wraparound model, and satisfaction with services. RESULTS: EHR-assigned facilitators from both sites demonstrated the robust use of the system. Facilitators in the EHR group reported spending significantly more time reviewing client progress (P=.03) in supervision, and less time overall sending reminders to youth/families (P=.04). A trend toward less time on administrative tasks (P=.098) in supervision was also found. Facilitators in both groups reported significantly increased use of measurement-based care strategies overall, which may reflect cross-group contamination (given that randomization of staff to the EHR occurred within agencies and supervisors supervised both types of staff). Although not significant at P<.05, there was a trend (P=.10) toward caregivers in the EHR group reporting poorer shared agreement on tasks on the measure of working alliance with providers. No other significant between-group differences were found. CONCLUSIONS: Results support the proposal that use of EHR systems can promote the use of client progress data and promote efficiency; however, there was little evidence of any impact (positive or negative) on overall service quality, fidelity, or client satisfaction. The field of children's behavioral health services would benefit from additional research on EHR systems using designs that include larger sample sizes and longer follow-up periods. TRIAL REGISTRATION: ClinicalTrials.gov NCT02421874; https://clinicaltrials.gov/ct2/show/NCT02421874 (Archived by WebCite at http://www.webcitation.org/6yyGPJ3NA).

Applying User Input to the Design and Testing of an Electronic Behavioral Health Information System for Wraparound Care Coordination.

Health information technology (HIT) and care coordination for individuals with complex needs are high priorities for quality improvement in health care. However, there is little empirical guidance about how best to design electronic health record systems and related technologies to facilitate implementation of care coordination models in behavioral health, or how best to apply user input to the design and testing process. In this paper, we describe an iterative development process that incorporated user/stakeholder perspectives at multiple points and resulted in an electronic behavioral health information system (EBHIS) specific to the wraparound care coordination model for youth with serious emotional and behavioral disorders. First, we review foundational HIT research on how EBHIS can enhance efficiency and outcomes of wraparound that was used to inform development. After describing the rationale for and functions of a prototype EBHIS for wraparound, we describe methods and results for a series of six small studies that informed system development across four phases of effort-predevelopment, development, initial user testing, and commercialization-and discuss how these results informed system design and refinement. Finally, we present next steps, challenges to dissemination, and guidance for others aiming to develop specialized behavioral health HIT. The research team's experiences reinforce the opportunity presented by EBHIS to improve care coordination for populations with complex needs, while also pointing to a litany of barriers and challenges to be overcome to implement such technologies.

Psychometrics, Reliability, and Validity of a Wraparound Team Observation Measure.

Wraparound is a widely-implemented team-based care coordination process for youth with serious emotional and behavioral needs. Wraparound has a positive evidence base; however, research has shown inconsistency in the quality of its implementation that can reduce its effectiveness. The current paper presents results of three studies used to examine psychometrics, reliability, and validity of a measure of wraparound fidelity as assessed during team meetings called the Team Observation Measure (TOM). Analysis of TOM results from 1,078 team observations across 59 sites found good overall internal consistency (alpha = .80), but constrained variability, with the average team rated as having 78% of indicators of model adherent wraparound present, 11% absent, and 11% not applicable. A study of N=23 pairs of raters found a pooled Kappa statistic of .733, indicating substantial inter-rater reliability. Higher agreement was found between external evaluators than for pairs of raters that included an external evaluator and an internal rater (e.g., supervisor or coach). A validity study found no correlation between the TOM and an alternate fidelity instrument, the Wraparound Fidelity Index (WFI), at the team level. However, positive correlations between mean program-level TOM and WFI scores provide support for TOM validity as a summative assessment of site- or program-level fidelity. Implications for TOM users, measure refinement, and future research are discussed.

Effectiveness of wraparound versus case management for children and adolescents: results of a randomized study.

In this study, we compared service experiences and outcomes for youths with serious emotional disorder (SED) randomly assigned to care coordination via a defined wraparound process (n = 47) versus more traditional intensive case management (ICM; n = 46) The wraparound group received more mean hours of care management and services; however, there ultimately were no group differences in restrictiveness of residential placement, emotional and behavioral symptoms, or functioning. Wraparound implementation fidelity was found to be poor. Organizational culture and climate, and worker morale, were poorer for the wraparound providers than the ICM group. Results suggest that, for less-impaired youths with SED, less intensive options such as ICM may be equally effective to poor-quality wraparound delivered in the absence of wraparound implementation supports and favorable system conditions.

Evaluating fidelity to the wraparound service model for youth: application of item response theory to the Wraparound Fidelity Index.

The wraparound process is a mechanism for multisystem planning and care coordination for youth with serious emotional and behavioral problems. Fidelity monitoring is critical to effective implementation of evidence-based practices in children's mental health, as it helps ensure that complex interventions like wraparound are implemented as intended. The 40-item Wraparound Fidelity Index, Version 4 (WFI-4; Bruns, Burchard, Suter, Leverentz-Brady, & Force, 2004) is the most frequently used measure of fidelity to the wraparound process, but analysis of its psychometric properties is insufficient. An item response theory approach, Rasch partial credit models for ordered polytomous data, was used on ratings from 1,234 facilitators, 1,006 caregivers, and 221 team members, focused on 1,478 youths (55% male). Results indicated the WFI-4 measured a unidimensional construct, with little evidence of item bias and good item and model fit. However, the item information curve was skewed, with most people endorsing high-fidelity responses, and several items had duplicative location estimates. A reduced 20-item measure is proposed. Internal reliability estimates for scores from this reduced measure were approximately equivalent to the longer measure. However, both versions would benefit from additional items located in the highest fidelity area of either version of the scale where scores by greater than half of our sample fall, but only 3 items are located.