RESUMEN
OBJECTIVE: The clinical decisions and actions of evidence-based practice in psychology (EBPP) are largely underspecified and poorly understood, in part due to the lack of measurement methods. We tested the reliability of a behavioral coding system that characterizes a flow of interrelated activities that includes problem detection and prioritization, intervention selection and implementation, and review of intervention integrity and impact. METHOD: The context included two publicly funded youth mental health service organizations located in geographically distinct and underresourced communities in the U.S. where service inequities are common. We sampled 84 digitally recorded and transcribed supervision events that included a sample of professionals who were mostly women (93.02%) and BIPOC (86.04%) whose self-reported race/ethnicity matched the youth populations they served. We coded these events for activities (e.g., considering) and their predicate content (i.e., problems or practices) and examined reliability of these codes applied to excerpts (i.e., small contiguous units of dialogue) as well as to complete events. RESULTS: Interrater reliability estimates showed that, overall, coders reliably rated the occurrence and extensiveness of activities and content. Excerpt coding was generally more reliable than event coding. However, mathematical aggregation of excerpt coding offered a superior method for estimating event codes reliably, reducing individual subjectivity while providing event level synthesis of activities that are grounded in excerpt level details. CONCLUSIONS: The assessment of clinical decisions and actions has the potential to unpack the black box of EBPP, with different methods best suited to different research questions and resource considerations.
RESUMEN
Positive organizational climate - employee perceptions of their work environment and the impact of this environment on well-being and functioning - is associated with desirable organizational and client-level outcomes in mental health organizations. Clinical supervisors are well-positioned to impact organizational climate, as they serve as intermediaries between higher-level administrators who drive the policies and procedures and the therapists impacted by such decisions. This cross-sectional study examined the role of clinical supervisors as drivers of therapist perceptions of organizational climate within supervisory teams. Specifically, the present study investigated: (1) shared perceptions of organizational climate among therapists on the same supervisory team; (2) predictors of therapist climate perceptions. Eighty-six therapists were supervised by 22 supervisors. Indices of interrater agreement and interrater reliability of therapists on the same supervisory team were examined to determine shared or distinct perceptions of organizational climate. Multi-level models were used to examine whether supervisor attitudes towards evidence-based practices and therapist perceptions of supervisor communication predicted perceived organizational climate. Results showed perceptions of organizational cohesion and autonomy were shared among therapists on the same supervisory team and distinct from therapists on different supervisory teams. Therapist perceptions of their supervisor's communication was positively associated with perceptions of organizational cohesion and autonomy. These findings align with emerging evidence that middle managers shape their employees' experience of their work environment through communication strategies. These findings also point to the potential for intervening at lower organizational levels to improve overall organizational climate.
RESUMEN
This study investigated coded data retrieved from clinical dashboards, which are decision-support tools that include a graphical display of clinical progress and clinical activities. Data were extracted from clinical dashboards representing 256 youth (M age = 11.9) from 128 practitioners who were trained in the Managing and Adapting Practice (MAP) system (Chorpita & Daleiden in BF Chorpita EL Daleiden 2014 Structuring the collaboration of science and service in pursuit of a shared vision. 43(2):323 338. 2014, Chorpita & Daleiden in BF Chorpita EL Daleiden 2018 Coordinated strategic action: Aspiring to wisdom in mental health service systems. 25(4):e12264. 2018) in 55 agencies across 5 regional mental health systems. Practitioners labeled up to 35 fields (i.e., descriptions of clinical activities), with the options of drawing from a controlled vocabulary or writing in a client-specific activity. Practitioners then noted when certain activities occurred during the episode of care. Fields from the extracted data were coded and reliability was assessed for Field Type, Practice Element Type, Target Area, and Audience (e.g., Caregiver Psychoeducation: Anxiety would be coded as Field Type = Practice Element; Practice Element Type = Psychoeducation; Target Area = Anxiety; Audience = Caregiver). Coders demonstrated moderate to almost perfect interrater reliability. On average, practitioners recorded two activities per session, and clients had 10 unique activities across all their sessions. Results from multilevel models showed that clinical activity characteristics and sessions accounted for the most variance in the occurrence, recurrence, and co-occurrence of clinical activities, with relatively less variance accounted for by practitioners, clients, and regional systems. Findings are consistent with patterns of practice reported in other studies and suggest that clinical dashboards may be a useful source of clinical information. More generally, the use of a controlled vocabulary for clinical activities appears to increase the retrievability and actionability of healthcare information and thus sets the stage for advancing the utility of clinical documentation.
Asunto(s)
Sistemas de Tablero , Servicios de Salud Mental , Adolescente , Humanos , Niño , Reproducibilidad de los Resultados , Trastornos de Ansiedad , DocumentaciónRESUMEN
This study explored predictors of community-based providers' adherence to MATCH, a modular cognitive behavioral therapy for children and adolescents. Provider-reported adherence to MATCH was measured using three increasingly strict criteria: (1) session content (whether the session covered MATCH content consistent with the client's target problem), (2) session content and sequencing (whether the session covered MATCH content in the expected sequence for the client's target problem), and (3) session content, sequencing, and participant (whether the session covered MATCH content in the expected sequence and with the expected participant(s) for the client's target problem). Session, client, provider, and organizational predictors of adherence to MATCH were assessed using multilevel modeling. Results revealed that nearly all providers delivered MATCH content that corresponded to the target problem, but only one-third of providers delivered MATCH content in the expected sequence and with the expected participant for the client's target problem. This difference underscores the need for nuanced adherence measurement to capture important implementation information that broad operationalizations of adherence miss. Regardless of the criteria used providers were most adherent to MATCH during sessions when clients presented with interfering comorbid mental health symptoms. This suggests that the design of MATCH, which offers flexibility and structured guidance to address comorbid mental health problems, may allow providers to personalize treatment to address interfering comorbidity symptoms while remaining adherent to evidence-based practices. Additional guidance for providers on managing other types of session interference (e.g., unexpected events) may improve treatment integrity in community settings.
Asunto(s)
Terapia Cognitivo-Conductual , Adhesión a Directriz , Humanos , Adolescente , Niño , Femenino , Masculino , Servicios Comunitarios de Salud Mental/organización & administración , Trastornos Mentales/terapiaRESUMEN
Cognitive behavioral therapy (CBT) is often referred to as the "gold standard" treatment for mental health problems, given the large body of evidence supporting its efficacy. However, there are persistent questions about the generalizability of CBTs to culturally diverse populations and whether culturally sensitive approaches are warranted. In this review, we synthesize the literature on CBT for ethnic minorities, with an emphasis on randomized trials that address cultural sensitivity within the context of CBT. In general, we find that CBT is effective for ethnic minorities with diverse mental health problems, although nonsignificant trends suggest that CBT effects may be somewhat weaker for ethnic minorities compared to Whites. We find mixed support for the cultural adaptation of CBTs, but evidence for cultural sensitivity training of CBT clinicians is lacking, given a dearth of relevant trials. Based on the limited evidence thus far, we summarize three broad models for addressing cultural issues when providing CBT to diverse populations.
Asunto(s)
Terapia Cognitivo-Conductual , Asistencia Sanitaria Culturalmente Competente , Minorías Étnicas y Raciales , HumanosRESUMEN
To improve the dissemination and actionability of mental health research, many mental health professionals have developed online informational resources to increase the general public's awareness of mental health difficulties and available treatments. Yet, limited information exists on the quality and scope of these resources. This study aimed to explore the scope and quantity of online, free, evidence-based mental health resources. Fifty-two mental health professionals nominated 178 resources, which predominantly consisted of homepages and links to more information. When reviewing the original nominations, our team identified an additional 290 resources (e.g., fact sheets linked from a nominated homepage). Of the 468 total nominated resources, 72 were screened out due to not meeting the inclusion criteria of being free (inter-screener reliability = 95%), evidence-based (inter-screener reliability = 94%), and online (inter-screener reliability = 96%). Nominated resources most commonly covered anxiety and obsessive-compulsive disorder (n = 67) and suicide (n = 60). Resources providing information about the mental health problem were most common (n = 210) and resources providing information about immediate help (e.g., hotline) were least common (n = 57). Our findings indicate many free, online, evidence-based resources are available and raise questions of whether efforts to disseminate mental health research are recreating the issue of information overload. Other considerations and future directions for improving the utilization and synthesizing of available resources are discussed.
Asunto(s)
Salud Mental , Suicidio , Humanos , Reproducibilidad de los Resultados , Trastornos de Ansiedad , Recursos en SaludRESUMEN
Online parenting programs are an effective way to teach behavioral management skills to parents in the absence of in-person resources. This community-engaged study aimed to examine strategies for disseminating online parenting resources in schools. Online resources were disseminated to parents in a Northern California school district. Dissemination strategies were informed by conversations with school principals, teachers, and parents and considered agent, message, and format. A total of 685 parents and teachers clicked on the online resources: 151 parents and 114 teachers attended synchronous classes. The use of dissemination strategies had a compounding influence on the number of synchronous class attendees and clicks. Emails sent by the school district yielded the greatest number of clicks, which was influenced by message content and format. A community-academic partnership (CAP) led to the dissemination of evidence-based online parenting resources to a large population and led to lessons learned that could inform future research involving CAPs.
Asunto(s)
COVID-19 , Responsabilidad Parental , Humanos , Pandemias , Padres , Instituciones AcadémicasRESUMEN
This study examined the accessibility of community resources (e.g., welfare programs and afterschool programs) for underserved youth and families with mental health needs. Mental health professionals (n = 52) from a large community mental health and welfare agency serving predominantly low-income, Latinx families completed a semistructured interview that asked about the accessibility of community resources. Participant responses were coded using an inductive thematic analysis. Results showed that 71% of participants endorsed availability barriers (e.g., limited local programs), 37% endorsed logistical barriers (e.g., waitlists), 27% endorsed attitudinal barriers (e.g., stigmatized beliefs about help-seeking), and 23% endorsed knowledge barriers (e.g., lacking awareness about local programs). Professionals' perceived availability barriers were mostly consistent with the actual availability of community resources. Findings highlight the compounding challenges that underserved communities face and point to opportunities for promoting enhanced well-being and functioning for youth and families with mental health needs.
Asunto(s)
Recursos Comunitarios , Servicios de Salud Mental , Adolescente , Humanos , Salud Mental , Pobreza , Investigación CualitativaRESUMEN
This study explored whether post-treatment symptom severity moderated the association between session attendance during an initial treatment episode and subsequent mental health service use. Data on attendance, symptom severity, and service use were gathered from an effectiveness trial testing a modular treatment for youth anxiety, depression, disruptive behavior, and traumatic stress. Multilevel logistic regression analyses showed a significant interaction between attendance and post-treatment symptom severity on subsequent service use, such that attendance significantly predicted subsequent service use when post-treatment symptom severity was in the normal range. Implications regarding the influence of treatment engagement on future help-seeking are discussed.
Asunto(s)
Trastornos Mentales/fisiopatología , Trastornos Mentales/terapia , Servicios de Salud Mental , Aceptación de la Atención de Salud , Adolescente , Niño , Bases de Datos Factuales , Femenino , Humanos , Entrevistas como Asunto , Modelos Logísticos , Masculino , Investigación Cualitativa , Índice de Severidad de la Enfermedad , Resultado del TratamientoRESUMEN
Use of evidence to inform clinical decisions has been shown to improve the quality and effectiveness of services. This study piloted an observational coding system for understanding providers and supervisors' use of evidence in their clinical decision-making. The Action Cycle and Use of Evidence Behavioral Observation Coding System (ACE-BOCS) is based on Graham et al. (Contin Educ Health Prof 26:13-24, 2006) conceptual framework for knowledge management, which articulates a sequence relevant to integrating evidence into decisions and actions, including identifying and selecting a problem and choosing, planning, and rehearsing a solution or action. Using the ACE-BOCS, two coders rated the extensiveness with which evidence was used to inform decisions made in clinical supervision sessions. In these clinical supervision sessions, supervisor-provider dyads discussed cases (N = 30; age range 8-19 years; 80% Latino/a or Hispanic ethnicity) that were identified as potentially being at risk for low treatment engagement in school mental health services. Results indicated that the ACE-BOCS can reliably and validly measure use of evidence and distinguish between strategic and indiscriminate use of evidence. The ACE-BOCS has value and utility for studying use of evidence, as it incorporates multiple actions related to service delivery and has the potential to be adapted for other aspects of mental healthcare decision-making beyond clinical supervision, as well as decision making within fields outside of mental health.
Asunto(s)
Toma de Decisiones , Medicina Basada en la Evidencia , Servicios de Salud Mental , Adulto , Femenino , Humanos , Masculino , Proyectos Piloto , Mejoramiento de la Calidad , Calidad de la Atención de Salud/normasRESUMEN
This study explored mental health professionals' perceptions about barriers and facilitators to engaging underserved populations. Responses were coded using an iterative thematic analysis based on grounded theory. Results revealed that many professionals endorsed barriers to engaging ethnic minorities and families receiving social services. Client-provider racial and linguistic matching, therapy processes and procedures (e.g., nonjudgmental stance), and implementation supports (e.g., supervision) were commonly nominated as engagement facilitators. Many professionals felt that an organizational culture focused on productivity is detrimental to client engagement. Findings shed light on professionals' perceived barriers to delivering high-quality care to underserved communities and illuminate potential engagement strategies.
Asunto(s)
Actitud del Personal de Salud , Servicios Comunitarios de Salud Mental , Personal de Salud/psicología , Área sin Atención Médica , Poblaciones Vulnerables , Adulto , Femenino , Disparidades en el Estado de Salud , Humanos , Entrevistas como Asunto , Masculino , Persona de Mediana Edad , Investigación CualitativaRESUMEN
Emergent life events (ELEs)-unexpected stressors disclosed in psychotherapy that have a significant negative impact on the client-commonly occur in community populations of youth and are associated with decreased provider adherence to evidence-based treatment (EBT) in session. The present study extends previous research by examining longer term associations of ELEs with (a) provider adherence to planned EBT practices in subsequent sessions and (b) clinical progress. Data were drawn from the modular EBT condition (MATCH) of the Child STEPs California trial conducted with primarily Latino youth, ages 5-15, who were 54% male (Chorpita et al., 2017). Study 1 utilized data from 57 MATCH participants who reported at least one ELE during treatment. Provider adherence was measured by identifying whether planned practices were covered in either the session in which the ELE was reported or the following session using the MATCH Consultation Record. In Study 2, clinical progress for 78 MATCH participants was assessed using weekly youth- and caregiver-ratings of symptomatology (Brief Problem Checklist) and functioning (Top Problems Assessment). Study 1 revealed that ELEs were associated with reduced adherence to planned practices for at least two sessions. Study 2 demonstrated that each disruptive ELE (i.e., an ELE for which no EBT content was covered) was associated with a 14%-20% slower rate of clinical improvement, with greater declines for functioning and externalizing symptoms. Findings suggest that ELEs can be a major barrier to the effectiveness of an EBT and require further research in order to be addressed effectively.
Asunto(s)
Práctica Clínica Basada en la Evidencia/métodos , Acontecimientos que Cambian la Vida , Psicoterapia/métodos , Niño , Preescolar , Femenino , Humanos , MasculinoRESUMEN
Codifying a complex treatment intervention into a manual necessarily involves simplifying procedures, possibly resulting in developer and expert adaptations that align with but are not fully consistent with the manualized content. This study evaluated the integrity of an evidence-based treatment to two information sources (i.e., an expert consultant and the treatment manual) within youth community mental health settings. Consultant recommendations were compared with manual-prescribed therapy procedures, which identified instances of agreement between the sources as well as consultant adaptations. To determine clinicians' integrity to each source, clinician-reported therapy procedures were compared with consultant recommendations and manual prescriptions. Participants consisted of 30 clinicians, five consultants, and 78 youths (53% male; M age = 9.73 years, SD = 2.84; 81% Latino). Findings indicated that consultants generally recommended content from the manual but also made frequent adaptations that involved changing the sequence or problem focus of practices and adding additional content. Clinicians followed both information sources in a majority of their sessions when these sources agreed but were more likely to follow only one or neither source when they disagreed. Findings suggest that consultants often make adaptations to the manual and that clinicians are more likely to employ recommendations from the manual and/or consultant when the information sources are in agreement. Although it is as yet unclear whether consultant recommendations are effective when they differ from the prescriptions of the manual, results from this study indicate that information sources that supplement treatment manuals may benefit clinical decision making.
Asunto(s)
Consultores/psicología , Psicoterapia/métodos , Niño , Femenino , Humanos , MasculinoRESUMEN
The goal of this study is to explore providers' patterns of implementation by investigating how community mental health providers selected therapy practice modules from a flexible, modular evidence-based treatment working with youths with comorbid mental health problems. Data were obtained from 57 youths, 5-15 years old, presenting with anxiety, depressive, and/or conduct problems and their 27 providers during their participation in an effectiveness trial involving a modular evidence-based treatment. Although all youths evidenced clinically elevated symptomatology in at least two problem areas, providers targeted youths' comorbid problems with only about half of their study cases. Practice modules indicated for youths' comorbid problems were typically used less frequently and with less depth relative to practice modules indicated for youths' principal clinical problem and were often transdiagnostic in nature (i.e., designed to target more than one problem area). To determine whether providers' decisions to target youths' comorbid problems were systematic, multilevel, logistic regression analyses were conducted and revealed that youths' pretreatment characteristics and time in therapy influenced providers' patterns of module selection. Providers tend to use, but not exploit, the flexibility allowed by modular EBTs and to focus treatment on youths' principal presenting problem. In addition, providers appear to make these practice choices in a systematic and rational manner, and whether and which choices are associated with improved outcomes is an important area of future study.
Asunto(s)
Servicios Comunitarios de Salud Mental/métodos , Personal de Salud , Salud Mental , Trastornos del Neurodesarrollo/epidemiología , Trastornos del Neurodesarrollo/terapia , Adolescente , Adulto , Anciano , Trastornos de Ansiedad/epidemiología , Trastornos de Ansiedad/psicología , Trastornos de Ansiedad/terapia , Niño , Preescolar , Servicios Comunitarios de Salud Mental/tendencias , Comorbilidad , Trastorno Depresivo/epidemiología , Trastorno Depresivo/psicología , Trastorno Depresivo/terapia , Femenino , Personal de Salud/tendencias , Humanos , Masculino , Salud Mental/tendencias , Persona de Mediana Edad , Trastornos del Neurodesarrollo/psicología , Adulto JovenRESUMEN
This study examined patterns of evidence-based treatment (EBT) delivery following a county-wide EBT reform initiative. Data were gathered from 60 youth and their 21 providers, who were instructed to deliver therapy as they normally would under the EBT initiative. Results showed limited applicability of county-supported EBTs to this service sample, and that most youth did not receive traditional delivery of EBTs. Findings suggest that it may be unrealistic to expect providers to deliver EBTs with fidelity with all clients, and that EBT implementation may be best thought of as a strategy for improving mental health services rather than a goal.
Asunto(s)
Práctica Clínica Basada en la Evidencia/normas , Política de Salud , Trastornos Mentales/terapia , Servicios de Salud Mental/normas , Mejoramiento de la Calidad , Calidad de la Atención de Salud , Adolescente , Adulto , Niño , Preescolar , Femenino , Humanos , Ciencia de la Implementación , Los Angeles , Masculino , Ensayos Clínicos Controlados Aleatorios como AsuntoRESUMEN
This study expands upon the Chorpita et al. (J Consult Clin Psychol 81:999-1009, 2013) findings by examining the impact of treatment protocol on youths' service utilization up to 2 years after starting an episode of: standard manualized treatment (Standard); modular treatment (Modular); or usual care (UC). Results showed that youths who received Modular accessed fewer service settings at their one-year follow-up relative to youths who received Standard or UC. Findings suggest that modular treatment may offer an advantage over standardized treatment manuals and UC in terms of sustained clinical benefits, and highlight the importance of treatment design considerations for service systems.
Asunto(s)
Atención Ambulatoria/estadística & datos numéricos , Servicios Comunitarios de Salud Mental/estadística & datos numéricos , Práctica Clínica Basada en la Evidencia , Hospitalización/estadística & datos numéricos , Trastornos Mentales/terapia , Psicoterapia/métodos , Servicios de Salud Escolar/estadística & datos numéricos , Niño , Femenino , Humanos , MasculinoRESUMEN
Measurement feedback systems (MFSs) can help improve clinical outcomes by enhancing clinical decision-making. Unfortunately, limited information exists to guide the use and interpretation of data from MFSs. This study examined the amount of data that would provide a reasonable and reliable prediction of a client's rate of symptomatology in order to help inform clinical decision-making processes. Results showed that use of more data predicted greater levels of accuracy. However, there were diminishing returns on the ability for additional data to improve predictive accuracy. Findings inform efforts to develop guidelines on the interpretation of data from MFSs.
Asunto(s)
Lista de Verificación , Servicios de Salud del Niño , Toma de Decisiones Clínicas , Retroalimentación , Trastornos Mentales/terapia , Servicios de Salud Mental , Adolescente , Niño , Femenino , Humanos , Estudios Longitudinales , Los Angeles , Masculino , Resultado del TratamientoRESUMEN
This study examined patterns of evidence-based treatment (EBT) implementation within community settings by evaluating integrity along separate dimensions of practice content (PC; a session included the prescribed procedure) and practice sequencing (a session occurred in the prescribed sequence) within a recent randomized effectiveness trial. We measured whether sessions showed integrity to PC and to flexible or linear practice sequences. Findings revealed that providers tended to incorporate content from the EBT protocol in most treatment sessions, but that the sequencing of the sessions was often modified, suggesting that providers are amenable to evidence-based procedures, but not necessarily their prescribed arrangement.
Asunto(s)
Terapia Cognitivo-Conductual/normas , Práctica Clínica Basada en la Evidencia , Adhesión a Directriz , Trastornos Mentales/terapia , Adulto , Trastornos de Ansiedad/terapia , Niño , Protocolos Clínicos , Trastorno de la Conducta/terapia , Trastorno Depresivo/terapia , Femenino , Humanos , Masculino , Persona de Mediana Edad , Psiquiatría/normas , Psicología/normas , Asistencia Social en Psiquiatría/normasRESUMEN
Treatment engagement is a significant challenge in school mental health services. Despite a growing evidence base on effective interventions for treatment engagement, attempts to address these challenges rarely leverage the available relevant research. To close this gap, this pilot study examined the feasibility, acceptability, and initial implementation outcomes of a coordinated knowledge system (CKS) designed to support the efforts of school mental health (SMH) professionals to address treatment engagement challenges. A sample of four supervisors and their supervisees (nâ¯=â¯17) with master's of social work degrees serving youth within an urban SMH program were randomly assigned to either (a) a CKS condition that offered a unified set of resources to structure decisions about treatment engagement or (b) a survey plus practice guidelines (SPG) condition, in which resources were provided without an explicit model for their coordinated use. Feasibility was assessed quantitatively, acceptability was assessed quantitatively and qualitatively, and initial implementation outcomes were evaluated using a behavioral observation coding system. Results provided strong support for the feasibility and acceptability of the study design, instrumentation, and interventions. It appears that the CKS can be feasibly implemented with good acceptability. Patterns of initial implementation strongly suggested that, relative to the SPG, the CKS supports identification of engagement problems as well as intervention selection and planning. Findings also identified modifications to study procedures that would be necessary in a future study. A larger trial using similar methodology and instrumentation could yield important findings about the effectiveness of the CKS in clinical practice and could articulate the mechanisms by which it operates.
Asunto(s)
Aceptación de la Atención de Salud , Servicios de Salud Mental Escolar , Adolescente , Adulto , Actitud del Personal de Salud , Niño , Estudios de Factibilidad , Femenino , Humanos , Masculino , Proyectos Piloto , Guías de Práctica Clínica como AsuntoRESUMEN
Cognitive behavioral therapies (CBT) for youth with anxiety, traumatic stress, and depression have demonstrated strong effects in individual studies and meta-analyses. Relatively more attention has been given to posttreatment effects, though, and assessment of follow-up effects has been limited at the meta-analytic level. The current meta-analysis aimed to (a) examine the effects of youth CBT at posttreatment, 1-month, 3-month, 6-month, 1-year, and long-term (2+ years) follow-up as well as (b) identify research-related variables (e.g., measure respondent type) that relate to effects. Using a random effects model across 110 child and adolescent CBT groups, within-group effect sizes were large at posttreatment (g = 1.24) and from 1-month through long-term follow-up (g = 1.23-1.82), and effect sizes did not significantly differ by treatment target (i.e., anxiety, traumatic stress, depression). However, availability of outcome data for effect sizes diminished across later follow-up assessments. Moreover, effect sizes were significantly associated with outcome respondent type across assessment timing, with outcome measures from caregiver and youth respondents associated with smaller effect sizes (B = -0.97, p < 0.001) relative to outcome measures that were evaluator-reported. Results provide initial support for the durability of treatment effects for youth CBTs and highlight the importance of some confounding variables. Implications for improving treatment research standards and prioritizing assessment of long-term follow-up assessment are discussed.