Twelve tips for Natural Language Processing in medical education program evaluation.

With the increasing application of Natural Language Processing (NLP) in Medicine at large, medical educators are urged to gain an understanding and implement NLP techniques within their own education programs to improve the workflow and make significant and rapid improvements in their programs. This paper aims to provide twelve essential tips inclusive of both conceptual and technical factors to facilitate the successful integration of NLP in medical education program evaluation. These twelve tips range from advising on various stages of planning the evaluation process, considerations for data collection, and reflections on preprocessing of data in preparation for analysis and interpretation of results. Using these twelve tips as a framework, medical researchers, educators, and administrators will have an understanding and reference to navigating applications of NLP and be able to unlock its potential for enhancing the evaluation of their own medical education programs.

Transfer Learning for Risk Classification of Social Media Posts: Model Evaluation Study.

BACKGROUND: Mental illness affects a significant portion of the worldwide population. Online mental health forums can provide a supportive environment for those afflicted and also generate a large amount of data that can be mined to predict mental health states using machine learning methods. OBJECTIVE: This study aimed to benchmark multiple methods of text feature representation for social media posts and compare their downstream use with automated machine learning (AutoML) tools. We tested on datasets that contain posts labeled for perceived suicide risk or moderator attention in the context of self-harm. Specifically, we assessed the ability of the methods to prioritize posts that a moderator would identify for immediate response. METHODS: We used 1588 labeled posts from the Computational Linguistics and Clinical Psychology (CLPsych) 2017 shared task collected from the Reachout.com forum. Posts were represented using lexicon-based tools, including Valence Aware Dictionary and sEntiment Reasoner, Empath, and Linguistic Inquiry and Word Count, and also using pretrained artificial neural network models, including DeepMoji, Universal Sentence Encoder, and Generative Pretrained Transformer-1 (GPT-1). We used Tree-based Optimization Tool and Auto-Sklearn as AutoML tools to generate classifiers to triage the posts. RESULTS: The top-performing system used features derived from the GPT-1 model, which was fine-tuned on over 150,000 unlabeled posts from Reachout.com. Our top system had a macroaveraged F1 score of 0.572, providing a new state-of-the-art result on the CLPsych 2017 task. This was achieved without additional information from metadata or preceding posts. Error analyses revealed that this top system often misses expressions of hopelessness. In addition, we have presented visualizations that aid in the understanding of the learned classifiers. CONCLUSIONS: In this study, we found that transfer learning is an effective strategy for predicting risk with relatively little labeled data and noted that fine-tuning of pretrained language models provides further gains when large amounts of unlabeled text are available.

The Mortality and Myocardial Effects of Antidepressants Are Moderated by Preexisting Cardiovascular Disease: A Meta-Analysis.

BACKGROUND: Antidepressants (ADs) are commonly prescribed medications, but their long-term health effects are debated. ADs disrupt multiple adaptive processes regulated by evolutionarily ancient biochemicals, potentially increasing mortality. However, many ADs also have anticlotting properties that can be efficacious in treating cardiovascular disease. We conducted a meta-analysis assessing the effects of ADs on all-cause mortality and cardiovascular events in general-population and cardiovascular-patient samples. METHODS: Two reviewers independently assessed articles from PubMed, EMBASE, and Google Scholar for AD-related mortality controlling for depression and other comorbidities. From these articles, we extracted information about cardiovascular events, cardiovascular risk status, and AD class. We conducted mixed-effect meta-analyses testing sample type and AD class as moderators of all-cause mortality and new cardiovascular events. RESULTS: Seventeen studies met our search criteria. Sample type consistently moderated health risks. In general-population samples, AD use increased the risks of mortality (HR = 1.33, 95% CI: 1.14-1.55) and new cardiovascular events (HR = 1.14, 95% CI: 1.08-1.21). In cardiovascular patients, AD use did not significantly affect risks. AD class also moderated mortality, but the serotonin reuptake inhibitors were not significantly different from tricyclic ADs (TCAs) (HR = 1.10, 95% CI: 0.93-1.31, p = 0.27). Only "other ADs" were differentiable from TCAs (HR = 1.35, 95% CI: 1.08-1.69). Mortality risk estimates increased when we analyzed the subset of studies controlling for premedication depression, suggesting the absence of confounding by indication. CONCLUSIONS: The results support the hypothesis that ADs are harmful in the general population but less harmful in cardiovascular patients.

Cognitive Outcomes After Antidepressant Pharmacotherapy for Late-Life Depression: A Systematic Review and Meta-Analysis.

OBJECTIVE: The authors evaluated whether treatment of late-life depression (LLD) with antidepressants leads to changes in cognitive function. METHODS: A systematic review and meta-analysis of prospective studies of antidepressant pharmacotherapy for adults age 50 or older (or mean age of 65 or older) with LLD was conducted. MEDLINE, EMBASE, and PsycInfo were searched through December 31, 2022. The primary outcome was a change on cognitive test scores from baseline to after treatment. Secondary outcomes included the effects of specific medications and the associations between changes in depressive symptoms and cognitive test scores. Participants with bipolar disorder, psychotic depression, dementia, or neurological disease were excluded. Findings from all eligible studies were synthesized at a descriptive level, and a random-effects model was used to pool the results for meta-analysis. RESULTS: Twenty-two studies were included. Thirteen of 19 studies showed an improvement on at least one cognitive test after antidepressant pharmacotherapy, with the most robust evidence for the memory and learning (nine of 16 studies) and processing speed (seven of 10 studies) domains and for sertraline (all five studies). Improvements in depressive symptoms were associated with improvement in cognitive test scores in six of seven relevant studies. The meta-analysis (eight studies; N=493) revealed a statistically significant overall improvement in memory and learning (five studies: effect size=0.254, 95% CI=0.103-0.404, SE=0.077); no statistically significant changes were seen in other cognitive domains. The evaluated risk of publication bias was low. CONCLUSION: Antidepressant pharmacotherapy of LLD appears to improve certain domains of cognitive function, particularly memory and learning. This effect may be mediated by an improvement in depressive symptoms. Studies comparing individuals receiving pharmacotherapy with untreated control participants are needed.

An experimental paradigm for triggering a depressive syndrome.

Research investigating whether depression is an adaptation or a disorder has been hindered by the lack of an experimental paradigm that can test causal relationships. Moreover, studies attempting to induce the syndrome often fail to capture the suite of feelings, thoughts, and behaviors that characterize depression. An experimental paradigm for triggering depressive symptoms can improve our etiological understanding of the syndrome. The present study attempts to induce core symptoms of depression, particularly those related to rumination, in a healthy, nonclinical sample through a controlled social experiment. These symptoms are sad or depressed mood, anhedonia, feelings of worthlessness or guilt, and difficulty concentrating. One hundred and thirty-four undergraduate students were randomly assigned to either an exclusion (E) or control (C) group. Participants in the exclusion group were exposed to a modified Cyberball paradigm, designed to make them feel socially excluded, followed by a dual-interference task to assess whether their exclusion interfered with their working memory. Excluded participants: (a) self-reported a significant increase in sadness and decrease in happiness, but not anxiety or calmness; (b) scored significantly higher in four of five variables related to depressive rumination; and (c) performed significantly worse on a dual-interference task, suggesting an impaired ability to concentrate. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Out with AI, in with the psychiatrist: a preference for human-derived clinical decision support in depression care.

Advancements in artificial intelligence (AI) are enabling the development of clinical support tools (CSTs) in psychiatry to facilitate the review of patient data and inform clinical care. To promote their successful integration and prevent over-reliance, it is important to understand how psychiatrists will respond to information provided by AI-based CSTs, particularly if it is incorrect. We conducted an experiment to examine psychiatrists' perceptions of AI-based CSTs for treating major depressive disorder (MDD) and to determine whether perceptions interacted with the quality of CST information. Eighty-three psychiatrists read clinical notes about a hypothetical patient with MDD and reviewed two CSTs embedded within a single dashboard: the note's summary and a treatment recommendation. Psychiatrists were randomised to believe the source of CSTs was either AI or another psychiatrist, and across four notes, CSTs provided either correct or incorrect information. Psychiatrists rated the CSTs on various attributes. Ratings for note summaries were less favourable when psychiatrists believed the notes were generated with AI as compared to another psychiatrist, regardless of whether the notes provided correct or incorrect information. A smaller preference for psychiatrist-generated information emerged in ratings of attributes that reflected the summary's accuracy or its inclusion of important information from the full clinical note. Ratings for treatment recommendations were also less favourable when their perceived source was AI, but only when recommendations were correct. There was little evidence that clinical expertise or familiarity with AI impacted results. These findings suggest that psychiatrists prefer human-derived CSTs. This preference was less pronounced for ratings that may have prompted a deeper review of CST information (i.e. a comparison with the full clinical note to evaluate the summary's accuracy or completeness, assessing an incorrect treatment recommendation), suggesting a role of heuristics. Future work should explore other contributing factors and downstream implications for integrating AI into psychiatric care.

Cocreating Meaning Through Expressive Writing and Reading for Cancer Caregivers.

PURPOSE: Caregivers of patients with cancer cope with socioemotional challenges, which can adversely affect their well-being. We developed an intervention, expressive writing and reading (EWR), to promote emotional processing and social connectedness among caregivers. In a single-arm pilot study, we assessed its feasibility and perceived usefulness. METHODS: Caregivers participated in weekly 1.5-hour EWR workshops offered over 20 weeks. After 4 sessions, they completed semistructured interviews, which were analyzed using qualitative descriptive analysis. FINDINGS: Of 65 caregivers approached, 25 were eligible, 18 consented, and 9 (50%) caregivers completed at least 4 workshops and the interview. Their responses revealed 3 themes: "inner processing," "interpersonal learning," and "enhanced processing and preparedness." Perceived benefits of EWR included emotional and cognitive processing (individual and collaborative), learning from the emotions and experiences of other caregivers, and preparing for upcoming challenges. CONCLUSIONS: Expressive writing and reading can be a safe and cost-effective supportive intervention for caregivers of patients with cancer.

Predictive care: a protocol for a computational ethnographic approach to building fair models of inpatient violence in emergency psychiatry.

INTRODUCTION: Managing violence or aggression is an ongoing challenge in emergency psychiatry. Many patients identified as being at risk do not go on to become violent or aggressive. Efforts to automate the assessment of risk involve training machine learning (ML) models on data from electronic health records (EHRs) to predict these behaviours. However, no studies to date have examined which patient groups may be over-represented in false positive predictions, despite evidence of social and clinical biases that may lead to higher perceptions of risk in patients defined by intersecting features (eg, race, gender). Because risk assessment can impact psychiatric care (eg, via coercive measures, such as restraints), it is unclear which patients might be underserved or harmed by the application of ML. METHODS AND ANALYSIS: We pilot a computational ethnography to study how the integration of ML into risk assessment might impact acute psychiatric care, with a focus on how EHR data is compiled and used to predict a risk of violence or aggression. Our objectives include: (1) evaluating an ML model trained on psychiatric EHRs to predict violent or aggressive incidents for intersectional bias; and (2) completing participant observation and qualitative interviews in an emergency psychiatric setting to explore how social, clinical and structural biases are encoded in the training data. Our overall aim is to study the impact of ML applications in acute psychiatry on marginalised and underserved patient groups. ETHICS AND DISSEMINATION: The project was approved by the research ethics board at The Centre for Addiction and Mental Health (053/2021). Study findings will be presented in peer-reviewed journals, conferences and shared with service users and providers.

Race and Racialization in Mental Health Research and Implications for Developing and Evaluating Machine Learning Models: A Rapid Review.

Machine learning models are often trained on sociodemographic features to predict mental health outcomes. Biases in the collection of race-related data can limit the development of useful and fair models. To assess the current state of this data in mental health research, we conducted a rapid review guided by Critical Race Theory. Findings reveal limitations in the measurement and reporting of race and ethnicity, potentially leading to models that amplify health inequities.

Conceptualising fairness: three pillars for medical algorithms and health equity.

OBJECTIVES: Fairness is a core concept meant to grapple with different forms of discrimination and bias that emerge with advances in Artificial Intelligence (eg, machine learning, ML). Yet, claims to fairness in ML discourses are often vague and contradictory. The response to these issues within the scientific community has been technocratic. Studies either measure (mathematically) competing definitions of fairness, and/or recommend a range of governance tools (eg, fairness checklists or guiding principles). To advance efforts to operationalise fairness in medicine, we synthesised a broad range of literature. METHODS: We conducted an environmental scan of English language literature on fairness from 1960-July 31, 2021. Electronic databases Medline, PubMed and Google Scholar were searched, supplemented by additional hand searches. Data from 213 selected publications were analysed using rapid framework analysis. Search and analysis were completed in two rounds: to explore previously identified issues (a priori), as well as those emerging from the analysis (de novo). RESULTS: Our synthesis identified 'Three Pillars for Fairness': transparency, impartiality and inclusion. We draw on these insights to propose a multidimensional conceptual framework to guide empirical research on the operationalisation of fairness in healthcare. DISCUSSION: We apply the conceptual framework generated by our synthesis to risk assessment in psychiatry as a case study. We argue that any claim to fairness must reflect critical assessment and ongoing social and political deliberation around these three pillars with a range of stakeholders, including patients. CONCLUSION: We conclude by outlining areas for further research that would bolster ongoing commitments to fairness and health equity in healthcare.

Evolutionary theory and the treatment of depression: It is all about the squids and the sea bass.

According to the analytical rumination hypothesis, depression is an evolved adaptation (like pain or anxiety) that served in our ancestral past to keep people focused on complex interpersonal problems until they could arrive at a resolution (spontaneous remission). If this is true, then those clinical treatments that most facilitate the functions that depression evolved to serve are likely to be more advantageous in the long run than others that simply relieve distress. For example, antidepressant medications may be efficacious in the treatment of depression but only work for so long as they are taken. They may also have an iatrogenic effect that prolongs the duration of the underlying episode. Cognitive and behavioral interventions are as efficacious as medications in terms of reducing acute distress and also appear to have an enduring effect that protects against the return of subsequent symptoms. However, the bulk of the evidence for this effect comes from comparisons to prior medication treatment and it remains unclear whether these psychosocial interventions are truly preventative, or antidepressant medications iatrogenic. A study is described that could resolve this issue and test evolutionary theory with respect to the purported role of rumination in bringing about spontaneous remission.

Apps and gaps in bipolar disorder: A systematic review on electronic monitoring for episode prediction.

BACKGROUND: Long-term clinical monitoring in bipolar disorder (BD) is an important therapeutic tool. The availability of smartphones and wearables has sparked the development of automated applications to remotely monitor patients. This systematic review focus on the current state of electronic (e-) monitoring for episode prediction in BD. METHODS: We systematically reviewed the literature on e-monitoring for episode prediction in adult BD patients. The systematic review was done according to the guidelines for reporting of systematic reviews and meta-analyses (PRISMA) and was registered in PROSPERO on April 29, 2020 (CRD42020155795). We conducted a search of Web of Science, MEDLINE, EMBASE, and PsycINFO (all 2000-2020) databases. We identified and extracted data from 17 published reports on 15 relevant studies. RESULTS: Studies were heterogeneous and most had substantial methodological and technical limitations. Models varied widely in their performance. Published metrics were too heterogeneous to lend themselves to a meta-analysis. Four studies reported sensitivity (range: 0.21 - 0.95); and two reported specificity for prediction of mood episodes (range: 0.36 - 0.99). Two studies reported accuracy (range: 0.64 - 0.88) and four reported area under the curve (AUC; range: 0.52-0.95). Overall, models were better in predicting manic or hypomanic episodes, but their performance depended on feature type. LIMITATIONS: Our conclusions are tempered by the lack of appropriate information impeding our ability to synthesize the available evidence. CONCLUSIONS: Given the clinical variability in BD, predicting mood episodes remains a challenging task. Emerging e-monitoring technology for episode prediction in BD requires more development before it can be adopted clinically.

Individual Differences in Response to Antidepressants: A Meta-analysis of Placebo-Controlled Randomized Clinical Trials.

Importance: Antidepressants are commonly used to treat major depressive disorder (MDD). Antidepressant outcomes can vary based on individual differences; however, it is unclear whether specific factors determine this variability or whether it is at random. Objective: To investigate the assumption of systematic variability in symptomatic response to antidepressants and to assess whether variability is associated with MDD severity, antidepressant class, or study publication year. Data Sources: Data used were updated from a network meta-analysis of treatment with licensed antidepressants in adults with MDD. The Cochrane Central Register of Controlled Trials, CINAHL, Embase, LILACS database, MEDLINE, MEDLINE In-Process, and PsycInfo were searched from inception to March 21, 2019. Additional sources were international trial registries and sponsors, drug companies and regulatory agencies' websites, and reference lists of published articles. Data were analyzed between June 8, 2020, and June 13, 2020. Study Selection: Analysis was restricted to double-blind, randomized placebo-controlled trials with depression scores available at the study's end point. Data Extraction and Synthesis: Baseline means, number of participants, end point means and SDs of total depression scores, antidepressant type, and publication year were extracted. Main Outcomes and Measures: Log SDs (bln σ̂) were derived for treatment groups (ie, antidepressant and placebo). A random-slope mixed-effects model was conducted to estimate the difference in bln σ̂ between treatment groups while controlling for end point mean. Secondary models determined whether differences in variability between groups were associated with baseline MDD severity; antidepressant class (selective serotonin reuptake inhibitors and other related drugs; serotonin and norepinephrine reuptake inhibitors; norepinephrine-dopamine reuptake inhibitors; noradrenergic agents; or other antidepressants); and publication year. Results: In the 91 eligible trials (18 965 participants), variability in response did not differ significantly between antidepressants and placebo (bln σ̂, 1.02; 95% CI, 0.99-1.05; P = .19). This finding is consistent with a range of treatment effect SDs (up to 16.10), depending on the association between the antidepressant and placebo effects. Variability was not associated with baseline MDD severity or publication year. Responses to noradrenergic agents were 11% more variable than responses to selective serotonin reuptake inhibitors (bln σ̂, 1.11; 95% CI, 1.01-1.21; P = .02). Conclusions and Relevance: Although this study cannot rule out the possibility of treatment effect heterogeneity, it does not provide empirical support for personalizing antidepressant treatment based solely on total depression scores. Future studies should explore whether individual symptom scores or biomarkers are associated with variability in response to antidepressants.

Disordered doctors or rational rats? Testing adaptationist and disorder hypotheses for melancholic depression and their relevance for clinical psychology.

Most clinicians view depression as a painful disorder in which motivation to pursue adaptive goals is lacking and cognition is impaired. An alternative hypothesis-grounded in a common evolutionary approach-suggests that depression is inherently motivational and evolved to motivate avoidant learning of harmful situations. Testing these hypotheses requires a clear definition of "disorder". Wakefield's harmful dysfunction evolution-based definition proposes that all unambiguous cases of disorder involve a malfunctioning adaptation. These hypotheses-functional adaptation and malfunctioning adaptation-are mutually exclusive and require a common research strategy. One must identify and map out the relevant adaptation-characterized by a high degree of non-random organization and coordination for promoting a function-which will eventually result in a conceptual blueprint of where and how the adaptation can malfunction. Using inescapable shock in rats and physicians' emotional responses to medical errors to provide context, we show how the symptoms of melancholic depression exhibit signs of adaptation for motivating a time-consuming, attentionally-demanding, energetically-expensive avoidant learning style after experiencing a harmful event. We discuss how this adaptationist approach may provide insight into spontaneous remission and the effects of psychotherapies and antidepressant medications.

Individual Differences in Response to Antidepressants: A Meta-analysis of Placebo-Controlled Randomized Clinical Trials.

Importance: Antidepressants are commonly used worldwide to treat major depressive disorder. Symptomatic response to antidepressants can vary depending on differences between individuals; however, this variability may reflect nonspecific or random factors. Objectives: To investigate the assumption of systematic variability in symptomatic response to antidepressants and to assess whether this variability is associated with severity of major depressive disorder, antidepressant class, or year of study publication. Data Sources: Data used were from a recent network meta-analysis of acute treatment with licensed antidepressants in adults with major depressive disorder. The following databases were searched from inception to January 8, 2016: the Cochrane Central Register of Controlled Trials, CINAHL, Embase, LILACS database, MEDLINE, MEDLINE In-Process, and PsycINFO. Additional sources were international trial registries, drug approval agency websites, and key scientific journals. Study Selection: Analysis was restricted to double-blind, randomized placebo-controlled trials with available data at the study's end point. Data Extraction and Synthesis: Baseline and end point means, SDs, number of participants in each group, antidepressant class, and publication year were extracted. The data were analyzed between August 14 and November 18, 2019. Main Outcomes and Measures: With the use of validated methods, coefficients of variation were derived for antidepressants and placebo, and their ratios were calculated to compare outcome variability between antidepressant and placebo. Ratios were entered into a random-effects model, with the expectation that response to antidepressants would be more variable than response to placebo. Analysis was repeated after stratifying by baseline severity of depression, antidepressant class (selective serotonin reuptake inhibitors: citalopram, escitalopram, fluoxetine, fluvoxamine, paroxetine, sertraline, and vilazodone; serotonin and norepinephrine reuptake inhibitors: desvenlafaxine and venlafaxine; norepinephrine-dopamine reuptake inhibitor: bupropion; noradrenergic agents: amitriptyline and reboxetine; and other antidepressants: agomelatine, mirtazapine, and trazodone), and publication year. Results: In the 87 eligible randomized placebo-controlled trials (17 540 unique participants), there was significantly more variability in response to antidepressants than to placebo (coefficients of variation ratio, 1.14; 95% CI, 1.11-1.17; P < .001). Baseline severity of depression did not moderate variability in response to antidepressants. Variability in response to selective serotonin reuptake inhibitors was lower than variability in response to noradrenergic agents (coefficients of variation ratio, 0.88; 95% CI, 0.80-0.97; P = .01), as was the variability in response to other antidepressants compared with noradrenergic agents (coefficients of variation ratio, 0.87; 95% CI, 0.79-0.97; P = .001). Variability also tended to be lower in studies that were published more recently, with coefficients of variation changing by a value of 0.005 (95% CI, 0.002-0.008; P = .003) for every year a study is more recent. Conclusions and Relevance: Individual differences may be systematically associated with responses to antidepressants in major depressive disorder beyond placebo effects or statistical factors. This study provides empirical support for identifying moderators and personalizing antidepressant treatment.

Testing the Analytical Rumination Hypothesis: Exploring the Longitudinal Effects of Problem Solving Analysis on Depression.

Depression is a mental health condition for which individuals commonly seek treatment. However, depressive episodes often resolve on their own, even without treatment. One evolutionary perspective, the analytical rumination hypothesis (ARH), suggests that depression occurs in response to complex problems. According to this perspective, depressive symptoms promote analytical rumination, i.e., distraction-resistant thoughts about the causes of problems [causal analysis (CA)] and how they can be solved [problem-solving analysis (PSA)]. By helping individuals solve complex problems, analytical rumination may contribute to remission from depression. The aim of this study was to investigate (1) whether clinically-depressed individuals have more complex problems and engage in more CA and PSA than non-depressed and (2) the effects of CA and PSA on decreases in problem complexity, depressive symptoms, and remission from the depression. Samples of 85 patients were treated for depression with antidepressants and psychotherapy, and 49 healthy subjects were assessed three times over a 4-month period (at Weeks 1, 5, and 16). At each assessment, they completed measures of depression, analytical rumination, and problem complexity. Depressed individuals reported having more complex problems and engaging in more CA than non-depressed participants. The two groups engaged in a similar degree of PSA. Findings from a multiple regression suggested that more PSA at Week 1 was related to a decrease in depressive symptoms at Week 5, even after controlling for baseline depression, problem number, and complexity. PSA at Week 1 did not predict the remission after hospitalization or at follow-up; however, having less complex problems at the baseline made it more likely that a patient would later remit. Engaging in more CA or PSA at Week 1 did not affect perceived problem complexity at Week 5 or at follow-up. However, these findings were not statistically significant when influential observations (or outliers) were included in the analysis. Our findings suggest that PSA may contribute to a decrease in symptoms of depression over time. However, alleviations in problem complexity and remission might only be achieved if problems are initially less complex. Future directions involve exploring how PSA might contribute to decreases in depressive symptoms and other mechanisms underlying remission from depression.

A randomized controlled dismantling trial of post-workshop consultation strategies to increase effectiveness and fidelity to an evidence-based psychotherapy for Posttraumatic Stress Disorder.

BACKGROUND: Posttraumatic Stress Disorder (PTSD) is a serious mental health condition with substantial costs to individuals and society. Among military veterans, the lifetime prevalence of PTSD has been estimated to be as high as 20%. Numerous research studies have demonstrated that short-term cognitive-behavioral psychotherapies, such as Cognitive Processing Therapy (CPT), lead to substantial and sustained improvements in PTSD symptoms. Despite known benefits, only a minority of clinicians provide these therapies. Transferring this research knowledge into clinical settings remains one of the largest hurdles to improving the health of veterans with PTSD. Attending a workshop alone is insufficient to promote adequate knowledge transfer and sustained skill; however, relatively little research has been conducted to identify effective post-training support strategies. METHODS: The current study investigates whether clinicians receiving post-workshop support (six-month duration) will deliver CPT with greater fidelity (i.e., psychotherapy adherence and competence) and have improved patient outcomes compared with clinicians receiving no formal post-workshop support. The study conditions are: technology-enhanced group tele-consultation; standard group tele-consultation; and fidelity assessment with no consultation. The primary outcome is independent assessment (via audio-recordings) of the clinicians' adherence and competence in delivering CPT. The secondary outcome is observed changes in patient symptoms during and following treatment as a function of clinician fidelity. Post-consultation interviews with clinicians will help identify facilitators and barriers to psychotherapy skill acquisition. The study results will inform how best to implement and transfer evidence-based psychotherapy (e.g., CPT) to clinical settings to attain comparable outcomes to those observed in research settings. DISCUSSION: Findings will deepen our understanding of how much and what type of support is needed following a workshop to help clinicians become proficient in delivering a new protocol. Several influences on clinician learning and patient outcomes will be discussed. An evidence-based model of clinical consultation will be developed, with the ultimate goal of informing policy and influencing best practice in clinical consultation. TRIAL REGISTRATION: ClinicalTrials.gov: NCT01861769.