RESUMEN
Purpose: A pragmatic, cluster-randomized controlled trial of a comprehensive practice-level, multi-staged practice transformation intervention aimed to increase behavioral health integration in primary care practices and improve patient outcomes. We examined association between the completion of intervention stages and patient outcomes across a heterogenous national sample of primary care practices. Methods: Forty-two primary care practices across the U.S. with co-located behavioral health and 2,426 patients with multiple chronic medical and behavioral health conditions completed surveys at baseline, midpoint and two year follow-up. Effects of the intervention on patient health and primary care integration outcomes were examined using multilevel mixed-effects models, while controlling for baseline outcome measurements. Results: No differences were found associated with the number of intervention stages completed in patient health outcomes were found for depression, anxiety, fatigue, sleep disturbance, pain, pain interference, social function, patient satisfaction with care or medication adherence. The completion of each intervention stage was associated with increases in Practice Integration Profile (PIP) domain scores and were confirmed with modeling using multiple imputation for: Workflow 3.5 (95% CI: 0.9-6.1), Integration Methods 4.6 (95% CI: 1.5-7.6), Patient Identification 2.9 (95% CI: 0.9-5.0), and Total Integration 2.7 (95% CI: 0.7-4.7). Conclusion: A practice-centric flexible practice transformation intervention improved integration of behavioral health in primary care across heterogenous primary care practices treating patients with multiple chronic conditions. Interventions that allow practices to flexibly improve care have potential to help complex patient populations. Future research is needed to determine how to best target patient health outcomes at a population level.
RESUMEN
PURPOSE: Patient outcomes can improve when primary care and behavioral health providers use a collaborative system of care, but integrating these services is difficult. We tested the effectiveness of a practice intervention for improving patient outcomes by enhancing integrated behavioral health (IBH) activities. METHODS: We conducted a pragmatic, cluster randomized controlled trial. The intervention combined practice redesign, quality improvement coaching, provider and staff education, and collaborative learning. At baseline and 2 years, staff at 42 primary care practices completed the Practice Integration Profile (PIP) as a measure of IBH. Adult patients with multiple chronic medical and behavioral conditions completed the Patient-Reported Outcomes Measurement Information System (PROMIS-29) survey. Primary outcomes were the change in 8 PROMIS-29 domain scores. Secondary outcomes included change in level of integration. RESULTS: Intervention assignment had no effect on change in outcomes reported by 2,426 patients who completed both baseline and 2-year surveys. Practices assigned to the intervention improved PIP workflow scores but not PIP total scores. Baseline PIP total score was significantly associated with patient-reported function, independent of intervention. Active practices that completed intervention workbooks (n = 13) improved patient-reported outcomes and practice integration (P ≤ .05) compared with other active practices (n = 7). CONCLUSION: Intervention assignment had no effect on change in patient outcomes; however, we did observe improved patient outcomes among practices that entered the study with greater IBH. We also observed more improvement of integration and patient outcomes among active practices that completed the intervention compared to active practices that did not. Additional research is needed to understand how implementation efforts to enhance IBH can best reach patients.
Asunto(s)
Afecciones Crónicas Múltiples , Adulto , Humanos , Atención Primaria de SaludRESUMEN
Comments on the original article by French, et al. (see record 2023-74550-001) regarding adapting psychotherapy in collaborative care for treating opioid use disorder and co-occurring psychiatric conditions in primary care. This article provided a potential treatment pathway for certain patients. However, it has left the question: What options are available for patients who decline to participate in this collaborative care approach? It is important to emphasize that we do not want to deny them access to treatment for their (opioid use disorder) OUD. Are there alternative interventions with a lower intensity that may still hold value for patients who opt out of this particular approach? In future research, the current authors would like to see expanded treatment pathways that match patients at all levels of engagement. By offering a more diverse range of options, we can better meet the needs of a broader spectrum of patients, ultimately improving treatment outcomes for individuals with OUD and psychiatric symptoms. (PsycInfo Database Record (c) 2023 APA, all rights reserved).
Asunto(s)
Trastornos Relacionados con Opioides , Psicoterapia , Humanos , Bases de Datos Factuales , Trastornos Relacionados con Opioides/complicaciones , Trastornos Relacionados con Opioides/terapia , Atención Primaria de SaludRESUMEN
INTRODUCTION: The Practice Integration Profile (PIP) is a reliable, valid, and broadly used measure of the integration of behavioral health (BH) into primary care. The PIP assesses operational and procedural elements that are grounded in the AHRQ Lexicon for Behavioral Health and Primary Care Integration. Prior analyses of PIP data and feedback from users suggested the measure was in need of revisions. This article describes the process used to improve readability, clarity, and pragmatic utility of the instrument. METHOD: Two rounds of structured cognitive interviews were conducted with clinicians in primary care settings. After each round, interview transcripts were coded by an analytic team using an iterative and consensus-driven process. Themes were identified based on codes. Themes and recommendations for revisions were reviewed and modified by committee. RESULTS: Based on feedback and a prior factor analysis of the PIP, revisions were undertaken to: (a) eliminate redundant or overlapping items; (b) clarify the meaning of items; (c) standardize the response categories, and (d) place items in the most appropriate domains. The resulting measure has 28 items in five domains. DISCUSSION: PIP 2.0 will need further examination to confirm its continuing use as a foundational tool for evaluating integrated care. (PsycInfo Database Record (c) 2023 APA, all rights reserved).
Asunto(s)
Comprensión , Psiquiatría , Humanos , Reproducibilidad de los Resultados , Evaluación de Resultado en la Atención de SaludRESUMEN
PURPOSE: Although many primary care clinics screen for behavioral health (BH) conditions using validated tools, it is not clear whether adult BH screening leads to better patient outcomes. Our objective was to determine the evidence base by reviewing effectiveness research for multiple strategies of BH screening in adult primary care identified in the Practice Integration Profile. METHOD: We conducted five rapid reviews of effectiveness research supporting BH screening strategies cited in the Practice Integration Profile. Each rapid review was conducted using an adapted REAL (Rapid Evidence Assessment of the Literature) methodology and a standardized search tailored for each screening strategy to identify evidence related to BH screening in primary care. RESULTS: The database search yielded 931 references. Following eligibility review and extraction, we evaluated data from 20 references examining five screening strategies. Results demonstrated limited support for all five strategies and high risk of bias within most studies. Outcomes associated with various BH screening strategies were rarely the focus of study. CONCLUSIONS: There is an absence of robust, well-structured evidence supporting many of the BH screening strategies advocated for in primary care. Stakeholders may wish to understand how to ensure value when developing a robust screening program that will improve patient health outcomes. Future research should advance the science of BH screen selection, timing, and implementation by answering new questions about screening strategies. (PsycInfo Database Record (c) 2022 APA, all rights reserved).
Asunto(s)
Tamizaje Masivo , Atención Primaria de Salud , Adulto , Atención a la Salud , Humanos , Tamizaje Masivo/métodos , Atención Primaria de Salud/métodosRESUMEN
Context: Most patients in need of behavioral health (BH) care are seen in primary care, which often has difficulty responding. Some practices integrate behavioral health care (IBH), with medical and BH providers at the same location, working as a team. However, it is difficult to achieve high levels of integration. Objective: Test the effectiveness of a practice intervention designed to increase BH integration. Study Design: Pragmatic, cluster-randomized controlled trial. Setting: 43 primary care practices with on-site BH services in 13 states. Population: 2,460 adults with multiple chronic medical and behavioral conditions. Intervention: 24-month practice change process including an online curriculum, a practice redesign and implementation workbook, remote quality improvement coaching services, and an online learning community. Outcomes: Primary outcomes were changes in the 8 Patient-Reported Outcomes Measurement Information System (PROMIS-29) domain scores. Secondary outcomes were changes in medication adherence, self-reported healthcare utilization, time lost due to disability, cardiovascular capacity, patient centeredness, provider empathy, and several condition-specific measures. A sample of practice staff completed the Practice Integration Profile at each time point to estimate the degree of BH integration in that site. Practice-level case studies estimated the typical costs of implementing the intervention. Results: The intervention had no significant effect on any of the primary or secondary outcomes. Subgroup analyses showed no convincing patterns of effect in any populations. COVID-19 was apparently not a moderating influence of the effect of the intervention on outcomes. The intervention had a modest effect on the degree of practice integration, reaching statistical significance in the Workflow domain. The median cost of the intervention was $18,204 per practice. In post-hoc analysis, level of BH integration was associated with improved patient outcomes independent of the intervention, both at baseline and longitudinally. Conclusions: The specific intervention tested in this study was inexpensive, but had only a small impact on the degree of BH integration, and none on patient outcomes. However, practices that had more integration at baseline had better patient outcomes, independent of the intervention. Although this particular intervention was ineffective, IBH remains an attractive strategy for improving patient outcomes.
Asunto(s)
Servicios de Salud Mental , Atención Primaria de Salud , Adulto , Humanos , Enfermedad Crónica , COVID-19 , Aceptación de la Atención de SaludRESUMEN
Valid measures of behavioral health integration have the potential to enable comparisons of various models of integration, contribute to the overall development of high-quality care, and evaluate outcomes that are strategically aligned with standard improvement efforts. The Practice Integration Profile has proven to discriminate among clinic types and integration efforts. We continued the validation of the measure's internal consistency, intra-rater consistency, and inter-rater consistency with a separate and larger sample from a broader array of practices. We found that the Practice Integration Profile demonstrated a high level of internal consistency, suggesting empirically sound measurement of independent attributes of integration, and high reliability over time. The Practice Integration Profile provides internally consistent and interpretable results and can serve as both a quality improvement and health services research tool.
Asunto(s)
Atención Primaria de Salud , Psiquiatría , Investigación sobre Servicios de Salud , Humanos , Atención Primaria de Salud/métodos , Mejoramiento de la Calidad , Reproducibilidad de los ResultadosRESUMEN
Background: Our rural health system sought to (1) increase the number of primary care clinicians waivered to prescribe buprenorphine for treatment of opioid use disorder (OUD) and (2) consequently increase the number of our patients receiving this treatment. Methods: We used the Project for Extension for Community Health Outcomes (ECHO) tele-education model as an implementation strategy. We examined the number of clinicians newly waivered, the number of patients treated with buprenorphine, the relationship between clinician engagement with ECHO training and rates of buprenorphine prescribing, and treatment retention at 180 days. Results: The number of clinicians with a waiver and number of patients treated increased during and after ECHO training. There was a moderate correlation between the number of ECHO sessions attended by a clinician and number of their buprenorphine prescriptions (r = 0.50, p = 0.01). The 180-day retention rate was 80.7%. Conclusions: Project ECHO was highly effective for increasing access to this evidence-based treatment. The high retention rate in this rural context indicates that most patients are increasing their likelihood of favorable outcomes.
Asunto(s)
Buprenorfina , Trastornos Relacionados con Opioides , Buprenorfina/uso terapéutico , Humanos , Tratamiento de Sustitución de Opiáceos , Trastornos Relacionados con Opioides/tratamiento farmacológico , Atención Primaria de Salud , Población RuralRESUMEN
The COVID-19 pandemic presented unique biological, psychological, and social threats to health care providers. The failure of local macrosystems placed providers at elevated risk of psychological and physical harm. To reduce the immediate risk of trauma to our local physician workforce, our team initiated a program of proactive psychological first aid in which physicians were regularly contacted by behavioral health colleagues to assess safety conditions and physician's well-being. When threats to the physician's safety were identified, these concerns were escalated to leadership and addressed when possible. When threats to well-being were identified, behavioral health team members provided supportive listening, and, if indicated, provided referral information for appropriate treatment resources. This paper reviews the rationale for this program, addresses ethical concerns, and proposes future directions for responding to threats to safety during events such as the COVID-19 pandemic. (PsycInfo Database Record (c) 2021 APA, all rights reserved).
Asunto(s)
COVID-19 , Trauma Psicológico , Personal de Salud , Humanos , Pandemias/prevención & control , Evaluación de Programas y Proyectos de Salud , SARS-CoV-2RESUMEN
Introduction: Many behavioral health providers have not received training in primary care practice during their education. Since 2007, the online Certificate in Primary Care Behavioral Health course has been completed by thousands of behavioral health providers. An evaluation of the course assessed whether learner's baseline confidence in their abilities to address behavioral health concerns in primary care settings changed over time, whether learning outcomes for live offerings of the course differed from asynchronous offerings, and whether learning outcomes for psychologists and social workers differed. Method: Learners were asked to rate their confidence in their abilities using 10 retrospective pre-post questions. Responses from 14 cohorts of learners, between 2011 and 2019 were assessed. T-tests and analyses of variance were used to compare groups. Results: Learners' baseline confidence in their abilities changed in three of the areas assessed. Those completing the course asynchronously reported outcomes equal to or greater than those completing the course synchronously. In all but one domain, psychologists and social workers reported equal increases in their confidence. Discussion: Learners reported significant improvements in confidence in their ability to work as behavioral health providers in primary care. With one exception, these findings did not differ for psychologists and social workers. Over time, baseline confidence of behavioral health providers enrolling in the course increased in some areas, particularly those focused on patients with substance use disorders. Learning outcomes for the asynchronous version of the course support the continued delivery of asynchronous training of behavioral health providers working in primary care. (PsycInfo Database Record (c) 2021 APA, all rights reserved).
Asunto(s)
Atención Primaria de Salud , Psiquiatría , Competencia Clínica , Humanos , Estudios RetrospectivosRESUMEN
BACKGROUND: Chronic diseases that drive morbidity, mortality, and health care costs are largely influenced by human behavior. Behavioral health conditions such as anxiety, depression, and substance use disorders can often be effectively managed. The majority of patients in need of behavioral health care are seen in primary care, which often has difficulty responding. Some primary care practices are providing integrated behavioral health care (IBH), where primary care and behavioral health providers work together, in one location, using a team-based approach. Research suggests there may be an association between IBH and improved patient outcomes. However, it is often difficult for practices to achieve high levels of integration. The Integrating Behavioral Health and Primary Care study responds to this need by testing the effectiveness of a comprehensive practice-level intervention designed to improve outcomes in patients with multiple chronic medical and behavioral health conditions by increasing the practice's degree of behavioral health integration. METHODS: Forty-five primary care practices, with existing onsite behavioral health care, will be recruited for this study. Forty-three practices will be randomized to the intervention or usual care arm, while 2 practices will be considered "Vanguard" (pilot) practices for developing the intervention. The intervention is a 24-month supported practice change process including an online curriculum, a practice redesign and implementation workbook, remote quality improvement coaching services, and an online learning community. Each practice's degree of behavioral health integration will be measured using the Practice Integration Profile. Approximately 75 patients with both chronic medical and behavioral health conditions from each practice will be asked to complete a series of surveys to measure patient-centered outcomes. Change in practice degree of behavioral health integration and patient-centered outcomes will be compared between the two groups. Practice-level case studies will be conducted to better understand the contextual factors influencing integration. DISCUSSION: As primary care practices are encouraged to provide IBH services, evidence-based interventions to increase practice integration will be needed. This study will demonstrate the effectiveness of one such intervention in a pragmatic, real-world setting. TRIAL REGISTRATION: ClinicalTrials.gov NCT02868983 . Registered on August 16, 2016.
Asunto(s)
Evaluación de Resultado en la Atención de Salud , Atención Primaria de Salud , Adulto , Costos de la Atención en Salud , Humanos , Atención Dirigida al Paciente , Ensayos Clínicos Controlados Aleatorios como Asunto , Encuestas y CuestionariosRESUMEN
OBJECTIVE: To perform a factor analysis of the Practice Integration Profile (PIP), a 30-item practice-level measure of primary care and behavioral health integration derived from the Agency for Healthcare Research and Quality's Lexicon for Behavioral Health and Primary Care Integration. DATA SOURCES: The PIP was completed by 735 individuals, representing 357 practices across the United States. STUDY DESIGN: The study design was a cross-sectional survey. An exploratory factor analysis and assessment of internal consistency reliability via Cronbach's alpha were performed. DATA COLLECTION METHODS: Participant responses were collected using REDCap, a secure, web-based data capture tool. PRINCIPAL FINDINGS: Five of the PIP's six domains had factor loadings for most items related to each factor representing the PIP of 0.50 or greater. However, one factor had items from two PIP domains that had loadings >0.50. A five-factor model with redistributed items resulted in improved factor loadings for all domains along with greater internal consistency reliability (>0.80). CONCLUSIONS: Five of the PIP's six domains demonstrated excellent internal consistency for measures of health care resources. Although minor improvements to strengthen the PIP are possible, it is a valid and reliable measure of the integration of primary care and behavioral health.
Asunto(s)
Encuestas de Atención de la Salud/normas , Servicios de Salud Mental/organización & administración , Atención Primaria de Salud/organización & administración , Integración de Sistemas , Conducta Cooperativa , Estudios Transversales , Análisis Factorial , Investigación sobre Servicios de Salud , Humanos , Grupo de Atención al Paciente/organización & administración , Psicometría , Reproducibilidad de los Resultados , Estados UnidosRESUMEN
Collaborative approaches to healthcare that integrate behavioural and biomedical interventions are more likely to enhance patient outcomes as well as provider satisfaction with care delivery than siloed approaches to care. The recognition for specific and targeted training for these models is growing among all health professions, although many in the field have not received systematized, interprofessional, and competency-based training that adequately prepared them for the work of integration. This article reviews some of the fundamental principles of biopsychosocially-oriented, team-based approaches to care that integrate behavioural and biomedical perspectives and delineates the need for targeted training efforts. It describes which specific elements must be addressed within it in order to promote effective integration, and highlights the array of options for training currently in existence. This review provides an overview of current models of training offered in the US, and concludes with a discussion of the challenges and barriers that may render training either ineffective or difficult to achieve.
Asunto(s)
Prestación Integrada de Atención de Salud/métodos , Fuerza Laboral en Salud , Servicios de Salud Mental/organización & administración , Enseñanza/educación , Conducta Cooperativa , Conocimientos, Actitudes y Práctica en Salud , Humanos , Atención Primaria de Salud/organización & administración , PsiquiatríaRESUMEN
The inaugural Collaborative Family Health Care (CFHA) regional conference took place March 17, 2017, in St. Louis, Missouri. The conference theme was opioid use disorders and the potential of medication-assisted treatment using team-based care to address this epidemic. The conference ended with an emphasis on the importance of and strategies for self-care among caregivers working with this population and their families. This intensive meeting was crafted as a proof in concept for CFHA; however, the content was timely with regard to its importance in health care as well as compelling in the opportunity for collaborative care to offer an effective approach. In this article, the authors highlight five impotance topics: (1) opioid use disorders should be addressed in primary care; (2) a team is essential; (3) harm reduction; (4) sustaining self through mindful practice and self-compassion; and (5) medication-assisted treatment. (PsycINFO Database Record
Asunto(s)
Prestación Integrada de Atención de Salud/métodos , Epidemias/estadística & datos numéricos , Trastornos Relacionados con Opioides/terapia , Atención Primaria de Salud/tendencias , Conducta Cooperativa , Prestación Integrada de Atención de Salud/tendencias , Humanos , Missouri , Atención Primaria de Salud/métodos , Evaluación de Programas y Proyectos de Salud/métodos , Evaluación de Programas y Proyectos de Salud/normas , AutocuidadoRESUMEN
INTRODUCTION: We developed the Practice Integration Profile (PIP) to measure the degree of behavioral health integration in clinical practices with a focus on primary care (PC). Its 30 items, completed by providers, managers, and staff, provide an overall score and 6 domain scores derived from the Lexicon of Collaborative Care. We describe its history and psychometric properties. METHOD: The PIP was tested in a convenience sample of practices. Linear regression compared scores across integration exemplars, PC with behavioral services, PC without behavioral services, and community mental health centers without PC. An additional sample rated 4 scenarios describing practices with varying degrees of integration. RESULTS: One hundred sixty-nine surveys were returned. Mean domain scores ran from 49 to 65. The mean total score was 55 (median 58; range 0-100) with high internal consistency (Cronbach's alpha = .95). The lowest total scores were for PC without behavioral health (27), followed by community mental health centers (44), PC with behavioral health (60), and the exemplars (86; p < .001). Eleven respondents rerated their practices 37 to 194 days later. The mean change was + 1.5 (standard deviation = 11.1). Scenario scores were highly correlated with the degree of integration each scenario was designed to represent (Spearman's ρ = -0.71; P = 0.0005). DISCUSSION: These data suggest that the PIP is useful, has face, content, and internal validity, and distinguishes among types of practices with known variations in integration. We discuss how the PIP may support practices and policymakers in their integration efforts and researchers assessing the degree to which integration affects patient health outcomes. (PsycINFO Database Record
Asunto(s)
Medicina de la Conducta/clasificación , Atención Primaria de Salud/métodos , Desarrollo de Programa/métodos , Evaluación de Programas y Proyectos de Salud/métodos , Medicina de la Conducta/métodos , Humanos , Modelos Lineales , Servicios de Salud Mental/clasificación , Atención Primaria de Salud/clasificación , Atención Primaria de Salud/tendencias , Reproducibilidad de los Resultados , Encuestas y Cuestionarios , Estados UnidosRESUMEN
Insufficient knowledge exists regarding how to measure the presence and degree of integrated care. Prior estimates of integration levels are neither grounded in theory nor psychometrically validated. They provide scant guidance to inform improvement activities, compare integration efforts, discriminate among practices by degree of integration, measure the effect of integration on quadruple aim outcomes, or address the needs of clinicians, regulators, and policymakers seeking new models of health care delivery and funding. We describe the development of the Practice Integration Profile (PIP), a novel instrument designed to measure levels of integrated behavioral health care within a primary care clinic. The PIP draws upon the Agency for Health care Research & Quality's (AHRQ) Lexicon of Collaborative Care which provides theoretic justification for a paradigm case of collaborative care. We used the key clauses of the Lexicon to derive domains of integration and generate measures corresponding to those key clauses. After reviewing currently used methods for identifying collaborative care, or integration, and identifying the need to improve on them, we describe a national collaboration to describe and evaluate the PIP. We also describe its potential use in practice improvement, research, responsiveness to multiple stakeholder needs, and other future directions. (PsycINFO Database Record
Asunto(s)
Prestación Integrada de Atención de Salud/clasificación , Atención Primaria de Salud/normas , Evaluación de Procesos, Atención de Salud/tendencias , Humanos , Atención Primaria de Salud/métodos , Atención Primaria de Salud/tendencias , Mejoramiento de la CalidadRESUMEN
INTRODUCTION: This study examines the outcomes of a 22-hr motivational interviewing (MI) course and compares online and in-person offerings of the course. It also evaluates clinicians' ability to accurately self-assess their MI skills. METHOD: 34 clinicians participated in this study and completed MI workshops either in-person or online. Use of MI in an acting patient encounter was recorded early in the training and again following the training. Recordings of these encounters were coded using the Motivational Interviewing Treatment Integrity (MITI) 3.1 coding system. After each acting patient encounter clinicians also self-evaluated their use of MI. RESULTS: Participants showed statistically significant improvement in MI skills measured by the MITI. There were no meaningful differences between the MI skills acquired by the participants in the online group compared with those who completed training in-person. There was little correlation between participants' self-assessment of MI skills and objective assessment. DISCUSSION: It is feasible to complete MI training through synchronous online workshops. Participant self-assessment of MI skill does not appear to be a useful approach for assessing MI skill. The acquisition of MI skills by health professionals is possible via the Internet. Learning should be assessed using objective measures rather than relying on self-report. (PsycINFO Database Record
Asunto(s)
Educación/normas , Personal de Salud/educación , Entrevista Motivacional/normas , Evaluación de Programas y Proyectos de Salud/métodos , Competencia Clínica/normas , Humanos , Internet , Entrevista Motivacional/métodos , Reproducibilidad de los Resultados , Autoevaluación (Psicología)RESUMEN
INTRODUCTION: Motivational interviewing (MI) skills are relevant for primary care providers (PCPs) who are responsible for caring for patients with diseases affected by behavior. There are significant challenges associated with developing PCP's MI skills. We report on an effort to document the acquisition of MI skills by PCPs using an objective measure of MI competence, the Motivational Interviewing Treatment Integrity (MITI) coding system. METHOD: Eleven PCPs volunteered to participate in 6 MI workshops over a period of 6 months and to submit work samples between each of these workshops to be assessed with the MITI coding system. RESULTS: Thirteen of the expected 55 work samples were submitted before the final workshop. A revised approach was implemented in which each participant completed 2 simulated patient encounters. None of the providers reached the MITI's Beginning Proficiency threshold of MI skill. DISCUSSION: Six MI workshops were not sufficient to help motivated PCPs achieve Beginning Proficiency as measured by the MITI. Participants failed to submit most of the work samples for feedback on their MI practice, which may have contributed to their limited acquisition of MI skills. Helping PCPs develop MI skills likely requires more than participation in a series of workshops totaling 18 h. Questions remain about the feasibility of training PCPs to be competent in MI. Approaches such as use of simulated patients, peer observation, or specific protected time for obtaining work samples may be required. (PsycINFO Database Record