Adjusting for race in metrics of organ procurement organization performance.

The Scientific Registry of Transplant Recipients has previously reported the effects of adjusting for demographic variables, including race, in the Centers for Medicare & Medicaid Services (CMS) organ procurement organization (OPO) performance metrics: donation rate and transplant rate. CMS chose not to adjust for most demographic variables other than age (for the transplant rate), arguing that there is no biological reason that these variables would affect the organ donation/utilization decision. However, organ donation is a process based on altruism and trust, not a simple biological phenomenon. Focusing only on biological impacts on health ignores other pathways through which demographic factors can influence OPO outcomes. In this study, we update analyses of demographic adjustment on the OPO metrics for 2020 with a specific focus on adjusting for race. We find that adjusting for race would lead to 8 OPOs changing their CMS tier rankings, including 2 OPOs that actually overperform the national rate among non-White donors improving from a tier 3 ranking (facing decertification without possibility of recompeting) to a tier 2 ranking (allowing the possibility of recompeting). Incorporation of stratified and risk-adjusted metrics in public reporting of OPO performance could help OPOs identify areas for improvement within specific demographic categories.

Expanding Access to Comprehensive Geriatric Evaluation via Telehealth: Development of Hybrid-Virtual Home Visits.

BACKGROUND: In response to the aging population, the Department of Veterans Affairs (VA) seeks to expand access to evidence-based practices which support community-dwelling older persons such as the Geriatric Resources for Assessment and Care of Elders (GRACE) program. GRACE is a multidisciplinary care model which provides home-based geriatric evaluation and management for older Veterans residing within a 20-mile drive radius from the hospital. We sought to expand the geographic reach of VA-GRACE by developing a hybrid-virtual home visit (TeleGRACE). OBJECTIVES: The objectives were to: (1) describe challenges encountered and solutions implemented during the iterative, pre-implementation program development process; and (2) illustrate potential successes of the program with two case examples. DESIGN: Quality improvement project with longitudinal qualitative data collection. PROGRAM DESCRIPTION: The hybrid-virtual home visit involved a telehealth technician travelling to patients' homes and connecting virtually to VA-GRACE team members who participated remotely. APPROACH & PARTICIPANTS: We collected multiple data streams throughout program development: TeleGRACE staff periodic reflections, fieldnotes, and team meeting notes; and VA-GRACE team member interviews. KEY RESULTS: The five program domains that required attention and problem-solving were: telehealth connectivity and equipment, virtual physical examination, protocols and procedures, staff training, and team integration. For each domain, we describe several challenges and solutions. An example from the virtual physical examination domain: several iterations were required to identify the combination of telehealth stethoscope with dedicated headphones that allowed remote nurse practitioners to hear heart and lung sounds. The two cases illustrate how this hybrid-virtual home visit model provided care for patients who would not otherwise have received timely healthcare services. CONCLUSIONS: These results provide a blueprint to translate an in-person home-based geriatrics program into a hybrid-virtual model and support the feasibility of using hybrid-virtual home visits to expand access to comprehensive geriatric evaluation and ongoing care for high-risk, community-dwelling older persons who reside geographically distant from the primary VA facility.

Synthetic Controls for Implementation Science: Opportunities for HIV Program Evaluation Using Routinely Collected Data.

PURPOSE OF REVIEW: HIV service delivery programs are some of the largest funded public health programs in the world. Timely, efficient evaluation of these programs can be enhanced with methodologies designed to estimate the effects of policy. We propose using the synthetic control method (SCM) as an implementation science tool to evaluate these HIV programs. RECENT FINDINGS: SCM, introduced in econometrics, shows increasing utility across fields. Key benefits of this methodology over traditional design-based approaches for evaluation stem from directly approximating pre-intervention trends by weighting of candidate non-intervention units. We demonstrate SCM to evaluate the effectiveness of a public health intervention targeting HIV health facilities with high numbers of recent infections on trends in pre-exposure prophylaxis (PrEP) enrollment. This test case demonstrates SCM's feasibility for effectiveness evaluations of site-level HIV interventions. HIV programs collecting longitudinal, routine service delivery data for many facilities, with only some receiving a time-specified intervention, are well-suited for evaluation using SCM.

Asynchronous online training increased knowledge, confidence, and intent to screen and treat substance use disorders among primary care clinicians practicing in Texas.

INTRODUCTION: Despite increasing prevalence of substance use disorders (SUD), few patients are diagnosed and treated in primary care settings. This study evaluated changes in knowledge, confidence, and intention to implement screening and brief intervention (SBI) and SUD treatment after clinicians participated in an asynchronous online education course. METHODS: A self-selected sample of primary care clinicians in Texas participated in online SBI and SUD education March 2021-July 2023. Baseline and post-training surveys evaluated changes in knowledge, confidence, and intent to implement SBI and SUD treatment. Changes were compared using paired t-tests. Multivariable linear regression examined factors potentially associated with confidence and intention to implement changes. Clinician feedback regarding the course was included. RESULTS: Of 613 respondents, 50.9% were practicing family medicine clinicians. Knowledge of adolescent screening tools increased from 21.9% to 75.7% (p < 0.001). Knowledge about the number of drinks that define excessive drinking among non-pregnant women increased from 24.5% at baseline to 64.9% (p < 0.001). Clinicians reported lowest confidence in providing opioid use disorder pharmacotherapy, which improved after program participation. Intent to implement SBI and medication for alcohol, nicotine and opioid use disorders increased (p < 0.001) after training. No factors were associated with change in confidence or intention to implement in multivariable models (p > 0.05). Satisfaction was high and nearly 60% reported intention to change their clinical practice because of the program. CONCLUSION: Knowledge, confidence, and intent to implement SBI and SUD treatment increased after completing the online course. Clinician satisfaction was high and demonstrated improved intention to implement SBI and SUD treatment.

Adapting Schools to Climate Change with Green, Blue, and Grey Measures in Barcelona: Study Protocol of a Mixed-Method Evaluation.

Under the framework of the Urban Innovative Actions program of the European Commission, in 2020, 11 primary schools in Barcelona were transformed into climate shelters by implementing green, blue, and grey measures. Schoolyards were also opened to the local community to be used during non-school periods. Here we present the study protocol of a mixed-method approach to evaluate the effectiveness of the interventions in terms of improving environmental quality and health for users. We evaluated school level through the following: (1) quantitative pre-post quasi-experimental study, and (2) qualitative evaluation. The quantitative study included measures of (a) environmental variables (collected via low-cost and non-low-cost sensors), (b) students' health and well-being (collected via health questionnaires, attention levels test, and systematic observations), and (c) teachers' health and well-being (collected via thermal comfort measurements and health questionnaires). The qualitative methods evaluated the perceptions about the effects of the interventions among students (using Photovoice) and teachers (through focus groups). The impact of the interventions was assessed at community level during summer non-school periods through a spontaneous ethnographic approach. Data collection started in August 2019 and ended in July 2022. The evaluation provides the opportunity to identify those solutions that worked and those that need to be improved for future experiences, as well as improve the evaluation methodology and replication for these kinds of interventions.

The Australian Health Care Homes trial: quality of care and patient outcomes. A propensity score-matched cohort study.

OBJECTIVE: To assess the impact of the Health Care Homes (HCH) primary health care initiative on quality of care and patient outcomes. DESIGN, SETTING: Quasi-experimental, matched cohort study; analysis of general practice data extracts and linked administrative data from ten Australian primary health networks, 1 October 2017 - 30 June 2021. PARTICIPANTS: People with chronic health conditions (practice data extracts: 9811; linked administrative data: 10 682) enrolled in the HCH 1 October 2017 - 30 June 2019; comparison groups of patients receiving usual care (1:1 propensity score-matched). INTERVENTION: Participants were involved in shared care planning, provided enhanced access to team care, and encouraged to seek chronic condition care at the HCH practice where they were enrolled. Participating practices received bundled payments based on clinical risk tier. MAIN OUTCOME MEASURES: Access to care, processes of care, diabetes-related outcomes, hospital service use, risk of death. RESULTS: During the first twelve months after enrolment, the mean numbers of general practitioner encounters (rate ratio, 1.14; 95% confidence interval [CI], 1.11-1.17) and Medicare Benefits Schedule claims for allied health services (rate ratio, 1.28; 95% CI, 1.24-1.33) were higher for the HCH than the usual care group. Annual influenza vaccinations (relative risk, 1.20; 95% CI, 1.17-1.22) and measurements of blood pressure (relative risk, 1.09; 95% CI, 1.08-1.11), blood lipids (relative risk, 1.19; 95% CI, 1.16-1.21), glycated haemoglobin (relative risk, 1.06; 95% CI, 1.03-1.08), and kidney function (relative risk, 1.13; 95% CI, 1.11-1.15) were more likely in the HCH than the usual care group during the twelve months after enrolment. Similar rate ratios and relative risks applied in the second year. The numbers of emergency department presentations (rate ratio, 1.09; 95% CI, 1.02-1.18) and emergency admissions (rate ratio, 1.13; 95% CI, 1.04-1.22) were higher for the HCH group during the first year; other differences in hospital use were not statistically significant. Differences in glycaemic and blood pressure control in people with diabetes in the second year were not statistically significant. By 30 June 2021, 689 people in the HCH group (6.5%) and 646 in the usual care group (6.1%) had died (hazard ratio, 1.07; 95% CI, 0.96-1.20). CONCLUSIONS: The HCH program was associated with greater access to care and improved processes of care for people with chronic diseases, but not changes in diabetes-related outcomes, most measures of hospital use, or risk of death.

Evaluation of the Cultural, Social and Emotional Wellbeing Program with Aboriginal women in the Boronia Pre-Release Centre for Women: a mixed methods study.

OBJECTIVE: To assess the effectiveness of the Cultural, Social and Emotional Wellbeing Program for reducing psychological distress and enhancing the social and emotional wellbeing of Aboriginal women preparing for release from prison. STUDY DESIGN: Mixed methods; qualitative study (adapted reflexive thematic analysis of stories of most significant change) and assessment of psychological distress. SETTING, PARTICIPANTS: Aboriginal and Torres Strait Islander women at the Boronia Pre-release Centre for Women, Perth, Western Australia, May and July 2021. INTERVENTION: Cultural, Social and Emotional Wellbeing Program (two days per week for six weeks). The Program involves presentations, workshops, activities, group discussions, and self-reflections designed to enhance social and emotional wellbeing. MAIN OUTCOME MEASURES: Themes and subthemes identified from reflexive thematic analysis of participants' stories of most significant change; change in mean psychological distress, as assessed with the 5-item Kessler Scale (K-5) before and after the Program. RESULTS: Fourteen of 16 invited women completed the Program; ten participated in its evaluation. They reported improved social and emotional wellbeing, reflected as enhanced connections to culture, family, and community. Mean psychological distress was lower after the Program (mean K-5 score, 11.3; 95% confidence interval [CI], 9.0-13.6) than before the Program (9.0; 95% CI, 6.5-11.5; P = 0.047). CONCLUSION: The women who participated in the Program reported personal growth, including acceptance of self and acceptance and pride in culture, reflecting enhanced social and emotional wellbeing through connections to culture and kinship. Our preliminary findings suggest that the Program could improve the resilience of Aboriginal and Torres Strait Islander in contact with the justice system.

Simulation based education in paediatric resuscitation.

There is increasing use of clinical Simulation Based Education (SBE) in healthcare due to an increased focus on patient safety, the call for a new training model not based solely on apprenticeship, a desire for standardised educational opportunities that are available on-demand, and a need to practice and hone skills in a controlled environment. SBE programs should be evaluated against Kirkpatrick level 3 or 4 criteria to ensure they improve patient or staff outcomes in the real world. SBE programs have been shown to improve outcomes in neonatology - reductions in hypoxic ischaemic encephalopathy, in brachial plexus injury, rates of school age cerebral palsy, reductions in 24hr mortality and improvements in first pass intubation rates. In paediatrics SBE programs have shown improvements in paediatric cardiac arrest survival, PICU survival, reduced PICU admissions, reduced PICU length of stay and reduced time to critical operations. SBE can improve the non-technical tasks of teamwork, leadership and communication (within the team and with patients and carers). Simulation is a useful tool in Quality and Safety and is used to identify latent safety issues that can be addressed by future programs. In high stakes assessment simulation can be a mode of assessment, however, care needs to be taken to ensure the tool is validated carefully.

Experiences and perceived benefits of remotely delivered dance/movement therapy for adult cancer patients: a multi-method program evaluation.

BACKGROUND: Dance/movement therapy (DMT) is increasingly used as a complementary treatment to address psychological and physical wellbeing. However, it is unknown how it can be leveraged in adult cancer care. This mixed methods program evaluation aimed to assess patient-reported benefits and satisfaction with the virtual DMT in an academic oncology setting. METHODS: We developed, implemented, and evaluated a 6-week virtual, synchronous DMT program aiming to improve physical health, address mental distress, and foster social connection for cancer patients. We used deidentified program evaluation data to assess impact of DMT on patient-reported outcomes and patients' satisfaction with the DMT program. Pre- and post-session data were analyzed using descriptive statistics and a paired t-test. Qualitative data were captured through semi-structured interviews and analyzed using thematic analysis. RESULTS: Results from 39 participants (mean age 64.7 ± 9.8), majority female (89.7%) with a history of breast cancer (43.6%), showed high satisfaction (100%) and unanimous program recommendation (100%). Significant improvements were noted in anxiety (- 0.42 ± 0.76, p = .009), distress (- 0.35 ± 0.80, p = .036), and sense of joy (0.73 ± 1.18, p = .004), with a non-significant trend in increased physical activity (0.38 ± 0.98, p = .057). Thematic findings indicated that DMT participation (1) facilitated engagement in physical activity for improved physical health, (2) fostered creative expression, (3) improved mental state, and (4) helped build social connections and support. CONCLUSION: Our DMT program shows promise as a component of integrative cancer care. The mixed-method evaluation provides insightful information to generate hypotheses for future RCT studies aiming to evaluate the specific effects of DMT on patient experience and outcomes.

Reducing risk for chronic disease: evaluation of a collective community approach to sustainable evidence-based health programming.

BACKGROUND: Community initiatives can shape health behaviors, such as physical activity and dietary habits, across a population and help reduce the risk of developing chronic disease. To achieve this goal and impact health outcomes, Pasadena Vibrant Community aimed to engage communities in an ongoing dialogue about the importance of healthy behaviors, implement and advance community-based strategies to promote health, and improve diet and physical activity behaviors. The initiative was centered around a collaboration between a backbone organization, steering committee, and 7 collaborating organizations funded to implement multicomponent, evidence-based programs.. The common agenda was detailed in a community action plan, which included 19 interventions targeting healthy eating and active living among adults and youth in Pasadena, Texas. METHODS: A mixed methods evaluation of the initiative was conducted over 4 years. Data sources included document reviews of quarterly progress reports (n = 86) and supplemental data reports (n = 16) provided by collaborating organizations, annual Steering Committee surveys (n = 4), and interviews conducted with staff from a subset of Collaborating Organizations (n = 4). RESULTS: The initiative reached over 50,000 community members per year through 19 evidence-based interventions and impacted health outcomes, including knowledge and adoption of healthy eating practices and increased physical activity. Thirty-one systems-level changes were implemented during the initiative, including 16 environmental changes. Steering Committee meetings and shared goals enabled connections, communication, and cooperation, which allowed Collaborating Organizations to address challenges and combine resources to deliver their programs. CONCLUSIONS: Community initiatives can effectively permeate the community by reaching individuals, improving physical activity and dietary habits, and ensuring sustainability. Based on the experience reported here, the success of a community initiative can be facilitated if collaborating organizations come together to implement evidence-based interventions and tailor them to the community, and if they are empowered by significant leadership and supportive collaboration and aligned by a common agenda.

Promoting healthful food purchases through in-store interventions: Empirical evidence from rural food deserts.

Effective ways to promote healthful food intake in rural areas are understudied. The paper evaluated whether a two-component, in-store intervention designed to encourage healthy food purchases was associated with improved healthfulness scores of food items purchased by shoppers in rural food deserts. One component introduced a point-of-sales label that assigned a single numerical score to each food item facilitating direct comparisons of the product's nutrition with those of other products shelved around it. The other component was a one-day nutrition education workshop promoted within the store. Interventions took place in 2015 at two stores in rural counties in the U.S. Midwest. Four stores in similar communities were selected as the control group. We applied a difference-in-difference model to estimate changes in the healthfulness of food items purchased attributable to the intervention among shoppers at the treatment stores (n = 486) and control stores (n = 10,759) using store transaction data. Healthfulness of food items was measured in terms of food scores published by the Environmental Working Group on a 1-10 scale. Both components had minimal impacts on the scores, although 0.2 and 0.1 points increases in the score per item and score per dollar were statistically significant at the 1% level respectively. A year after the intervention, these small effects of the intervention further diminished compared to the immediately after implementation. Results suggest the average effects of intervention across the study communities had limited practical significance but benefited some rural residents who were exposed to the intervention.

Implementation of a patient reminder system in Haiti in a socio-political crisis context: evaluation of outcomes.

BACKGROUND: In Haiti, patient's adherence to treatment and compliance with medical appointments are very challenging due to different local factors. We aimed to assess the effectiveness of a reminder system implemented in health facilities in Haiti in a context of socio-political crisis. METHODS: We used appointment data from patients aged 15 years and older between January 2021 and November 2023 from four healthcare centers in the Port-au-Prince metropolitan area. We performed descriptive analysis, crossing covariates with appointment attendance. We performed Pearson's Chi-squared test, and multivariate regression analysis using a mixed-effect logistic regression model in order to explore the association between sending reminders and appointment attendance, with and without adjustment for other patient-level covariates. RESULTS: A total of 14 108 appointments were registered on the reminder systems, with 2 479 (17.6%) attendances. Among those to whom reminders were sent, the number of attendances was 167 (17.4%) for email recipients only, 199 (36.7%) for SMS recipients only, and 19 (42.2%) for both SMS and email recipients - versus 2 094 (16.7%) for non-reminders. After adjusting on all other covariates, we found that patients to whom a reminder was sent via email (aOR: 1.45; CI: 1.08, 1.94), SMS (aOR: 2.95; CI: 2.41, 3.60), and both SMS and email (aOR: 2.86, CI: 1.37, 5.96) were more likely to show up on their appointment day compared to those who did not receive any reminder. Other socio-demographic factors such as being 50 years and older (aOR: 1.31; CI: 1.10, 1.56) compared to under 30 years, living as couple (aOR: 1.23; CI: 1.10, 1.37), and not having children (aOR: 1.21; CI: 1.07, 1.37) were significantly associated with appointment attendance. CONCLUSIONS: Our study suggests that patient reminder systems may be used to reduce non-attendance in Haiti, even in a context of socio-political crisis.

An integrated primary care service to reduce cardiovascular disease risk in people with severe mental illness: Primrose-A - thematic analysis of its acceptability, feasibility, and implementation.

BACKGROUND: Cardiovascular disease among patients with severe mental illness in England is a major preventable contributor to premature mortality. To address this, a nurse and peer-coach delivered service (Primrose-A) was implemented in three London general practices from 2019 (implementation continued during COVID-19). This study aimed to conduct interviews with patient and staff to determine the acceptability of, and experiences with, Primrose-A. METHODS: Semi-structured audio-recorded interviews with eight patients who had received Primrose-A, and 3 nurses, 1 GP, and 1 peer-coach who had delivered Primrose-A in three London-based GP surgeries were conducted. Reflexive thematic analysis was used to identify themes from the transcribed interviews. FINDINGS: Overall, Primrose-A was viewed positively by patients and staff, with participants describing success in improving patients' mental health, isolation, motivation, and physical health. Therapeutic relationships between staff and patients, and long regular appointments were important facilitators of patient engagement and acceptance of the intervention. Several barriers to the implementation of Primrose-A were identified, including training, administrative and communication issues, burden of time and resources, and COVID-19. CONCLUSIONS: Intervention acceptability could be enhanced by providing longer-term continuity of care paired with more peer-coaching sessions to build positive relationships and facilitate sustained health behaviour change. Future implementation of Primrose-A or similar interventions should consider: (1) training sufficiency (covering physical and mental health, including addiction), (2) adequate staffing to deliver the intervention, (3) facilitation of clear communication pathways between staff, and (4) supporting administrative processes.

The State of National Institute of Health Awards for funding genetic counseling research, resources, and training over the past decade.

Research related to the practice of genetic counseling has historically been accomplished through student projects, small private grants, or as a part of a larger research study. Yet, recent initiatives supported by the National Society of Genetic Counselors and the National Human Genome Research Institute have recognized and promoted the need for additional genetic counseling research funding and training. In this study, we aimed to characterize awards from the United States' National Institutes of Health (NIH) over the past 10 years that support research related to genetic counseling. A search of the NIH RePORTER database conducted on April 25, 2022, using terms broadly related to genetic counseling, identified 3993 awards from fiscal years 2011 to 2022. After deduplication, 1231 grants were reviewed for screening. The final dataset included 43 awards of various types with the majority being research or R series grants [R01 (n = 17), R21 (n = 7)]. The remaining awards were Cooperative Agreements (U01, n = 6), intramural (Z-grants, n = 5), Center Core (P30, n = 2), Specialized Center (P50, n = 1), Career Development Award (K01, n = 1), Other Transactions (OT2, n = 1), Resource Program (G13, n = 1), and Research Training and Fellowship (T32, n = 1). Most grants were awarded between 2019 and 2022 (n = 24, 55.8%). The majority of awards were categorized as studies that included a comparison of different genetic counseling models (n = 23), outcomes of implementing genetic counseling (n = 11), core resources for genetic counselors (n = 5), and genetic counseling training programs (n = 4). Thirteen of the awards (30.2%) had a specific aim/goal/focus on underserved or underrepresented populations. The topics were predominantly related to cancer (n = 26). Nine awards were led or co-led by a genetic counselor (20.9%). Despite significant growth in genetic counseling research support from US-based funders over the past 3 years, major gaps related to funding exist, including that most award recipients are not genetic counselors.

Proposed use of entrustable professional activities (EPAs) in genetic counseling for clinical training and assessment.

We introduce Entrustable Professional Activities (EPAs) as a potential framework for clinical training and assessment in genetic counseling. We discuss advantages of this approach, review how EPAs complement Practice-Based Competencies (PBCs), describe our process of generating proposed "core" EPAs, provide examples of specialty-specific EPAs, discuss the concept of entrustment in clinical training, and propose an approach to implementation.

The Saturation- and Dose-Dependent Effects of a Teen Sexual Harassment Prevention Program: Findings from a Randomized Controlled Trial.

Using a randomized controlled trial, we investigated changes in both sexual harassment (SH) perpetration and victimization of 2104 middle school students in New York City who received divergent saturation and dosage levels of Shifting Boundaries, an SH prevention program, which was represented by the length of the program. We assessed the saturation effect of the program by comparing the outcomes across respondents from 26 schools in which there were varying percentages of students enrolled in the program. The data suggested that, overall, the program was effective in reducing sexual harassment victimization but achieved a null effect against respondents' SH perpetration and that neither the length nor the school-saturation level of the program exerted a significant effect on SH perpetration. Although the data indicated a significant difference in SH victimization between the treatment and control group, when comparing subgroups who received treatment with divergent saturation and dosage levels, no statistically significant difference was identified. Our results suggested that the program effect was not contingent on the portion of students in a school who enrolled in the program, nor was it contingent on the dosage.

Twelve tips for Natural Language Processing in medical education program evaluation.

With the increasing application of Natural Language Processing (NLP) in Medicine at large, medical educators are urged to gain an understanding and implement NLP techniques within their own education programs to improve the workflow and make significant and rapid improvements in their programs. This paper aims to provide twelve essential tips inclusive of both conceptual and technical factors to facilitate the successful integration of NLP in medical education program evaluation. These twelve tips range from advising on various stages of planning the evaluation process, considerations for data collection, and reflections on preprocessing of data in preparation for analysis and interpretation of results. Using these twelve tips as a framework, medical researchers, educators, and administrators will have an understanding and reference to navigating applications of NLP and be able to unlock its potential for enhancing the evaluation of their own medical education programs.

Competency based medical education implementation at the institutional level: A cross-discipline comparative program evaluation.

INTRODUCTION: As an early adopter of competency-based medical education (CBME) our postgraduate institution was uniquely positioned to analyze implementation experience data across programs, while keeping institutional factors constant. We described participants' experiences related to CBME implementation across programs derived from early program evaluation efforts within our setting. METHODS: This evaluation focused on eight residency programs at a medium-sized academic institution in Canada. Participants (n = 175) included program leaders, faculty, and residents. The study consisted of 3 phases: (1) describing intended implementation; (2) documenting enacted implementation; and (3) comparing intended with enacted implementation to inform adaptations. Each program's findings were summarized in technical reports which were then analyzed thematically. Cross program data were organized by themes. RESULTS: Six themes were identified. All groups emphasized the need for ongoing refinement of CBME resulting from shared tensions such as increased assessment burden. However, there were some disparate CBME-related experiences between programs such as the experience with entrustable professional activities, the interpretation of retrospective entrustment anchors, and quality of feedback. CONCLUSION: We detected several cross-program successes and important challenges related to CBME. Our experience can inform other programs engaging in implementation and evaluation of CBME.

Using focused ethnography to inform biomedical research infrastructure enhancement at primarily undergraduate institutions.

The National Institutes of Health (NIH) supports 24 IDeA Networks of Biomedical Research Excellence (INBRE) Programs that help develop university-based biomedical research capacity in states that historically receive low levels of extramural grant support. To assess the effectiveness of the Arkansas INBRE in meeting its biomedical research capacity-building goals, we evaluated how the context (i.e., local and institutional settings) at two undergraduate institutions impacted variability in science faculty use of program resources. Data were collected by in-depth interviews with faculty and administrators (N = 9), focused observations, a review of Arkansas INBRE databases, and internet searches. Content analysis was used to code interview transcripts and field notes, and then qualitative data were integrated with data from databases and internet searches to construct two institutional case summaries. Constant comparison was used to identify similarities and differences between the institutions that helped to explain variability in how frequently faculty used Arkansas INBRE resources, including an enrollment crisis at undergraduate institutions in the United States and the presence or absence of a robust research culture at each institution. These findings were used to suggest program improvements (e.g., classroom-based research) that could further strengthen biomedical research capacity in Arkansas. As some barriers to program effectiveness are likely found in other IDeA-eligible states, improvements suggested for the Arkansas INBRE could apply to INBRE programs elsewhere.NEW & NOTEWORTHY This article describes results from an approach to program evaluation (i.e., focused ethnography) that has not been previously used to evaluate grant mechanisms. This "experience near" approach, which involved qualitative interviews and firsthand observations, lent valuable insights into how broader and institutional contexts at two primarily undergraduate institutions hindered or facilitated use of Arkansas INBRE resources. The insights gained can be used to enhance the Arkansas INBRE, which aims to strengthen the statewide biomedical infrastructure.

Program evaluation of postgraduate obstetrics and gynecology training in Lao people's democratic republic - using the CIPP model.

BACKGROUND: The obstetrics and gynaecology (OB-GYN) residency training program in Lao People's Democratic Republic (PDR) began in 2003 based on the Millennium Development Goals (MDGs) and 'Reproductive, maternal, newborn, and child health interventions (RMNCH) strategies and action plan'. However, the training program had not been properly evaluated previously. The purpose of this study is to evaluate the current postgraduate OB-GYN residency training program in Lao PDR by using CIPP model to identify the current problems (the strengths and weaknesses) and suggest a future plan to promote continuous improvement. METHOD: The context, input, process, and product classification (CIPP) model was used to develop criteria and indicators. A mixed-methods approach was used for this study. To capture instructional material for quantitative analysis, a Google survey with 38 items and a t-test were used to determine a significant difference in responses between residents and lecturers (N = 120). Based on qualitative analysis, an in-depth interview was done (four questions based on study outcomes, including satisfaction, strengths and weaknesses, and future opportunities), and six interviews provided different viewpoints on the course. The SPSS software program was used to measure validity, with p-values = 0.05. RESULTS: The overall average response rate was 97.5%. Two significant differences in program perspectives were revealed between lecturers and residents, difficulties in maintaining the course (professors 3.66 ± 1.03 and residents 3.27 ± 0.98, p = 0.04) and learning outcomes achieved (professors 3.57 ± 0.85 and residents 3.14 ± 0.95, p = 0.01 The overall average for the context part of the questionnaire was under 3.00, with the lowest scores for overlapped learning outcomes and difficulties in maintaining the course. The input part, lack of the classroom, skills lab and staff; the process part, lecturer to collect student opinions and the product part on learning outcomes. CONCLUSION: Curriculum improvement based on the program evaluation results, including regular evaluation and feedback, will advance the residency training program based on the RMNCH strategy and contribute to the promotion of maternal health in the Lao PDR.