Training and evaluating simulation debriefers in low-resource settings: lessons learned from Bihar, India.

BACKGROUND: To develop effective and sustainable simulation training programs in low-resource settings, it is critical that facilitators are thoroughly trained in debriefing, a critical component of simulation learning. However, large knowledge gaps exist regarding the best way to train and evaluate debrief facilitators in low-resource settings. METHODS: Using a mixed methods approach, this study explored the feasibility of evaluating the debriefing skills of nurse mentors in Bihar, India. Videos of obstetric and neonatal post-simulation debriefs were assessed using two known tools: the Center for Advanced Pediatric and Perinatal Education (CAPE) tool and Debriefing Assessment for Simulation in Healthcare (DASH). Video data was used to evaluate interrater reliability and changes in debriefing performance over time. Additionally, twenty semi-structured interviews with nurse mentors explored perceived barriers and enablers of debriefing in Bihar. RESULTS: A total of 73 debriefing videos, averaging 18 min each, were analyzed by two raters. The CAPE tool demonstrated higher interrater reliability than the DASH; 13 of 16 CAPE indicators and two of six DASH indicators were judged reliable (ICC > 0.6 or kappa > 0.40). All indicators remained stable or improved over time. The number of 'instructors questions,' the amount of 'trainee responses,' and the ability to 'organize the debrief' improved significantly over time (p < 0.01, p < 0.01, p = 0.04). Barriers included fear of making mistakes, time constraints, and technical challenges. Enablers included creating a safe learning environment, using contextually appropriate debriefing strategies, and team building. Overall, nurse mentors believed that debriefing was a vital aspect of simulation-based training. CONCLUSION: Simulation debriefing and evaluation was feasible among nurse mentors in Bihar. Results demonstrated that the CAPE demonstrated higher interrater reliability than the DASH and that nurse mentors were able to maintain or improve their debriefing skills overtime. Further, debriefing was considered to be critical to the success of the simulation training. However, fear of making mistakes and logistical challenges must be addressed to maximize learning. Teamwork, adaptability, and building a safe learning environment enhanced the quality enhanced the quality of simulation-based training, which could ultimately help to improve maternal and neonatal health outcomes in Bihar.

Simulation-enhanced nurse mentoring to improve preeclampsia and eclampsia care: an education intervention study in Bihar, India.

BACKGROUND: Inadequately treated, preeclampsia and eclampsia (PE/E) may rapidly lead to severe complications in both mothers and neonates, and are estimated to cause 60,000 global maternal deaths annually. Simulation-based training on obstetric and neonatal emergency management has demonstrated promising results in low- and middle-income countries. However, the impact of simulation training on use of evidence-based practices for PE/E diagnosis and management in low-resource settings remains unknown. METHODS: This study was based on a statewide, high fidelity in-situ simulation training program developed by PRONTO International and implemented in collaboration with CARE India on PE/E management in Bihar, India. Using a mixed methods approach, we evaluated changes over time in nurse mentees' use of evidence-based practices during simulated births at primary health clinics. We compared the proportion and efficiency of evidence-based practices completed during nurse mentees' first and last participation in simulated PE/E cases. Twelve semi-structured interviews with nurse mentors explored barriers and enablers to high quality PE/E care in Bihar. RESULTS: A total of 39 matched first and last simulation videos, paired by facility, were analyzed. Videos occurred a median of 62 days apart and included 94 nurses from 33 primary health centers. Results showed significant increases in the median number of 'key history questions asked,' (1.0 to 2.0, p = 0.03) and 'key management steps completed,' (2.0 to 3.0, p = 0.03). The time from BP measured to magnesium sulfate given trended downwards by 3.2 min, though not significantly (p = 0.06). Key barriers to high quality PE/E care included knowledge gaps, resource shortages, staff hierarchy between physicians and nurses, and poor relationships with patients. Enablers included case-based and simulation learning, promotion of teamwork and communication, and effective leadership. CONCLUSION: Simulation training improved the use of evidence-based practices in PE/E simulated cases and has the potential to increase nurse competency in diagnosing and managing complex maternal complications such as PE/E. However, knowledge gaps, resource limitations, and interpersonal barriers must be addressed in order to improve care. Teamwork, communication, and leadership are key mechanisms to facilitate high quality PE/E care in Bihar.

Barriers and facilitators to the provision of optimal obstetric and neonatal emergency care and to the implementation of simulation-enhanced mentorship in primary care facilities in Bihar, India: a qualitative study.

BACKGROUND: Globally, an estimated 275,000 maternal deaths, 2.7 million neonatal deaths, and 2.6 million third trimester stillbirths occurred in 2015. Major improvements could be achieved by providing effective care in low- and middle-income countries, where the majority of these deaths occur. Mentoring programs have become a popular modality to improve knowledge and skills among providers in low-resource settings. Thus, a detailed understanding of interrelated factors affecting care provision and mentorship is necessary both to improve the quality of care and to maximize the impact of mentoring programs. METHODS: In partnership with the Government of Bihar, CARE India and PRONTO International implemented simulation-enhanced mentoring in 320 primary health clinics (PHC) across the state of Bihar, India from 2015 to 2017, within the context of the AMANAT mobile nurse mentoring program. Between June and August 2016, we conducted semi-structured interviews with 20 AMANAT nurse mentors to explore barriers and facilitators to optimal care provision and to implementation of simulation-enhanced mentorship in PHCs in Bihar. Data were analyzed using the thematic content approach. RESULTS: Mentors identified numerous factors affecting care provision and mentorship, many of which were interdependent. Such barriers included human resource shortages, nurse-nurse hierarchy, distance between labor and training rooms, cultural norms, and low skill level and resistance to change among mentees. In contrast, physical resource shortages, doctor-nurse hierarchy, corruption, and violence against providers posed barriers to care provision alone. Facilitators included improved skills and confidence among providers, inclusion of doctors in training, increased training frequency, establishment of strong mentor-mentee relationships, administrative support, and nursing supervision and feedback. CONCLUSIONS: This study has identified many interrelated factors affecting care provision and mentorship in Bihar. The mentoring program was not designed to address several barriers, including resource shortages, facility infrastructure, corruption, and cultural norms. These require government support, community awareness, and other systemic changes. Programs may be adapted to address some barriers beyond knowledge and skill deficiencies, notably hierarchy, violence against providers, and certain cultural taboos. An in-depth understanding of barriers and facilitators is essential to enable the design of targeted interventions to improve maternal and neonatal survival in Bihar and related contexts.

Logistical, cultural, and structural barriers to immediate neonatal care and neonatal resuscitation in Bihar, India.

BACKGROUND: In India, the neonatal mortality rate is nearly double the Sustainable Development Goal target with more than half of neonatal deaths occurring in only four states, one of which is Bihar. Evaluations of immediate neonatal care and neonatal resuscitation skills in Bihar have demonstrated a need for significant improvement. However, barriers to evidence based practices in clinical care remain incompletely characterized. METHODS: To better understand such barriers, semi-structured interviews were conducted with 18 nurses who participated as mentors in the AMANAT maternal and child health quality improvement project, implemented by CARE India and the Government of Bihar. Nurse-mentors worked in primary health centers throughout Bihar facilitating PRONTO International emergency obstetric and neonatal simulations for nurse-mentees in addition to providing direct supervision of clinical care. Interviews focused on mentors' perceptions of barriers to evidence based practices in immediate neonatal care and neonatal resuscitation faced by mentees employed at Bihar's rural primary health centers. Data was analyzed using the thematic content approach. RESULTS: Mentors identified numerous interacting logistical, cultural, and structural barriers to care. Logistical barriers included poor facility layout, supply issues, human resource shortages, and problems with the local referral system. Cultural barriers included norms such as male infant preference, traditional clinical practices, hierarchy in the labor room, and interpersonal relations amongst staff as well as with patients' relatives. Poverty was described as an overarching structural barrier. CONCLUSION: Interacting logistical, cultural and structural barriers affect all aspects of immediate neonatal care and resuscitation in Bihar. These barriers must be addressed in any intervention focused on improving providers' clinical skills. Strategic local partnerships are vital to addressing such barriers and to contextualizing skills-based trainings developed in Western contexts to achieve the desired impact of reducing neonatal mortality.

The power of practice: simulation training improving the quality of neonatal resuscitation skills in Bihar, India.

BACKGROUND: Globally, neonatal mortality accounts for nearly half of under-five mortality, and intrapartum related events are a leading cause. Despite the rise in neonatal resuscitation (NR) training programs in low- and middle-income countries, their impact on the quality of NR skills amongst providers with limited formal medical education, particularly those working in rural primary health centers (PHCs), remains incompletely understood. METHODS: This study evaluates the impact of PRONTO International simulation training on the quality of NR skills in simulated resuscitations and live deliveries in rural PHCs throughout Bihar, India. Further, it explores barriers to performance of key NR skills. PRONTO training was conducted within CARE India's AMANAT intervention, a maternal and child health quality improvement project. Performance in simulations was evaluated using video-recorded assessment simulations at weeks 4 and 8 of training. Performance in live deliveries was evaluated in real time using a mobile-phone application. Barriers were explored through semi-structured interviews with simulation facilitators. RESULTS: In total, 1342 nurses participated in PRONTO training and 226 NR assessment simulations were matched by PHC and evaluated. From week 4 to 8 of training, proper neck extension, positive pressure ventilation (PPV) with chest rise, and assessment of heart rate increased by 14%, 19%, and 12% respectively (all p ≤ 0.01). No difference was noted in stimulation, suction, proper PPV rate, or time to completion of key steps. In 252 live deliveries, identification of non-vigorous neonates, use of suction, and use of PPV increased by 21%, 25%, and 23% respectively (all p < 0.01) between weeks 1-3 and 4-8. Eighteen interviews revealed individual, logistical, and cultural barriers to key NR skills. CONCLUSION: PRONTO simulation training had a positive impact on the quality of key skills in simulated and live resuscitations throughout Bihar. Nevertheless, there is need for ongoing improvement that will likely require both further clinical training and addressing barriers that go beyond the scope of such training. In settings where clinical outcome data is unreliable, data triangulation, the process of synthesizing multiple data sources to generate a better-informed evaluation, offers a powerful tool for guiding this process.

Care of the mother-infant dyad: a novel approach to conducting and evaluating neonatal resuscitation simulation training in Bihar, India.

BACKGROUND: As the global under-five mortality rate declines, an increasing percentage is attributable to early neonatal mortality. A quarter of early neonatal deaths are due to perinatal asphyxia. However, neonatal resuscitation (NR) simulation training in low-resource settings, where the majority of neonatal deaths occur, has achieved variable success. In Bihar, India, the poorest region in South Asia, there is tremendous need for a new approach to reducing neonatal morality. METHODS: This analysis aims to assess the impact of a novel in-situ simulation training program, developed by PRONTO International and implemented in collaboration with CARE India, on NR skills of nurses in Bihar. Skills were evaluated by clinical complexity of the simulated scenario, which ranged from level 1, requiring NR without a maternal complication, to level 3, requiring simultaneous management of neonatal and maternal complications. A total of 658 nurses at 80 facilities received training 1 week per month for 8 months. Simulations were video-recorded and coded for pre-defined clinical skills using Studiocode™. RESULTS: A total of 298 NR simulations were analyzed. As simulation complexity increased, the percentage of simulations in which nurses completed key steps of NR did not change, even with only 1-2 providers in the simulation. This suggests that with PRONTO training, nurses were able to maintain key skills despite higher clinical demands. As simulation complexity increased from level 1 to 3, time to completion of key NR steps decreased non-significantly. Median time to infant drying decreased by 7.5 s (p = 0.12), time to placing the infant on the warmer decreased by 21.7 s (p = 0.27), and time to the initiation of positive pressure ventilation decreased by 20.8 s (p = 0.12). Nevertheless, there remains a need for improvement in absolute time elapsed between delivery and completion of key NR tasks. CONCLUSIONS: PRONTO simulation training enabled nurses in Bihar to maintain core NR skills in simulation despite demands for higher-level triage and management. Although further evaluation of the PRONTO methodology is necessary to understand the full scope of its impact, this analysis highlights the importance of conducting and evaluating simulation training in low-resource settings based on simultaneous care of the mother-infant dyad.

Simulation and team training embedded nurse mentoring programme and improvement in intrapartum and newborn care in a low-resource setting in Bihar, India.

BACKGROUND: Improvement of the quality of maternal and child health care remains a focus in India. Working with the Government of Bihar, CARE-India facilitated a comprehensive set of quality of care improvement initiatives. PRONTO's simulation and team-training was incorporated into the large-scale Apatkaleen Matritva evam Navjat Tatparta (AMANAT)nurse-mentoring program of the Government of Bihar supported by CARE-India to improve maternal and child health outcomes. Along-with the AMANAT program, the PRONTO components provided training on nontechnical and technical competencies for managing a variety of obstetric and neonatal conditions, as a team. This study assessed the effectiveness of nurse-mentoring including simulations on intrapartum and newborn care practices in 320 basic emergency obstetric and neonatal care (BEmONC) facilities. METHODS: Deliveries were observed to obtain specific information on evidence-based practice (EBP) indicators before and after the intervention. Intrapartum and newborn care composite scores - were calculated using those EBP indicators. A web-based routine monitoring system provided total training days, weeks and days/week of training and counts of simulation and teamwork-communication activities. Multilevel linear regression was used to examine the exposure-outcome associations. RESULTS: The final analysis included 668 normal spontaneous vaginal deliveries (NSVDs) from 289 public health facilities in Bihar. Facility-level intrapartum and newborn scores improved by 37 and 26-percentage points, respectively, from baseline to endline. Compared to the bottom one-third facilities that performed fewest NSVD simulations, the top one-third had 6 (95% confidence interval (CI) = 1-12) percentage points higher intrapartum score. Similar comparison using maternal complication simulations yielded 7 (95% CI = 1-12) percentage point higher scores. The highest newborn scores were observed in the middle one-third of facilities relative to the bottom one-third that did the fewest NSVD simulations (5, 95% CI: 1-10). CONCLUSIONS: Findings suggest significant overall improvement in intrapartum and newborn care practices after the AMANAT nurse-mentoring program in public sector BEmONC facilities. Simulation and team-training likely contributed towards the overall improvement, especially for intrapartum care. STUDY REGISTRATION: ClinicalTrials.gov number NCT02726230.

Tracking and debriefing birth data at scale: A mobile phone application to improve obstetric and neonatal care in Bihar, India.

AIM: This analysis assessed changes over time in skill and knowledge related to the use of evidence-based practices associated with quality of maternal and neonatal care during a nurse midwife mentoring intervention at primary health clinics (PHCs) in Bihar, India. DESIGN: Nurse midwife mentors (NMMs) entered live birth observation data into a mobile App from 320 PHCs. METHODS: The NMMs completed prompted questions in the App after every live birth witnessed. The App consisted of questions around three main themes, "What went well?", "What needed improvement?" and "What can be done differently next time?". RESULTS: Observational data from 5,799 births was recorded by 120 NMMs in 320 PHCs. Knowledge and skill during normal spontaneous vaginal deliveries and complicated deliveries with either a postpartum haemorrhage or non-vigorous infant all showed statistically significant improvement (p < .001) over time using a Chi-squared test for trend with a mean increase of 41% across all indicators.

Video Monitoring a Simulation-Based Quality Improvement Program in Bihar, India.

BACKGROUND: Simulation-based training has become an accepted clinical training andragogy in high-resource settings with its use increasing in low-resource settings. Video recordings of simulated scenarios are commonly used by facilitators. Beyond using the videos during debrief sessions, researchers can also analyze the simulation videos to quantify technical and nontechnical skills during simulated scenarios over time. Little is known about the feasibility and use of large-scale systems to video record and analyze simulation and debriefing data for monitoring and evaluation in low-resource settings. METHODS: This manuscript describes the process of designing and implementing a large-scale video monitoring system. Mentees and Mentors were consented and all simulations and debriefs conducted at 320 Primary Health Centers (PHCs) were video recorded. The system design, number of video recordings, and inter-rater reliability of the coded videos were assessed. RESULTS: The final dataset included a total of 11,278 videos. Overall, a total of 2,124 simulation videos were coded and 183 (12%) were blindly double-coded. For the double-coded sample, the average inter-rater reliability (IRR) scores were 80% for nontechnical skills, and 94% for clinical technical skills. Among 4,450 long debrief videos received, 216 were selected for coding and all were double-coded. Data quality of simulation videos was found to be very good in terms of recorded instances of "unable to see" and "unable to hear" in Phases 1 and 2. CONCLUSION: This study demonstrates that video monitoring systems can be effectively implemented at scale in resource limited settings. Further, video monitoring systems can play several vital roles within program implementation, including monitoring and evaluation, provision of actionable feedback to program implementers, and assurance of program fidelity.