Methods to identify and prioritize research projects and perform embedded research in learning healthcare systems.

BACKGROUND: The Embedded Healthcare Research Conference aimed to promote and enhance research-operations partnerships in diverse health care settings. Within this conference, the Priorities and Methods Workgroup set out to define a vision of embedded research that leverages diverse methods to address clearly articulated research questions of importance to health systems. METHODS: The Workgroup session involved a combination of small and large group discussions around three broadly focused topics: the integration of embedded research within the existing quality improvement (QI) ecosystem; the identification, prioritization and formulation of embedded research questions; the creation of an embedded research "tool kit." RESULTS: Workgroup participants envisioned a future for embedded research that is characterized by authentic engagement between researchers and health system leaders; seamless integration between research, QI and clinical operations; clear and explicit articulation of research questions; an appropriate balance between rigor and relevance in applied methodology; alignment between study design, available resources and the importance of the knowledge to be gained; efficient processes; and bi-directional communication. Important barriers to achieving this vision include limited access to executive leaders, silos that discourage integration of research and QI, generally low tolerance for disruption in high-risk clinical settings, limited access to data, and limited availability of researchers with requisite skills and training. CONCLUSIONS: Embedded research holds potential to enhance the relevance, value and use of research, while also creating generalizable knowledge. Key recommendations include building authentic relationships, discouraging silos, encouraging innovation and experimentation, and expanding opportunities for funding research in delivery systems.

Rethinking ethical oversight in the era of the learning health system.

Opportunities to advance science increasingly arise through investigations embedded within routine clinical practice in the form of learning health systems. Such activities challenge conventional approaches to research regulation that have not caught up with those opportunities, often imposing burdens generalized from riskier research. We analyze the rules and conventions in the US, demonstrating how even those rules are compatible with a much more flexible approach to participant risk, institutional oversight, participant consent, and disclosure for low-risk learning activities in all jurisdictions.

Evaluation of Adaptive Feedback in a Smartphone-Based Game on Health Care Providers' Learning Gain: Randomized Controlled Trial.

BACKGROUND: Although smartphone-based emergency care training is more affordable than traditional avenues of training, it is still in its infancy, remains poorly implemented, and its current implementation modes tend to be invariant to the evolving learning needs of the intended users. In resource-limited settings, the use of such platforms coupled with gamified approaches remains largely unexplored, despite the lack of traditional training opportunities, and high mortality rates in these settings. OBJECTIVE: The primary aim of this randomized experiment is to determine the effectiveness of offering adaptive versus standard feedback, on the learning gains of clinicians, through the use of a smartphone-based game that assessed their management of a simulated medical emergency. A secondary aim is to examine the effects of learner characteristics and learning spacing with repeated use of the game on the secondary outcome of individualized normalized learning gain. METHODS: The experiment is aimed at clinicians who provide bedside neonatal care in low-income settings. Data were captured through an Android app installed on the study participants' personal phones. The intervention, which was based on successful attempts at a learning task, included adaptive feedback provided within the app to the experimental arm, whereas the control arm received standardized feedback. The primary end point was completion of the second learning session. Of the 572 participants enrolled between February 2019 and July 2019, 247 (43.2%) reached the primary end point. The primary outcome was standardized relative change in learning gains between the study arms as measured by the Morris G effect size. The secondary outcomes were the participants individualized normalized learning gains. RESULTS: The effect of adaptive feedback on care providers' learning gain was found to be g=0.09 (95% CI -0.31 to 0.46; P=.47). In exploratory analysis, using normalized learning gains, when subject-treatment interaction and differential time effect was controlled for, this effect increased significantly to 0.644 (95% CI 0.35 to 0.94; P<.001) with immediate repetition, which is a moderate learning effect, but reduced significantly by 0.28 after a week. The overall learning change from the app use in both arms was large and may have obscured a direct effect of feedback. CONCLUSIONS: There is a considerable learning gain between the first two rounds of learning with both forms of feedback and a small added benefit of adaptive feedback after controlling for learner differences. We suggest that linking the adaptive feedback provided to care providers to how they space their repeat learning session(s) may yield higher learning gains. Future work might explore in more depth the feedback content, in particular whether or not explanatory feedback (why answers were wrong) enhances learning more than reflective feedback (information about what the right answers are). TRIAL REGISTRATION: Pan African Clinical Trial Registry (PACTR) 201901783811130; https://pactr.samrc.ac.za/TrialDisplay.aspx?TrialID=5836. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): RR2-10.2196/13034.