Balancing best practice and reality in behavioral intervention development: A survey of principal investigators funded by the National Institutes of Health.

ABSTRACT

Preliminary studies play a prominent role in the development of large-scale behavioral interventions. Though recommendations exist to guide the execution and interpretation of preliminary studies, these assume optimal scenarios which may clash with realities faced by researchers. The purpose of this study was to explore how principal investigators (PIs) balance expectations when conducting preliminary studies. We surveyed PIs funded by the National Institutes of Health to conduct preliminary behavioral interventions between 2000 and 2020. Four hundred thirty-one PIs (19% response rate) completed the survey (November 2021 to January 2022, 72% female, mean 21 years post-terminal degree). Most PIs were aware of translational models and believed preliminary studies should precede larger trials but also believed a single preliminary study provided sufficient evidence to scale. When asked about the relative importance of preliminary efficacy (i.e. changes in outcomes) and feasibility (i.e. recruitment, acceptance/adherence) responses varied. Preliminary studies were perceived as necessary to successfully compete for research funding, but among PIs who had peer-reviewed federal-level grants applications (n = 343 [80%]), responses varied about what should be presented to secure funding. Confusion surrounding the definition of a successful, informative preliminary study poses a significant challenge when developing behavior interventions. This may be due to a mismatch between expectations surrounding preliminary studies and the realities of the research enterprise in which they are conducted. To improve the quality of preliminary studies and advance the field of behavioral interventions, additional funding opportunities, more transparent criteria in grant reviews, and additional training for grant reviewers are suggested.

Initial testing of behavioral interventions can provide valuable information about the methods of the intervention and whether it is effective. However, recommendations that provide researchers with guidance on how to best conduct pilot studies assume ideal circumstances. The mismatch between what can be realistically accomplished in a preliminary study, and what researchers expect from preliminary studies creates confusion. As a result, it is difficult for researchers to judge the quality, relevance, and potential of preliminary studies. This study suggests more research funding opportunities, clearer rules for reviewing grant applications, and more training for the people who review these applications could help improve preliminary studies and create more effective health behavior programs.