The use of implementation science theories, models, and frameworks in implementation research for medicinal products: A scoping review.

BACKGROUND: The uptake, adoption and integration of new medicines and treatment regimens within healthcare delivery can take a decade or more. Increasingly, implementation science (IS) research is being used to bridge this gap between the availability of new therapeutic evidence and its actual application in clinical practice. Little is known, however, about the quality of IS research in this area, including the degree to which theories, models and frameworks (TMFs) are being used. The objective of this study was to conduct a scoping review of the use of TMFs in implementation research involving medicinal products. METHODS: A search was conducted for English language abstracts and manuscripts describing the application of TMFs in IS studies for medicinal products. Eligible publications were those published between 1 January 1974 and 12 December 2022. All records were screened at the title and abstract stage; included full-text papers were abstracted using data extraction tables designed for the study. Study quality was appraised using the Implementation Research Development Tool. RESULTS: The initial scoping search identified 2697 publications, of which 9 were ultimately eligible for inclusion in the review. Most studies were published after 2020 and varied in their objectives, design and therapeutic area. Most studies had sample sizes of fewer than 50 participants, and all focused on the post-marketing phase of drug development. The TMF most frequently used was the Consolidated Framework for Implementation Research (CFIR). Although most studies applied all TMF domains, TMF use was limited to instrument development and/or qualitative analysis. Quality appraisals indicated the need for engaging patients and other stakeholders in the implementation research, reporting on the cost of implementation strategies, and evaluating the unintended consequences of implementation efforts. CONCLUSIONS: We found that few IS studies involving medicinal products reported using TMFs. Those that did encompassed a wide variety of therapeutic indications and medicinal products; all were in the post-marketing phase and involved limited application of the TMFs. Researchers should consider conducting IS in earlier phases of drug development and integrating the TMFs throughout the research process. More consistent and in-depth use of TMFs may help advance research in this area.

Health literacy-listening skill and patient questions following cancer prevention and screening discussions.

OBJECTIVE: Patient question-asking is essential to shared decision making. We sought to describe patients' questions when faced with cancer prevention and screening decisions, and to explore differences in question-asking as a function of health literacy with respect to spoken information (health literacy-listening). METHODS: Four-hundred and thirty-three (433) adults listened to simulated physician-patient interactions discussing (i) prophylactic tamoxifen for breast cancer prevention, (ii) PSA testing for prostate cancer and (iii) colorectal cancer screening, and identified questions they would have. Health literacy-listening was assessed using the Cancer Message Literacy Test-Listening (CMLT-Listening). Two authors developed a coding scheme, which was applied to all questions. Analyses examined whether participants scoring above or below the median on the CMLT-Listening asked a similar variety of questions. RESULTS: Questions were coded into six major function categories: risks/benefits, procedure details, personalizing information, additional information, decision making and credibility. Participants who scored higher on the CMLT-Listening asked a greater variety of risks/benefits questions; those who scored lower asked a greater variety of questions seeking to personalize information. This difference persisted after adjusting for education. CONCLUSION: Patients' health literacy-listening is associated with distinctive patterns of question utilization following cancer screening and prevention counselling. Providers should not only be responsive to the question functions the patient favours, but also seek to ensure that the patient is exposed to the full range of information needed for shared decision making.

E-mail to Promote Colorectal Cancer Screening Within Social Networks: Acceptability and Content.

Effective techniques to encourage colorectal cancer screening in underscreened populations have included social support interventions and e-mail reminders from physicians. Personalized e-mail messages to promote colorectal cancer screening within social networks could be even more effective but have not been studied. The authors interviewed 387 e-mail users, aged 42-73 years in Georgia, Hawaii, and Massachusetts. Participants were asked to edit a sample message in which the sender shares a recent colonoscopy experience and urges the recipient to discuss colorectal cancer screening with a doctor. For those reporting willingness to send this message, changes to the message and suggested subject lines were recorded. Edited text was analyzed for content and concordance with original message. The majority of participants (74.4%) were willing to e-mail a modifiable message. Of those willing, 63.5% edited the message. Common edits included deletion (17.7%) or modification (17.4%) of a negatively framed sentence on colon cancer risks and addition or modification of personalizing words (15.6%). Few edits changed the meaning of the message (5.6%), and even fewer introduced factual inaccuracies (1.7%). Modifiable e-mail messages offer a way for screened individuals to promote colorectal cancer screening to social network members. The accuracy and effects of such messages should be further studied.

The RE-AIM framework: a systematic review of use over time.

We provided a synthesis of use, summarized key issues in applying, and highlighted exemplary applications in the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework. We articulated key RE-AIM criteria by reviewing the published literature from 1999 to 2010 in several databases to describe the application and reporting on various RE-AIM dimensions. After excluding nonempirical articles, case studies, and commentaries, 71 articles were identified. The most frequent publications were on physical activity, obesity, and disease management. Four articles reported solely on 1 dimension compared with 44 articles that reported on all 5 dimensions of the framework. RE-AIM was broadly applied, but several criteria were not reported consistently.

Patients' and family members' views on patient-centered communication during cancer care.

OBJECTIVES: To explore patients' and family members' views on communication during cancer care and to identify those aspects of clinician-patient communication which were most important to patients and family members. METHODS: We conducted a secondary data analysis of qualitative data from 137 patients with cancer and family members of patients with cancer. We used a modified version of the constant comparative method and coding paradigm of grounded theory. RESULTS: Patients want sensitive, caring clinicians who provide information that they need, when they need it, in a way that they can understand; who listen and respond to questions and concerns, and who attempt to understand the patient's experience. Effective information exchange and a positive interpersonal relationship with the clinician were of fundamental importance to patients and family members. These were interrelated; for instance, failure to provide information a patient needed could damage the relationship, whereas excellent listening could foster the relationship. Information exchange and relationship were also integral to decision-making, managing uncertainty, responding to emotions, and self-management. Clinicians who were responsive to patients' needs beyond the immediate medical encounter were valued. CONCLUSIONS: The complexity of cancer care today suggests that efforts to improve communication must be multilevel, acknowledging and addressing patient, clinician, organizational and policy barriers, and facilitators. Measurement tools are needed to assess cancer patients' and family members' experiences with communication over the course of cancer care to provide meaningful, actionable feedback to those seeking to optimize their effectiveness in communicating with patients with cancer.

The association between health literacy and cancer-related attitudes, behaviors, and knowledge.

Using a multidimensional assessment of health literacy (the Cancer Message Literacy Test-Listening, the Cancer Message Literacy Test-Reading, and the Lipkus Numeracy Scale), the authors assessed a stratified random sample of 1013 insured adults (40-70 years of age). The authors explored whether low health literacy across all 3 domains (n =111) was associated with sets of variables likely to affect engagement in cancer prevention and screening activities: (a) attitudes and behaviors relating to health care encounters and providers, (b) attitudes toward cancer and health, (c) knowledge of cancer screening tests, and (d) attitudes toward health related media and actual media use. Adults with low health literacy were more likely to report avoiding doctor's visits, to have more fatalistic attitudes toward cancer, to be less accurate in identifying the purpose of cancer screening tests, and more likely to avoid information about diseases they did not have. Compared with other participants, those with lower health literacy were more likely to say that they would seek information about cancer prevention or screening from a health care professional and less likely to turn to the Internet first for such information. Those with lower health literacy reported reading on fewer days and using the computer on fewer days than did other participants. The authors assessed the association of low health literacy with colorectal cancer screening in an age-appropriate subgroup for which colorectal cancer screening is recommended. In these insured subjects receiving care in integrated health care delivery systems, those with low health literacy were less likely to be up to date on screening for colorectal cancer, but the difference was not statistically significant.

Do patient preferences for health information vary by health literacy or numeracy? A qualitative assessment.

Seeking health information can be a complicated process for a patient. Patients must know the topic of interest, where to look or ask, how to assess and comprehend, and how to evaluate the credibility and trustworthiness of the sources. In this study, the authors describe preferences of patients with multiple risk factors for cardiovascular disease with varying health literacy and numeracy abilities for receiving health information. Participants were recruited from 2 health care systems. Health literacy and numeracy were assessed and participants completed an orally administered survey consisting of open-ended questions about obtaining health information and preferences for health information. In-depth interviews were conducted with a subset of participants. A diverse sample of 150 individuals (11.3% Latino, 37.3% African American, 44.7% with income less than $15,000/year) participated. Most participants had adequate functional health literacy, while 65% had low numeracy skills. Regardless of health literacy or numeracy ability, participants overwhelmingly preferred to receive health information during a face-to-face conversation with their health care provider. While individuals with adequate functional health literacy identified a variety of health information sources, actions are needed to ensure multiple modalities are available and are in plain, clear language that reinforces patients' understanding and application of information to health behavior.

A citation analysis and scoping systematic review of the operationalization of the Practical, Robust Implementation and Sustainability Model (PRISM).

BACKGROUND: The Practical, Robust Implementation and Sustainability Model (PRISM) was developed in 2008 as a contextually expanded version of the broadly used Reach, Adoption, Effectiveness, Implementation, and Maintenance (RE-AIM) framework. PRISM provides researchers a pragmatic and intuitive model to improve translation of research interventions into clinical and community practice. Since 2008, the use of PRISM increased across diverse topics, populations, and settings. This citation analysis and scoping systematic review aimed to assess the use of the PRISM framework and to make recommendations for future research. METHODS: A literature search was conducted using three databases (PubMed, Web of Science, Scopus) for the period of 2008 and September 2020. After exclusion, reverse citation searches and invitations to experts in the field were used to identify and obtain recommendations for additional articles not identified in the original search. Studies that integrated PRISM into their study design were selected for full abstraction. Unique research studies were abstracted for information on study characteristics (e.g., setting/population, design), PRISM contextual domains, and RE-AIM outcomes. RESULTS: A total of 180 articles were identified to include PRISM to some degree. Thirty-two articles representing 23 unique studies integrated PRISM within their study design. Study characteristics varied widely and included studies conducted in diverse contexts, but predominately in high-income countries and in clinical out-patient settings. With regards to use, 19 used PRISM for evaluation, 10 for planning/development, 10 for implementation, four for sustainment, and one for dissemination. There was substantial variation across studies in how and to what degree PRISM contextual domains and RE-AIM outcomes were operationalized and connected. Only two studies directly connected individual PRISM context domains with RE-AIM outcomes, and another four included RE-AIM outcomes without direct connection to PRISM domains. CONCLUSIONS: This is the first systematic review of the use of PRISM in various contexts. While there were low levels of 'integrated' use of PRISM and few reports on linkage to RE-AIM outcomes, most studies included important context domains of implementation and sustainability infrastructure and external environment. Recommendations are provided for more consistent and comprehensive use of and reporting on PRISM to inform both research and practice on contextual factors in implementation.

Understanding and applying the RE-AIM framework: Clarifications and resources.

INTRODUCTION: Understanding, categorizing, and using implementation science theories, models, and frameworks is a complex undertaking. The issues involved are even more challenging given the large number of frameworks and that some of them evolve significantly over time. As a consequence, researchers and practitioners may be unintentionally mischaracterizing frameworks or basing actions and conclusions on outdated versions of a framework. METHODS: This paper addresses how the RE-AIM (Reach, Effectiveness, Adoption, Implementation, and Maintenance) framework has been described, summarizes how the model has evolved over time, and identifies and corrects several misconceptions. RESULTS: We address 13 specific areas where misconceptions have been noted concerning the use of RE-AIM and summarize current guidance on these issues. We also discuss key changes to RE-AIM over the past 20 years, including the evolution to Pragmatic Robust Implementation and Sustainability Model, and provide resources for potential users to guide application of the framework. CONCLUSIONS: RE-AIM and many other theories and frameworks have evolved, been misunderstood, and sometimes been misapplied. To some degree, this is inevitable, but we conclude by suggesting some actions that reviewers, framework developers, and those selecting or applying frameworks can do to prevent or alleviate these problems.

Using theory and technology to design a practical and generalizable smoking reduction intervention.

The aim of this article is to describe the process of using theory to form strategies for a generalizable smoking reduction intervention delivered through multiple intervention modalities. This report describes the process of integrating theory, data from diverse sources, staff from three different organizations, and different intervention modalities into an efficient, large-scale smoking reduction program featuring automated data from electronic medical records, computer-assisted telephone interviews, and tailored newsletters. The authors successfully developed a program that was consistently implemented as planned for 320 smokers in a managed care organization. The mapping of theory to intervention, data transfer and security procedures, and processes and strategies used to overcome challenges to intervention implementation should provide lessons learned for similar health promotion projects. Few intervention studies discuss details of how they translate theory into practice or how they integrate different modalities and collaborating institutions, but such integration is critical for project success.

Long-term results of a smoking reduction program.

INTRODUCTION: There have been few comprehensive evaluations of smoking reduction, especially in health care delivery systems, and little is known about its cost, maintenance of reduced smoking, or robustness across patient subgroups. METHODS: A generally representative sample of 320 adult smokers from an HMO scheduled for outpatient surgery or a diagnostic procedure was randomized to enhanced usual care or a theory-based smoking reduction intervention that combined telephone counseling and tailored newsletters. Outcomes included cigarettes smoked, carbon monoxide levels, and costs. RESULTS: Both intervention and control conditions continued to improve from 3- to 12-month assessments. Between-condition differences using intent-to-treat analyses on both self-report and carbon monoxide measures were nonsignificant by the 12-month follow-up (25% vs. 19% achieved 50% or greater reductions in cigarettes smoked). The intervention was implemented consistently despite logistical constraints and was generally robust across patient characteristics (eg, education, ethnicity, health literacy, dependence). CONCLUSIONS: In the absence of nicotine replacement therapy, the long-term effects of this smoking reduction intervention seem modest and nonsignificant. Future research is indicated to enhance intervention effects and conduct more comprehensive economic analyses of program variations.

Costing behavioral interventions: a practical guide to enhance translation.

BACKGROUND/PURPOSE: Cost and cost effectiveness of behavioral interventions are critical parts of dissemination and implementation into non-academic settings. Due to the lack of indicative data and policy makers' increasing demands for both program effectiveness and efficiency, cost analyses can serve as valuable tools in the evaluation process. METHODS: To stimulate and promote broader use of practical techniques that can be used to efficiently estimate the implementation costs of behavioral interventions, we propose a set of analytic steps that can be employed across a broad range of interventions. RESULTS/CONCLUSIONS: Intervention costs must be distinguished from research, development, and recruitment costs. The inclusion of sensitivity analyses is recommended to understand the implications of implementation of the intervention into different settings using different intervention resources. To illustrate these procedures, we use data from a smoking reduction practical clinical trial to describe the techniques and methods used to estimate and evaluate the costs associated with the intervention. Estimated intervention costs per participant were $419, with a range of $276 to $703, depending on the number of participants.

Comparative Clinical Effectiveness Research Focused on Community-Based Delivery of Palliative Care: Overview of the Patient-Centered Outcomes Research Institute's Funding Initiative.

Palliative care is a growing specialty that addresses the needs of individuals diagnosed with advanced illness and their caregivers. Although palliative care has been shown to improve a variety of patient- and caregiver-centered outcomes, access to comprehensive palliative care services for patients is often limited. There is a need to identify the most effective approaches to delivering palliative care to patients in community settings. In fiscal year 2017, based on extensive input from a diverse set of stakeholders, the Patient-Centered Outcomes Research Institute (PCORI) funded nine multisite comparative clinical effectiveness research (CER) trials focused on community-based delivery of palliative care for a total investment of $80 million. These studies, focusing on advance care planning and models of palliative care delivery, represent some of the largest most complex palliative care trials funded to date. Each study evaluates both patient and caregiver outcomes, and together, these trials include a broad range of health conditions, interventions, and settings of care. PCORI has also fostered a learning network of the funded awardees to facilitate the successful conduct of these CER studies and to support awardee efforts to develop collaborative products relevant to advancing the field of palliative care research and practice. The protocols of each of the nine trials, detailed in this issue, demonstrate the expansive reach of the investment PCORI has made in an effort to further the research agenda and provide substantive research evidence in stakeholder-identified areas of need in the field of palliative care.

National Working Group on the RE-AIM Planning and Evaluation Framework: Goals, Resources, and Future Directions.

The National Working Group on RE-AIM Planning and Evaluation Framework (herein Workgroup) was established in 2004 to support the application of the framework and advance dissemination and implementation science (D&I). Workgroup members developed and disseminated products and resources (and continue to do so) to advocate for consistent application of RE-AIM and allow for cross study comparisons. The purpose of this paper is to summarize key Workgroup activities, products, and services (e.g., webinars, consultations, planning tools) and enhance bidirectional communication between the Workgroup and RE-AIM users. The ultimate goal of this work is to serve as a forum for dissemination to improve the balance between RE-AIM user demand (needs) and the currently limited RE-AIM Workgroup supply (consultation and resources) to demonstrate and expand the utility of RE-AIM as a D&I planning and evaluation framework. A summary of resources is provided as well as specific examples of how the Workgroup has been responsive to user needs.

RE-AIM Planning and Evaluation Framework: Adapting to New Science and Practice With a 20-Year Review.

The RE-AIM planning and evaluation framework was conceptualized two decades ago. As one of the most frequently applied implementation frameworks, RE-AIM has now been cited in over 2,800 publications. This paper describes the application and evolution of RE-AIM as well as lessons learned from its use. RE-AIM has been applied most often in public health and health behavior change research, but increasingly in more diverse content areas and within clinical, community, and corporate settings. We discuss challenges of using RE-AIM while encouraging a more pragmatic use of key dimensions rather than comprehensive applications of all elements. Current foci of RE-AIM include increasing the emphasis on cost and adaptations to programs and expanding the use of qualitative methods to understand "how" and "why" results came about. The framework will continue to evolve to focus on contextual and explanatory factors related to RE-AIM outcomes, package RE-AIM for use by non-researchers, and integrate RE-AIM with other pragmatic and reporting frameworks.

Evaluating initial reach and robustness of a practical randomized trial of smoking reduction.

OBJECTIVE: This study evaluated the reach, initial effectiveness, and potential moderators and mediators of results of a smoking reduction program. DESIGN: A generally representative sample of 320 adult smokers from an HMO, scheduled for outpatient surgery or a diagnostic procedure, were randomized to enhanced usual care or a theory-based smoking reduction intervention that combined telephone counseling and tailored newsletters. MAIN OUTCOME MEASURES: Self-reported number of cigarettes smoked and carbon monoxide levels. RESULTS: The intervention enrolled 30% of known eligible smokers and produced reductions of 3 cigarettes per day greater than enhanced usual care. Intervention participants were significantly more likely than control participants to achieve at least a 50% reduction in self-reported number of cigarettes using complete cases, imputation analyses, and intent-to-treat procedures. Similar patterns were seen for carbon monoxide results but were significant only in complete case analyses. The intervention was generally robust across patient characteristics (e.g., education, ethnicity, health literacy, and dependence) and phone counselors. CONCLUSION: Initial results suggest that this program has potential to reach and assist smokers who may not participate in cessation programs. Additional research is indicated to enhance intervention effects, assess maintenance, and evaluate public health impact.

Development, Feasibility, and Small-Scale Implementation of a Web-Based Prognostic Tool-Surveillance, Epidemiology, and End Results Cancer Survival Calculator.

BACKGROUND: Population datasets and the Internet are playing an ever-growing role in the way cancer information is made available to providers, patients, and their caregivers. The Surveillance, Epidemiology, and End Results Cancer Survival Calculator (SEER*CSC) is a Web-based cancer prognostic tool that uses SEER data, a large population dataset, to provide physicians with highly valid, evidence-based prognostic estimates for increasing shared decision-making and improving patient-provider communication of complex health information. OBJECTIVE: The aim of this study was to develop, test, and implement SEER*CSC. METHODS: An iterative approach was used to develop the SEER*CSC. Based on input from cancer patient advocacy groups and physicians, an initial version of the tool was developed. Next, providers from 4 health care delivery systems were recruited to do formal usability testing of SEER*CSC. A revised version of SEER*CSC was then implemented in two health care delivery sites using a real-world clinical implementation approach, and usage data were collected. Post-implementation follow-up interviews were conducted with site champions. Finally, patients from two cancer advocacy groups participated in usability testing. RESULTS: Overall feedback of SEER*CSC from both providers and patients was positive, with providers noting that the tool was professional and reliable, and patients finding it to be informational and helpful to use when discussing their diagnosis with their provider. However, use during the small-scale implementation was low. Reasons for low usage included time to enter data, not having treatment options in the tool, and the tool not being incorporated into the electronic health record (EHR). Patients found the language in its current version to be too complex. CONCLUSIONS: The implementation and usability results showed that participants were enthusiastic about the use and features of SEER*CSC, but sustained implementation in a real-world clinical setting faced significant challenges. As a result of these findings, SEER*CSC is being redesigned with more accessible language for a public facing release. Meta-tools, which put different tools in context of each other, are needed to assist in understanding the strengths and limitations of various tools and their place in the clinical decision-making pathway. The continued development and eventual release of prognostic tools should include feedback from multidisciplinary health care teams, various stakeholder groups, patients, and caregivers.

Do behavioral smoking reduction approaches reach more or different smokers? Two studies; similar answers.

There is a need for innovative approaches capable of reaching smokers who would not otherwise participate in efforts to modify their smoking. This paper reports on two studies to determine whether a smoking reduction intervention would appeal to additional or different types of smokers than do cessation interventions. Study 1 attempted to contact 160 HMO smokers scheduled for outpatient surgeries. In Study 2, actual pilot reduction and cessation programs were offered to 531 smokers about to undergo out-patient surgeries or procedures. In Study 1, 39% of those eligible elected smoking reduction; and 38% selected cessation. In Study 2 of those eligible, 22% began participation in the smoking reduction program; 12% preferred a cessation approach; and 65% declined. There were few demographic or smoking history differences among those who elected smoking reduction, cessation, or declined. Among this understudied population, a sizable proportion in both studies agreed to participate in smoking reduction. If replicated, this suggests that comprehensive programs that include a smoking reduction component could substantially increase their reach.

The role of rapport: lessons learned from conducting research in a primary care setting.

Conducting research in a primary care setting is difficult because of the competing demands primary care offices have to manage in the current medical environment. Every primary care office has its own relationship and reporting structures, norms, and ways of conducting day-to-day affairs. Regardless of the setting, when carrying out qualitative and/or mixed-methods research, researchers must have rapport with the individuals or group of individuals they will be working with to carry out the research. In this article, the authors describe the different approaches that research staff have used to create rapport with practice staff that were not only participants in the study but were also responsible for the delivery and implementation of this effectiveness study.