Translating a motivational interviewing intervention for childhood cancer survivors into an eHealth tool: A user-centered design process.

Childhood cancer survivors have a higher risk of developing cardiomyopathy than members of the general population. Screening echocardiograms can facilitate early detection and treatment of cardiomyopathy. Furthermore, motivational interviewing can increase uptake of cardiac screening. However, such approaches are time- and resource-intensive, which limits their reach to the survivors who need them. We describe how we utilized a user-centered design process to translate an in-person motivational interviewing intervention into an eHealth tool to improve cardiac screening among childhood cancer survivors. We used an iterative, three-phase, user-centered design approach: (i) setting the stage (convening advisory boards and reviewing the original intervention), (ii) content programming and development (writing and programming intervention text and flow), and (iii) intervention testing (research team testing and cognitive interviews.) For cognitive interviews, participants were recruited via institutional participant registries and medical records. Data were analyzed using rapid qualitative analysis. During Phase 1, we identified survivor and provider advisors and outlined elements of the in-person intervention to change for the eHealth tool. During Phases 2 and 3, advisors recommended several modifications that guided the final intervention content and flow. Examples include: acknowledging potential hesitation or apprehension surrounding medical screenings, addressing barriers and facilitators to obtaining screening, and improving the tool's usability and appeal. In Phase 3, cognitive interview participants suggested additional refinements to the intervention language. This translation process shows that continued in-depth engagement of community advisors and iterative testing can improve the applicability of an eHealth to survivors' lived experiences and social contexts.

Childhood cancer survivors have a higher-than-average risk for developing heart damage compared to the general population. One-on-one interviews aimed at educating survivors about the importance of screening for heart damage can increase engagement in screening, but these programs are often too resource-intensive to be made available to large groups of survivors. Programs delivered using digital technology, like websites and smartphone apps, can be a more accessible alternative. In this article, we describe how we translated an in-person counseling program into a digital tool. We convened advisors who were childhood cancer survivors and healthcare providers to review the tool throughout the three-phase translation process: (i) setting the stage (convening advisory boards and reviewing original intervention), (ii) content programming and development (writing, and programming intervention text and flow), and (iii) intervention testing (research team testing and cognitive interviews.). Our translation process shows that continuously engaging with advisory boards and testing apps with participants can improve health programs in line with communities' diverse perspectives.

Understanding disruptions in cancer care to reduce increased cancer burden.

Background: This study seeks to understand how and for whom COVID-19 disrupted cancer care to understand the potential for cancer health disparities across the cancer prevention and control continuum. Methods: In this cross-sectional study, participants age 30+residing in an 82-county region in Missouri and Illinois completed an online survey from June-August 2020. Descriptive statistics were calculated for all variables separately and by care disruption status. Logistic regression modeling was conducted to determine the correlates of care disruption. Results: Participants (N=680) reported 21% to 57% of cancer screening or treatment appointments were canceled/postponed from March 2020 through the end of 2020. Approximately 34% of residents stated they would need to know if their doctor's office is taking the appropriate COVID-related safety precautions to return to care. Higher education (OR = 1.26, 95% CI:1.11-1.43), identifying as female (OR = 1.60, 95% CI:1.12-2.30), experiencing more discrimination in healthcare settings (OR = 1.40, 95% CI:1.13-1.72), and having scheduled a telehealth appointment (OR = 1.51, 95% CI:1.07-2.15) were associated with higher odds of care disruption. Factors associated with care disruption were not consistent across races. Higher odds of care disruption for White residents were associated with higher education, female identity, older age, and having scheduled a telehealth appointment, while higher odds of care disruption for Black residents were associated only with higher education. Conclusions: This study provides an understanding of the factors associated with cancer care disruption and what patients need to return to care. Results may inform outreach and engagement strategies to reduce delayed cancer screenings and encourage returning to cancer care. Funding: This study was supported by the National Cancer Institute's Administrative Supplements for P30 Cancer Center Support Grants (P30CA091842-18S2 and P30CA091842-19S4). Kia L. Davis, Lisa Klesges, Sarah Humble, and Bettina Drake were supported by the National Cancer Institute's P50CA244431 and Kia L. Davis was also supported by the Breast Cancer Research Foundation. Callie Walsh-Bailey was supported by NIMHD T37 MD014218. The content does not necessarily represent the official view of these funding agencies and is solely the responsibility of the authors.

Inclusive Recruitment Strategies to Maximize Sociodemographic Diversity among Participants: A St. Louis Case Study.

Background. Sociodemographically diverse study samples are critical for research related to health decision making. However, not all researchers have the training, capacity, and funding to engage research methods that recruit the most diverse populations. Objective and Methods. We used participant-generated data, staff salary data, and participant observation to examine the effectiveness and cost of strategies that we used for screening, enrolling, and retaining a sociodemographically diverse sample for a risk communication and behavior change randomized controlled trial. Results. It took approximately 646 hours to contact 1,626 individuals and enroll 554 participants (505 of whom completed the baseline survey; 45.2% were members of a underrepresented racial/ethnic group, 19.4% had no college education, 49.5% were age 30-49 y). Retention at 90-d follow-up was 93%. The total cost was USD$19,898.50. The average cost was $35.92 per participant enrolled. In-person recruitment was most successful in identifying the largest proportion of screened and eligible participants who were members of underrepresented racial/ethnic populations (32.8% and 27.8%, respectively) and with no college experience (39.7% and 33.5%, respectively); it also had the highest total cost ($8,079.17). Existing research pools identified the largest proportion of younger participants (ages 30-49 y; 39.3% and 43.4% for screened and eligible, respectively). Existing listservs yielded the smallest proportion of individuals with no college experience and the fewest members of underrepresented racial/ethnic populations but had the lowest total cost ($290.33). Newspaper ads identified the fewest younger individuals and also had the highest cost per participant enrolled ($166.21). Word of mouth had the lowest cost per participant enrolled ($10.47). Conclusion. Results help medical decision-making researchers formulate recruitment plans that increase sociodemographic diversity in study samples. We also ask funders to accommodate increased costs required to maximize sociodemographic diversity in medical decision-making research. Highlights: We provide concrete strategies for recruiting, enrolling, and retaining a sociodemographically diverse study sample.We offer cost estimates for all stages of study recruitment and found that in-person recruitment was the most effective, but also the most expensive, way to identify Black participants and participants with no college experience.It is critical for investigators to have access to institutional infrastructure and resources to support conducting research that is inclusive of diverse sociodemographic groups.An intentionally diverse recruitment staff supports a diverse study sample.

Testing Explanations for Skepticism of Personalized Risk Information.

BACKGROUND: The promise of precision medicine could be stymied if people do not accept the legitimacy of personalized risk information. We tested 4 explanations for skepticism of personalized diabetes risk information. METHOD: We recruited participants (N = 356; Mage = 48.6 [s = 9.8], 85.1% women, 59.0% non-Hispanic white) from community locations (e.g., barbershops, churches) for a risk communication intervention. Participants received personalized information about their risk of developing diabetes and heart disease, stroke, colon cancer, and/or breast cancer (women). Then they completed survey items. We combined 2 items (recalled risk, perceived risk) to create a trichotomous risk skepticism variable (acceptance, overestimation, underestimation). Additional items assessed possible explanations for risk skepticism: 1) information evaluation skills (education, graph literacy, numeracy), 2) motivated reasoning (negative affect toward the information, spontaneous self-affirmation, information avoidance); 3) Bayesian updating (surprise), and 4) personal relevance (racial/ethnic identity). We used multinomial logistic regression for data analysis. RESULTS: Of the participants, 18% believed that their diabetes risk was lower than the information provided, 40% believed their risk was higher, and 42% accepted the information. Information evaluation skills were not supported as a risk skepticism explanation. Motivated reasoning received some support; higher diabetes risk and more negative affect toward the information were associated with risk underestimation, but spontaneous self-affirmation and information avoidance were not moderators. For Bayesian updating, more surprise was associated with overestimation. For personal relevance, belonging to a marginalized racial/ethnic group was associated with underestimation. CONCLUSION: There are likely multiple cognitive, affective, and motivational explanations for risk skepticism. Understanding these explanations and developing interventions that address them will increase the effectiveness of precision medicine and facilitate its widespread implementation.

Community Research Fellows Training Program: Evaluation of a COVID-19-Precipitated Virtual Adaptation.

Community engagement is important for promoting health equity. However, effective community engagement requires trust, collaboration, and the opportunity for all stakeholders to share in decision-making. Community-based training in public health research can build trust and increase community comfort with shared decision-making in academic and community partnerships. The Community Research Fellows Training (CRFT) Program is a community-based training program that promotes the role of underserved populations in research by enhancing participant knowledge and understanding of public health research and other relevant topics in health. This paper describes the process of modifying the original 15-week in-person training program to a 12-week online, virtual format to assure program continuation. In addition, we provide program evaluation data of the virtual training. Average post-test scores were higher than pre-test scores for every session, establishing the feasibility of virtual course delivery. While the knowledge gains observed were not as strong as those observed for the in-person training program, findings suggest the appropriateness of continuing to adapt CRFT for virtual formats.

Adapting a Research and Community Capacity-Building Program to Address Rural Cancer Burden and Facilitate Partnership Development Between Rural Community Stakeholders and an Urban Comprehensive Cancer Center.

While rural-urban cancer disparities persist, the research building capacity between rural communities and high-quality cancer centers remains limited. Thus, we describe how a National Cancer Institute-designated cancer center partnered with rural community stakeholders to adapt a cancer prevention-focused research and community capacity-building workshop. The workshop's goal was to strengthen community-academic partnerships and facilitate the development of sustainable well-resourced rural cancer-focused research. Researchers from the Siteman Cancer Center partnered with community leaders from rural counties in southern Illinois. We adapted the workshop from an existing evidence-based program. We analyzed changes in knowledge and research capacity and relevance to their community work. From February to May 2019, community partners guided all elements of the workshop development. Workshop participants were mostly White race (93%), had a college degree or beyond (75%), reported living in a rural community (93%), and represented an academic, faith-based, or healthcare institution (78%). Participants' mean knowledge scores of the presented content increased significantly after each session, from 9.3 to 9.9 for session 1 (p = 0.05) and 6.8 to 9.7 (p < 0.001) for session two. Through the workshop, participant scores also increased in research capacity skills, confidence, and their understanding of conducting research in the community. The workshop, co-curated and led by rural community leaders and researchers from Siteman Cancer Center, successfully increased knowledge of and interest in building cancer research capacity. Lessons from our work can inform the implementation of similar programs that address rural cancer health through research and community capacity building between rural community partners and urban cancer centers.

Construct validation of the Research Engagement Survey Tool (REST).

BACKGROUND: The Research Engagement Survey Tool (REST) was developed to examine the level of partner (e.g., patients, caregivers, advocates, clinicians, community members) engagement in research studies. The REST is aligned with eight engagement principles based on the literature and consensus reached through a five round Delphi process. Each of the engagement principles has three-five corresponding items that are assessed on two Likert type scales quantity (how often: never, rarely, sometimes, often, always, not applicable) and quality (how well: poor, fair, good, very good, excellent, not applicable). We conducted a comprehensive validation of the REST. Despite the importance of partner engagement in research, currently no gold standard measure exists. METHODS: Multiple strategies were employed to validate the REST. Here, we examine the internal consistency of items for each of the eight engagement principles. In addition, we examine the convergent validity of the comprehensive (32-item) REST with other measures (e.g., medical mistrust, Community Engagement in Research Index, Partnership Self-Assessment Tool, Wilder collaboration inventory, Partnership Assessment In community-based Research). We propose two scoring approaches for the REST; one aligned with the engagement principles and the other aligned with levels of community engagement: (1) outreach and education, (2) consultation, (3) cooperation, (4) collaboration, and (5) partnership. RESULTS: The REST has strong internal consistency (Cronbach's alpha > 0.75) for each of the eight engagement principals measured on both scales (quality and quantity). The REST had negligible (e.g., medical mistrust, community engagement in research index), low (e.g., Partnership Assessment In community-based Research, Partnership Self-Assessment Tool- benefits scale), and moderate (e.g., Wilder collaboration inventory, Partnership Self-Assessment Tool- synergy scale) statistically significant correlations with other measures based on the Spearman rank correlation coefficient. These results suggest the REST is measuring something similar and correlated to the existing measures, but it captures a different construct (perceived research engagement). CONCLUSIONS: The REST is a valid and reliable tool to assess research engagement of community health stakeholders in the research process. Valid tools to assess research engagement are necessary to examine the impact of engagement on the scientific process and scientific discovery and move the field of stakeholder engagement from best practices and lessons learned to evidence-based approaches based on empirical data.

Researchers often conduct studies with partners (e.g., patients, caregivers, advocates, clinicians, community members) who also have an interest in the research topic. Depending on the research study the level of partner engagement in the research process may be high or low. Partners may be involved from the beginning including determining what topic to study and what questions the study should examine. They may suggest who should be included in the study, the geographic area of focus, and the outcome measures to be examined. In addition, they may help recruit study participants, interpret study results, and plan for how to share the results with those that need to know. No standard way exists to find out how involved a partner has been in a study from the partner's perspective. Here we develop and validate survey questions to measure the level of partner engagement in research studies. We looked at existing survey questions used to measure similar topics to make sure that a person who takes the survey gets consistent scores. We tested the survey with community health stakeholders (e.g., patients, caregivers, advocates, clinicians, community members) who are research partners for studies at universities across the United States. Over 2 years, the partners took different versions of the survey online four times. We used the data we collected from each survey to revise the questions and make sure that it measures partner involvement accurately and reliably. The Research Engagement Survey Tool (REST) has 32 questions to examine eight engagement principles on two scales: quantity (how much) and quality (how well). The REST is a valid and reliable tool to examine partner engagement in research.

A Study Examining the Usefulness of a New Measure of Research Engagement.

INTRODUCTION: Engagement of relevant stakeholders' ideas, opinions, and concerns is critical to the success of modern research projects. We have developed a tool to measure stakeholder engagement, called the Research Engagement Survey Tool (REST). The purpose of this paper is to present the implementation and uptake of the stakeholder engagement measure REST among research teams, including the assessment of barriers and facilitating factors for use of the new research engagement measure in practice. METHODS: In this implementation study, project team members participated in baseline and follow-up web-based surveys. Web-based interviews were conducted with a subset of project teams that implemented the REST. On the baseline survey, project teams were asked to provide details about up to three ongoing or recently completed projects, were asked if they agreed with compensation for REST completion, and were asked if they would like to send the survey to stakeholders or would prefer our project team to email their project stakeholders. Follow-up surveys contained questions on reactions to implementing REST and results of REST. RESULTS: Project team members/researchers who completed the baseline survey (n=86) were mostly female (79%) and Non-Hispanic/Latino(a) White (76%). Those who implemented REST were also mostly female (86%) and Non-Hispanic/Latino(a) White (71%), with an average of 11 years in academic research. About 98% of all participants completing the baseline survey had the capacity to survey partners, while 100% of all teams who implemented REST did. A small portion of respondents indicated the time commitment of REST would be a barrier (29% of baseline survey respondents, 10% of those who implemented REST) and indicated workload would be a barrier (31% of baseline survey respondents, 14% of those who implemented REST). DISCUSSION: The data presented here indicate that REST implementation is feasible in a volunteer group of ongoing research projects.

Joanne Knight Breast Health Cohort at Siteman Cancer Center.

PURPOSE: The Joanne Knight Breast Health Cohort was established to link breast cancer risk factors, mammographic breast density, benign breast biopsies and associated tissue markers, and blood markers in a diverse population of women undergoing routine mammographic screening to study risk factors and validate models for breast cancer risk prediction. METHODS: Women were recruited from November 2008 to April 2012 through the mammography service at the Joanne Knight Breast Health Center at Washington University in St. Louis, Missouri. Baseline questionnaire risk factors, blood, and screening mammograms were collected from 12,153 women. Of these, 1,672 were excluded for prior history of any cancer (except non-melanoma skin) or diagnosis of breast cancer within 6 months of blood draw/registration for the study, for a total of 10,481 women. Follow-up is through linking to electronic health records, tumor registry, and death register. Routine screening mammograms are collected every 1-2 years and incident benign breast biopsies and cancers are identified through record linkage to pathology and tumor registries. Formal fixed tissue samples are retrieved and stored for analysis. County-level measures of structural inequality were derived from publicly available resources. RESULTS: Cohort Composition: median age at entry was 54.8 years and 26.7% are African American. Through 2020, 74% of participants have had a medical center visit within the past year and 80% within the past 2 years representing an average of 9.7 person-years of follow-up from date of blood draw per participant. 9,997 women are continuing in follow-up. Data collected at baseline include breast cancer risk factors, plasma and white blood cells, and mammograms prior to baseline, at baseline, and during follow-up. CONCLUSION: This cohort assembled and followed in a routine mammography screening and care setting that serves a diverse population of women in the St. Louis region now provides opportunities to integrate study of questionnaire measures, plasma and DNA markers, benign and malignant tissue markers, and repeated breast image features into prospective evaluation for breast cancer etiology and outcomes.

Mental imagery-based self-regulation: Effects on physical activity behaviour and its cognitive and affective precursors over time.

OBJECTIVES: (1) Test whether a mental imagery-based self-regulation intervention increases physical activity behaviour over 90 days; (2) Examine cognitive and affective precursors of change in physical activity behaviour. DESIGN: A randomized control trial with participants (N = 500) randomized to one of six intervention conditions in a 3 (risk communication format: bulleted list, table, risk ladder) x 2 (mental imagery behaviour: physical activity, active control [sleep hygiene]) factorial design. METHODS: After receiving personalized risk estimates via a website on a smartphone, participants listened to an audiorecording that guided them through a mental imagery activity related to improving physical activity (intervention group) or sleep hygiene behaviour (active control). Participants received text message reminders to complete the imagery for 3 weeks post-intervention, 4 weekly text surveys to assess behaviour and its cognitive and affective precursors, and a mailed survey 90 days post-baseline. RESULTS: Physical activity increased over 90 days by 19.5 more minutes per week (95%CI: 2.0, 37.1) in the physical activity than the active control condition. This effect was driven by participants in the risk ladder condition, who exercised 54.8 more minutes (95%CI 15.6, 94.0) in the physical activity condition than participants in the active control sleep hygiene group. Goal planning positively predicted physical activity behaviour (b = 12.2 minutes per week, p = 0.002), but self-efficacy, image clarity, and affective attitudes towards behaviours did not (p > 0.05). CONCLUSIONS: Mental imagery-based self-regulation interventions can increase physical activity behaviour, particularly when supported by personalized disease risk information presented in an easy-to-understand format.

10-year weight loss outcomes after Roux-en-Y gastric bypass and attendance at follow-up visits: a single-center study.

BACKGROUND: Long-term durability of weight loss is a prerequisite for a greater acceptance of bariatric surgery. OBJECTIVES: To examine long-term weight trajectory in patients undergoing Roux-en-Y gastric bypass (RYGB) and determine factors predicting long-term follow-up and weight outcomes. SETTING: University hospital. METHODS: A retrospective cohort of adults who underwent RYGB during 1997-2010 were identified and followed until 2017. Predictors for attendance at periodic follow-up visits, reduction in body mass index (BMI), and percent excess BMI lost were determined using multivariable logistic regression and linear mixed-effects models. The latter was used to predict long-term weight outcomes for a typical patient. RESULTS: The study included 1104 patients with a mean age of 45.5 (standard deviation [SD] 9.9) years and a preoperative BMI of 54.7 (SD 10.9) kg/m2. Follow-up data were available for 92.8% of the patients after 1 year, 50.0% after 5 years, and 35.2% after 10 years post-surgery. Black patients, compared with White patients, were less likely to attend follow-up visits. Attendance at follow-up visits at least every other year was not associated with larger weight loss, but higher preoperative BMI, being White (versus Black), and female sex were. Predicted BMI reduction for a typical patient, a 45-year-old White female with a preoperative BMI of 54.7 kg/m2 and private health insurance, undergoing laparoscopic RYGB in 2004, was 18.3 (standard error [SE] .36) kg/m2 at year 5 and 17.6 (SE .49) kg/m2 at year 10. CONCLUSION: RYGB results in clinically significant and durable weight loss. Attendance at periodic follow-up visits does not appear to be associated with long-term weight loss outcomes. Future work should focus on strategies to remove barriers to post-operative care.

Development and Validation of a Brief Version of the Research Engagement Survey Tool.

The Research Engagement Survey Tool (REST) examines the level of partner engagement in research studies. This study used mixed methods, including web-based surveys (N = 336), a modified Delphi process (N = 18), and cognitive response interviews (N = 16), with convenience sampling to develop and validate a short version of the REST. We conducted factor analysis and calculated internal consistency for the condensed REST. We validated the condensed REST against the comprehensive REST. All analyses were carried out on two scales (quality and quantity) based on Likert-type response options. We examined convergent validity with other measures theoretically associated with the REST (e.g., the Community Engagement Research Index and the Partnership Self-Assessment Tool). This study produced a 9-item condensed version of the REST. The condensed REST loads on 1 factor, has high internal consistency (Cronbach's alpha = 0.92 for the quantity scale; 0.94 for the quality scale), is significantly correlated (ρ = 0.97; p < 0.001 for both scales) with the comprehensive (32-item) REST, and has negligible, low, and moderate correlation with other measures (e.g., the Partnership Assessment In community-based Research, trust in medical researchers, and the Coalition Self-Assessment Survey). Use of the condensed REST will reduce participant burden and time to complete. This standardized and validated quantitative measure is useful to compare engagement across projects or within a project over time.

Understanding the Current Role of Robotic-Assisted Bariatric Surgery.

BACKGROUND: The role of robotic surgery in bariatrics remains controversial. Patient selection for robotic surgery is not well-studied. The objective of this study was to identify factors associated with robotic surgery and its temporal trends. METHODS: The Metabolic and Bariatric Surgery Accreditation and Quality Improvement Program database from 2015 to 2018 was used. Adult patients undergoing primary sleeve gastrectomy (SG) and Roux-en-Y gastric bypass (RYGB) using the laparoscopic or robotic approach were identified. Revisional, hybrid, or those with concomitant procedures were excluded. Logistic regression was conducted to identify factors associated with undergoing robotic-assisted surgery. RESULTS: Among 211,568 patients who underwent SG, 9.1% underwent a robotic SG; among 76,805 patients who underwent RYGB, 7.9% of patients underwent a robotic RYGB. During 2015-2018, robotics increased from 7.1 to 11.3% for SG and 7.4 to 8.6% for RYGB. After controlling for patient characteristics, there was still an increasing trend in the use of robotic surgery: SG (multivariable-adjusted odds ratio, aOR, 1.18; 95% confidence interval, CI, 1.17-1.20) and RYGB (aOR, 1.05; 95% CI, 1.03-1.08). For both robotic SG and RYGB, functional status and African American race were associated with undergoing robotic surgery, while races other than White or African American and Hispanic ethnicity were not. Pre-operative IVC filter was associated with robotic SG, while the presence of GERD, diabetes, and COPD were associated with robotic RYGB. CONCLUSIONS: Robotic bariatric surgery has increased over time. Our findings identified factors associated with the receipt of robotic surgery. Reasons for these factors require further investigation to better delineate indications for this technology.

Risk Ladder, Table, or Bulleted List? Identifying Formats That Effectively Communicate Personalized Risk and Risk Reduction Information for Multiple Diseases.

BACKGROUND: Personalized medicine may increase the amount of probabilistic information patients encounter. Little guidance exists about communicating risk for multiple diseases simultaneously or about communicating how changes in risk factors affect risk (hereafter "risk reduction"). PURPOSE: To determine how to communicate personalized risk and risk reduction information for up to 5 diseases associated with insufficient physical activity in a way laypeople can understand and that increases intentions. METHODS: We recruited 500 participants with <150 min weekly of physical activity from community settings. Participants completed risk assessments for diabetes, heart disease, stroke, colon cancer, and breast cancer (women only) on a smartphone. Then, they were randomly assigned to view personalized risk and risk reduction information organized as a bulleted list, a simplified table, or a specialized vertical bar graph ("risk ladder"). Last, they completed a questionnaire assessing outcomes. Personalized risk and risk reduction information was presented as categories (e.g., "very low"). Our analytic sample (N = 372) included 41.3% individuals from underrepresented racial/ethnic backgrounds, 15.9% with vocational-technical training or less, 84.7% women, 43.8% aged 50 to 64 y, and 71.8% who were overweight/obese. RESULTS: Analyses of covariance with post hoc comparisons showed that the risk ladder elicited higher gist comprehension than the bulleted list (P = 0.01). There were no significant main effects on verbatim comprehension or physical activity intentions and no moderation by sex, race/ethnicity, education, numeracy, or graph literacy (P > 0.05). Sequential mediation analyses revealed a small beneficial indirect effect of risk ladder versus list on intentions through gist comprehension and then through perceived risk (bIndirectEffect = 0.02, 95% confidence interval: 0.00, 0.04). CONCLUSION: Risk ladders can communicate the gist meaning of multiple pieces of risk information to individuals from many sociodemographic backgrounds and with varying levels of facility with numbers and graphs.

Strategies of community engagement in research: definitions and classifications.

Engagement activities are defined along a continuum that analyzes and represents nonacademic stakeholder activities and interactions with academic researchers. Proposed continua begin with none to limited stakeholder inclusion and input into research and continue with descriptions of increasing presence, input, and participation in decision-making. Despite some agreement in the literature, development of consistent terminology and definitions has been recommended to promote the common understanding of strategies in engaged research. This paper sought to develop and understand classifications and definitions of community-engaged research that can serve as the foundation of a measure of engaged research that permits comparisons among engagement strategies and the outcomes that they produce in health- and healthcare-related research studies. Data on academic and stakeholder perceptions and understandings of classifications and definitions were obtained using Delphi process (N = 19) via online and face-to-face survey and cognitive response interviews (N = 16). Participants suggested the need for more nuanced understanding of engagement along portions of the continuum, with active involvement and decision-making as engagement progressed. Cognitive interview responses suggested that outreach and education is a more advanced level of engagement than previously discussed in the literature and viewed consultation negatively because it required work without guaranteeing community benefit. It is possible to define a continuum of patient- and community-engaged research that is understood and accepted by both academic researchers and community members. However, future research should revisit the understanding and depiction of the strategies that are to be used in measure development.

Community partners' responses to items assessing stakeholder engagement: Cognitive response testing in measure development.

BACKGROUND: Despite recognition of the importance of stakeholder input into research, there is a lack of validated measures to assess how well constituencies are engaged and their input integrated into research design. Measurement theory suggests that a community engagement measure should use clear and simple language and capture important components of underlying constructs, resulting in a valid measure that is accessible to a broad audience. OBJECTIVE: The primary objective of this study was to evaluate how community members understood and responded to a measure of community engagement developed to be reliable, valid, easily administered, and broadly usable. METHOD: Cognitive response interviews were completed, during which participants described their reactions to items and how they processed them. Participants were asked to interpret item meaning, paraphrase items, and identify difficult or problematic terms and phrases, as well as provide any concerns with response options while responding to 16 of 32 survey items. RESULTS: The results of the cognitive response interviews of participants (N = 16) suggest concerns about plain language and literacy, clarity of question focus, and the lack of context clues to facilitate processing in response to items querying research experience. Minimal concerns were related to response options. Participants suggested changes in words and terms, as well as item structure. CONCLUSION: Qualitative research can improve the validity and accessibility of measures that assess stakeholder experience of community-engaged research. The findings suggest wording and sentence structure changes that improve ability to assess implementation of community engagement and its impact on research outcomes.

Reaching Consensus on Principles of Stakeholder Engagement in Research.

BACKGROUND: Stakeholder-engaged research is an umbrella term for the types of research that have community, patient, and/or stakeholder engagement, feedback, and bidirectional communication as approaches used in the research process. The level of stakeholder engagement across studies can vary greatly, from minimal engagement to fully collaborative partnerships. OBJECTIVES: To present the process of reaching consensus among stakeholder and academic experts on the stakeholder engagement principles (EPs) and to identify definitions for each principle. METHODS: We convened 19 national experts, 18 of whom remained engaged in a five-round Delphi process. The Delphi panel consisted of a broad range of stakeholders (e.g., patients, caregivers, advocacy groups, clinicians, researchers). We used web-based surveys for most rounds (1-3 and 5) and an in-person meeting for round 4. Panelists evaluated EP titles and definitions with a goal of reaching consensus (>80% agreement). Panelists' comments guided modifications, with greater weight given to non-academic stakeholder input. CONCLUSIONS: EP titles and definitions were modified over five Delphi rounds. The panel reached consensus on eight EPs (dropping four, modifying four, and adding one) and corresponding definitions. The Delphi process allowed for a stakeholder-engaged approach to methodological research. Stakeholder engagement in research is time consuming and requires greater effort but may yield a better, more relevant outcome than more traditional scientist-only processes. This stakeholder-engaged process of reaching consensus on EPs and definitions provides a key initial step for the content validation of a survey tool to examine the level of stakeholder engagement in research studies.

Specifying Future Behavior When Assessing Risk Perceptions: Implications for Measurement and Theory.

Background. Many theories assert that high perceived risk motivates health behavior change; the empirical literature shows mixed findings. Purpose. To determine whether, for whom, and under what circumstances specifying a future behavior when assessing perceived risk (i.e., "conditioning" risk perception items on behavior) improves data quality and strengthens the perceived risk-intentions/behavior relationship. Methods. Internet panel participants (N = 787, 58.8% no college experience, 44.4% racial/ethnic minority, 43.7% men, 67.3% aged 18-49 years, 59.0% nonadherent to physical activity guidelines) answered 8 colon cancer perceived risk items in a within-subjects design. Participants answered 4 types of risk perception items: absolute and comparative perceived likelihood and absolute and comparative feelings of risk. Participants answered each type of item twice: once conditioned on not engaging in physical activity and once unconditioned. Results. Compared to unconditioned items, conditioned items elicited fewer "don't know" (DK) responses (OR = 0.80; 95% CI, 0.68-0.93), higher risk perceptions (b = 0.55; 95% CI, 0.49-0.61) and stronger positive correlations with intentions (zSteiger = 5.46, P < 0.001) and behavior (zSteiger = 5.10, P < 0.001). The effect of conditioning was more pronounced for perceived likelihood than feelings of risk items (OR = 2.21; 95% CI, 1.63-3.01 and b = 0.14; 95% CI, 0.08-0.20 for DK responding and risk perception magnitude, respectively). The effect on risk perception magnitude (except absolute feelings of risk) was higher among people with higher health literacy (χ2(3) = 8.11, P = 0.04). Conclusions. Researchers who examine whether perceived risk motivates precautionary behavior should consider conditioning risk perception items on behavior to increase the validity of the statistical conclusions they draw and to gain insight into the nature of perceived risk and its relation to behavior.

Content validation of a quantitative stakeholder engagement measure.

AIM: Using a stakeholder-engaged approach, this study conducted content validation and item reduction of a quantitative measure of research engagement. METHODS: A five-round modified Delphi process was used to reach consensus on items. Rounds 1-3 and 5 were conducted using web-based surveys. Round 4 consisted of a 2-day, in-person meeting. Delphi panelists received individualized reports outlining individual and aggregate group responses after rounds 1-3. RESULTS: Over the five-round process, items were added, dropped, modified, and moved from one engagement principle to another. The number of items was reduced from 48 to 32, with three to five items corresponding to eight engagement principles. CONCLUSIONS: Research that develops standardized, reliable, and accurate measures to assess stakeholder engagement is essential to understanding the impact of engagement on scientific discovery and the scientific process. Valid quantitative measures to assess stakeholder engagement in research are necessary to assess associations between engagement and research outcomes.