Mixed methods evaluation of the inaugural year of the Cancer Prevention and Control Research Network's (CPCRN) scholars program.

PURPOSE: A diverse workforce trained in dissemination & implementation (D&I) science is critical for improving cancer outcomes and reducing cancer-related health disparities. This study aims to describe and evaluate impact of the Cancer Prevention and Control Research Network (CPCRN) Scholars Program in preparing scholars for collaborative careers in cancer control and implementation research and practice, and offers evaluation-driven recommendations for program improvements. METHODS: The CPCRN Scholars Workgroup conducted a sequential, mixed methods evaluation. We collected baseline and follow-up surveys and invited all 20 scholars and ten mentors to participate in an exit interview. We assessed the experience with the Scholar's program, ratings of D&I competences, progress on their project, feedback about the curriculum, and understanding of implementation science. RESULTS: Over 86% partially or fully completed their project within 9 months; 78% of scholars engaged with a CPCRN workgroup. Scholars rated the following program components as valuable: the Putting Public Health Evidence in Action (PPHEIA) training (88.9%), D&I training modules (83.3%), and webinars (kickoff webinar-88.9% and selecting theories/models-88.9%). There was an increase in D&I competencies from baseline to posttest, with the greatest in community engagement topics. About 78% reported that they were satisfied with format of the activities and increased confidence in ability to discuss D&I concepts. From the qualitative interviews, the benefit of the program was becoming more knowledgeable about D&I research and networking. CONCLUSION: The inaugural year of the program yielded positive results, particularly related to increasing knowledge about D&I science and cancer control. This program builds the capacity of students, researchers and practitioners in D&I science.

Designing for Dissemination and Sustainability to Promote Equitable Impacts on Health.

Designing for dissemination and sustainability (D4DS) refers to principles and methods for enhancing the fit between a health program, policy, or practice and the context in which it is intended to be adopted. In this article we first summarize the historical context of D4DS and justify the need to shift traditional health research and dissemination practices. We present a diverse literature according to a D4DS organizing schema and describe a variety of dissemination products, design processes and outcomes, and approaches to messaging, packaging, and distribution. D4DS design processes include stakeholder engagement, participatory codesign, and context and situation analysis, and leverage methods and frameworks from dissemination and implementation science, marketing and business, communications and visualarts, and systems science. Finally, we present eight recommendations to adopt a D4DS paradigm, reflecting shifts in ways of thinking, skills and approaches, and infrastructure and systems for training and evaluation.

The Effect of the Affordable Care Act on Women's Postpartum Insurance and Depression in 5 States That Did Not Expand Medicaid, 2012-2015.

BACKGROUND: Before the Affordable Care Act (ACA), most women who gained pregnancy-related Medicaid were not eligible for Medicaid as parents postpartum. The ACA aimed to expand health insurance coverage, in part, by expanding Medicaid; introducing mandates; reforming regulations; and establishing exchanges with federal subsidies. Federal subsidies offer a means to coverage for individuals with income at 100%-400% of the federal poverty level who do not qualify for Medicaid. OBJECTIVE: The objective of this study was to identify the effects of the ACA's non-Medicaid provisions on women's postpartum insurance coverage and depressive symptoms in nonexpansion states with low parental Medicaid thresholds. PARTICIPANTS: Women with incomes at 100%-400% of the federal poverty level who had prenatal insurance and completed the Pregnancy Risk Assessment Monitoring System (2012-2015). SETTING: Five non-Medicaid expansion states with Medicaid parental eligibility thresholds below the federal poverty level. DESIGN: Interrupted time-series analyses were conducted to examine changes between pre-ACA (January 2012-November 2013) and post-ACA (December 2013-December 2015) trends for self-reported loss of postpartum insurance and symptoms of postpartum depression. RESULTS: The sample included 9,472 women. Results showed significant post-ACA improvements where the: (1) trend for loss of postpartum insurance reversed (change of -0.26 percentage points per month, P=0.047) and (2) level of postpartum depressive symptoms decreased (change of -3.5 percentage points, P=0.042). CONCLUSIONS: In these 5 states, the ACA's non-Medicaid provisions were associated with large increases in retention of postpartum insurance and reductions in postpartum depressive symptoms, although depressive symptoms findings are sensitive to model specification.

Dissemination of an evidence-based behavioral intervention to alleviate distress in caregivers of children recently diagnosed with cancer: Bright IDEAS.

BACKGROUND: Four multisite randomized clinical trials of > 1400 caregivers of children newly diagnosed with cancer showed that the Bright IDEAS (BI) paradigm of problem-solving skills training is an acceptable and efficacious approach to alleviating the high levels of distress they experience. To facilitate providing evidence-based caregiver support as recommended in the pediatric oncology standards of care, the project described here was designed to disseminate BI to 200 psychosocial professionals. PROCEDURE: We partnered with the Children's Oncology Group (COG), Association of Pediatric Oncology Social Workers (APOSW), Association of Pediatric Hematology/Oncology Nurses (APHON), and special interest group in pediatric hematology/oncology of the Society for Pediatric Psychology (SPP). Membership surveys revealed substantial enthusiasm for training in BI. We structured training to include review of the evidence base for BI, role plays, and strategies for implementation at individual sites. Four conference calls designed to enhance implementation were held one, two, three, and five months after training. RESULTS: Ten 1.5-day workshops were held in conjunction with annual meetings of COG, APOSW, APHON, and SPP. A total of 209 psychosocial clinicians from 134 sites were trained. Evaluations were highly favorable. Trainees had provided BI to 545 individuals as of the last conference call. CONCLUSIONS: Initial dissemination goals were met. BI is now available at numerous pediatric oncology centers, but it has not become part of routine care. Future work focused on implementation might consider top-down approaches that include direct communication with pediatric oncologists and hospital leaders about the benefits of incorporating this evidence-based intervention systemically.

A flexible mixed-data model applied to claims data for post-market surveillance of prescription drug safety behavior.

We develop a new modeling framework for jointly modeling first prescription times and the presence of risk-mitigating behavior for prescription drugs using real-world data. We are interested in active surveillance of clinical quality improvement programs, especially for drugs which enter the market under an FDA-mandated Risk Evaluation and Mitigation Strategy (REMS). Our modeling framework attempts to jointly model two important aspects of prescribing, the time between a drug's initial marketing and a patient's first prescription of that drug, and the presence of risk-mitigating behavior at the first prescription. First prescription times can be flexibly modeled as a mixture of component distributions to accommodate different subpopulations and allow the proportion of prescriptions that exhibit risk-mitigating behavior to change for each component. Risk-mitigating behavior is defined in the context of each drug. We develop a joint model using a mixture of positive unimodal distributions to model first prescription times, and a logistic regression model conditioned on component membership to model the presence of risk-mitigating behavior. We apply our model to two recently approved extended release/long-acting (ER/LA) opioids, which have an FDA-approved blueprint for best prescribing practices to inform our definition of risk-mitigating behavior. We also apply our methods to simulated data to evaluate their performance under various conditions such as clustering.

Pragmatic applications of implementation science frameworks to regulatory science: an assessment of FDA Risk Evaluation and Mitigation Strategies (REMS) (2014-2018).

BACKGROUND: A Risk Evaluation and Mitigation Strategy (REMS) is a drug safety program for certain medications with serious safety concerns required by the U.S. Food and Drug Administration (FDA) of manufacturers to implement to help ensure the benefits of the medication outweigh its risks. FDA is encouraging "the research community to develop novel methods for assessing REMS," conveying the unmet need for a standardized evaluation method of these regulatory-mandated healthcare programs. The objective of this research is to evaluate FDA REMS assessment plans using established implementation science frameworks and identify opportunities for strengthening REMS evaluation. METHODS: A content analysis was conducted of publicly available assessment plans for all REMS programs (N = 23) approved 1/1/2014-12/31/2018 for new drug applications (NDAs) and biologics license applications (BLAs) requiring FDA-mandated Elements to Assure Safe Use (ETASU). Blinded reviewers critically appraised REMS assessment measures (n = 674) using three established implementation science frameworks: RE-AIM (Reach, Effectiveness, Adoption, Implementation, Maintenance); PRECEDE-PROCEED (Predisposing, Reinforcing, and Enabling Constructs in Educational/Environmental Diagnosis and Evaluation - Policy, Regulatory, and Organizational Constructs in Educational and Environmental Development); and CFIR (Consolidated Framework for Implementation Research). Framework constructs were mapped to REMS Assessment categories as defined by FDA Guidance for Industry to evaluate congruence. RESULTS: REMS assessment measures demonstrated strong congruence (> 90% mapping rate) with the evaluative constructs of RE-AIM, PRECEDE-PROCEED, and CFIR. Application of the frameworks revealed that REMS assessment measures heavily emphasize implementation and operations, focus less on health outcomes, and do not evaluate program context and design assumptions. CONCLUSIONS: Implementation science frameworks have utility for evaluating FDA-mandated drug safety programs including the selection of primary measures to determine whether REMS goals are being met and of secondary measures to evaluate contextual factors affecting REMS effectiveness in varying organizational settings.

Child characteristics and early intervention referral and receipt of services: a retrospective cohort study.

BACKGROUND: Early Intervention (EI) is a federally mandated, state-administered system of care for children with developmental delays and disabilities under the age of three. Gaps exist in the process of accessing EI through pediatric primary care, and low rates of EI access are well documented and disproportionately affect poor and minority children. The aims of this paper are to examine child characteristics associated with gaps in EI (1) referral, (2) access and (3) service use. To our knowledge, this is the first study to leverage linked safety net health system pediatric primary care and EI records data to follow EI-referred children longitudinally to understand EI service use gaps from EI referral to EI service utilization. METHODS: In a retrospective cohort design (14,710 children with developmental disability or delay), we linked pediatric primary care records between a large, integrated safety net health system in metro Denver and its corresponding EI program (2014-2016). Using adjusted marginal effects [ME, (95% CI)], we estimated gaps in EI referral, access, and service type (i.e., physical [PT], occupational [OT], speech therapy [ST] and developmental intervention [DI]). Analyses accounted for child characteristics including socio-demographics, diagnosis, condition severity, and baseline function. RESULTS: Only 18.7% of EI-eligible children (N = 2726) received a referral; 26% of those (N = 722) received services for a net enrollment rate of 5% among EI-eligible children. Having the most severe developmental condition was positively associated with EI referral [ME = 0.334 [0.249, 0.420]) and Individualized Family Services Plan (IFSP) receipt [ME = 0.156 [0.088, 0.223]). Children less likely to be EI-referred were Black, non-Hispanic (BNH) [ME = -0.029 (- 0.054, - 0.004)] and had a diagnosed condition ([ME = - 0.046 (- 0.087, - 0.005)]. Children with a diagnosis and those with higher income were more likely to receive PT or OT. Higher baseline cognitive and adaptive skills were associated with lower likelihood of PT [ME = -0.029 (- 0.054, - 0.004)], OT [ME = -0.029 (- 0.054, - 0.004)], and ST [ME = -0.029 (- 0.054, - 0.004)]. CONCLUSIONS: We identified and characterized gaps in EI referral, access, and service use in an urban safety-net population of children with high rates of developmental delay. Interventions are needed to improve integrated systems of care affecting primary care and EI processes and coordination.

Family-centred care in early intervention: Examining caregiver perceptions of family-centred care and early intervention service use intensity.

BACKGROUND: Family-centred care (FCC) is an approach to paediatric rehabilitation service delivery endorsing shared decision making and effective communication with families. There is great need to understand how early intervention (EI) programmes implement these processes, how EI caregivers perceive them, and how they relate to EI service use. Therefore, the purpose of this study is to examine (a) parent and provider perceptions about EI FCC processes and (b) the association between FCC perceptions and EI service intensity. METHODS: In this cross-sectional study, parent perceptions of EI FCC were measured using the electronically administered Measures of Processes of Care (MPOC-56 and MPOC-SP; using 7-point scales). Participants included EI parents (n = 29) and providers (n = 9) from one urban EI programme (1/1/18-6/1/18). We linked survey responses with child characteristics and service use ascertained through EI records. We estimated parent-provider MPOC score correlations and the association between EI service intensity (hr/month) and parent MPOC scores using adjusted linear regression accounting for child characteristics. RESULTS: Parents (M = 4.2, SD = 1.1) and providers (M = 5.8, SD = 1.3) reported low involvement related to general information exchange. Parent and provider subscale scores were not correlated except that parent-reported receipt of specific information was inversely associated with provider-reported provision of general information (r = -0.4, P < .05). In adjusted models, parent perceptions related to respectful and supportive (b = 1.57, SE = 0.56) and enabling (b = 1.42, SE = 0.67) care were positively associated with EI intensity, whereas specific information exchange and general information exchange were not associated with intensity. CONCLUSION: We found that EI parents and providers reported high levels of investment in the family centredness of their EI care, with the exception of information sharing. Greater EI service intensity was associated with higher perception of involvement with some metrics of family centredness.

Ethnoracial Disparities in Medicare Annual Wellness Visit Utilization: Evidence From a Nationally Representative Database.

INTRODUCTION: In 2011, Medicare began offering annual preventive care visits (annual wellness visit; AWV) to beneficiaries at no charge. Providing free preventive care supports primary and secondary prevention of chronic disease and may reduce ethnoracial disparities in health outcomes. OBJECTIVES: To estimate AWV utilization trends by ethnoracial group in a nationally representative sample of the Medicare population. RESEARCH DESIGN: We estimated the probability of AWV utilization using probit regression models with beneficiary-reported ethnoracial group as the primary predictor and demographics, socioeconomic indicators, and factors related to access and utilization of health care as covariates. SUBJECTS: In total, 14,687 fee-for-service Medicare beneficiaries aged 66 years or older who participated in the Medicare Current Beneficiary Survey 2011-2013. MEASURES: AWV utilization was identified using procedure codes. RESULTS: Overall AWV utilization increased from 8.1% (2011) to 13.4% (2013). In 2011, utilization was highest in non-Hispanic white (8.5%) and lowest in non-Hispanic black (4.5%) beneficiaries. Utilization increased the most in non-Hispanic black beneficiaries, to 15.4% in 2013. Significant differences in AWV utilization by non-Hispanic black and Hispanic/Latino beneficiaries were found in unadjusted models, but did not persist after controlling for income and education. Having a usual (nonemergent) place of care and a nonrural residence were strong predictors of utilization. CONCLUSIONS: Utilization of the AWV has increased modestly since its introduction, but remains low. Utilization varies by ethnoracial group, with disparities largely explained by differences in income and education. Further efforts are needed to evaluate AWV utilization and effectiveness, especially among low socioeconomic status ethnoracial minorities.

Using Social Network Analysis to Examine the Effect of Care Management Structure on Chronic Disease Management Communication Within Primary Care.

BACKGROUND: Care management and care managers are becoming increasingly prevalent in primary care medical practice as a means of improving population health and reducing unnecessary care. Care managers are often involved in chronic disease management and associated transitional care. In this study, we examined the communication regarding chronic disease care within 24 primary care practices in Michigan and Colorado. We sought to answer the following questions: Do care managers play a key role in chronic disease management in the practice? Does the prominence of the care manager's connectivity within the practice's communication network vary by the type of care management structure implemented? METHODS: Individual written surveys were given to all practice members in the participating practices. Survey questions assessed demographics as well as practice culture, quality improvement, care management activities, and communication regarding chronic disease care. Using social network analysis and other statistical methods, we analyzed the communication dynamics related to chronic disease care for each practice. RESULTS: The structure of chronic disease communication varies greatly from practice to practice. Care managers who were embedded in the practice or co-located were more likely to be in the core of the communication network than were off-site care managers. These care managers also had higher in-degree centrality, indicating that they acted as a hub for communication with team members in many other roles. DISCUSSION: Social network analysis provided a useful means of examining chronic disease communication in practice, and highlighted the central role of care managers in this communication when their role structure supported such communication. Structuring care managers as embedded team members within the practice has important implications for their role in chronic disease communication within primary care.

Benefit-Risk Assessment, Communication, and Evaluation (BRACE) throughout the life cycle of therapeutic products: overall perspective and role of the pharmacoepidemiologist.

PURPOSE: Optimizing a therapeutic product's benefit-risk profile is an on-going process throughout the product's life cycle. Different, yet related, benefit-risk assessment strategies and frameworks are being developed by various regulatory agencies, industry groups, and stakeholders. This paper summarizes current best practices and discusses the role of the pharmacoepidemiologist in these activities, taking a life-cycle approach to integrated Benefit-Risk Assessment, Communication, and Evaluation (BRACE). METHODS: A review of the medical and regulatory literature was performed for the following steps involved in therapeutic benefit-risk optimization: benefit-risk evidence generation; data integration and analysis; decision making; regulatory and policy decision making; benefit-risk communication and risk minimization; and evaluation. Feedback from International Society for Pharmacoepidemiology members was solicited on the role of the pharmacoepidemiologist. The case example of natalizumab is provided to illustrate the cyclic nature of the benefit-risk optimization process. RESULTS: No single, globally adopted benefit-risk assessment process exists. The BRACE heuristic offers a way to clarify research needs and to promote best practices in a cyclic and integrated manner and highlight the critical importance of cross-disciplinary input. Its approach focuses on the integration of BRACE activities for risk minimization and optimization of the benefit-risk profile. CONCLUSION: The activities defined in the BRACE heuristic contribute to the optimization of the benefit-risk profile of therapeutic products in the clinical world at both the patient and population health level. With interdisciplinary collaboration, pharmacoepidemiologists are well suited for bringing in methodology expertise, relevant research, and public health perspectives into the BRACE process.

Adjusting for geographic variation in observational comparative effectiveness studies: a case study of antipsychotics using state Medicaid data.

BACKGROUND: Area-level variation in treatment and outcomes may be a potential source of confounding bias in observational comparative effectiveness studies. This paper demonstrates how to use exploratory spatial data analysis (ESDA) and spatial statistical methods to investigate and control for these potential biases. The case presented compares the effectiveness of two antipsychotic treatment strategies: oral second-generation antipsychotics (SGAs) vs. long-acting paliperiodone palmitate (PP). METHODS: A new-start cohort study was conducted analyzing patient-level administrative claims data (8/1/2008-4/30/2011) from Missouri Medicaid. ESDA techniques were used to examine spatial patterns of antipsychotic prescriptions and outcomes (hospitalization and emergency department (ED) visits). Likelihood of mental health-related outcomes were compared between patients starting PP (N = 295) and oral SGAs (N = 8,626) using multilevel logistic regression models adjusting for patient composition (demographic and clinical factors) and geographic region. RESULTS: ESDA indicated significant spatial variation in antipsychotic prescription patterns and moderate variation in hospitalization and ED visits thereby indicating possible confounding by geography. In the multilevel models for this antipsychotic case example, patient composition represented a stronger source of confounding than geographic context. CONCLUSION: Because geographic variation in health care delivery is ubiquitous, it could be a comparative effectiveness research (CER) best practice to test for possible geographic confounding in observational data. Though the magnitude of the area-level geography effects were small in this case, they were still statistically significant and should therefore be examined as part of this observational CER study. More research is needed to better estimate the range of confounding due to geography across different types of observational comparative effectiveness studies and healthcare utilization outcomes.

The Reporting Recommendations Intended for Pharmaceutical Risk Minimization Evaluation Studies: Standards for Reporting of Implementation Studies Extension (RIMES-SE).

INTRODUCTION: The Reporting recommendations Intended for pharmaceutical risk Minimization Evaluation Studies (RIMES) was developed to improve the quality of reporting of risk minimization program evaluations. In light of continued inadequacies in study reporting, and high-profile program implementation failures, we updated the RIMES Checklist to incorporate additional concepts from the Standards for Reporting of Implementation studies (StaRI). METHODS: The development of the updated checklist, the RIMES-StaRI Extension (RIMES-SE), entailed developing a study protocol and drafting an initial pool of items based on a mapping of the RIMES against the StaRI checklist. A modified e-Delphi exercise was then conducted to determine the importance and understandability of items for checklist inclusion. An expert workshop and an online commentary period for additional feedback followed. RESULTS: The RIMES-SE contains 27 items. It includes two signature features of the StaRI Checklist: 1) a dual strand of items (represented in two columns) describing the risk minimization program (the 'intervention') and the corresponding implementation strategy; and 2) applicable to an array of different research methodologies. CONCLUSIONS: The RIMES-SE Statement and Checklist extends the reporting guidelines set forth in the original RIMES Checklist via inclusion of key implementation science concepts. It is intended to improve the quality and transparency of reporting of risk minimization evaluation studies so as to advance drug safety science.

Behavioral Science: Enhancing Our Approach to the Development of Effective Additional Risk Minimization Strategies.

Additional risk minimization strategies may be required to assure a positive benefit-risk balance for some therapeutic products associated with serious adverse drug reactions/risks of use, without which these products may be otherwise unavailable to patients. The goals of risk minimization strategies are often fundamentally to influence the behavior of healthcare professionals (HCPs) and/or patients and can include appropriate patient selection, provision of education and counselling, appropriate medication use, adverse drug reaction monitoring, and adoption of other elements to assure safe use, such as pregnancy prevention. Current approaches to additional risk minimization strategy development rely heavily on information provision, without full consideration of the contextual factors and multi-level influences on patient and HCP behaviors that impact adoption and long-term adherence to these interventions. Application of evidence-based behavioral science methods are urgently needed to improve the quality and effectiveness of these strategies. Evidence from the fields of adherence, health promotion, and drug utilization research underscores the value and necessity for using established behavioral science frameworks and methods if we are to achieve clinical safety goals for patients. The current paper aims to enhance additional risk minimization strategy development and effectiveness by considering how a behavioral science approach can be applied, drawing from evidence in understanding of engagement with pharmaceutical medicines as well as wider public health interventions for patients and HCPs.

Impact of electronic health records on malpractice claims in a sample of physician offices in Colorado: a retrospective cohort study.

BACKGROUND: Electronic health records (EHRs) might reduce medical liability claims and potentially justify premium credits from liability insurers, but the evidence is limited. OBJECTIVES: To evaluate the association between EHR use and medical liability claims in a population of office-based physicians, including claims that could potentially be directly prevented by features available in EHRs ("EHR-sensitive" claims). DESIGN: Retrospective cohort study of medical liability claims and analysis of claim abstracts. PARTICIPANTS: The 26 % of Colorado office-based physicians insured through COPIC Insurance Company who responded to a survey on EHR use (894 respondents out of 3,502 invitees). MAIN MEASURES: Claims incidence rate ratio (IRR); prevalence of "EHR-sensitive" claims. KEY RESULTS: 473 physicians (53 % of respondents) used an office-based EHR. After adjustment for sex, birth cohort, specialty, practice setting and use of an EHR in settings other than an office, IRR for all claims was not significantly different between EHR users and non-users (0.88, 95 % CI 0.52-1.46; p = 0.61), or for users after EHR implementation as compared to before (0.73, 95 % CI 0.41-1.29; p = 0.28). Of 1,569 claim abstracts reviewed, 3 % were judged "Plausibly EHR-sensitive," 82 % "Unlikely EHR-sensitive," and 15 % "Unable to determine." EHR-sensitive claims occurred in six out of 633 non-users and two out of 251 EHR users. Incidence rate ratios were 0.01 for both groups. CONCLUSIONS: Colorado physicians using office-based EHRs did not have significantly different rates of liability claims than non-EHR users; nor were rates different for EHR users before and after EHR implementation. The lack of significant effect may be due to a low prevalence of EHR-sensitive claims. Further research on EHR use and medical liability across a larger population of physicians is warranted.

Variability in spinal surgery outcomes among children's hospitals in the United States.

BACKGROUND: We performed a retrospective cohort study of 7637 spinal fusion surgical cases from 2004 to 2006 at 38 children's hospitals participating in the Pediatric Health Information System database to evaluate the variability of in-hospital outcomes by patient factors and between facilities in children who underwent spinal surgery. METHODS: Outcomes were stratified by whether children did or did not have neurological impairment. Multilevel multivariate logistic regression models were used to determine patient and hospital factors associated with in-hospital infections, surgical complications, and length of stay (LOS)≥10 days. RESULTS: Neurologically impaired (NI) children (N=2117 out of 7637) represented 28% of the cases. The interhospital interquartile range of LOS for NI children was 6 to 8 days (median 7 d) and for non-neurologically impaired (NNI) children was 5 to 6 days (median 5 d). Children with NI had roughly 6 times higher rates of in-hospital infection and 3 times higher complication rates: major interhospital variation was seen for both of these outcomes. Hospital rates of infection ranged from 0% to 27% (median 10%) for NI and from 0% to 14% (median 2%) for NNI children. Complication rates ranged from 0% to 89% (median 33%) for NI and from 3% to 68% (median 9%) for NNI children. The following factors were associated with a LOS≥10 days: in-hospital infection (P<0.0001), surgical complication (P<0.0001), and anterior/posterior versus posterior-only surgery (P<0.0001). Hospital case volume was not associated with infection, surgical complication, or LOS≥10 days. CONCLUSIONS: Substantial variation exists in reported outcomes for children undergoing spinal surgery in children's hospitals within the United States. Further study is needed to characterize hospital-level factors related to surgical outcome to direct future quality improvement.

Evolution of Cross-Sectional Survey Protocol Quality Over Time: A Case Series of Index U.S. REMS Knowledge Survey Protocols (2007-2020).

INTRODUCTION: Surveys are commonly used to assess effectiveness of FDA-required risk evaluation and mitigation strategies (REMS) for drugs and biologics in the United States. OBJECTIVE: The aim of this study was to assess the scientific rigor of REMS knowledge survey protocols submitted to FDA and compare protocols before and after FDA's 2012 public workshop and 2019 draft guidance. METHOD: A content analysis of index survey protocols submitted to FDA (2007-2020) for single-product REMS with elements to assure safe use (39 programs, 78 protocols) was conducted. Each protocol was scored against 52 core essential elements (CEE), abstracted from FDA's guidance and grouped into six domains: study objective (n = 5), study design (n = 18), survey instrument (n = 9), participant recruitment (n = 7), survey administration (n = 9), and statistical analysis plan (n = 4). Scores were collected by time periods: (A) Oct 2007 to Jul 2012; (B) Aug 2012 to Feb 2019; (C) Mar 2019 to Dec 2020; and compared using logistic generalized linear mixed models adjusting for domain, survey population, vendor, program, and protocol. RESULTS: There were 30 (38.5%), 40 (51.3%), and 8 (10.3%) protocols submitted in time period A, B, and C, respectively. Adjusted marginal means of elements present (on the probability scale) by time period were 0.5816 (SE = 0.0242), 0.6429 (SE = 0.0229), and 0.7543 (SE = 0.0394). The likelihood of missing a CEE declined over time (adjusted p-value = 0.0094, time period A vs C). The statistical analysis plan domain had the most improvement; study design remained the weakest domain with the scientific justification CEE particularly underrepresented. CONCLUSION: The rigor of REMS knowledge survey protocols improved over time consistent with FDA's efforts to advance regulatory science, but gaps remain.

Applying Customer Discovery Method to a Chronic Disease Self-Management Mobile App: Qualitative Study.

BACKGROUND: A significant health challenge is evident in the United States, with 6 in 10 adults having a chronic disease and 4 in 10 adults having 2 or more. Chronic disease self-management aims to prevent or delay disease progression and disability and reduce mortality risk. The evidence to support the use of information technology tools, including mobile apps, web-based portals, and web-based educational interventions, that support disease self-management and improve clinical outcomes is growing. Customer discovery and value proposition design methodology is a form of stakeholder engagement and is based on marketing and lean start-up business methods. As applied in health care, customer discovery and value proposition methodology can be used to understand the clinical problem and articulate the product's hypothesized unique value proposition relative to alternative options that are available to end users. OBJECTIVE: This study aims to describe the experience and findings of academic researchers applying the customer discovery and value proposition methodology to identify stakeholders, needs, adaptability, and sustainability of a chronic disease self-management mobile app (CDapp). The motivation of the work is to make mobile health app interventions accessible and acceptable for all segments of patients' chronic diseases. METHODS: Data were obtained through key informant interviews and analyzed using rapid qualitative analysis techniques. The value proposition framework was used to build the interview guide. The aim was to identify the needs, challenges (pains), and potential benefits (gains) of the CDapp for our stakeholders. RESULTS: Our results showed that the primary consumers (end users) of a CDapp were the patients. The app adopters (decision makers) can be medical center leaders including population health department managers or insurance providers, while the consumer adoption influencers (influencers or saboteurs) are clinicians and patient caregivers. We developed an ecosystem map to visualize the clinical practice workflow and how an app for chronic disease management might integrate within an academic health care center or system. A value proposition for the identified customer segments was generated. Each stakeholder segment was working within a different framework to improve patient self-management. Patients needed help to adhere to self-care activities and they needed tailored health education. Health care leaders aim to improve the quality of care while reducing costs and workload. Clinicians wanted to improve patient education and care while reducing the time burden. Our results also showed that within academic medical centers, there were variations regarding patients' self-reported abilities to manage their diseases. CONCLUSIONS: Customer discovery is a useful form of stakeholder engagement when designing studies that seek to implement, adapt, and sustain an intervention. The customer discovery and value proposition methodology can be used as an alternative or complementary approach to formative research to generate valuable information in a brief period.

Adaptation of a Mobile Interactive Obesity Treatment Approach for Early Severe Mental Illness: Protocol for a Mixed Methods Implementation and Pilot Randomized Controlled Trial.

BACKGROUND: Obesity is common in individuals with severe mental illness (SMI), contributing to a significantly shortened lifespan when compared to the general population. Available weight loss treatments have attenuated efficacy in this population, underscoring the importance of prevention and early intervention. OBJECTIVE: Here, we describe a type 1 hybrid study design for adapting and pilot-testing an existing mobile health intervention for obesity prevention in individuals with early SMI and Class I or early-stage obesity, defined as a BMI of 30-35. METHODS: An existing, evidence-based interactive obesity treatment approach using low-cost, semiautomated SMS text messaging was selected for adaptation. Community mental health clinics and Clubhouse settings in Eastern Missouri and South Florida were identified to participate. This study has the following 3 aims. First, using the Enhanced Framework for Reporting Adaptations and Modifications to Evidence-based interventions, contextual aspects of the clinical and digital treatment environments are identified for adaptation, considering 5 main stakeholder groups (clinical administrators, prescribing clinicians, case managers, nurses, and patients). Following a 2-week trial of unadapted SMS text messaging, Innovation Corps methods are used to discover needed intervention adaptations by stakeholder group and clinical setting. Second, adaptations to digital functionality and intervention content will be made based on themes identified in aim 1, followed by rapid usability testing with key stakeholders. A process for iterative treatment adaptation will be developed for making unplanned modifications during the aim 3 implementation pilot study. Individuals working in partner community mental health clinics and Clubhouse settings will be trained in intervention delivery. Third, in a randomized pilot and feasibility trial, adults with 5 years or less of treatment for an SMI diagnosis will be randomized 2:1 to 6 months of an adapted interactive obesity treatment approach or to an attentional control condition, followed by a 3-month extension phase of SMS text messages only. Changes in weight, BMI, and behavioral outcomes, as well as implementation challenges, will be evaluated at 6 and 9 months. RESULTS: Institutional review board approval for aims 1 and 2 was granted on August 12, 2018, with 72 focus group participants enrolled; institutional review board approval for aim 3 was granted on May 6, 2020. To date, 52 participants have been enrolled in the study protocol. CONCLUSIONS: In this type 1 hybrid study design, we apply an evidence-based treatment adaptation framework to plan, adapt, and feasibility test a mobile health intervention in real-world treatment settings. Resting at the intersection of community mental health treatment and physical health promotion, this study aims to advance the use of simple technology for obesity prevention in individuals with early-stage mental illness. TRIAL REGISTRATION: ClinicalTrials.gov NCT03980743; https://clinicaltrials.gov/ct2/show/NCT03980743. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/42114.

The Evolve to Next-Gen ACT Network: An evolving open-access, real-world data resource primed for real-world evidence research across the Clinical and Translational Science Award Consortium.

The ACT Network was funded by NIH to provide investigators from across the Clinical and Translational Science Award (CTSA) Consortium the ability to directly query national federated electronic health record (EHR) data for cohort discovery and feasibility assessment of multi-site studies. NIH refunded the program for expanded research application to become "Evolve to Next-Gen ACT" (ENACT). In parallel, the US Food and Drug Administration has been evaluating the use of real-world data (RWD), including EHR data, as sources of real-world evidence (RWE) for its regulatory decisions involving drug and biological products. Using insights from implementation science, six lessons learned from ACT for developing and sustaining RWD/RWE infrastructures and networks across the CTSA Consortium are presented in order to inform ENACT's development from the outset. Lessons include intentional institutional relationship management, end-user engagement, beta-testing, and customer-driven adaptation. The ENACT team is also conducting customer discovery interviews with CTSA hub and investigators using Innovation-Corps@NCATS (I-Corps™) methodology for biomedical entrepreneurs to uncover unmet RWD needs. Possible ENACT value proposition hypotheses are presented by stage of research. Developing evidence about methods for sustaining academically derived data infrastructures and support can advance the science of translation and support our nation's RWD/RWE research capacity.