Building to learn: Information technology innovations to enable rapid pragmatic evaluation in a learning health system.

Background: Learning health systems (LHSs) iteratively generate evidence that can be implemented into practice to improve care and produce generalizable knowledge. Pragmatic clinical trials fit well within LHSs as they combine real-world data and experiences with a degree of methodological rigor which supports generalizability. Objectives: We established a pragmatic clinical trial unit ("RapidEval") to support the development of an LHS. To further advance the field of LHS, we sought to further characterize the role of health information technology (HIT), including innovative solutions and challenges that occur, to improve LHS project delivery. Methods: During the period from December 2021 to February 2023, eight projects were selected out of 51 applications to the RapidEval program, of which five were implemented, one is currently in pilot testing, and two are in planning. We evaluated pre-study planning, implementation, analysis, and study closure approaches across all RapidEval initiatives to summarize approaches across studies and identify key innovations and learnings by gathering data from study investigators, quality staff, and IT staff, as well as RapidEval staff and leadership. Implementation Results: Implementation approaches spanned a range of HIT capabilities including interruptive alerts, clinical decision support integrated into order systems, patient navigators, embedded micro-education, targeted outpatient hand-off documentation, and patient communication. Study approaches include pre-post with time-concordant controls (1), randomized stepped-wedge (1), cluster randomized across providers (1) and location (3), and simple patient level randomization (2). Conclusions: Study selection, design, deployment, data collection, and analysis required close collaboration between data analysts, informaticists, and the RapidEval team.

Implementing Telestroke in the Inpatient Setting: Identifying Factors for Success.

BACKGROUND: Inpatient telestroke programs have emerged as a solution to provide timely stroke care in underserved areas, but their successful implementation and factors influencing their effectiveness remain underexplored. This study aimed to qualitatively evaluate the perspectives of inpatient clinicians located at spoke hospitals participating in a newly established inpatient telestroke program to identify implementation barriers and facilitators. METHODS: This was a formative evaluation relying on semistructured qualitative interviews with 16 inpatient providers (physicians and nurse practitioners) at 5 spoke sites of a hub-and-spoke inpatient telestroke program. The Integrated-Promoting Action on Research Implementation in Health Services framework guided data analysis, focusing on the innovation, recipients, context, and facilitation aspects of implementation. Interviews were transcribed and coded using thematic analysis. RESULTS: Fifteen themes were identified in the data and mapped to the Integrated-Promoting Action on Research Implementation in Health Services framework. Themes related to the innovation (the telestroke program) included easy access to stroke specialists, the benefits of limiting patient transfers, concerns about duplicating tests, and challenges of timing inpatient telestroke visits and notes to align with discharge workflow. Themes pertaining to recipients (care team members and patients) were communication gaps between teams, concern about the supervision of inpatient telestroke advanced practice providers and challenges with nurse empowerment. With regard to the context (hospital and system factors), providers highlighted familiarity with telehealth technologies as a facilitator to implementing inpatient telestroke, yet highlighted resource limitations in smaller facilities. Facilitation (program implementation) was recognized as crucial for education, standardization, and buy-in. CONCLUSIONS: Understanding barriers and facilitators to implementation is crucial to determining where programmatic changes may need to be made to ensure the success and sustainment of inpatient telestroke services.

Evaluation of a Clinical Decision Support Tool to Guide Adoption of the American Heart Association Telemetry Monitoring Practice Standards.

BACKGROUND: The objectives of this study were to (1) evaluate telemetry use pre- and postimplementation of clinical decision support tools to support American Heart Association practice standards for telemetry monitoring and (2) understand the factors that may contribute to variation of telemetry monitoring in practice. METHODS AND RESULTS: First, we captured overall variability in telemetry use pre- and postimplementation of the clinical decision support intervention. We then conducted semistructured interviews with telemetry-ordering providers to identify key barriers and facilitators to adoption. During the study period, 399 physicians met criteria for inclusion and were divided into excessive and nonexcessive orderers. Distribution of telemetry use was bimodal. Among nonexcessive users, 24.4% of patient days were with telemetry compared with 51.6% among excessive users. On average, both excessive (6.1% reduction) and nonexcessive users (2.8% reduction) decreased telemetry use postimplementation, and these reductions were sustained over a 16-month period. Sixteen interviews were conducted. Physicians believed that the tool was successful because it caused them to more closely consider if telemetry was indicated for each patient. Physicians also voiced frustration with interruptions to their workflow, and some noted that they commonly use telemetry outside of practice standards to monitor patients who were acutely but not critically ill. CONCLUSIONS: Embedding telemetry practice standards into the electronic health record in the form of clinical decision support is effective at reducing excess telemetry use. Although the intervention was well received, there are persistent barriers, such as preexisting views on telemetry and existing workflow habits, that may inhibit higher adoption of standards.

Characteristics and co-morbidities associated with hospital discharges for opioid and methamphetamine co-use, United States 2016-2019.

Introduction: The US overdose crisis is increasingly characterized by opioid and methamphetamine co-use. Hospitalization is an important opportunity to engage patients in substance use treatment. Understanding characteristics of co-use-related hospital stays can inform the development of services to better support this growing patient population. Methods: We used 2016-2019 National Inpatient Sample data to conduct a cross sectional analysis of hospitalizations involving use of opioids, methamphetamine, or both. We used bivariate analysis to compare patient demographics. We then used multinomial logistic regressions to compare the proportion of hospital stays which indicated co-morbid diagnosis. To account for correlated data, we used generalized linear models to compare outcomes in hospital mortality, patient-directed discharge, and length of stay. Results: Co-use-related stays had a higher proportion of co-morbid mental health (60.7%; 95% CI: 59.9-61.4%) and infectious diseases (41.5%; 95% CI: 40.8-42.2%), than opioid- or methamphetamine-related stays. Co-use-related stays increased between 2016 and 2019 and were associated with a higher proportion of patient directed discharge (10.7%; 95% CI: 10.4-11.0%) and longer length of stay (6.3 days; 95% CI: 6.2-6.4 days) compared to opioid (8.1%; 95% CI: 7.9-8.3% and 5.8 days; 95% CI: 5.8-5.9 days) and methamphetamine-related stays (6.5%; 95% CI: 6.3-6.6% and 5.5 days; 95% CI: 5.4-5.5 days). Conclusion: Patients discharged with co-use differ from patients with opioid or methamphetamine use alone, representing a range of challenges and opportunities. In addition to offering treatment for both substance use disorders, hospital-based services that address co-occurring conditions may better support patients with co-use through targeted and tailored approaches.

Introducing a Measure of Hospital Community Orientation.

Policymakers have long sought to encourage hospitals to assume a more collaborative role in improving community health. By urging hospitals to interact with community stakeholders, more integrative relationships may result that can better address local health issues. This study establishes a composite measure of hospital community orientation, defined as the extent to which a hospital uses community resources and knowledge in its community benefit (CB) work, based on an expansion of CB regulations that require nonprofit hospitals (NPHs) to develop strategies to address prioritized health issues. We collected data on each proposed intervention from 125 randomly selected NPHs over three reporting periods. Confirmatory factor analysis was used to assess how well a single-factor model approximated community orientation. We conclude that using hospital community orientation measurement is a useful metric to assess the effects of expanded CB regulations, as well as to determine how NPHs have interacted with communities over time.

Providers' Experiences and Perspectives in Treating Patients With Co-Occurring Opioid and Stimulant Use Disorders in the Hospital.

BACKGROUND: The overdose crisis is increasingly characterized by opioid and stimulant co-use. Despite effective pharmacologic treatment for both opioid use disorder (OUD) and contingency management for stimulant use disorders, most individuals with these co-occurring conditions are not engaged in treatment. Hospitalization is an important opportunity to engage patients and initiate treatment, however existing hospital addiction care is not tailored for patients with co-use and may not meet the needs of this population. METHODS: Semi-structured interviews were conducted with hospital providers about their experiences and perspectives treating patients with opioid and stimulant co-use. We used directed content analysis to identify common experiences and opportunities to improve hospital-based treatment for patients with co-use. RESULTS: From qualitative interviews with 20 providers, we identified 4 themes describing how co-use complicated hospital-based substance use treatment: (1) patients' unstable circumstances impacting the treatment plan, (2) co-occurring withdrawals are difficult to identify and treat, (3) providers holding more stigmatizing views of patients with co-use, and (4) stimulant use is often "ignored" in the treatment plans. Participants also described a range of potential opportunities to improve hospital-based treatment of co-use that fall into 3 categories: (1) provider practice changes, (2) healthcare system changes, and (3) development and validation of clinical tools and treatment approaches. CONCLUSIONS: We identified unique challenges providing hospital addiction medicine care to patients who use both opioids and stimulants. These findings inform the development, implementation, and testing of hospital-based interventions for patients with co-use.

Engaging Public Health Alumni in the Tracking of Career Trends: Results From a Large-Scale Experiment on Survey Fielding Mode.

OBJECTIVE: We sought to understand the relative impact of fielding mode on response rate among public health alumni. METHODS: As part of the 2021 Career Trends Survey of alumni from the University of Minnesota School of Public Health, we designed a fielding mode experiment to ascertain whether a paper survey, a postcard with a custom survey link ("postcard push-to-web"), a mobile telephone call or text (mobile), or an email invitation would garner the highest response rates. Invitations were randomly assigned from available contact information. RESULTS: Of 8531 alumni invited, 1671 alumni (19.6%) completed the survey. Among the initial fielding modes, the paper survey had the highest response rate (28%), followed by mobile (19%), email (10%), and postcard push-to-web (10%). More robust recent engagement with alumni relations, paper survey invitation or mode switch, and recent graduation were all significantly associated with a higher likelihood of response. CONCLUSIONS: Paper and mobile invitations had the highest response rates to our survey among public health alumni. Findings from this fielding mode experiment are relevant to schools and programs of public health seeking to capture similar information among their alumni, especially given current trends in investment in the public health workforce.

Demographic, social, and behavioral correlates of SARS-CoV-2 seropositivity in a representative, population-based study of Minnesota residents.

BACKGROUND: Monitoring COVID-19 infection risk in the general population is a public health priority. Few studies have measured seropositivity using representative, probability samples. The present study measured seropositivity in a representative population of Minnesota residents prior to vaccines and assess the characteristics, behaviors, and beliefs of the population at the outset of the pandemic and their association with subsequent infection. METHODS: Participants in the Minnesota COVID-19 Antibody Study (MCAS) were recruited from residents of Minnesota who participated in the COVID-19 Household Impact Survey (CIS), a population-based survey that collected data on physical health, mental health, and economic security information between April 20 and June 8 of 2020. This was followed by collection of antibody test results between December 29, 2020 and February 26, 2021. Demographic, behavioral, and attitudinal exposures were assessed for association with the outcome of interest, SARS-CoV-2 seroprevalence, using univariate and multivariate logistic regression. RESULTS: Of the 907 potential participants from the CIS, 585 respondents then consented to participate in the antibody testing (64.4% consent rate). Of these, results from 537 test kits were included in the final analytic sample, and 51 participants (9.5%) were seropositive. The overall weighted seroprevalence was calculated to be 11.81% (95% CI, 7.30%-16.32%) at of the time of test collection. In adjusted multivariate logistic regression models, significant associations between seroprevalence and the following were observed; being from 23-64 and 65+ age groups were both associated with higher odds of COVID-19 seropositivity compared to the 18-22 age group (17.8 [1.2-260.1] and 24.7 [1.5-404.4] respectively). When compared to a less than $30k annual income reference group, all higher income groups had significantly lower odds of seropositivity. Reporting practicing a number of 10 (median reported value in sample) or more of 19 potential COVID-19 mitigation factors (e.g. handwashing and mask wearing) was associated with lower odds of seropositivity (0.4 [0.1-0.99]) Finally, the presence of at least one household member in the age range of 6 to 17 years old was associated with higher odds of seropositivity (8.3 [1.2-57.0]). CONCLUSIONS: The adjusted odds ratio of SARS-CoV-2 seroprevalence was significantly positively associated with increasing age and having household member(s) in the 6-17 year age group, while increasing income levels and a mitigation score at or above the median were shown to be significantly protective factors.

COVID-19 Vaccine Hesitancy in a Population-Based Study of Minnesota Residents.

COVID-19 continues to be a public health concern in the United States. Although safe and effective vaccines have been developed, a significant proportion of the US population has not received a COVID-19 vaccine. This cross-sectional study aimed to describe the demographics and behaviors of Minnesota adults who have not received the primary series of the COVID-19 vaccine, or the booster shot using data from the Minnesota COVID-19 Antibody Study (MCAS) collected through a population-based sample between September and December 2021. Data were collected using a web-based survey sent to individuals that responded to a similar survey in 2020 and their adult household members. The sample was 51% female and 86% White/Non-Hispanic. A total of 9% of vaccine-eligible participants had not received the primary series and 23% of those eligible to receive a booster had not received it. Older age, higher education, better self-reported health, $75,000 to $100,000 annual household income, mask-wearing, and social distancing were associated with lower odds of hesitancy. Gender, race, and previous COVID-19 infection were not associated with hesitancy. The most frequently reported reason for not receiving a COVID-19 vaccination was safety concerns. Mask-wearing and being age 65 or older were the only strong predictors of lower odds of vaccine hesitancy for both the primary series and booster analyses.

Development and use of a novel tool for assessing and improving researcher embeddedness in learning health systems and applied system improvements.

This paper outlines the development, deployment and use, and testing of a tool for measuring and improving healthcare researcher embeddedness - i.e., being connected to and engaged with key leverage points and stakeholders in a health system. Despite the widely acknowledged importance of embeddedness for learning health systems and late-stage translational research, we were not aware of useful tools for addressing and improving embeddedness in scholar training programs. We developed the MN-LHS Embeddedness Tool covering connections to committees, working groups, leadership, and other points of contact across four domains: patients and caregivers; local practice (e.g., operations and workflows); local institutional research (e.g., research committees and agenda- or initiative-setting groups); and national (strategic connections within professional groups, conferences, etc.). We used qualitative patterns and narrative findings from 11 learning health system training program scholars to explore variation in scholar trajectories and the embeddedness tool's usefulness in scholar professional development. Tool characteristics showed moderate evidence of construct validity; secondarily, we found significant differences in embeddedness, as a score, from baseline through program completion. The tool has demonstrated simple, practical utility in making embeddedness an explicit (rather than hidden) part of applied and learning health system researcher training, alongside emerging evidence for validity.

Training the next generation of learning health system scientists.

Introduction: The learning health system (LHS) aligns science, informatics, incentives, stakeholders, and culture for continuous improvement and innovation. The Agency for Healthcare Research and Quality and the Patient-Centered Outcomes Research Institute designed a K12 initiative to grow the number of LHS scientists. We describe approaches developed by 11 funded centers of excellence (COEs) to promote partnerships between scholars and health system leaders and to provide mentored research training. Methods: Since 2018, the COEs have enlisted faculty, secured institutional resources, partnered with health systems, developed and implemented curricula, recruited scholars, and provided mentored training. Program directors for each COE provided descriptive data on program context, scholar characteristics, stakeholder engagement, scholar experiences with health system partnerships, roles following program completion, and key training challenges. Results: To date, the 11 COEs have partnered with health systems to train 110 scholars. Nine (82%) programs partner with a Veterans Affairs health system and 9 (82%) partner with safety net providers. Clinically trained scholars (n = 87; 79%) include 70 physicians and 17 scholars in other clinical disciplines. Non-clinicians (n = 29; 26%) represent diverse fields, dominated by population health sciences. Stakeholder engagement helps scholars understand health system and patient/family needs and priorities, enabling opportunities to conduct embedded research, improve outcomes, and grow skills in translating research methods and findings into practice. Challenges include supporting scholars through roadblocks that threaten to derail projects during their limited program time, ranging from delays in access to data to COVID-19-related impediments and shifts in organizational priorities. Conclusions: Four years into this novel training program, there is evidence of scholars' accomplishments, both in traditional academic terms and in terms of moving along career trajectories that hold the potential to lead and accelerate transformational health system change. Future LHS training efforts should focus on sustainability, including organizational support for scholar activities.

Changes in opioid marketing practices after release of the CDC guidelines.

OBJECTIVES: After the release of the CDC guidelines in March 2016, the rate of opioid prescriptions decreased. How or whether pharmaceutical companies changed their opioid marketing practices post release of the CDC guidelines is unknown. Our objectives were to (1) evaluate whether the release of the guidelines was associated with changes in total monthly marketing spending received per physician, monthly marketing encounter frequency per physician, and spending per encounter during opioid marketing; and (2) evaluate whether such changes in marketing differed between specialist physicians and primary care physicians (PCPs) and between urban and rural primary care service areas (PCSAs). STUDY DESIGN: Retrospective observational cross-sectional study using opioid marketing spending data from the CMS Open Payments database between August 2013 and December 2017. METHODS: Single-group and multiple-group interrupted time series analyses were used to evaluate differences in the immediate changes in level and trend over time in opioid marketing practices post release of the CDC guidelines. RESULTS: Post release of the CDC guidelines, the monthly number of marketing encounters per physician and total monthly amount received per physician decreased. However, the amount spent at each marketing encounter increased. The release of the CDC guidelines was associated with an immediate increase in level of opioid marketing spending per encounter by $0.59 (95% CI, $0.51-$0.68; P < .001) and an over-time increase in rate of spending per encounter of $0.04 per month (95% CI, $0.03-$0.05; P < .001). These changes differed between specialists and PCPs and between urban and rural PCSAs. CONCLUSIONS: It is important to continue ongoing education for physicians on changes in pharmaceutical opioid marketing practices.

COVID-19 vaccine confidence and reasons for vaccination among health care workers and household members.

BACKGROUND: The majority of healthcare workers (HCW) in the US report being fully vaccinated against COVID-19, yet little is known about vaccine decision-making for their household members, including children. METHODS: Cross-sectional survey July-August 2021 of HCW and their household members in Minnesota. RESULTS: 94 % of eligible participants were vaccinated with the most common reasons being wanting to protect oneself, family and loved ones. Safety concerns were the most commonly reported reasons for not being vaccinated; a significantly higher proportion of unvaccinated compared to vaccinated HCW (58 % vs 12 %, p = 0.0035) and household adults (25 % vs 5 %, p = 0.03) reported prior SARS-CoV-2 infection. Nearly half of unvaccinated adults and two-thirds of unvaccinated children would be vaccinated if a vaccine mandate were in place. CONCLUSIONS: Despite high COVID-19 vaccine acceptance among HCWs, more research is required to identify and address the needs and concerns of healthcare workers who decline COVID-19 vaccination despite availability.

Identification of Natural SARS-CoV-2 Infection in Seroprevalence Studies Among Vaccinated Populations.

Most SARS-CoV-2 antibody assays cannot distinguish between antibodies that developed after natural infection and those that developed after vaccination. We assessed the accuracy of a nucleocapsid-containing assay in identifying natural infection among vaccinated individuals. A longitudinal cohort composed of health care workers in the Minneapolis/St. Paul area was enrolled. Two rounds of seroprevalence studies separated by 1 month were conducted from November 2020 to January 2021 among 81 participants. Capillary blood from rounds 1 and 2 was tested for IgG antibodies against spike proteins by enzyme-linked immunosorbent assay (spike-only assay). During round 2, IgGs reactive to SARS-CoV-2 nucleocapsid protein (nucleocapsid-containing assay) were assessed. Vaccination status at round 2 was determined by self-report. Area under the curve was computed to determine the discriminatory ability of the nucleocapsid-containing assay for identification of recent infection. Participants had a mean age of 40 years (range, 23 to 66 years); 83% were female. Round 1 seroprevalence was 9.5%. Before round 2 testing, 46% reported vaccination. Among those not recently infected, in comparing vaccinated vs unvaccinated individuals, elevated levels of spike 1 (P<.001) and spike 2 (P=.01) were observed, whereas nucleocapsid levels were not statistically significantly different (P=.90). Among all participants, nucleocapsid response predicted recent infection with an area under the curve of 0.93 (95% CI, 0.88 to 0.99). Among individuals vaccinated more than 10 days before antibody testing, the specificity of the nucleocapsid-containing assay was 92%, whereas the specificity of the spike-only assay was 0%. An IgG assay identifying reactivity to nucleocapsid protein is an accurate predictor of natural infection among a partially vaccinated population, whereas a spike-only assay performed poorly.

SARS-CoV-2 seroprevalence among healthcare workers.

BACKGROUND: Monitoring COVID-19 infection risk among health care workers (HCWs) is a public health priority. We examined the seroprevalence of SARS-CoV-2 among HCWs following the fall infection surge in Minnesota, and before and after COVID-19 vaccination. Additionally, we assessed demographic and occupational risk factors for SARS-CoV-2 infection. METHODS: We conducted two rounds of seroprevalence testing among a cohort of HCWs: samples in round 1 were collected from 11/22/20-02/21/21 and in round 2 from 12/18/20-02/15/21. Demographic and occupational exposures assessed with logistic regression were age, sex, healthcare role and setting, and number of children in the household. The primary outcome was SARS-CoV-2 IgG seropositivity. A secondary outcome, SARS-CoV-2 infection, included both seropositivity and self-reported SARS-CoV-2 test positivity. RESULTS: In total, 459 HCWs were tested. 43/454 (9.47%) had a seropositive sample 1 and 75/423 (17.7%) had a seropositive sample 2. By time of sample 2 collection, 54% of participants had received at least one vaccine dose and seroprevalence was 13% among unvaccinated individuals. Relative to physicians, the odds of SARS-CoV-2 infection in other roles were increased (Nurse Practitioner: OR[95%CI] 1.93[0.57,6.53], Physician's Assistant: 1.69[0.38,7.52], Nurse: 2.33[0.94,5.78], Paramedic/EMTs: 3.86[0.78,19.0], other: 1.68[0.58,4.85]). The workplace setting was associated with SARS-CoV-2 infection (p = 0.04). SARS-CoV-2 seroprevalence among HCWs reporting duties in the ICU vs. those working in an ambulatory clinic was elevated: OR[95%CI] 2.17[1.01,4.68]. CONCLUSIONS: SARS-CoV-2 seroprevalence in HCW increased during our study period which was consistent with community infection rates. HCW role and setting-particularly working in the ICU-is associated with higher risk for SARS-CoV-2 infection.

SARS-CoV-2 Seroprevalence Survey in Grocery Store Workers-Minnesota, 2020-2021.

Grocery workers were essential to the workforce and exempt from lockdown requirements as per Minnesota Executive Order 20-20. The risk of COVID-19 transmission in grocery settings is not well documented. This study aimed to determine which factors influenced seropositivity among grocery workers. We conducted a cross-sectional study of Minnesota grocery workers aged 18 and older using a convenience sample. Participants were recruited using a flyer disseminated electronically via e-mail, social media, and newspaper advertising. Participants were directed to an electronic survey and were asked to self-collect capillary blood for IgG antibody testing. Data were analyzed using logistic regression and adjusted for urbanicity, which confounded the relationship between number of job responsibilities in a store and seropositivity. Of 861 Minnesota grocery workers surveyed, 706 (82%) were tested as part of this study, of which 56 (7.9%) tested positive for IgG antibodies. Participants aged 65-74 years had the highest percent positivity. Having multiple job responsibilities in a store was significantly associated with seropositivity in our adjusted model (OR: 1.14 95% CI: 1.01-1.27). Workplace factors influenced seropositivity among Minnesota grocery workers. Future research will examine other potential factors (e.g., in-store preventive measures and access to PPE) that may contribute to increased seropositivity.

Impact of different cover letter content and incentives on non-response bias in a sample of Veterans applying for Department of Veterans Affairs disability benefits: a randomized, 3X2X2 factorial trial.

BACKGROUND: Non-random non-response bias in surveys requires time-consuming, complicated, post-survey analyses. Our goal was to see if modifying cover letter information would prevent non-random non-response bias altogether. Our secondary goal tested whether larger incentives would reduce non-response bias. METHODS: A mailed, survey of 480 male and 480 female, nationally representative, Operations Enduring Freedom, Iraqi Freedom, or New Dawn (OEF/OIF/OND) Veterans applying for Department of Veterans Affairs (VA) disability benefits for posttraumatic stress disorder (PTSD). Cover letters conveyed different information about the survey's topics (combat, unwanted sexual attention, or lifetime and military experiences), how Veterans' names had been selected (list of OEF/OIF/OND Veterans or list of Veterans applying for disability benefits), and what incentive Veterans would receive ($20 or $40). The main outcome, non-response bias, measured differences between survey respondents' and sampling frame's characteristics on 8 administrative variables, including Veterans' receipt of VA disability benefits and exposure to combat or military sexual trauma. Analysis was intention to treat. We used ANOVA for factorial block-design, logistic, mixed-models to assess bias and multiple imputation and expectation-maximization algorithms to assess potential missing mechanisms (missing completely at random, missing at random, or not random) of two self-reported variables: combat and military sexual assault. RESULTS: Regardless of intervention, men with any VA disability benefits, women with PTSD disability benefits, and women with combat exposure were over-represented among respondents. Interventions explained 0.0 to 31.2% of men's variance and 0.6 to 30.5% of women's variance in combat non-response bias and 10.2 to 43.0% of men's variance and 0.4 to 31.9% of women's variance in military sexual trauma non-response bias. Non-random assumptions showed that men's self-reported combat exposure was overestimated by 19.0 to 28.8 percentage points and their self-reported military sexual assault exposure was underestimated by 14.2 to 28.4 percentage points compared to random missingness assumptions. Women's self-reported combat exposure was overestimated by 8.6 to 10.6 percentage points and military sexual assault exposure, by 1.2 to 6.9 percentage points. CONCLUSIONS: Our interventions reduced bias in some characteristics, leaving others unaffected or exacerbated. Regardless of topic, researchers are urged to present estimates that include all three assumptions of missingness.

'Trying to fly the plane while we were building it'. Applying a learning health systems approach to evaluate early-stage barriers and facilitators to implementing primary care transformation: a qualitative study.

OBJECTIVE: A learning health system (LHS) uses data to generate evidence and answer questions required to continually improve system performance and patient care. Given the complexities of practice transformation, an area where LHS is particularly important is the study of primary care transformation (PCT) as PCT generates several practice-level questions that require study where the findings can be readily implemented. In May 2019, a large integrated health delivery system in Minnesota began implementation of a population management PCT in two of its 40 primary care clinics. In this model of care, patients are grouped into one of five service bundles based on their complexity of care; patient appointment lengths and services provided are then tailored to each service bundle. The objective of this study was to examine the use of a LHS in PCT by utilising the Consolidated Framework for Implementation Research (CFIR) to categorise implementation lessons from the initial two PCT clinics to inform further implementation of the PCT within the health system. DESIGN: This was a formative evaluation in which semistructured qualitative interviews were carried out. Observational field notes were also taken. Inductive coding of the data was performed and resultant codes were mapped to the CFIR. SETTING: Two suburban primary care clinics in the Twin Cities, Minnesota. PARTICIPANTS: Twenty-two care team members from the first two clinics to adopt the PCT. RESULTS: Seventeen codes emerged to describe care team members' perceived implementation influences. Codes occurred in each of the five CFIR domains (intervention characteristics, outer setting, inner setting, characteristics of individuals and process), with most codes occurring in the 'inner setting' domain. CONCLUSIONS: Using an LHS approach to determine early-stage implementation influences is key to guiding further PCT implementation, understanding modifications that need to be made and additional research that needs to occur.

Effects of patient-reported outcome assessment order.

BACKGROUND: In clinical trials and clinical practice, patient-reported outcomes are almost always assessed using multiple patient-reported outcome measures at the same time. This raises concerns about whether patient responses are affected by the order in which the patient-reported outcome measures are administered. METHODS: This questionnaire-based study of order effects included adult cancer patients from five cancer centers. Patients were randomly assigned to complete questionnaires via paper booklets, interactive voice response system, or tablet web survey. Linear Analogue Self-Assessment, Patient-Reported Outcomes Version of the Common Terminology Criteria for Adverse Events, and Patient-Reported Outcomes Measurement Information System assessment tools were each used to measure general health, physical function, social function, emotional distress/anxiety, emotional distress/depression, fatigue, sleep, and pain. The order in which the three tools, and domains within tools, were presented to patients was randomized. Rates of missing data, scale scores, and Cronbach's alpha coefficients were compared by the order in which they were assessed. Analyses included Cochran-Armitage trend tests and mixed models adjusted for performance score, age, sex, cancer type, and curative intent. RESULTS: A total of 1830 patients provided baseline patient-reported outcome assessments. There were no significant trends in rates of missing values by whether a scale was assessed earlier or later. The largest order effect for scale scores was due to a large mean score at one assessment time point. The largest difference in Cronbach's alpha between the versions for the Patient-Reported Outcomes Measurement Information System scales was 0.106. CONCLUSION: The well-being of a cancer patient has many different aspects such as pain, fatigue, depression, and anxiety. These are assessed using a variety of surveys often collected at the same time. This study shows that the order in which the different aspects are collected from the patient is not important.

Association of opioid fills with centers for disease control and prevention opioid guidelines and payer coverage policies: physician, insurance and geographic factors.

Background The Centers for Disease Control and Prevention (CDC) issued guidelines and certain healthcare payers have made pharmacy coverage changes (PCC) focusing on regulating prescription opioids. Aim We evaluated differences in the rate of first-time opioid fills at doses ≥ 50 morphine milligram equivalents (MME)/day and first-time opioid fills with benzodiazepine fill overlap following the CDC guidelines and following a PCC between provider types, geographic locations, and insurance types. Method We used OptumLabs® Data Warehouse claims data between 2014 and 2018. Subjects were opioid naïve non-cancer care patients, 18 years and older who had an identified chronic pain condition ICD diagnosis within 2 weeks prior to their first-time opioid fill. We used multiple treatment period segmented regression analysis with interaction terms to test the differences between primary care providers (PCPs) and specialist providers (SPs), urban and rural primary care service areas (PCSAs), and Medicare Advantage (MA) and commercially insured patients (CIPs) in their first-time opioid fill patterns. Results Prescribing first-time opioid fills at doses ≥ 50MME/day declined following the CDC guidelines and PCC, the decline was greater among SPs than PCPs and in rural PCSAs than urban PCSAs. Also, following the CDC guidelines, the decline was greater among MA patients however following the PCC the decline was greater among CIPs. There were no differences in rate of first-time opioid fill with benzodiazepine overlap between groups. Conclusion Responses to the CDC opioid guidelines and a PCC differed between PCPs and SPs, urban and rural PCSAs, and when prescribing to MA and CIPs. Understanding these differences is important to help inform future guidelines.