Clinician perspectives on how situational context and augmented intelligence design features impact perceived usefulness of sepsis prediction scores embedded within a simulated electronic health record.

OBJECTIVE: Obtain clinicians' perspectives on early warning scores (EWS) use within context of clinical cases. MATERIAL AND METHODS: We developed cases mimicking sepsis situations. De-identified data, synthesized physician notes, and EWS representing deterioration risk were displayed in a simulated EHR for analysis. Twelve clinicians participated in semi-structured interviews to ascertain perspectives across four domains: (1) Familiarity with and understanding of artificial intelligence (AI), prediction models and risk scores; (2) Clinical reasoning processes; (3) Impression and response to EWS; and (4) Interface design. Transcripts were coded and analyzed using content and thematic analysis. RESULTS: Analysis revealed clinicians have experience but limited AI and prediction/risk modeling understanding. Case assessments were primarily based on clinical data. EWS went unmentioned during initial case analysis; although when prompted to comment on it, they discussed it in subsequent cases. Clinicians were unsure how to interpret or apply the EWS, and desired evidence on its derivation and validation. Design recommendations centered around EWS display in multi-patient lists for triage, and EWS trends within the patient record. Themes included a "Trust but Verify" approach to AI and early warning information, dichotomy that EWS is helpful for triage yet has disproportional signal-to-high noise ratio, and action driven by clinical judgment, not the EWS. CONCLUSIONS: Clinicians were unsure of how to apply EWS, acted on clinical data, desired score composition and validation information, and felt EWS was most useful when embedded in multi-patient views. Systems providing interactive visualization may facilitate EWS transparency and increase confidence in AI-generated information.

Genetically guided precision medicine clinical decision support tools: a systematic review.

OBJECTIVES: Patient care using genetics presents complex challenges. Clinical decision support (CDS) tools are a potential solution because they provide patient-specific risk assessments and/or recommendations at the point of care. This systematic review evaluated the literature on CDS systems which have been implemented to support genetically guided precision medicine (GPM). MATERIALS AND METHODS: A comprehensive search was conducted in MEDLINE and Embase, encompassing January 1, 2011-March 14, 2023. The review included primary English peer-reviewed research articles studying humans, focused on the use of computers to guide clinical decision-making and delivering genetically guided, patient-specific assessments, and/or recommendations to healthcare providers and/or patients. RESULTS: The search yielded 3832 unique articles. After screening, 41 articles were identified that met the inclusion criteria. Alerts and reminders were the most common form of CDS used. About 27 systems were integrated with the electronic health record; 2 of those used standards-based approaches for genomic data transfer. Three studies used a framework to analyze the implementation strategy. DISCUSSION: Findings include limited use of standards-based approaches for genomic data transfer, system evaluations that do not employ formal frameworks, and inconsistencies in the methodologies used to assess genetic CDS systems and their impact on patient outcomes. CONCLUSION: We recommend that future research on CDS system implementation for genetically GPM should focus on implementing more CDS systems, utilization of standards-based approaches, user-centered design, exploration of alternative forms of CDS interventions, and use of formal frameworks to systematically evaluate genetic CDS systems and their effects on patient care.

Vision Loss in Children from Immigrant and Nonimmigrant Households: Evidence from the National Survey of Children's Health 2018-2020.

The aim of this study was to determine whether immigrant generation is associated with caregiver-reported vision loss in children adjusting for sociodemographic characteristics. Nationally representative data from the National Survey of Children's Health (2018-2020) was used. The primary exposure was immigrant generation defined as: first (child and all reported parents were born outside the United States); second (child was born in the United States and at least one parent was born outside the United States); third or higher (all parents in the household were born in the United States). The main outcome was caregiver-reported vision loss in child. Adjusted odds ratios (aOR) and 95% confidence intervals were computed based on immigration generation. The study sample included 84,860 US children aged 3-17 years. First generation children had higher adjusted odds of caregiver-reported vision loss (aOR 2.30; 95% CI 1.21, 4.35) than third or higher generation children after adjusting for demographic characteristics and social determinants of health. For Hispanic families, first generation (aOR 2.99; 95% CI 1.34, 6.66), and second-generation children (aOR 1.70; 95% CI 1.06, 2.74) had a higher adjusted odds of vision loss compared with third or higher generation children. Even when adjusting for sociodemographic characteristics, first generation children had greater odds of vision loss, especially in Hispanic households, than third generation children. Immigration generation should be treated as an independent risk factor for vision loss for children and is a social determinant of eye health.

SCALE-UP II: protocol for a pragmatic randomised trial examining population health management interventions to increase the uptake of at-home COVID-19 testing in community health centres.

INTRODUCTION: SCALE-UP II aims to investigate the effectiveness of population health management interventions using text messaging (TM), chatbots and patient navigation (PN) in increasing the uptake of at-home COVID-19 testing among patients in historically marginalised communities, specifically, those receiving care at community health centres (CHCs). METHODS AND ANALYSIS: The trial is a multisite, randomised pragmatic clinical trial. Eligible patients are >18 years old with a primary care visit in the last 3 years at one of the participating CHCs. Demographic data will be obtained from CHC electronic health records. Patients will be randomised to one of two factorial designs based on smartphone ownership. Patients who self-report replying to a text message that they have a smartphone will be randomised in a 2×2×2 factorial fashion to receive (1) chatbot or TM; (2) PN (yes or no); and (3) repeated offers to interact with the interventions every 10 or 30 days. Participants who do not self-report as having a smartphone will be randomised in a 2×2 factorial fashion to receive (1) TM with or without PN; and (2) repeated offers every 10 or 30 days. The interventions will be sent in English or Spanish, with an option to request at-home COVID-19 test kits. The primary outcome is the proportion of participants using at-home COVID-19 tests during a 90-day follow-up. The study will evaluate the main effects and interactions among interventions, implementation outcomes and predictors and moderators of study outcomes. Statistical analyses will include logistic regression, stratified subgroup analyses and adjustment for stratification factors. ETHICS AND DISSEMINATION: The protocol was approved by the University of Utah Institutional Review Board. On completion, study data will be made available in compliance with National Institutes of Health data sharing policies. Results will be disseminated through study partners and peer-reviewed publications. TRIAL REGISTRATION NUMBER: ClinicalTrials.gov: NCT05533918 and NCT05533359.

Use of implementation mapping in the planning of a hybrid type 1 pragmatic clinical trial: the BeatPain Utah study.

BACKGROUND: Considerable disparities in chronic pain management have been identified. Persons in rural, lower income, and minoritized communities are less likely to receive evidence-based, nonpharmacologic care. Telehealth delivery of nonpharmacologic, evidence-based interventions for persons with chronic pain is a promising strategy to lessen disparities, but implementation comes with many challenges. The BeatPain Utah study is a hybrid type 1 effectiveness-implementation pragmatic clinical trial investigating telehealth strategies to provide nonpharmacologic care from physical therapists to persons with chronic back pain receiving care in ommunity health centers (CHCs). CHCs provide primary care to all persons regardless of ability to pay. This paper outlines the use of implementation mapping to develop a multifaceted implementation plan for the BeatPain study. METHODS: During a planning year for the BeatPain trial, we developed a comprehensive logic model including the five-step implementation mapping process informed by additional frameworks and theories. The five iterative implementation mapping steps were addressed in the planning year: (1) conduct needs assessments for involved groups; (2) identify implementation outcomes, performance objectives, and determinants; (3) select implementation strategies; (4) produce implementation protocols and materials; and (5) evaluate implementation outcomes. RESULTS: CHC leadership/providers, patients, and physical therapists were identified as involved groups. Barriers and assets were identified across groups which informed identification of performance objectives necessary to implement two key processes: (1) electronic referral of patients with back pain in CHC clinics to the BeatPain team and (2) connecting patients with physical therapists providing telehealth. Determinants of the performance objectives for each group informed our choice of implementation strategies which focused on training, education, clinician support, and tailoring physical therapy interventions for telehealth delivery and cultural competency. We selected implementation outcomes for the BeatPain trial to evaluate the success of our implementation strategies. CONCLUSIONS: Implementation mapping provided a comprehensive and systematic approach to develop an implementation plan during the planning phase for our ongoing hybrid effectiveness-implementation trial. We will be able to evaluate the implementation strategies used in the BeatPain Utah study to inform future efforts to implement telehealth delivery of evidence-based pain care in CHCs and other settings. TRIAL REGISTRATION: ClinicalTrials.gov Identifier: NCT04923334 . Registered June 11, 2021.

The Business Process Management for Healthcare (BPM+ Health) Consortium: motivation, methodology, and deliverables for enabling clinical knowledge interoperability (CKI).

OBJECTIVES: To enhance the Business Process Management (BPM)+ Healthcare language portfolio by incorporating knowledge types not previously covered and to improve the overall effectiveness and expressiveness of the suite to improve Clinical Knowledge Interoperability. METHODS: We used the BPM+ Health and Object Management Group (OMG) standards development methodology to develop new languages, following a gap analysis between existing BPM+ Health languages and clinical practice guideline knowledge types. Proposal requests were developed based on these requirements, and submission teams were formed to respond to them. The resulting proposals were submitted to OMG for ratification. RESULTS: The BPM+ Health family of languages, which initially consisted of the Business Process Model and Notation, Decision Model and Notation, and Case Model and Notation, was expanded by adding 5 new language standards through the OMG. These include Pedigree and Provenance Model and Notation for expressing epistemic knowledge, Knowledge Package Model and Notation for supporting packaging knowledge, Shared Data Model and Notation for expressing ontic knowledge, Party Model and Notation for representing entities and organizations, and Specification Common Elements, a language providing a standard abstract and reusable library that underpins the 4 new languages. DISCUSSION AND CONCLUSION: In this effort, we adopted a strategy of separation of concerns to promote a portfolio of domain-agnostic, independent, but integrated domain-specific languages for authoring medical knowledge. This strategy is a practical and effective approach to expressing complex medical knowledge. These new domain-specific languages offer various knowledge-type options for clinical knowledge authors to choose from without potentially adding unnecessary overhead or complexity.

Enhanced family history-based algorithms increase the identification of individuals meeting criteria for genetic testing of hereditary cancer syndromes but would not reduce disparities on their own.

OBJECTIVE: This study aimed to 1) investigate algorithm enhancements for identifying patients eligible for genetic testing of hereditary cancer syndromes using family history data from electronic health records (EHRs); and 2) assess their impact on relative differences across sex, race, ethnicity, and language preference. MATERIALS AND METHODS: The study used EHR data from a tertiary academic medical center. A baseline rule-base algorithm, relying on structured family history data (structured data; SD), was enhanced using a natural language processing (NLP) component and a relaxed criteria algorithm (partial match [PM]). The identification rates and differences were analyzed considering sex, race, ethnicity, and language preference. RESULTS: Among 120,007 patients aged 25-60, detection rate differences were found across all groups using the SD (all P < 0.001). Both enhancements increased identification rates; NLP led to a 1.9 % increase and the relaxed criteria algorithm (PM) led to an 18.5 % increase (both P < 0.001). Combining SD with NLP and PM yielded a 20.4 % increase (P < 0.001). Similar increases were observed within subgroups. Relative differences persisted across most categories for the enhanced algorithms, with disproportionately higher identification of patients who are White, Female, non-Hispanic, and whose preferred language is English. CONCLUSION: Algorithm enhancements increased identification rates for patients eligible for genetic testing of hereditary cancer syndromes, regardless of sex, race, ethnicity, and language preference. However, differences in identification rates persisted, emphasizing the need for additional strategies to reduce disparities such as addressing underlying biases in EHR family health information and selectively applying algorithm enhancements for disadvantaged populations. Systematic assessment of differences in algorithm performance across population subgroups should be incorporated into algorithm development processes.

Collecting patient-reported outcome measures in the electronic health record: Lessons from the NIH pragmatic trials Collaboratory.

The NIH Pragmatic Trials Collaboratory supports the design and conduct of 27 embedded pragmatic clinical trials, and many of the studies collect patient reported outcome measures as primary or secondary outcomes. Study teams have encountered challenges in the collection of these measures, including challenges related to competing health care system priorities, clinician's buy-in for adoption of patient-reported outcome measures, low adoption and reach of technology in low resource settings, and lack of consensus and standardization of patient-reported outcome measure selection and administration in the electronic health record. In this article, we share case examples and lessons learned, and suggest that, when using patient-reported outcome measures for embedded pragmatic clinical trials, investigators must make important decisions about whether to use data collected from the participating health system's electronic health record, integrate externally collected patient-reported outcome data into the electronic health record, or collect these data in separate systems for their studies.

Increasing the reach of evidence-based interventions for weight management and diabetes prevention among Medicaid patients: study protocol for a pilot Sequential Multiple Assignment Randomised Trial.

INTRODUCTION: Over 40% of US adults meet criteria for obesity, a major risk factor for chronic disease. Obesity disproportionately impacts populations that have been historically marginalised (eg, low socioeconomic status, rural, some racial/ethnic minority groups). Evidence-based interventions (EBIs) for weight management exist but reach less than 3% of eligible individuals. The aims of this pilot randomised controlled trial are to evaluate feasibility and acceptability of dissemination strategies designed to increase reach of EBIs for weight management. METHODS AND ANALYSIS: This study is a two-phase, Sequential Multiple Assignment Randomized Trial, conducted with 200 Medicaid patients. In phase 1, patients will be individually randomised to single text message (TM1) or multiple text messages (TM+). Phase 2 is based on treatment response. Patients who enrol in the EBI within 12 weeks of exposure to phase 1 (ie, responders) receive no further interventions. Patients in TM1 who do not enrol in the EBI within 12 weeks of exposure (ie, TM1 non-responders) will be randomised to either TM1-Continued (ie, no further TM) or TM1 & MAPS (ie, no further TM, up to 2 Motivation And Problem Solving (MAPS) navigation calls) over the next 12 weeks. Patients in TM+ who do not enrol in the EBI (ie, TM+ non-responders) will be randomised to either TM+Continued (ie, monthly text messages) or TM+ & MAPS (ie, monthly text messages, plus up to 2 MAPS calls) over the next 12 weeks. Descriptive statistics will be used to characterise feasibility (eg, proportion of patients eligible, contacted and enrolled in the trial) and acceptability (eg, participant opt-out, participant engagement with dissemination strategies, EBI reach (ie, the proportion of participants who enrol in EBI), adherence, effectiveness). ETHICS AND DISSEMINATION: Study protocol was approved by the University of Utah Institutional Review Board (#00139694). Results will be disseminated through study partners and peer-reviewed publications. TRIAL REGISTRATION NUMBER: clinicaltrials.gov; NCT05666323.

Information displays for automated surveillance algorithms of in-hospital patient deterioration: a scoping review.

OBJECTIVE: Surveillance algorithms that predict patient decompensation are increasingly integrated with clinical workflows to help identify patients at risk of in-hospital deterioration. This scoping review aimed to identify the design features of the information displays, the types of algorithm that drive the display, and the effect of these displays on process and patient outcomes. MATERIALS AND METHODS: The scoping review followed Arksey and O'Malley's framework. Five databases were searched with dates between January 1, 2009 and January 26, 2022. Inclusion criteria were: participants-clinicians in inpatient settings; concepts-intervention as deterioration information displays that leveraged automated AI algorithms; comparison as usual care or alternative displays; outcomes as clinical, workflow process, and usability outcomes; and context as simulated or real-world in-hospital settings in any country. Screening, full-text review, and data extraction were reviewed independently by 2 researchers in each step. Display categories were identified inductively through consensus. RESULTS: Of 14 575 articles, 64 were included in the review, describing 61 unique displays. Forty-one displays were designed for specific deteriorations (eg, sepsis), 24 provided simple alerts (ie, text-based prompts without relevant patient data), 48 leveraged well-accepted score-based algorithms, and 47 included nurses as the target users. Only 1 out of the 10 randomized controlled trials reported a significant effect on the primary outcome. CONCLUSIONS: Despite significant advancements in surveillance algorithms, most information displays continue to leverage well-understood, well-accepted score-based algorithms. Users' trust, algorithmic transparency, and workflow integration are significant hurdles to adopting new algorithms into effective decision support tools.

Preliminary Reach of an Information Technology Approach to Support COVID-19 Testing in Schools.

OBJECTIVES: SCALE-UP Counts tests population health management interventions to promote coronavirus disease 2019 (COVID-19) testing in kindergarten through 12th-grade schools that serve populations that have been historically marginalized. METHODS: Within 6 participating schools, we identified 3506 unique parents/guardians who served as the primary contact for at least 1 student. Participants were randomized to text messaging (TM), text messaging + health navigation (HN) (TM + HN), or usual care. Bidirectional texts provided COVID-19 symptom screening, along with guidance on obtaining and using tests as appropriate. If parents/guardians in the TM + HN group were advised to test their child but either did not test or did not respond to texts, they were called by a trained health navigator to address barriers. RESULTS: Participating schools served a student population that was 32.9% non-white and 15.4% Hispanic, with 49.6% of students eligible to receive free lunches. Overall, 98.8% of parents/guardians had a valid cell phone, of which 3.8% opted out. Among the 2323 parents/guardians included in the intervention, 79.6% (n = 1849) were randomized to receive TM, and 19.1% (n = 354) engaged with TM (ie, responded to at least 1 message). Within the TM + HN group (40.1%, n = 932), 1.3% (n = 12) qualified for HN at least once, of which 41.7% (n = 5) talked to a health navigator. CONCLUSIONS: TM and HN are feasible ways to reach parents/guardians of kindergarten through 12th-grade students to provide COVID-19 screening messages. Strategies to improve engagement may strengthen the impact of the intervention.

Evaluation of software impact designed for biomedical research: Are we measuring what's meaningful?

Software is vital for the advancement of biology and medicine. Through analysis of usage and impact metrics of software, developers can help determine user and community engagement. These metrics can be used to justify additional funding, encourage additional use, and identify unanticipated use cases. Such analyses can help define improvement areas and assist with managing project resources. However, there are challenges associated with assessing usage and impact, many of which vary widely depending on the type of software being evaluated. These challenges involve issues of distorted, exaggerated, understated, or misleading metrics, as well as ethical and security concerns. More attention to the nuances, challenges, and considerations involved in capturing impact across the diverse spectrum of biological software is needed. Furthermore, some tools may be especially beneficial to a small audience, yet may not have comparatively compelling metrics of high usage. Although some principles are generally applicable, there is not a single perfect metric or approach to effectively evaluate a software tool's impact, as this depends on aspects unique to each tool, how it is used, and how one wishes to evaluate engagement. We propose more broadly applicable guidelines (such as infrastructure that supports the usage of software and the collection of metrics about usage), as well as strategies for various types of software and resources. We also highlight outstanding issues in the field regarding how communities measure or evaluate software impact. To gain a deeper understanding of the issues hindering software evaluations, as well as to determine what appears to be helpful, we performed a survey of participants involved with scientific software projects for the Informatics Technology for Cancer Research (ITCR) program funded by the National Cancer Institute (NCI). We also investigated software among this scientific community and others to assess how often infrastructure that supports such evaluations is implemented and how this impacts rates of papers describing usage of the software. We find that although developers recognize the utility of analyzing data related to the impact or usage of their software, they struggle to find the time or funding to support such analyses. We also find that infrastructure such as social media presence, more in-depth documentation, the presence of software health metrics, and clear information on how to contact developers seem to be associated with increased usage rates. Our findings can help scientific software developers make the most out of the evaluations of their software so that they can more fully benefit from such assessments.

Potential bias and lack of generalizability in electronic health record data: reflections on health equity from the National Institutes of Health Pragmatic Trials Collaboratory.

Embedded pragmatic clinical trials (ePCTs) play a vital role in addressing current population health problems, and their use of electronic health record (EHR) systems promises efficiencies that will increase the speed and volume of relevant and generalizable research. However, as the number of ePCTs using EHR-derived data grows, so does the risk that research will become more vulnerable to biases due to differences in data capture and access to care for different subsets of the population, thereby propagating inequities in health and the healthcare system. We identify 3 challenges-incomplete and variable capture of data on social determinants of health, lack of representation of vulnerable populations that do not access or receive treatment, and data loss due to variable use of technology-that exacerbate bias when working with EHR data and offer recommendations and examples of ways to actively mitigate bias.

Machine Learning Algorithms to Predict Breast Cancer Recurrence Using Structured and Unstructured Sources from Electronic Health Records.

Recurrence is a critical aspect of breast cancer (BC) that is inexorably tied to mortality. Reuse of healthcare data through Machine Learning (ML) algorithms offers great opportunities to improve the stratification of patients at risk of cancer recurrence. We hypothesized that combining features from structured and unstructured sources would provide better prediction results for 5-year cancer recurrence than either source alone. We collected and preprocessed clinical data from a cohort of BC patients, resulting in 823 valid subjects for analysis. We derived three sets of features: structured information, features from free text, and a combination of both. We evaluated the performance of five ML algorithms to predict 5-year cancer recurrence and selected the best-performing to test our hypothesis. The XGB (eXtreme Gradient Boosting) model yielded the best performance among the five evaluated algorithms, with precision = 0.900, recall = 0.907, F1-score = 0.897, and area under the receiver operating characteristic AUROC = 0.807. The best prediction results were achieved with the structured dataset, followed by the unstructured dataset, while the combined dataset achieved the poorest performance. ML algorithms for BC recurrence prediction are valuable tools to improve patient risk stratification, help with post-cancer monitoring, and plan more effective follow-up. Structured data provides the best results when fed to ML algorithms. However, an approach based on natural language processing offers comparable results while potentially requiring less mapping effort.

Implementation of Lung Cancer Screening in Primary Care and Pulmonary Clinics: Pragmatic Clinical Trial of Electronic Health Record-Integrated Everyday Shared Decision-Making Tool and Clinician-Facing Prompts.

BACKGROUND: Although low-dose CT (LDCT) scan imaging lung cancer screening (LCS) can reduce lung cancer mortality, it remains underused. Shared decision-making (SDM) is recommended to assess the balance of benefits and harms for each patient. RESEARCH QUESTION: Do clinician-facing electronic health record (EHR) prompts and an EHR-integrated everyday SDM tool designed to support routine incorporation of SDM into primary care improve LDCT scan imaging ordering and completion? STUDY DESIGN AND METHODS: A preintervention and postintervention analysis was conducted in 30 primary care and four pulmonary clinics for visits with patients who met United States Preventive Services Task Force criteria for LCS. Propensity scores were used to adjust for covariates. Subgroup analyses were conducted based on the expected benefit from screening (high benefit vs intermediate benefit), pulmonologist involvement (ie, whether the patient was seen in a pulmonary clinic in addition to a primary care clinic), sex, and race and ethnicity. RESULTS: In the 12-month preintervention phase among 1,090 eligible patients, 77 patients (7.1%) had LDCT scan imaging orders and 48 patients (4.4%) completed screenings. In the 9-month intervention phase among 1,026 eligible patients, 280 patients (27.3%) had LDCT scan imaging orders and 182 patients (17.7%) completed screenings. Adjusted ORs were 4.9 (95% CI, 3.4-6.9; P < .001) and 4.7 (95% CI, 3.1-7.1; P < .001) for LDCT imaging ordering and completion, respectively. Subgroup analyses showed increases in ordering and completion for all patient subgroups. In the intervention phase, the SDM tool was used by 23 of 102 ordering providers (22.5%) and for 69 of 274 patients (25.2%) for whom LDCT scan imaging was ordered and who needed SDM at the time of ordering. INTERPRETATION: Clinician-facing EHR prompts and an EHR-integrated everyday SDM tool are promising approaches to improving LCS in the primary care setting. However, room for improvement remains. As such, further research is warranted. TRIAL REGISTRY: ClinicalTrials.gov; No.: NCT04498052; URL: www. CLINICALTRIALS: gov.

Equity and bias in electronic health records data.

Embedded pragmatic clinical trials (ePCTs) are conducted during routine clinical care and have the potential to increase knowledge about the effectiveness of interventions under real world conditions. However, many pragmatic trials rely on data from the electronic health record (EHR) data, which are subject to bias from incomplete data, poor data quality, lack of representation from people who are medically underserved, and implicit bias in EHR design. This commentary examines how the use of EHR data might exacerbate bias and potentially increase health inequities. We offer recommendations for how to increase generalizability of ePCT results and begin to mitigate bias to promote health equity.

Rapid-cycle designs to adapt interventions for COVID-19 in safety-net healthcare systems.

Racial/ethnic minority, low socioeconomic status, and rural populations are disproportionately affected by COVID-19. Developing and evaluating interventions to address COVID-19 testing and vaccination among these populations are crucial to improving health inequities. The purpose of this paper is to describe the application of a rapid-cycle design and adaptation process from an ongoing trial to address COVID-19 among safety-net healthcare system patients. The rapid-cycle design and adaptation process included: (a) assessing context and determining relevant models/frameworks; (b) determining core and modifiable components of interventions; and (c) conducting iterative adaptations using Plan-Do-Study-Act (PDSA) cycles. PDSA cycles included: Plan. Gather information from potential adopters/implementers (e.g., Community Health Center [CHC] staff/patients) and design initial interventions; Do. Implement interventions in single CHC or patient cohort; Study. Examine process, outcome, and context data (e.g., infection rates); and, Act. If necessary, refine interventions based on process and outcome data, then disseminate interventions to other CHCs and patient cohorts. Seven CHC systems with 26 clinics participated in the trial. Rapid-cycle, PDSA-based adaptations were made to adapt to evolving COVID-19-related needs. Near real-time data used for adaptation included data on infection hot spots, CHC capacity, stakeholder priorities, local/national policies, and testing/vaccine availability. Adaptations included those to study design, intervention content, and intervention cohorts. Decision-making included multiple stakeholders (e.g., State Department of Health, Primary Care Association, CHCs, patients, researchers). Rapid-cycle designs may improve the relevance and timeliness of interventions for CHCs and other settings that provide care to populations experiencing health inequities, and for rapidly evolving healthcare challenges such as COVID-19.

Racial/ethnic minority, low socioeconomic status, and rural populations experience a disproportionate burden of COVID-19. Finding ways to address COVID-19 among these populations is crucial to improving health inequities. The purpose of this paper is to describe the rapid-cycle design process for a research project to address COVID-19 testing and vaccination among safety-net healthcare system patients. The project used real-time information on changes in COVID-19 policy (e.g., vaccination authorization), local case rates, and the capacity of safety-net healthcare systems to iteratively change interventions to ensure interventions were relevant and timely for patients. Key changes that were made to interventions included a change to the study design to include vaccination as a focus of the interventions after the vaccine was authorized; change in intervention content according to the capacity of local Community Health Centers to provide testing to patients; and changes to intervention cohorts such that priority groups of patients were selected for intervention based on characteristics including age, residency in an infection "hot spot," or race/ethnicity. Iteratively improving interventions based on real-time data collection may increase intervention relevance and timeliness, and rapid-cycle adaptions can be successfully implemented in resource constrained settings like safety-net healthcare systems.

Leveraging Natural Language Processing to Extract Features of Colorectal Polyps From Pathology Reports for Epidemiologic Study.

PURPOSE: Histopathologic features are critical for studying risk factors of colorectal polyps, but remain deeply embedded within unstructured pathology reports, requiring costly and time-consuming manual abstraction for research. In this study, we developed and evaluated a natural language processing (NLP) pipeline to automatically extract histopathologic features of colorectal polyps from pathology reports, with an emphasis on individual polyp size. These data were then linked with structured electronic health record (EHR) data, creating an analysis-ready epidemiologic data set. METHODS: We obtained 24,584 pathology reports from colonoscopies performed at the University of Utah's Gastroenterology Clinic. Two investigators annotated 350 reports to determine inter-rater agreement, develop an annotation scheme, and create a reference standard for performance evaluation. The pipeline was then developed, and performance was compared against the reference for extracting polyp location, histology, size, shape, dysplasia, and the number of polyps. Finally, the pipeline was applied to 24,225 unseen reports and NLP-extracted data were linked with structured EHR data. RESULTS: Across all features, our pipeline achieved a precision of 98.9%, a recall of 98.0%, and an F1-score of 98.4%. In patients with polyps, the pipeline correctly extracted 95.6% of sizes, 97.2% of polyp locations, 97.8% of histology, 98.3% of shapes, and 98.3% of dysplasia levels. When applied to unseen data, the pipeline classified 12,889 patients as having polyps, 4,907 patients without polyps, and extracted the features of 28,387 polyps. Tubular adenomas were the most common subtype (55.9%), 8.1% of polyps were advanced adenomas, and the mean polyp size was 0.57 (±0.4) cm. CONCLUSION: Our pipeline extracted histopathologic features of colorectal polyps from colonoscopy pathology reports, most notably individual polyp sizes, with considerable accuracy. This study demonstrates the utility of NLP for extracting polyp features and linking these data with EHR data to create an epidemiologic data set to study colorectal polyp risk factors and outcomes.