Uptake of Cancer Genetic Services for Chatbot vs Standard-of-Care Delivery Models: The BRIDGE Randomized Clinical Trial.

Importance: Increasing numbers of unaffected individuals could benefit from genetic evaluation for inherited cancer susceptibility. Automated conversational agents (ie, chatbots) are being developed for cancer genetics contexts; however, randomized comparisons with standard of care (SOC) are needed. Objective: To examine whether chatbot and SOC approaches are equivalent in completion of pretest cancer genetic services and genetic testing. Design, Setting, and Participants: This equivalence trial (Broadening the Reach, Impact, and Delivery of Genetic Services [BRIDGE] randomized clinical trial) was conducted between August 15, 2020, and August 31, 2023, at 2 US health care systems (University of Utah Health and NYU Langone Health). Participants were aged 25 to 60 years, had had a primary care visit in the previous 3 years, were eligible for cancer genetic evaluation, were English or Spanish speaking, had no prior cancer diagnosis other than nonmelanoma skin cancer, had no prior cancer genetic counseling or testing, and had an electronic patient portal account. Intervention: Participants were randomized 1:1 at the patient level to the study groups at each site. In the chatbot intervention group, patients were invited in a patient portal outreach message to complete a pretest genetics education chat. In the enhanced SOC control group, patients were invited to complete an SOC pretest appointment with a certified genetic counselor. Main Outcomes and Measures: Primary outcomes were completion of pretest cancer genetic services (ie, pretest genetics education chat or pretest genetic counseling appointment) and completion of genetic testing. Equivalence hypothesis testing was used to compare the study groups. Results: This study included 3073 patients (1554 in the chatbot group and 1519 in the enhanced SOC control group). Their mean (SD) age at outreach was 43.8 (9.9) years, and most (2233 of 3063 [72.9%]) were women. A total of 204 patients (7.3%) were Black, 317 (11.4%) were Latinx, and 2094 (75.0%) were White. The estimated percentage point difference for completion of pretest cancer genetic services between groups was 2.0 (95% CI, -1.1 to 5.0). The estimated percentage point difference for completion of genetic testing was -1.3 (95% CI, -3.7 to 1.1). Analyses suggested equivalence in the primary outcomes. Conclusions and Relevance: The findings of the BRIDGE equivalence trial support the use of chatbot approaches to offer cancer genetic services. Chatbot tools can be a key component of sustainable and scalable population health management strategies to enhance access to cancer genetic services. Trial Registration: ClinicalTrials.gov Identifier: NCT03985852.

Chatbot for the Return of Positive Genetic Screening Results for Hereditary Cancer Syndromes: a Prompt Engineering Study.

Background: The growing demand for genomic testing and limited access to experts necessitate innovative service models. While chatbots have shown promise in supporting genomic services like pre-test counseling, their use in returning positive genetic results, especially using the more recent large language models (LLMs) remains unexplored. Objective: This study reports the prompt engineering process and intrinsic evaluation of the LLM component of a chatbot designed to support returning positive population-wide genomic screening results. Methods: We used a three-step prompt engineering process, including Retrieval-Augmented Generation (RAG) and few-shot techniques to develop an open-response chatbot. This was then evaluated using two hypothetical scenarios, with experts rating its performance using a 5-point Likert scale across eight criteria: tone, clarity, program accuracy, domain accuracy, robustness, efficiency, boundaries, and usability. Results: The chatbot achieved an overall score of 3.88 out of 5 across all criteria and scenarios. The highest ratings were in Tone (4.25), Usability (4.25), and Boundary management (4.0), followed by Efficiency (3.88), Clarity and Robustness (3.81), and Domain Accuracy (3.63). The lowest-rated criterion was Program Accuracy, which scored 3.25. Discussion: The LLM handled open-ended queries and maintained boundaries, while the lower Program Accuracy rating indicates areas for improvement. Future work will focus on refining prompts, expanding evaluations, and exploring optimal hybrid chatbot designs that integrate LLM components with rule-based chatbot components to enhance genomic service delivery.

Precision public health in the era of genomics and big data.

Precision public health (PPH) considers the interplay between genetics, lifestyle and the environment to improve disease prevention, diagnosis and treatment on a population level-thereby delivering the right interventions to the right populations at the right time. In this Review, we explore the concept of PPH as the next generation of public health. We discuss the historical context of using individual-level data in public health interventions and examine recent advancements in how data from human and pathogen genomics and social, behavioral and environmental research, as well as artificial intelligence, have transformed public health. Real-world examples of PPH are discussed, emphasizing how these approaches are becoming a mainstay in public health, as well as outstanding challenges in their development, implementation and sustainability. Data sciences, ethical, legal and social implications research, capacity building, equity research and implementation science will have a crucial role in realizing the potential for 'precision' to enhance traditional public health approaches.

Best practices to evaluate the impact of biomedical research software-metric collection beyond citations.

MOTIVATION: Software is vital for the advancement of biology and medicine. Impact evaluations of scientific software have primarily emphasized traditional citation metrics of associated papers, despite these metrics inadequately capturing the dynamic picture of impact and despite challenges with improper citation. RESULTS: To understand how software developers evaluate their tools, we conducted a survey of participants in the Informatics Technology for Cancer Research (ITCR) program funded by the National Cancer Institute (NCI). We found that although developers realize the value of more extensive metric collection, they find a lack of funding and time hindering. We also investigated software among this community for how often infrastructure that supports more nontraditional metrics were implemented and how this impacted rates of papers describing usage of the software. We found that infrastructure such as social media presence, more in-depth documentation, the presence of software health metrics, and clear information on how to contact developers seemed to be associated with increased mention rates. Analysing more diverse metrics can enable developers to better understand user engagement, justify continued funding, identify novel use cases, pinpoint improvement areas, and ultimately amplify their software's impact. Challenges are associated, including distorted or misleading metrics, as well as ethical and security concerns. More attention to nuances involved in capturing impact across the spectrum of biomedical software is needed. For funders and developers, we outline guidance based on experience from our community. By considering how we evaluate software, we can empower developers to create tools that more effectively accelerate biological and medical research progress. AVAILABILITY AND IMPLEMENTATION: More information about the analysis, as well as access to data and code is available at https://github.com/fhdsl/ITCR_Metrics_manuscript_website.

Lung Cancer Screening Before and After a Multifaceted Electronic Health Record Intervention: A Nonrandomized Controlled Trial.

Importance: Lung cancer is the deadliest cancer in the US. Early-stage lung cancer detection with lung cancer screening (LCS) through low-dose computed tomography (LDCT) improves outcomes. Objective: To assess the association of a multifaceted clinical decision support intervention with rates of identification and completion of recommended LCS-related services. Design, Setting, and Participants: This nonrandomized controlled trial used an interrupted time series design, including 3 study periods from August 24, 2019, to April 27, 2022: baseline (12 months), period 1 (11 months), and period 2 (9 months). Outcome changes were reported as shifts in the outcome level at the beginning of each period and changes in monthly trend (ie, slope). The study was conducted at primary care and pulmonary clinics at a health care system headquartered in Salt Lake City, Utah, among patients aged 55 to 80 years who had smoked 30 pack-years or more and were current smokers or had quit smoking in the past 15 years. Data were analyzed from September 2023 through February 2024. Interventions: Interventions in period 1 included clinician-facing preventive care reminders, an electronic health record-integrated shared decision-making tool, and narrative LCS guidance provided in the LDCT ordering screen. Interventions in period 2 included the same clinician-facing interventions and patient-facing reminders for LCS discussion and LCS. Main Outcome and Measure: The primary outcome was LCS care gap closure, defined as the identification and completion of recommended care services. LCS care gap closure could be achieved through LDCT completion, other chest CT completion, or LCS shared decision-making. Results: The study included 1865 patients (median [IQR] age, 64 [60-70] years; 759 female [40.7%]). The clinician-facing intervention (period 1) was not associated with changes in level but was associated with an increase in slope of 2.6 percentage points (95% CI, 2.4-2.7 percentage points) per month in care gap closure through any means and 1.6 percentage points (95% CI, 1.4-1.8 percentage points) per month in closure through LDCT. In period 2, introduction of patient-facing reminders was associated with an immediate increase in care gap closure (2.3 percentage points; 95% CI, 1.0-3.6 percentage points) and closure through LDCT (2.4 percentage points; 95% CI, 0.9-3.9 percentage points) but was not associated with an increase in slope. The overall care gap closure rate was 175 of 1104 patients (15.9%) at the end of the baseline period vs 588 of 1255 patients (46.9%) at the end of period 2. Conclusions and Relevance: In this study, a multifaceted intervention was associated with an improvement in LCS care gap closure. Trial Registration: ClinicalTrials.gov Identifier: NCT04498052.

Genetically guided precision medicine clinical decision support tools: a systematic review.

OBJECTIVES: Patient care using genetics presents complex challenges. Clinical decision support (CDS) tools are a potential solution because they provide patient-specific risk assessments and/or recommendations at the point of care. This systematic review evaluated the literature on CDS systems which have been implemented to support genetically guided precision medicine (GPM). MATERIALS AND METHODS: A comprehensive search was conducted in MEDLINE and Embase, encompassing January 1, 2011-March 14, 2023. The review included primary English peer-reviewed research articles studying humans, focused on the use of computers to guide clinical decision-making and delivering genetically guided, patient-specific assessments, and/or recommendations to healthcare providers and/or patients. RESULTS: The search yielded 3832 unique articles. After screening, 41 articles were identified that met the inclusion criteria. Alerts and reminders were the most common form of CDS used. About 27 systems were integrated with the electronic health record; 2 of those used standards-based approaches for genomic data transfer. Three studies used a framework to analyze the implementation strategy. DISCUSSION: Findings include limited use of standards-based approaches for genomic data transfer, system evaluations that do not employ formal frameworks, and inconsistencies in the methodologies used to assess genetic CDS systems and their impact on patient outcomes. CONCLUSION: We recommend that future research on CDS system implementation for genetically GPM should focus on implementing more CDS systems, utilization of standards-based approaches, user-centered design, exploration of alternative forms of CDS interventions, and use of formal frameworks to systematically evaluate genetic CDS systems and their effects on patient care.

Clinician perspectives on how situational context and augmented intelligence design features impact perceived usefulness of sepsis prediction scores embedded within a simulated electronic health record.

OBJECTIVE: Obtain clinicians' perspectives on early warning scores (EWS) use within context of clinical cases. MATERIAL AND METHODS: We developed cases mimicking sepsis situations. De-identified data, synthesized physician notes, and EWS representing deterioration risk were displayed in a simulated EHR for analysis. Twelve clinicians participated in semi-structured interviews to ascertain perspectives across four domains: (1) Familiarity with and understanding of artificial intelligence (AI), prediction models and risk scores; (2) Clinical reasoning processes; (3) Impression and response to EWS; and (4) Interface design. Transcripts were coded and analyzed using content and thematic analysis. RESULTS: Analysis revealed clinicians have experience but limited AI and prediction/risk modeling understanding. Case assessments were primarily based on clinical data. EWS went unmentioned during initial case analysis; although when prompted to comment on it, they discussed it in subsequent cases. Clinicians were unsure how to interpret or apply the EWS, and desired evidence on its derivation and validation. Design recommendations centered around EWS display in multi-patient lists for triage, and EWS trends within the patient record. Themes included a "Trust but Verify" approach to AI and early warning information, dichotomy that EWS is helpful for triage yet has disproportional signal-to-high noise ratio, and action driven by clinical judgment, not the EWS. CONCLUSIONS: Clinicians were unsure of how to apply EWS, acted on clinical data, desired score composition and validation information, and felt EWS was most useful when embedded in multi-patient views. Systems providing interactive visualization may facilitate EWS transparency and increase confidence in AI-generated information.

Vision Loss in Children from Immigrant and Nonimmigrant Households: Evidence from the National Survey of Children's Health 2018-2020.

The aim of this study was to determine whether immigrant generation is associated with caregiver-reported vision loss in children adjusting for sociodemographic characteristics. Nationally representative data from the National Survey of Children's Health (2018-2020) was used. The primary exposure was immigrant generation defined as: first (child and all reported parents were born outside the United States); second (child was born in the United States and at least one parent was born outside the United States); third or higher (all parents in the household were born in the United States). The main outcome was caregiver-reported vision loss in child. Adjusted odds ratios (aOR) and 95% confidence intervals were computed based on immigration generation. The study sample included 84,860 US children aged 3-17 years. First generation children had higher adjusted odds of caregiver-reported vision loss (aOR 2.30; 95% CI 1.21, 4.35) than third or higher generation children after adjusting for demographic characteristics and social determinants of health. For Hispanic families, first generation (aOR 2.99; 95% CI 1.34, 6.66), and second-generation children (aOR 1.70; 95% CI 1.06, 2.74) had a higher adjusted odds of vision loss compared with third or higher generation children. Even when adjusting for sociodemographic characteristics, first generation children had greater odds of vision loss, especially in Hispanic households, than third generation children. Immigration generation should be treated as an independent risk factor for vision loss for children and is a social determinant of eye health.

SCALE-UP II: protocol for a pragmatic randomised trial examining population health management interventions to increase the uptake of at-home COVID-19 testing in community health centres.

INTRODUCTION: SCALE-UP II aims to investigate the effectiveness of population health management interventions using text messaging (TM), chatbots and patient navigation (PN) in increasing the uptake of at-home COVID-19 testing among patients in historically marginalised communities, specifically, those receiving care at community health centres (CHCs). METHODS AND ANALYSIS: The trial is a multisite, randomised pragmatic clinical trial. Eligible patients are >18 years old with a primary care visit in the last 3 years at one of the participating CHCs. Demographic data will be obtained from CHC electronic health records. Patients will be randomised to one of two factorial designs based on smartphone ownership. Patients who self-report replying to a text message that they have a smartphone will be randomised in a 2×2×2 factorial fashion to receive (1) chatbot or TM; (2) PN (yes or no); and (3) repeated offers to interact with the interventions every 10 or 30 days. Participants who do not self-report as having a smartphone will be randomised in a 2×2 factorial fashion to receive (1) TM with or without PN; and (2) repeated offers every 10 or 30 days. The interventions will be sent in English or Spanish, with an option to request at-home COVID-19 test kits. The primary outcome is the proportion of participants using at-home COVID-19 tests during a 90-day follow-up. The study will evaluate the main effects and interactions among interventions, implementation outcomes and predictors and moderators of study outcomes. Statistical analyses will include logistic regression, stratified subgroup analyses and adjustment for stratification factors. ETHICS AND DISSEMINATION: The protocol was approved by the University of Utah Institutional Review Board. On completion, study data will be made available in compliance with National Institutes of Health data sharing policies. Results will be disseminated through study partners and peer-reviewed publications. TRIAL REGISTRATION NUMBER: ClinicalTrials.gov: NCT05533918 and NCT05533359.

The Business Process Management for Healthcare (BPM+ Health) Consortium: motivation, methodology, and deliverables for enabling clinical knowledge interoperability (CKI).

OBJECTIVES: To enhance the Business Process Management (BPM)+ Healthcare language portfolio by incorporating knowledge types not previously covered and to improve the overall effectiveness and expressiveness of the suite to improve Clinical Knowledge Interoperability. METHODS: We used the BPM+ Health and Object Management Group (OMG) standards development methodology to develop new languages, following a gap analysis between existing BPM+ Health languages and clinical practice guideline knowledge types. Proposal requests were developed based on these requirements, and submission teams were formed to respond to them. The resulting proposals were submitted to OMG for ratification. RESULTS: The BPM+ Health family of languages, which initially consisted of the Business Process Model and Notation, Decision Model and Notation, and Case Model and Notation, was expanded by adding 5 new language standards through the OMG. These include Pedigree and Provenance Model and Notation for expressing epistemic knowledge, Knowledge Package Model and Notation for supporting packaging knowledge, Shared Data Model and Notation for expressing ontic knowledge, Party Model and Notation for representing entities and organizations, and Specification Common Elements, a language providing a standard abstract and reusable library that underpins the 4 new languages. DISCUSSION AND CONCLUSION: In this effort, we adopted a strategy of separation of concerns to promote a portfolio of domain-agnostic, independent, but integrated domain-specific languages for authoring medical knowledge. This strategy is a practical and effective approach to expressing complex medical knowledge. These new domain-specific languages offer various knowledge-type options for clinical knowledge authors to choose from without potentially adding unnecessary overhead or complexity.

Use of implementation mapping in the planning of a hybrid type 1 pragmatic clinical trial: the BeatPain Utah study.

BACKGROUND: Considerable disparities in chronic pain management have been identified. Persons in rural, lower income, and minoritized communities are less likely to receive evidence-based, nonpharmacologic care. Telehealth delivery of nonpharmacologic, evidence-based interventions for persons with chronic pain is a promising strategy to lessen disparities, but implementation comes with many challenges. The BeatPain Utah study is a hybrid type 1 effectiveness-implementation pragmatic clinical trial investigating telehealth strategies to provide nonpharmacologic care from physical therapists to persons with chronic back pain receiving care in ommunity health centers (CHCs). CHCs provide primary care to all persons regardless of ability to pay. This paper outlines the use of implementation mapping to develop a multifaceted implementation plan for the BeatPain study. METHODS: During a planning year for the BeatPain trial, we developed a comprehensive logic model including the five-step implementation mapping process informed by additional frameworks and theories. The five iterative implementation mapping steps were addressed in the planning year: (1) conduct needs assessments for involved groups; (2) identify implementation outcomes, performance objectives, and determinants; (3) select implementation strategies; (4) produce implementation protocols and materials; and (5) evaluate implementation outcomes. RESULTS: CHC leadership/providers, patients, and physical therapists were identified as involved groups. Barriers and assets were identified across groups which informed identification of performance objectives necessary to implement two key processes: (1) electronic referral of patients with back pain in CHC clinics to the BeatPain team and (2) connecting patients with physical therapists providing telehealth. Determinants of the performance objectives for each group informed our choice of implementation strategies which focused on training, education, clinician support, and tailoring physical therapy interventions for telehealth delivery and cultural competency. We selected implementation outcomes for the BeatPain trial to evaluate the success of our implementation strategies. CONCLUSIONS: Implementation mapping provided a comprehensive and systematic approach to develop an implementation plan during the planning phase for our ongoing hybrid effectiveness-implementation trial. We will be able to evaluate the implementation strategies used in the BeatPain Utah study to inform future efforts to implement telehealth delivery of evidence-based pain care in CHCs and other settings. TRIAL REGISTRATION: ClinicalTrials.gov Identifier: NCT04923334 . Registered June 11, 2021.

Enhanced family history-based algorithms increase the identification of individuals meeting criteria for genetic testing of hereditary cancer syndromes but would not reduce disparities on their own.

OBJECTIVE: This study aimed to 1) investigate algorithm enhancements for identifying patients eligible for genetic testing of hereditary cancer syndromes using family history data from electronic health records (EHRs); and 2) assess their impact on relative differences across sex, race, ethnicity, and language preference. MATERIALS AND METHODS: The study used EHR data from a tertiary academic medical center. A baseline rule-base algorithm, relying on structured family history data (structured data; SD), was enhanced using a natural language processing (NLP) component and a relaxed criteria algorithm (partial match [PM]). The identification rates and differences were analyzed considering sex, race, ethnicity, and language preference. RESULTS: Among 120,007 patients aged 25-60, detection rate differences were found across all groups using the SD (all P < 0.001). Both enhancements increased identification rates; NLP led to a 1.9 % increase and the relaxed criteria algorithm (PM) led to an 18.5 % increase (both P < 0.001). Combining SD with NLP and PM yielded a 20.4 % increase (P < 0.001). Similar increases were observed within subgroups. Relative differences persisted across most categories for the enhanced algorithms, with disproportionately higher identification of patients who are White, Female, non-Hispanic, and whose preferred language is English. CONCLUSION: Algorithm enhancements increased identification rates for patients eligible for genetic testing of hereditary cancer syndromes, regardless of sex, race, ethnicity, and language preference. However, differences in identification rates persisted, emphasizing the need for additional strategies to reduce disparities such as addressing underlying biases in EHR family health information and selectively applying algorithm enhancements for disadvantaged populations. Systematic assessment of differences in algorithm performance across population subgroups should be incorporated into algorithm development processes.

Collecting patient-reported outcome measures in the electronic health record: Lessons from the NIH pragmatic trials Collaboratory.

The NIH Pragmatic Trials Collaboratory supports the design and conduct of 27 embedded pragmatic clinical trials, and many of the studies collect patient reported outcome measures as primary or secondary outcomes. Study teams have encountered challenges in the collection of these measures, including challenges related to competing health care system priorities, clinician's buy-in for adoption of patient-reported outcome measures, low adoption and reach of technology in low resource settings, and lack of consensus and standardization of patient-reported outcome measure selection and administration in the electronic health record. In this article, we share case examples and lessons learned, and suggest that, when using patient-reported outcome measures for embedded pragmatic clinical trials, investigators must make important decisions about whether to use data collected from the participating health system's electronic health record, integrate externally collected patient-reported outcome data into the electronic health record, or collect these data in separate systems for their studies.

Increasing the reach of evidence-based interventions for weight management and diabetes prevention among Medicaid patients: study protocol for a pilot Sequential Multiple Assignment Randomised Trial.

INTRODUCTION: Over 40% of US adults meet criteria for obesity, a major risk factor for chronic disease. Obesity disproportionately impacts populations that have been historically marginalised (eg, low socioeconomic status, rural, some racial/ethnic minority groups). Evidence-based interventions (EBIs) for weight management exist but reach less than 3% of eligible individuals. The aims of this pilot randomised controlled trial are to evaluate feasibility and acceptability of dissemination strategies designed to increase reach of EBIs for weight management. METHODS AND ANALYSIS: This study is a two-phase, Sequential Multiple Assignment Randomized Trial, conducted with 200 Medicaid patients. In phase 1, patients will be individually randomised to single text message (TM1) or multiple text messages (TM+). Phase 2 is based on treatment response. Patients who enrol in the EBI within 12 weeks of exposure to phase 1 (ie, responders) receive no further interventions. Patients in TM1 who do not enrol in the EBI within 12 weeks of exposure (ie, TM1 non-responders) will be randomised to either TM1-Continued (ie, no further TM) or TM1 & MAPS (ie, no further TM, up to 2 Motivation And Problem Solving (MAPS) navigation calls) over the next 12 weeks. Patients in TM+ who do not enrol in the EBI (ie, TM+ non-responders) will be randomised to either TM+Continued (ie, monthly text messages) or TM+ & MAPS (ie, monthly text messages, plus up to 2 MAPS calls) over the next 12 weeks. Descriptive statistics will be used to characterise feasibility (eg, proportion of patients eligible, contacted and enrolled in the trial) and acceptability (eg, participant opt-out, participant engagement with dissemination strategies, EBI reach (ie, the proportion of participants who enrol in EBI), adherence, effectiveness). ETHICS AND DISSEMINATION: Study protocol was approved by the University of Utah Institutional Review Board (#00139694). Results will be disseminated through study partners and peer-reviewed publications. TRIAL REGISTRATION NUMBER: clinicaltrials.gov; NCT05666323.

Use of implementation mapping in the planning of a hybrid type 1 pragmatic clinical trial: the BeatPain Utah study.

Background: Considerable disparities in chronic pain management have been identified. Persons in rural, lower income and minoritized communities are less likely to receive evidence-based, nonpharmacologic care. Telehealth delivery of nonpharmacologic, evidence-based interventions for persons with chronic pain is a promising strategy to lessen disparities, but implementation comes with many challenges. The BeatPain Utah study is a hybrid type I effectiveness-implementation pragmatic clinical trial investigating telehealth strategies to provide nonpharmacologic care from physical therapists to persons with chronic back pain receiving care in Community Health Centers (CHCs). CHCs provide primary care to all persons regardless of ability to pay. This paper outlines the use of implementation mapping to develop a multifaceted implementation plan for the BeatPain study. Methods: During a planning year for the BeatPain trial we developed a comprehensive logic model including the 5-step implementation mapping process informed by additional frameworks and theories. The five iterative implementation mapping steps were addressed in the planning year; 1) conduct needs assessments for involved groups; 2) identify implementation outcomes, performance objectives and determinants; 3) select implementation strategies; 4) produce implementation protocols and materials; and 5) evaluate implementation outcomes. Results: CHC leadership/providers, patients and physical therapists were identified as involved groups. Barriers and assets were identified across groups which informed identification of performance objectives necessary to implement two key processes; 1) electronic referral of patients with back pain in CHC clinics to the BeatPain team; and 2) connecting patients with physical therapists providing telehealth. Determinants of the performance objectives for each group informed our choice of implementation strategies which focused on training, education, clinician support and tailoring physical therapy interventions for telehealth delivery and cultural competency. We selected implementation outcomes for the BeatPain trial to evaluate the success of our implementation strategies. Conclusions: Implementation mapping provided a comprehensive and systematic approach to develop an implementation plan during the planning phase for our ongoing hybrid effectiveness-implementation trial. We will be able to evaluate the implementation strategies used in the BeatPain Utah study to inform future efforts to implement telehealth delivery of evidence-based pain care in CHCs and other settings. Trial registration: Clinicaltrials.gov Identifier: NCT04923334. Registered June 11, 2021 (https://clinicaltrials.gov/study/NCT04923334.

Information displays for automated surveillance algorithms of in-hospital patient deterioration: a scoping review.

OBJECTIVE: Surveillance algorithms that predict patient decompensation are increasingly integrated with clinical workflows to help identify patients at risk of in-hospital deterioration. This scoping review aimed to identify the design features of the information displays, the types of algorithm that drive the display, and the effect of these displays on process and patient outcomes. MATERIALS AND METHODS: The scoping review followed Arksey and O'Malley's framework. Five databases were searched with dates between January 1, 2009 and January 26, 2022. Inclusion criteria were: participants-clinicians in inpatient settings; concepts-intervention as deterioration information displays that leveraged automated AI algorithms; comparison as usual care or alternative displays; outcomes as clinical, workflow process, and usability outcomes; and context as simulated or real-world in-hospital settings in any country. Screening, full-text review, and data extraction were reviewed independently by 2 researchers in each step. Display categories were identified inductively through consensus. RESULTS: Of 14 575 articles, 64 were included in the review, describing 61 unique displays. Forty-one displays were designed for specific deteriorations (eg, sepsis), 24 provided simple alerts (ie, text-based prompts without relevant patient data), 48 leveraged well-accepted score-based algorithms, and 47 included nurses as the target users. Only 1 out of the 10 randomized controlled trials reported a significant effect on the primary outcome. CONCLUSIONS: Despite significant advancements in surveillance algorithms, most information displays continue to leverage well-understood, well-accepted score-based algorithms. Users' trust, algorithmic transparency, and workflow integration are significant hurdles to adopting new algorithms into effective decision support tools.

Preliminary Reach of an Information Technology Approach to Support COVID-19 Testing in Schools.

OBJECTIVES: SCALE-UP Counts tests population health management interventions to promote coronavirus disease 2019 (COVID-19) testing in kindergarten through 12th-grade schools that serve populations that have been historically marginalized. METHODS: Within 6 participating schools, we identified 3506 unique parents/guardians who served as the primary contact for at least 1 student. Participants were randomized to text messaging (TM), text messaging + health navigation (HN) (TM + HN), or usual care. Bidirectional texts provided COVID-19 symptom screening, along with guidance on obtaining and using tests as appropriate. If parents/guardians in the TM + HN group were advised to test their child but either did not test or did not respond to texts, they were called by a trained health navigator to address barriers. RESULTS: Participating schools served a student population that was 32.9% non-white and 15.4% Hispanic, with 49.6% of students eligible to receive free lunches. Overall, 98.8% of parents/guardians had a valid cell phone, of which 3.8% opted out. Among the 2323 parents/guardians included in the intervention, 79.6% (n = 1849) were randomized to receive TM, and 19.1% (n = 354) engaged with TM (ie, responded to at least 1 message). Within the TM + HN group (40.1%, n = 932), 1.3% (n = 12) qualified for HN at least once, of which 41.7% (n = 5) talked to a health navigator. CONCLUSIONS: TM and HN are feasible ways to reach parents/guardians of kindergarten through 12th-grade students to provide COVID-19 screening messages. Strategies to improve engagement may strengthen the impact of the intervention.

Potential bias and lack of generalizability in electronic health record data: reflections on health equity from the National Institutes of Health Pragmatic Trials Collaboratory.

Embedded pragmatic clinical trials (ePCTs) play a vital role in addressing current population health problems, and their use of electronic health record (EHR) systems promises efficiencies that will increase the speed and volume of relevant and generalizable research. However, as the number of ePCTs using EHR-derived data grows, so does the risk that research will become more vulnerable to biases due to differences in data capture and access to care for different subsets of the population, thereby propagating inequities in health and the healthcare system. We identify 3 challenges-incomplete and variable capture of data on social determinants of health, lack of representation of vulnerable populations that do not access or receive treatment, and data loss due to variable use of technology-that exacerbate bias when working with EHR data and offer recommendations and examples of ways to actively mitigate bias.

Evaluation of software impact designed for biomedical research: Are we measuring what's meaningful?

Software is vital for the advancement of biology and medicine. Through analysis of usage and impact metrics of software, developers can help determine user and community engagement. These metrics can be used to justify additional funding, encourage additional use, and identify unanticipated use cases. Such analyses can help define improvement areas and assist with managing project resources. However, there are challenges associated with assessing usage and impact, many of which vary widely depending on the type of software being evaluated. These challenges involve issues of distorted, exaggerated, understated, or misleading metrics, as well as ethical and security concerns. More attention to the nuances, challenges, and considerations involved in capturing impact across the diverse spectrum of biological software is needed. Furthermore, some tools may be especially beneficial to a small audience, yet may not have comparatively compelling metrics of high usage. Although some principles are generally applicable, there is not a single perfect metric or approach to effectively evaluate a software tool's impact, as this depends on aspects unique to each tool, how it is used, and how one wishes to evaluate engagement. We propose more broadly applicable guidelines (such as infrastructure that supports the usage of software and the collection of metrics about usage), as well as strategies for various types of software and resources. We also highlight outstanding issues in the field regarding how communities measure or evaluate software impact. To gain a deeper understanding of the issues hindering software evaluations, as well as to determine what appears to be helpful, we performed a survey of participants involved with scientific software projects for the Informatics Technology for Cancer Research (ITCR) program funded by the National Cancer Institute (NCI). We also investigated software among this scientific community and others to assess how often infrastructure that supports such evaluations is implemented and how this impacts rates of papers describing usage of the software. We find that although developers recognize the utility of analyzing data related to the impact or usage of their software, they struggle to find the time or funding to support such analyses. We also find that infrastructure such as social media presence, more in-depth documentation, the presence of software health metrics, and clear information on how to contact developers seem to be associated with increased usage rates. Our findings can help scientific software developers make the most out of the evaluations of their software so that they can more fully benefit from such assessments.