Integrating decision modeling and machine learning to inform treatment stratification.

There is increasing interest in moving away from "one size fits all (OSFA)" approaches toward stratifying treatment decisions. Understanding how expected effectiveness and cost-effectiveness varies with patient covariates is a key aspect of stratified decision making. Recently proposed machine learning (ML) methods can learn heterogeneity in outcomes without pre-specifying subgroups or functional forms, enabling the construction of decision rules ('policies') that map individual covariates into a treatment decision. However, these methods do not yet integrate ML estimates into a decision modeling framework in order to reflect long-term policy-relevant outcomes and synthesize information from multiple sources. In this paper, we propose a method to integrate ML and decision modeling, when individual patient data is available to estimate treatment-specific survival time. We also propose a novel implementation of policy tree algorithms to define subgroups using decision model output. We demonstrate these methods using the SPRINT (Systolic Blood Pressure Intervention Trial), comparing outcomes for "standard" and "intensive" blood pressure targets. We find that including ML into a decision model can impact the estimate of incremental net health benefit (INHB) for OSFA policies. We also find evidence that stratifying treatment using subgroups defined by a tree-based algorithm can increase the estimates of the INHB.

Passing the Test: A Model-Based Analysis of Safe School-Reopening Strategies.

BACKGROUND: The COVID-19 pandemic has induced historic educational disruptions. In April 2021, about 40% of U.S. public school students were not offered full-time in-person education. OBJECTIVE: To assess the risk for SARS-CoV-2 transmission in schools. DESIGN: An agent-based network model was developed to simulate transmission in elementary and high school communities, including home, school, and interhousehold interactions. SETTING: School structure was parametrized to reflect average U.S. classrooms, with elementary schools of 638 students and high schools of 1451 students. Daily local incidence was varied from 1 to 100 cases per 100 000 persons. PARTICIPANTS: Students, faculty, staff, and adult household members. INTERVENTION: Isolation of symptomatic individuals, quarantine of an infected individual's contacts, reduced class sizes, alternative schedules, staff vaccination, and weekly asymptomatic screening. MEASUREMENTS: Transmission was projected among students, staff, and families after a single infection in school and over an 8-week quarter, contingent on local incidence. RESULTS: School transmission varies according to student age and local incidence and is substantially reduced with mitigation measures. Nevertheless, when transmission occurs, it may be difficult to detect without regular testing because of the subclinical nature of most children's infections. Teacher vaccination can reduce transmission to staff, and asymptomatic screening improves understanding of local circumstances and reduces transmission. LIMITATION: Uncertainty exists about the susceptibility and infectiousness of children, and precision is low regarding the effectiveness of specific countermeasures, particularly with new variants. CONCLUSION: With controlled community transmission and moderate mitigation, elementary schools can open safety, but high schools require more intensive mitigation. Asymptomatic screening can facilitate reopening at higher local incidence while minimizing transmission risk. PRIMARY FUNDING SOURCE: Centers for Disease Control and Prevention through the Council of State and Territorial Epidemiologists, National Institute of Allergy and Infectious Diseases, National Institute on Drug Abuse, and Facebook.

Fatty acid metabolism and colon cancer protection by dietary methyl donor restriction.

INTRODUCTION: A methyl donor depleted (MDD) diet dramatically suppresses intestinal tumor development in Apc-mutant mice, but the mechanism of this prevention is not entirely clear. OBJECTIVES: We sought to gain insight into the mechanisms of cancer suppression by the MDD diet and to identify biomarkers of cancer risk reduction. METHODS: A plasma metabolomic analysis was performed on ApcΔ14/+ mice maintained on either a methyl donor sufficient (MDS) diet or the protective MDD diet. A group of MDS animals was also pair-fed with the MDD mice to normalize caloric intake, and another group was shifted from an MDD to MDS diet to determine the durability of the metabolic changes. RESULTS: In addition to the anticipated changes in folate one-carbon metabolites, plasma metabolites related to fatty acid metabolism were generally decreased by the MDD diet, including carnitine, acylcarnitines, and fatty acids. Some fatty acid selectivity was observed; the levels of cancer-promoting arachidonic acid and 2-hydroxyglutarate were decreased by the MDD diet, whereas eicosapentaenoic acid (EPA) levels were increased. Machine-learning elastic net analysis revealed a positive association between the fatty acid-related compounds azelate and 7-hydroxycholesterol and tumor development, and a negative correlation with succinate and ß-sitosterol. CONCLUSION: Methyl donor restriction causes dramatic changes in systemic fatty acid metabolism. Regulating fatty acid metabolism through methyl donor restriction favorably effects fatty acid profiles to achieve cancer protection.

Assessing the quality of shared decision making for elective orthopedic surgery across a large healthcare system: cross-sectional survey study.

BACKGROUND: Clinical guidelines recommend engaging patients in shared decision making for common orthopedic procedures; however, limited work has assessed what is occurring in practice. This study assessed the quality of shared decision making for elective hip and knee replacement and spine surgery at four network-affiliated hospitals. METHODS: A cross-sectional sample of 875 adult patients undergoing total hip or knee joint replacement (TJR) for osteoarthritis or spine surgery for lumbar herniated disc or lumbar spinal stenosis was selected. Patients were mailed a survey including measures of Shared Decision Making (SDMP scale) and Informed, Patient-Centered (IPC) decisions. We examined decision-making across sites, surgeons, and conditions, and whether the decision-making measures were associated with better health outcomes. Analyses were adjusted for clustering of patients within surgeons. RESULTS: Six hundred forty-six surveys (74% response rate) were returned with sufficient responses for analysis. Patients who had TJR reported lower SDMP scores than patients who had spine surgery (2.2 vs. 2.8; p < 0.001). Patients who had TJR were more likely to make IPC decisions (OA = 70%, Spine = 41%; p < 0.001). SDMP and IPC scores varied widely across surgeons, but the site was not predictive of SDMP scores or IPC decisions (all p > 0.09). Higher SDMP scores and IPC decisions were associated with larger improvements in global health outcomes for patients who had TJR, but not patients who had spine surgery. CONCLUSIONS: Measures of shared decision making and decision quality varied among patients undergoing common elective orthopedic procedures. Routine measurement of shared decision making provides insight into areas of strength across these different orthopedic conditions as well as areas in need of improvement.

Validation of an electronic coding algorithm to identify the primary indication of orthopedic surgeries from administrative data.

BACKGROUND: Determining the primary indication of a surgical procedure can be useful in identifying patients undergoing elective surgery where shared decision-making is recommended. The purpose of this study was to develop and validate an algorithm to identify patients receiving the following combinations of surgical procedure and primary indication as part of a study to promote shared decision-making: (1) knee arthroplasty to treat knee osteoarthritis (KOA); (2) hip arthroplasty to treat hip osteoarthritis (HOA); (3) spinal surgery to treat lumbar spinal stenosis (SpS); and (4) spinal surgery to treat lumbar herniated disc (HD). METHODS: Consecutive surgical procedures performed by participating spine, hip, and knee surgeons at four sites within an integrated care network were included. Study staff reviewed electronic medical records to ascertain a "gold standard" determination of the procedure and primary indication status. Electronic algorithms consisting of ICD-10 and CPT codes for each combination of procedure and indication were then applied to records for each case. The primary measures of validity for the algorithms were the sensitivity and specificity relative to the gold standard review. RESULTS: Participating surgeons performed 790 procedures included in this study. The sensitivity of the algorithms in determining whether a surgical case represented one of the combinations of procedure and primary indication ranged from 0.70 (HD) to 0.92 (KOA). The specificity ranged from 0.94 (SpS) to 0.99 (HOA, KOA). CONCLUSION: The electronic algorithm was able to identify all four procedure/primary indication combinations of interest with high specificity. Additionally, the sensitivity for the KOA cases was reasonably high. For HOA and the spine conditions, additional work is needed to improve the sensitivity of the algorithm to identify the primary indication for each case.

Screening Strategies to Reduce COVID-19 Mortality in Nursing Homes.

Importance: Nursing home residents continue to bear a disproportionate share of COVID-19 morbidity and mortality, accounting for 9% of all US COVID-19 deaths in 2023, despite comprising only 0.4% of the population. Objective: To evaluate the cost-effectiveness of screening strategies in reducing COVID-19 mortality in nursing homes. Design and Setting: An agent-based model was developed to simulate SARS-CoV-2 transmission in the nursing home setting. Parameters were determined using SARS-CoV-2 virus data and COVID-19 data from the Centers for Medicare & Medicaid Services and US Centers for Disease Control and Prevention that were published between 2020 and 2023, as well as data on nursing homes published between 2010 and 2023. The model used in this study simulated interactions and SARS-CoV-2 transmission between residents, staff, and visitors in a nursing home setting. The population used in the simulation model was based on the size of the average US nursing home and recommended staffing levels, with 90 residents, 90 visitors (1 per resident), and 83 nursing staff members. Exposure: Screening frequency (none, weekly, and twice weekly) was varied over 30 days against varying levels of COVID-19 community incidence, booster uptake, and antiviral use. Main Outcomes and Measures: The main outcomes were SARS-CoV-2 infections, detected cases per 1000 tests, and incremental cost of screening per life-year gained. Results: Nursing home interactions were modeled between 90 residents, 90 visitors, and 83 nursing staff over 30 days, completing 4000 to 8000 simulations per parameter combination. The incremental cost-effectiveness ratios of weekly and twice-weekly screening were less than $150 000 per resident life-year with moderate (50 cases per 100 000) and high (100 cases per 100 000) COVID-19 community incidence across low-booster uptake and high-booster uptake levels. When COVID-19 antiviral use reached 100%, screening incremental cost-effectiveness ratios increased to more than $150 000 per life-year when booster uptake was low and community incidence was high. Conclusions and Relevance: The results of this cost-effectiveness analysis suggest that screening may be effective for reducing COVID-19 mortality in nursing homes when COVID-19 community incidence is high and/or booster uptake is low. Nursing home administrators can use these findings to guide planning in the context of widely varying levels of SARS-CoV-2 transmission and intervention measures across the US.

Cost-Effectiveness of Screening Asymptomatic Carotid Stenosis by Atherosclerotic Cardiovascular Risk.

Importance: Extracranial internal carotid artery stenosis (50-99% arterial narrowing) is an important risk factor for ischemic stroke. Yet, the benefits and harms of targeted screening for asymptomatic carotid artery stenosis (ACAS) have not been assessed in population-based studies. Objective: To estimate the cost-effectiveness of one-time, targeted ACAS screening stratified by atherosclerotic cardiovascular disease (ASCVD) risk using the American Heart Association's Pooled Cohort Equations. Design Setting and Participants: We developed a lifetime microsimulation model of ACAS and stroke for a hypothetical cohort representative of US adults aged 50-80 years without stroke history. We used the Cardiovascular Health Study to estimate the probability and severity of ACAS based on individual characteristics (e.g., age, sex, smoking status, blood pressure, and cholesterol). Stroke risks were functions of these characteristics and ACAS severity. In the model, individuals testing positive for >70% stenosis with Duplex ultrasound and a confirmatory diagnostic test undergo revascularization, which may reduce the risk of stroke but also introduces complication risks. Diagnostic performance parameters, revascularization benefits and risks, utility weights, and costs were estimated from published sources. Cost-effectiveness was assessed from the health care sector perspective using a $100,000/quality-adjusted life year (QALY) threshold. Main Outcomes and Measures: Estimated stroke events prevented, lifetime costs, QALYs, and incremental cost-effectiveness ratios (ICERs) associated with ACAS screening. Costs (2023 USD) and QALYs were discounted at 3% annually. Results: We found that screening individuals with a 10-year ASCVD risk >30% was the most cost-effective strategy, with an ICER of $89,000/QALY. This strategy would make approximately 11.9% of the population eligible for screening, averting an estimated 24,084 strokes. Results were sensitive to variations in the efficacy and complication risk of revascularization. In probabilistic sensitivity analysis, screening those in lower ASCVD risk groups (0-20%) only had a 0.6% chance of being cost-effective. Conclusion and Relevance: A one-time screening may only be cost-effective for adults at a relatively high ASCVD risk. Our findings provide a framework that can be adapted as future clinical trial data continue to improve our understanding of the role of revascularization and intensive medical therapy in contemporary stroke prevention secondary to carotid disease.

Estimated Transmission Outcomes and Costs of SARS-CoV-2 Diagnostic Testing, Screening, and Surveillance Strategies Among a Simulated Population of Primary School Students.

Importance: In addition to illness, the COVID-19 pandemic has led to historic educational disruptions. In March 2021, the federal government allocated $10 billion for COVID-19 testing in US schools. Objective: Costs and benefits of COVID-19 testing strategies were evaluated in the context of full-time, in-person kindergarten through eighth grade (K-8) education at different community incidence levels. Design, Setting, and Participants: An updated version of a previously published agent-based network model was used to simulate transmission in elementary and middle school communities in the United States. Assuming dominance of the delta SARS-CoV-2 variant, the model simulated an elementary school (638 students in grades K-5, 60 staff) and middle school (460 students grades 6-8, 51 staff). Exposures: Multiple strategies for testing students and faculty/staff, including expanded diagnostic testing (test to stay) designed to avoid symptom-based isolation and contact quarantine, screening (routinely testing asymptomatic individuals to identify infections and contain transmission), and surveillance (testing a random sample of students to identify undetected transmission and trigger additional investigation or interventions). Main Outcomes and Measures: Projections included 30-day cumulative incidence of SARS-CoV-2 infection, proportion of cases detected, proportion of planned and unplanned days out of school, cost of testing programs, and childcare costs associated with different strategies. For screening policies, the cost per SARS-CoV-2 infection averted in students and staff was estimated, and for surveillance, the probability of correctly or falsely triggering an outbreak response was estimated at different incidence and attack rates. Results: Compared with quarantine policies, test-to-stay policies are associated with similar model-projected transmission, with a mean of less than 0.25 student days per month of quarantine or isolation. Weekly universal screening is associated with approximately 50% less in-school transmission at one-seventh to one-half the societal cost of hybrid or remote schooling. The cost per infection averted in students and staff by weekly screening is lowest for schools with less vaccination, fewer other mitigation measures, and higher levels of community transmission. In settings where local student incidence is unknown or rapidly changing, surveillance testing may detect moderate to large in-school outbreaks with fewer resources compared with schoolwide screening. Conclusions and Relevance: In this modeling study of a simulated population of primary school students and simulated transmission of COVID-19, test-to-stay policies and/or screening tests facilitated consistent in-person school attendance with low transmission risk across a range of community incidence. Surveillance was a useful reduced-cost option for detecting outbreaks and identifying school environments that would benefit from increased mitigation.

Model-Estimated Association Between Simulated US Elementary School-Related SARS-CoV-2 Transmission, Mitigation Interventions, and Vaccine Coverage Across Local Incidence Levels.

Importance: With recent surges in COVID-19 incidence and vaccine authorization for children aged 5 to 11 years, elementary schools face decisions about requirements for masking and other mitigation measures. These decisions require explicit determination of community objectives (eg, acceptable risk level for in-school SARS-CoV-2 transmission) and quantitative estimates of the consequences of changing mitigation measures. Objective: To estimate the association between adding or removing in-school mitigation measures (eg, masks) and COVID-19 outcomes within an elementary school community at varying student vaccination and local incidence rates. Design, Setting, and Participants: This decision analytic model used an agent-based model to simulate SARS-CoV-2 transmission within a school community, with a simulated population of students, teachers and staff, and their household members (ie, immediate school community). Transmission was evaluated for a range of observed local COVID-19 incidence (0-50 cases per 100 000 residents per day, assuming 33% of all infections detected). The population used in the model reflected the mean size of a US elementary school, including 638 students and 60 educators and staff members in 6 grades with 5 classes per grade. Exposures: Variant infectiousness (representing wild-type virus, Alpha variant, and Delta variant), mitigation effectiveness (0%-100% reduction in the in-school secondary attack rate, representing increasingly intensive combinations of mitigations including masking and ventilation), and student vaccination levels were varied. Main Outcomes and Measures: The main outcomes were (1) probability of at least 1 in-school transmission per month and (2) mean increase in total infections per month among the immediate school community associated with a reduction in mitigation; multiple decision thresholds were estimated for objectives associated with each outcome. Sensitivity analyses on adult vaccination uptake, vaccination effectiveness, and testing approaches (for selected scenarios) were conducted. Results: With student vaccination coverage of 70% or less and moderate assumptions about mitigation effectiveness (eg, masking), mitigation could only be reduced when local case incidence was 14 or fewer cases per 100 000 residents per day to keep the mean additional cases associated with reducing mitigation to 5 or fewer cases per month. To keep the probability of any in-school transmission to less than 50% per month, the local case incidence would have to be 4 or fewer cases per 100 000 residents per day. Conclusions and Relevance: In this study, in-school mitigation measures (eg, masks) and student vaccinations were associated with substantial reductions in transmissions and infections, but the level of reduction varied across local incidence. These findings underscore the potential role for responsive plans that deploy mitigation strategies based on local COVID-19 incidence, vaccine uptake, and explicit consideration of community objectives.

Passing the Test: A model-based analysis of safe school-reopening strategies.

BACKGROUND: The COVID-19 pandemic has induced historic educational disruptions. In December 2020, at least two-thirds of US public school students were not attending full-time in-person education. The Biden Administration has expressed that reopening schools is a priority. OBJECTIVE: To compare risks of SARS-COV-2 transmission in schools across different school-based prevention strategies and levels of community transmission. DESIGN: We developed an agent-based network model to simulate transmission in elementary and high school communities, including home, school, and inter-household interactions. SETTING: We parameterized school structure based on average US classrooms, with elementary schools of 638 students and high schools of 1,451 students. We varied daily community incidence from 1 to 100 cases per 100,000 population. Patients (or Participants). We simulated students, faculty/staff, and adult household members. INTERVENTIONS: We evaluated isolation of symptomatic individuals, quarantine of an infected individual's contacts, reduced class sizes, alternative schedules, staff vaccination, and weekly asymptomatic screening. MEASUREMENTS: We projected transmission among students, staff and families during one month following introduction of a single infection into a school. We also calculated the number of infections expected for a typical 8-week quarter, contingent on community incidence rate. RESULTS: School transmission risk varies according to student age and community incidence and is substantially reduced with effective, consistent mitigation measures. Nevertheless, when transmission occurs, it may be difficult to detect without regular, frequent testing due to the subclinical nature of most infections in children. Teacher vaccination can reduce transmission to staff, while asymptomatic screening both improves understanding of local circumstances and reduces transmission, facilitating five-day schedules at full classroom capacity. LIMITATIONS: There is uncertainty about susceptibility and infectiousness of children and low precision regarding the effectiveness of specific prevention measures, particularly with emergence of new variants. CONCLUSION: With controlled community transmission and moderate school-based prevention measures, elementary schools can open with few in-school transmissions, while high schools require more intensive mitigation. Asymptomatic screening can both reduce transmission and provide useful information for decision-makers.

Model-estimated relationship between elementary school-related SARS-CoV-2 transmission, mitigation interventions, and vaccination coverage across community incidence levels.

BACKGROUND: While CDC guidance for K-12 schools recommends indoor masking regardless of vaccination status, final decisions about masking in schools will be made at the local and state level. The impact of the removal of mask restrictions, however, on COVID-19 outcomes for elementary students, educators/staff, and their households is not well known. METHODS: We used a previously published agent-based dynamic transmission model of SARS-CoV-2 in K-12 schools to simulate an elementary school with 638 students across 12 scenarios: combinations of three viral infectiousness levels (reflecting wild-type virus, alpha variant, and delta variant) and four student vaccination levels (0%, 25%, 50% and 70% coverage). For each scenario, we varied observed community COVID-19 incidence (0 to 50 cases/100,000 people/day) and mitigation effectiveness (0-100% reduction to in-school secondary attack rate), and evaluated two outcomes over a 30 day period: (1) the probability of at least one in-school transmission, and (2) average increase in total infections among students, educators/staff, and their household members associated with moving from more to less intensive mitigation measures. RESULTS: Over 30 days in the simulated elementary school, the probability of at least one in-school SARS-CoV-2 transmission and the number of estimated additional infections in the immediate school community associated with changes in mitigation measures varied widely. In one scenario with the delta variant and no student vaccination, assuming that baseline mitigation measures of simple ventilation and handwashing reduce the secondary attack rate by 40%, if decision-makers seek to keep the monthly probability of an in-school transmission below 50%, additional mitigation (e.g., masking) would need to be added at a community incidence of approximately 2/100,000/day. Once students are vaccinated, thresholds shift substantially higher. LIMITATIONS: The interpretation of model results should be limited by the uncertainty in many of the parameters, including the effectiveness of individual mitigation interventions and vaccine efficacy against the delta variant, and the limited scope of the model beyond the school community. Additionally, the assumed case detection rate (33% of cases detected) may be too high in areas with decreased testing capacity. CONCLUSION: Despite the assumption of high adult vaccination, the risks of both in-school SARS-CoV-2 transmission and resulting infections among students, educators/staff, and their household members remain high when the delta variant predominates and students are unvaccinated. Mitigation measures or vaccinations for students can substantially reduce these risks. These findings underscore the potential role for responsive plans, where mitigation is deployed based on local COVID-19 incidence and vaccine uptake.

SARS-CoV-2 testing strategies to contain school-associated transmission: model-based analysis of impact and cost of diagnostic testing, screening, and surveillance.

Background: In March 2021, the Biden administration allocated $10 billion for COVID-19 testing in schools. We evaluate the costs and benefits of testing strategies to reduce the infection risks of full-time in-person K-8 education at different levels of community incidence. Methods: We used an agent-based network model to simulate transmission in elementary and middle school communities, parameterized to a US school structure and assuming dominance of the delta COVID-19 variant. We assess the value of different strategies for testing students and faculty/staff, including expanded diagnostic testing ("test to stay" policies that take the place of isolation for symptomatic students or quarantine for exposed classrooms); screening (routinely testing asymptomatic individuals to identify infections and contain transmission); and surveillance (testing a random sample of students to signaling undetected transmission and trigger additional investigation or interventions). Main outcome measures: We project 30-day cumulative incidence of SARS-CoV-2 infection; proportion of cases detected; proportion of planned and unplanned days out of school; and the cost of testing programs and of childcare costs associated with different strategies. For screening policies, we further estimate cost per SARS-CoV-2 infection averted in students and staff, and for surveillance, probability of correctly or falsely triggering an outbreak response at different incidence and attack rates. Results: Accounting for programmatic and childcare costs, "test to stay" policies achieve similar model-projected transmission to quarantine policies, with reduced overall costs. Weekly universal screening prevents approximately 50% of in-school transmission, with a lower projected societal cost than hybrid or remote schooling. The cost per infection averted in students and staff by weekly screening is lower for older students and schools with higher mitigation and declines as community transmission rises. In settings where local student incidence is unknown or rapidly changing, surveillance may trigger detection of moderate-to-large in-school outbreaks with fewer resources compared to screening. Conclusions: "Test to stay" policies and/or screening tests can facilitate consistent in-person school attendance with low transmission risk across a range of community incidence. Surveillance may be a useful reduced-cost option for detecting outbreaks and identifying school environments that may benefit from increased mitigation.