Measuring vaccine protection when the population is mostly vaccinated.

OBJECTIVES: This study aims to address limitations in assessing vaccine protection using the classical vaccine effectiveness (VE) measure, especially in contexts where a significant portion of the population is already vaccinated or infected. STUDY DESIGN AND SETTING: We propose using the adjusted number of cases (ANC) as a building block for deriving vaccine effectiveness measures. This approach accounts for biases arising from small and unrepresentative unvaccinated reference groups with incomplete data. We demonstrate the use of these measures for assessing the protection conferred by a booster dose against severe COVID-19 using data from Israel. RESULTS: The use of ANC and the derived measures reveals a more comprehensive understanding of the complex immunity landscape compared to traditional VE measures. This approach enables meaningful comparisons between different vaccination categories and provides insights to inform policy decisions. CONCLUSION: In situations with widespread vaccination and prior infections, traditional VE measures can be limited in their informative value. Using the ANC offers a more robust and insightful assessment of vaccine effectiveness. A demonstration of the evaluation of booster dose protection against severe COVID-19 in Israel underscores the importance of adopting complementary measures to guide public health strategies.

Protection against Omicron BA.1/BA.2 severe disease 0-7 months after BNT162b2 booster.

Following evidence of waning immunity against both infection and severe disease after 2 doses of the BNT162b2 vaccine, Israel began administering a 3rd BNT162b2 dose (booster) in July 2021. Recent studies showed that the 3rd dose provides a much lower protection against infection with the Omicron variant compared to the Delta variant and that this protection wanes quickly. However, there is little evidence regarding the protection of the 3rd dose against Omicron (BA.1/BA.2) severe disease. In this study, we estimate the preservation of immunity from severe disease up to 7 months after receiving the booster dose. We calculate rates of severe SARS-CoV-2 disease between groups of individuals aged 60 and above, comparing those who received two doses at least 4 months previously to those who received the 3rd dose (stratified by the time from vaccination), and to those who received a 4th dose. The analysis shows that protection conferred by the 3rd dose against Omicron severe disease did not wane over a 7-month period. Moreover, a 4th dose further improved protection, with a severe disease rate approximately 3-fold lower than in the 3-dose cohorts.

Initial protection against SARS-CoV-2 omicron lineage infection in children and adolescents by BNT162b2 in Israel: an observational study.

BACKGROUND: The BNT162b2 (Pfizer-BioNTech) two-dose vaccine regiment for children and the BNT162b2 third dose for adolescents were approved shortly before the SARS-CoV-2 omicron (B.1.1.529) outbreak in Israel. We aimed to estimate the effects of these vaccines on the rates of confirmed infection against the omicron variant in children and adolescents. METHODS: In this observational cohort study, we extracted data for the omicron-dominated (sublineage BA.1) period. We compared rates of confirmed SARS-CoV-2 infection between children aged 5-10 years 14-35 days after receiving the second vaccine dose with an internal control group of children 3-7 days after receiving the first dose (when the vaccine is not yet effective). Similarly, we compared confirmed infection rates in adolescents aged 12-15 years 14-60 days after receiving a booster dose with an internal control group of adolescents 3-7 days after receiving the booster dose. We used Poisson regression, adjusting for age, sex, socioeconomic status, calendar week, and exposure. FINDINGS: Between Dec 26, 2021, and Jan 8, 2022, we included 1 158 289 participants. In children aged 5-10 years, the adjusted rate of confirmed infection was 2·3 times (95% CI 2·0-2·5) lower in children who received a second dose than in the internal control group. The adjusted infection rate in children who received a second dose was 102 infections per 100 000 risk-days (94-110) compared with 231 infections per 100 000 risk-days (215-248) in the corresponding internal control cohort. In adolescents aged 12-15 years, the booster dose decreased confirmed infection rates by 3·3 times (2·8-4·0) compared with in the internal control group. The adjusted infection rate of the booster cohort was 70 per 100 000 risk-days (60-81) compared with 232 per 100 000 risk-days (212-254) in the internal control cohort. INTERPRETATION: A recent two-dose vaccination regimen with BNT162b2 and a recent booster dose in adolescents substantially reduced the rate of confirmed infection compared with the internal control groups. Future studies are needed to assess the duration of this protection and protection against other outcomes such as paediatric inflammatory multisystem syndrome temporally associated with SARS-CoV-2 and long-COVID. FUNDING: None.

Adaptive search space pruning in complex strategic problems.

People have limited computational resources, yet they make complex strategic decisions over enormous spaces of possibilities. How do people efficiently search spaces with combinatorially branching paths? Here, we study players' search strategies for a winning move in a "k-in-a-row" game. We find that players use scoring strategies to prune the search space and augment this pruning by a "shutter" heuristic that focuses the search on the paths emanating from their previous move. This strong pruning has its costs-both computational simulations and behavioral data indicate that the shutter size is correlated with players' blindness to their opponent's winning moves. However, simulations of the search while varying the shutter size, complexity levels, noise levels, branching factor, and computational limitations indicate that despite its costs, a narrow shutter strategy is the dominant strategy for most of the parameter space. Finally, we show that in the presence of computational limitations, the shutter heuristic enhances the performance of deep learning networks in these end-game scenarios. Together, our findings suggest a novel adaptive heuristic that benefits search in a vast space of possibilities of a strategic game.

Barriers and Facilitators to the Implementation of Family-Centered Technology in Complex Care: Feasibility Study.

BACKGROUND: Care coordination is challenging but crucial for children with medical complexity (CMC). Technology-based solutions are increasingly prevalent but little is known about how to successfully deploy them in the care of CMC. OBJECTIVE: The aim of this study was to assess the feasibility and acceptability of GoalKeeper (GK), an internet-based system for eliciting and monitoring family-centered goals for CMC, and to identify barriers and facilitators to implementation. METHODS: We used the Consolidated Framework for Implementation Research (CFIR) to explore the barriers and facilitators to the implementation of GK as part of a clinical trial of GK in ambulatory clinics at a children's hospital (NCT03620071). The study was conducted in 3 phases: preimplementation, implementation (trial), and postimplementation. For the trial, we recruited providers at participating clinics and English-speaking parents of CMC<12 years of age with home internet access. All participants used GK during an initial clinic visit and for 3 months after. We conducted preimplementation focus groups and postimplementation semistructured exit interviews using the CFIR interview guide. Participant exit surveys assessed GK feasibility and acceptability on a 5-point Likert scale. For each interview, 3 independent coders used content analysis and serial coding reviews based on the CFIR qualitative analytic plan and assigned quantitative ratings to each CFIR construct (-2 strong barrier to +2 strong facilitator). RESULTS: Preimplementation focus groups included 2 parents (1 male participant and 1 female participant) and 3 providers (1 in complex care, 1 in clinical informatics, and 1 in neurology). From focus groups, we developed 3 implementation strategies: education (parents: 5-minute demo; providers: 30-minute tutorial and 5-minute video on use in a clinic visit; both: instructional manual), tech support (in-person, virtual), and automated email reminders for parents. For implementation (April 1, 2019, to December 21, 2020), we enrolled 11 providers (7 female participants, 5 in complex care) and 35 parents (mean age 38.3, SD 7.8 years; n=28, 80% female; n=17, 49% Caucasian; n=16, 46% Hispanic; and n=30, 86% at least some college). One parent-provider pair did not use GK in the clinic visit, and few used GK after the visit. In 18 parent and 9 provider exit interviews, the key facilitators were shared goal setting, GK's internet accessibility and email reminders (parents), and GK's ability to set long-term goals and use at the end of visits (providers). A key barrier was GK's lack of integration into the electronic health record or patient portal. Most parents (13/19) and providers (6/9) would recommend GK to their peers. CONCLUSIONS: Family-centered technologies like GK are feasible and acceptable for the care of CMC, but sustained use depends on integration into electronic health records. TRIAL REGISTRATION: ClinicalTrials.gov NCT03620071; https://clinicaltrials.gov/ct2/show/NCT03620071.

Tell me something interesting: Clinical utility of machine learning prediction models in the ICU.

In recent years, extensive resources are dedicated to the development of machine learning (ML) based clinical prediction models for intensive care unit (ICU) patients. These models are transforming patient care into a collaborative human-AI task, yet prediction of patient-related events is mostly treated as a standalone goal, without considering clinicians' roles, tasks or workflow in depth. We conducted a mixed methods study aimed at understanding clinicians' needs and expectations from such systems, informing the design of machine learning based prediction models. Our findings identify several areas of focus where clinicians' needs deviate from current practice, including desired prediction targets, timescales stemming from actionability requirements, and concerns regarding the evaluation and trust in these algorithms. Based on our findings, we suggest several design implications for ML-based prediction tools in the ICU.

Protection by a Fourth Dose of BNT162b2 against Omicron in Israel.

BACKGROUND: On January 2, 2022, Israel began administering a fourth dose of BNT162b2 vaccine to persons 60 years of age or older. Data are needed regarding the effect of the fourth dose on rates of confirmed severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection and of severe coronavirus disease 2019 (Covid-19). METHODS: Using the Israeli Ministry of Health database, we extracted data on 1,252,331 persons who were 60 years of age or older and eligible for the fourth dose during a period in which the B.1.1.529 (omicron) variant of SARS-CoV-2 was predominant (January 10 through March 2, 2022). We estimated the rate of confirmed infection and severe Covid-19 as a function of time starting at 8 days after receipt of a fourth dose (four-dose groups) as compared with that among persons who had received only three doses (three-dose group) and among persons who had received a fourth dose 3 to 7 days earlier (internal control group). For the estimation of rates, we used quasi-Poisson regression with adjustment for age, sex, demographic group, and calendar day. RESULTS: The number of cases of severe Covid-19 per 100,000 person-days (unadjusted rate) was 1.5 in the aggregated four-dose groups, 3.9 in the three-dose group, and 4.2 in the internal control group. In the quasi-Poisson analysis, the adjusted rate of severe Covid-19 in the fourth week after receipt of the fourth dose was lower than that in the three-dose group by a factor of 3.5 (95% confidence interval [CI], 2.7 to 4.6) and was lower than that in the internal control group by a factor of 2.3 (95% CI, 1.7 to 3.3). Protection against severe illness did not wane during the 6 weeks after receipt of the fourth dose. The number of cases of confirmed infection per 100,000 person-days (unadjusted rate) was 177 in the aggregated four-dose groups, 361 in the three-dose group, and 388 in the internal control group. In the quasi-Poisson analysis, the adjusted rate of confirmed infection in the fourth week after receipt of the fourth dose was lower than that in the three-dose group by a factor of 2.0 (95% CI, 1.9 to 2.1) and was lower than that in the internal control group by a factor of 1.8 (95% CI, 1.7 to 1.9). However, this protection waned in later weeks. CONCLUSIONS: Rates of confirmed SARS-CoV-2 infection and severe Covid-19 were lower after a fourth dose of BNT162b2 vaccine than after only three doses. Protection against confirmed infection appeared short-lived, whereas protection against severe illness did not wane during the study period.

Protection following BNT162b2 booster in adolescents substantially exceeds that of a fresh 2-dose vaccine.

Israel began administering a BNT162b2 booster dose to restore protection following the waning of the 2-dose vaccine. Biological studies have shown that a "fresh" booster dose leads to increased antibody levels compared to a fresh 2-dose vaccine, which may suggest increased effectiveness. To compare the real-world effectiveness of a fresh (up to 60 days) booster dose with that of a fresh 2-dose vaccine, we took advantage of a quasi-experimental study that compares populations that were eligible to receive the vaccine at different times due to age-dependent policies. Specifically, we compared the confirmed infection rates in adolescents aged 12-14 (215,653 individuals) who received the 2-dose vaccine and in adolescents aged 16-18 (103,454 individuals) who received the booster dose. Our analysis shows that the confirmed infection rate was lower by a factor of 3.7 (95% CI: 2.7 to 5.2) in the booster group.

Parent Perspectives in Shared Decision-Making for Children With Medical Complexity.

OBJECTIVE: Shared decision-making (SDM) may improve outcomes for children with medical complexity (CMC). CMC have lower rates of SDM than other children, but little is known about how to improve SDM for CMC. The objective of this study is to describe parent perspectives of SDM for CMC and identify opportunities to improve elements of SDM specific to this vulnerable population. METHODS: Interviews with parents of CMC explored SDM preferences and experiences. Eligible parents were ≥18 years old, English- or Spanish-speaking, with a CMC <12 years old. Interviews were recorded, transcribed, and analyzed by independent coders for shared themes using modified grounded theory. Codes were developed using an iterative process, beginning with open-coding of a subset of transcripts followed by discussion with all team members, and distillation into preliminary codes. Subsequent coding reviews were conducted until no new themes emerged and existing themes were fully explored. RESULTS: We conducted interviews with 32 parents (27 in English, mean parent age 34 years, standard deviation = 7; mean child age 4 years, standard deviation = 4; 50% with household income <$50,000, 47% with low health literacy) in inpatient and outpatient settings. Three categories of themes emerged: participant, knowledge, and context. Key opportunities to improve SDM included: providing a shared decision timeline, purposefully integrating patient preferences and values, and addressing uncertainty in decisions. CONCLUSION: Our results provide insight into parent experiences with SDM for CMC. We identified unique opportunities to improve SDM for CMC that will inform future research and interventions to improve SDM for CMC.

Exploring Computational User Models for Agent Policy Summarization.

AI agents support high stakes decision-making processes from driving cars to prescribing drugs, making it increasingly important for human users to understand their behavior. Policy summarization methods aim to convey strengths and weaknesses of such agents by demonstrating their behavior in a subset of informative states. Some policy summarization methods extract a summary that optimizes the ability to reconstruct the agent's policy under the assumption that users will deploy inverse reinforcement learning. In this paper, we explore the use of different models for extracting summaries. We introduce an imitation learning-based approach to policy summarization; we demonstrate through computational simulations that a mismatch between the model used to extract a summary and the model used to reconstruct the policy results in worse reconstruction quality; and we demonstrate through a human-subject study that people use different models to reconstruct policies in different contexts, and that matching the summary extraction model to these can improve performance. Together, our results suggest that it is important to carefully consider user models in policy summarization.

Toward Robust policy Summarization: Extended Abstract.

AI agents are being developed to help people with high stakes decision-making processes from driving cars to prescribing drugs. It is therefore becoming increasingly important to develop "explainable AI" methods that help people understand the behavior of such agents. Summaries of agent policies can help human users anticipate agent behavior and facilitate more effective collaboration. Prior work has framed agent summarization as a machine teaching problem where examples of agent behavior are chosen to maximize reconstruction quality under the assumption that people do inverse reinforcement learning to infer an agent's policy from demonstrations. We compare summaries generated under this assumption to summaries generated under the assumption that people use imitation learning. We show through simulations that in some domains, there exist summaries that produce high-quality reconstructions under different models, but in other domains, only matching the summary extraction model to the reconstruction model produces high-quality reconstructions. These results highlight the importance of assuming correct computational models for how humans extrapolate from a summary, suggesting human-in-the-loop approaches to summary extraction.

The more the merrier? Increasing group size may be detrimental to decision-making performance in nominal groups.

Demonstrability-the extent to which group members can recognize a correct solution to a problem-has a significant effect on group performance. However, the interplay between group size, demonstrability and performance is not well understood. This paper addresses these gaps by studying the joint effect of two factors-the difficulty of solving a problem and the difficulty of verifying the correctness of a solution-on the ability of groups of varying sizes to converge to correct solutions. Our empirical investigations use problem instances from different computational complexity classes, NP-Complete (NPC) and PSPACE-complete (PSC), that exhibit similar solution difficulty but differ in verification difficulty. Our study focuses on nominal groups to isolate the effect of problem complexity on performance. We show that NPC problems have higher demonstrability than PSC problems: participants were significantly more likely to recognize correct and incorrect solutions for NPC problems than for PSC problems. We further show that increasing the group size can actually decrease group performance for some problems of low demonstrability. We analytically derive the boundary that distinguishes these problems from others for which group performance monotonically improves with group size. These findings increase our understanding of the mechanisms that underlie group problem-solving processes, and can inform the design of systems and processes that would better facilitate collective decision-making.

Economic games on the internet: the effect of $1 stakes.

Online labor markets such as Amazon Mechanical Turk (MTurk) offer an unprecedented opportunity to run economic game experiments quickly and inexpensively. Using Mturk, we recruited 756 subjects and examined their behavior in four canonical economic games, with two payoff conditions each: a stakes condition, in which subjects' earnings were based on the outcome of the game (maximum earnings of $1); and a no-stakes condition, in which subjects' earnings are unaffected by the outcome of the game. Our results demonstrate that economic game experiments run on MTurk are comparable to those run in laboratory settings, even when using very low stakes.