Putting computational models of immunity to the test - an invited challenge to predict B. pertussis vaccination outcomes.

Systems vaccinology studies have been used to build computational models that predict individual vaccine responses and identify the factors contributing to differences in outcome. Comparing such models is challenging due to variability in study designs. To address this, we established a community resource to compare models predicting B. pertussis booster responses and generate experimental data for the explicit purpose of model evaluation. We here describe our second computational prediction challenge using this resource, where we benchmarked 49 algorithms from 53 scientists. We found that the most successful models stood out in their handling of nonlinearities, reducing large feature sets to representative subsets, and advanced data preprocessing. In contrast, we found that models adopted from literature that were developed to predict vaccine antibody responses in other settings performed poorly, reinforcing the need for purpose-built models. Overall, this demonstrates the value of purpose-generated datasets for rigorous and open model evaluations to identify features that improve the reliability and applicability of computational models in vaccine response prediction.

Th1 polarization in Bordetella pertussis vaccine responses is maintained through a positive feedback loop.

Outbreaks of Bordetella pertussis (BP), the causative agent of whooping cough, continue despite broad vaccination coverage and have been increasing since vaccination switched from whole-BP (wP) to acellular BP (aP) vaccines. wP vaccination has been associated with more durable protective immunity and an induced Th1 polarized memory T cell response. Here, a multi-omics approach was applied to profile the immune response of 30 wP and 31 aP-primed individuals and identify correlates of T cell polarization before and after Tdap booster vaccination. We found that transcriptional changes indicating an interferon response on day 1 post-booster along with elevated plasma concentrations of IFN-γ and interferon-induced chemokines that peaked at day 1-3 post-booster correlated best with the Th1 polarization of the vaccine-induced memory T cell response on day 28. Our studies suggest that wP-primed individuals maintain their Th1 polarization through this early memory interferon response. This suggests that stimulating the interferon pathway during vaccination could be an effective strategy to elicit a predominant Th1 response in aP-primed individuals that protects better against infection.

A multi-omics systems vaccinology resource to develop and test computational models of immunity.

Systems vaccinology studies have identified factors affecting individual vaccine responses, but comparing these findings is challenging due to varying study designs. To address this lack of reproducibility, we established a community resource for comparing Bordetella pertussis booster responses and to host annual contests for predicting patients' vaccination outcomes. We report here on our experiences with the "dry-run" prediction contest. We found that, among 20+ models adopted from the literature, the most successful model predicting vaccination outcome was based on age alone. This confirms our concerns about the reproducibility of conclusions between different vaccinology studies. Further, we found that, for newly trained models, handling of baseline information on the target variables was crucial. Overall, multiple co-inertia analysis gave the best results of the tested modeling approaches. Our goal is to engage community in these prediction challenges by making data and models available and opening a public contest in August 2024.

A systems vaccinology resource to develop and test computational models of immunity.

Computational models that predict an individual's response to a vaccine offer the potential for mechanistic insights and personalized vaccination strategies. These models are increasingly derived from systems vaccinology studies that generate immune profiles from human cohorts pre- and post-vaccination. Most of these studies involve relatively small cohorts and profile the response to a single vaccine. The ability to assess the performance of the resulting models would be improved by comparing their performance on independent datasets, as has been done with great success in other areas of biology such as protein structure predictions. To transfer this approach to system vaccinology studies, we established a prototype platform that focuses on the evaluation of Computational Models of Immunity to Pertussis Booster vaccinations (CMI-PB). A community resource, CMI-PB generates experimental data for the explicit purpose of model evaluation, which is performed through a series of annual data releases and associated contests. We here report on our experience with the first such 'dry run' for a contest where the goal was to predict individual immune responses based on pre-vaccination multi-omic profiles. Over 30 models adopted from the literature were tested, but only one was predictive, and was based on age alone. The performance of new models built using CMI-PB training data was much better, but varied significantly based on the choice of pre-vaccination features used and the model building strategy. This suggests that previously published models developed for other vaccines do not generalize well to Pertussis Booster vaccination. Overall, these results reinforced the need for comparative analysis across models and datasets that CMI-PB aims to achieve. We are seeking wider community engagement for our first public prediction contest, which will open in early 2024.