Factors influencing clinician and patient interaction with machine learning-based risk prediction models: a systematic review.

Machine learning (ML)-based risk prediction models hold the potential to support the health-care setting in several ways; however, use of such models is scarce. We aimed to review health-care professional (HCP) and patient perceptions of ML risk prediction models in published literature, to inform future risk prediction model development. Following database and citation searches, we identified 41 articles suitable for inclusion. Article quality varied with qualitative studies performing strongest. Overall, perceptions of ML risk prediction models were positive. HCPs and patients considered that models have the potential to add benefit in the health-care setting. However, reservations remain; for example, concerns regarding data quality for model development and fears of unintended consequences following ML model use. We identified that public views regarding these models might be more negative than HCPs and that concerns (eg, extra demands on workload) were not always borne out in practice. Conclusions are tempered by the low number of patient and public studies, the absence of participant ethnic diversity, and variation in article quality. We identified gaps in knowledge (particularly views from under-represented groups) and optimum methods for model explanation and alerts, which require future research.

Health Economics Research on Non-surgical Biomedical HIV Prevention: Identifying Gaps and Proposing a Way Forward.

BACKGROUND AND OBJECTIVE: Although HIV prevention science has advanced over the last four decades, evidence suggests that prevention technologies do not always reach their full potential. Critical health economics evidence at appropriate decision-making junctures, particularly early in the development process, could help identify and address potential barriers to the eventual uptake of future HIV prevention products. This paper aims to identify key evidence gaps and propose health economics research priorities for the field of HIV non-surgical biomedical prevention. METHODS: We used a mixed-methods approach with three distinct components: (i) three systematic literature reviews (costs and cost effectiveness, HIV transmission modelling and quantitative preference elicitation) to understand health economics evidence and gaps in the peer-reviewed literature; (ii) an online survey with researchers working in this field to capture gaps in yet-to-be published research (recently completed, ongoing and future); and (iii) a stakeholder meeting with key global and national players in HIV prevention, including experts in product development, health economics research and policy uptake, to uncover further gaps, as well as to elicit views on priorities and recommendations based on (i) and (ii). RESULTS: Gaps in the scope of available health economics evidence were identified. Little research has been carried out on certain key populations (e.g. transgender people and people who inject drugs) and other vulnerable groups (e.g. pregnant people and people who breastfeed). Research is also lacking on preferences of community actors who often influence or enable access to health services among priority populations. Oral pre-exposure prophylaxis, which has been rolled out in many settings, has been studied in depth. However, research on newer promising technologies, such as long-acting pre-exposure prophylaxis formulations, broadly neutralising antibodies and multipurpose prevention technologies, is lacking. Interventions focussing on reducing intravenous and vertical transmission are also understudied. A disproportionate amount of evidence on low- and middle-income countries comes from two countries (South Africa and Kenya); evidence from other countries in sub-Saharan Africa as well as other low- and middle-income countries is needed. Further, data are needed on non-facility-based service delivery modalities, integrated service delivery and ancillary services. Key methodological gaps were also identified. An emphasis on equity and representation of heterogeneous populations was lacking. Research rarely acknowledged the complex and dynamic use of prevention technologies over time. Greater efforts are needed to collect primary data, quantify uncertainty, systematically compare the full range of prevention options available, and validate pilot and modelling data once interventions are scaled up. Clarity on appropriate cost-effectiveness outcome measures and thresholds is also lacking. Lastly, research often fails to reflect policy-relevant questions and approaches. CONCLUSIONS: Despite a large body of health economics evidence on non-surgical biomedical HIV prevention technologies, important gaps in the scope of evidence and methodology remain. To ensure that high-quality research influences key decision-making junctures and facilitates the delivery of prevention products in a way that maximises impact, we make five broad recommendations related to: improved study design, an increased focus on service delivery, greater community and stakeholder engagement, the fostering of an active network of partners across sectors and an enhanced application of research.

Infectious Disease Modelling of HIV Prevention Interventions: A Systematic Review and Narrative Synthesis of Compartmental Models.

BACKGROUND: The HIV epidemic remains a major public health problem. Critical to transmission control are HIV prevention strategies with new interventions continuing to be developed. Mathematical models are important for understanding the potential impact of these interventions and supporting policy decisions. This systematic review aims to answer the following question: when a new HIV prevention intervention is being considered or designed, what information regarding it is necessary to include in a compartmental model to provide useful insights to policy makers? The primary objective of this review is therefore to assess suitability of current compartmental HIV prevention models for informing policy development. METHODS: Articles published in EMBASE, Medline, Econlit, and Global Health were screened. Included studies were identified using permutations of (i) HIV, (ii) pre-exposure prophylaxis (PrEP), circumcision (both voluntary male circumcision [VMMC] and early-infant male circumcision [EIMC]), and vaccination, and (iii) modelling. Data extraction focused on study design, model structure, and intervention incorporation into models. Article quality was assessed using the TRACE (TRAnsparent and Comprehensive Ecological modelling documentation) criteria for mathematical models. RESULTS: Of 837 articles screened, 48 articles were included in the review, with 32 unique mathematical models identified. The substantial majority of studies included PrEP (83%), whilst fewer modelled circumcision (54%), and only a few focussed on vaccination (10%). Data evaluation, implementation verification, and model output corroboration were identified as areas of poorer model quality. Parameters commonly included in the mathematical models were intervention uptake and effectiveness, with additional intervention-specific common parameters identified. We identified key modelling gaps; critically, models insufficiently incorporate multiple interventions acting simultaneously. Additionally, population subgroups were generally poorly represented-with future models requiring improved incorporation of ethnicity and sexual risk group stratification-and many models contained inappropriate data in parameterisation which will affect output accuracy. CONCLUSIONS: This review identified gaps in compartmental models to date and suggests areas of improvement for models focusing on new prevention interventions. Resolution of such gaps within future models will ensure greater robustness and transparency, and enable more accurate assessment of the impact that new interventions may have, thereby providing more meaningful guidance to policy makers.

Costs and Cost-Effectiveness of Biomedical, Non-Surgical HIV Prevention Interventions: A Systematic Literature Review.

BACKGROUND: Considerable evidence on the costs and cost-effectiveness of biomedical, non-surgical interventions to prevent human immunodeficiency virus (HIV) transmission has been generated over the last decade. This study aims to synthesize findings and identify remaining knowledge gaps to suggest future research priorities. METHODS: A systematic literature review was carried out in August 2020 using the MEDLINE, Embase, Global Health and EconLit databases to retrieve economic evaluations and costing studies of oral pre-exposure prophylaxis (PrEP), injectable long-acting PrEP, vaginal microbicide rings and gels, HIV vaccines and broadly neutralizing antibodies. Studies reporting costs from the provider or societal perspective were included in the analysis. Those reporting on behavioural methods of prevention, condoms and surgical approaches (voluntary medical male circumcision) were excluded. The quality of reporting of the included studies was assessed using published checklists. RESULTS: We identified 3007 citations, of which 87 studies were retained. Most were set in low- and middle-income countries (LMICs; n = 53) and focused on the costs and/or cost-effectiveness of oral PrEP regimens (n = 70). Model-based economic evaluations were the most frequent study design; only two trial-based cost-effectiveness analyses and nine costing studies were found. Less than half of the studies provided practical details on how the intervention would be delivered by the health system, and only three of these, all in LMICs, explicitly focused on service integration and its implication for delivery costs. 'Real-world' programme delivery mechanisms and costs of intervention delivery were rarely considered. PrEP technologies were generally found to be cost-effective only when targeting high-risk subpopulations. Single-dose HIV vaccines are expected to be cost-effective for all groups despite substantial uncertainty around pricing. CONCLUSIONS: A lack of primary, detailed and updated cost data, including above-service level costs, from a variety of settings makes it difficult to evaluate the cost-effectiveness of specific delivery modes at scale, or to evaluate strategies for services integration. Closing this evidence gap around real-world implementation is vital, not least because the strategies targeting high-risk groups that are recommended by PrEP models may incur substantially higher costs and be of limited practical feasibility in some settings.

Duration of vaccine effectiveness against SARS-CoV-2 infection, hospitalisation, and death in residents and staff of long-term care facilities in England (VIVALDI): a prospective cohort study.

Background: Residents and staff in long-term care facilities have been prioritised for vaccination against SARS-CoV-2, but data on potential waning of vaccine effectiveness and the effect of booster doses in this vulnerable population are scarce. We aimed to evaluate effectiveness of one, two, and three vaccine doses against infection and severe clinical outcomes in staff and residents of long-term care facilities in England over the first year following vaccine roll-out. Methods: The VIVALDI study is a prospective cohort study done in 331 long-term care facilities in England. Residents aged 65 years or older and staff aged 18 years or older were eligible for participation. Participants had routine PCR testing throughout the study period between Dec 8, 2020, and Dec 11, 2021. We retrieved all PCR results and cycle threshold values for PCR-positive samples from routine testing in long-term care facilities, and positive PCR results from clinical testing in hospitals through the UK's COVID-19 Datastore. PCR results were linked to participants using pseudo-identifiers based on individuals' unique UK National Health Service (NHS) numbers, which were also used to retrieve vaccination records from the National Immunisation Management Service, hospitalisation records from NHS England, and deaths data from the Office for National Statistics through the COVID-19 Datastore. In a Cox proportional hazards regression, we estimated vaccine effectiveness against SARS-CoV-2 infection, COVID-19-related hospitalisation, and COVID-19-related death after one, two, and three vaccine doses, separately by previous SARS-CoV-2 exposure. This study is registered with the ISRCTN Registry, ISRCTN 14447421. Findings: 80 186 residents and staff of long-term care facilities had records available for the study period, of whom 15 518 eligible residents and 19 515 eligible staff were included in the analysis. For residents without evidence of previous SARS-CoV-2 exposure, vaccine effectiveness decreased from 61·7% (95% CI 35·1 to 77·4) to 22·0% (-14·9 to 47·0) against infection; from 89·0% (70·6 to 95·9) to 56·3% (30·1 to 72·6) against hospitalisation; and from 96·4% (84·3 to 99·2) to 64·4% (36·1 to 80·1) against death, when comparing 14-83 days after dose two and 84 days or more after dose two. For staff without evidence of previous exposure, vaccine effectiveness against infection decreased slightly from 57·9% (43·1 to 68·9) at 14-83 days after dose two to 42·1% (29·9 to 52·2) at 84 days or more after dose two. There were no hospitalisations or deaths among unexposed staff at 14-83 days, but seven hospitalisations (vaccine effectiveness 91·0% [95% CI 74·3 to 96·8]) and one death were observed at 84 days or more after dose two. High vaccine effectiveness was restored following a third vaccine dose, with vaccine effectiveness in unexposed residents of 72·7% (55·8 to 83·1) against infection, 90·1% (80·6 to 95·0) against hospitalisation, and 97·5% (88·1 to 99·5) against death; and vaccine effectiveness in unexposed staff of 78·2% (70·0 to 84·1) against infection and 95·8% (49·9 to 99·6) against hospitalisation. There were no COVID-19-related deaths among unexposed staff after the third vaccine dose. Interpretation: Our findings showed substantial waning of SARS-CoV-2 vaccine effectiveness against all outcomes in residents of long-term care facilities from 12 weeks after a primary course of ChAdOx1-S or mRNA vaccines. Boosters restored protection, and maximised immunity across all outcomes. These findings show the importance of boosting and the need for ongoing surveillance in this vulnerable cohort. Funding: UK Government Department of Health and Social Care.

Antibody and cellular immune responses following dual COVID-19 vaccination within infection-naive residents of long-term care facilities: an observational cohort study.

Background: Older age and frailty are risk factors for poor clinical outcomes following SARS-CoV-2 infection. As such, COVID-19 vaccination has been prioritised for individuals with these factors, but there is concern that immune responses might be impaired due to age-related immune dysregulation and comorbidity. We aimed to study humoral and cellular responses to COVID-19 vaccines in residents of long-term care facilities (LTCFs). Methods: In this observational cohort study, we assessed antibody and cellular immune responses following COVID-19 vaccination in members of staff and residents at 74 LTCFs across the UK. Staff and residents were eligible for inclusion if it was possible to link them to a pseudo-identifier in the COVID-19 datastore, if they had received two vaccine doses, and if they had given a blood sample 6 days after vaccination at the earliest. There were no comorbidity exclusion criteria. Participants were stratified by age (<65 years or ≥65 years) and infection status (previous SARS-CoV-2 infection [infection-primed group] or SARS-CoV-2 naive [infection-naive group]). Anticoagulated edetic acid (EDTA) blood samples were assessed and humoral and cellular responses were quantified. Findings: Between Dec 11, 2020, and June 27, 2021, blood samples were taken from 220 people younger than 65 years (median age 51 years [IQR 39-61]; 103 [47%] had previously had a SARS-CoV-2 infection) and 268 people aged 65 years or older of LTCFs (median age 87 years [80-92]; 144 [43%] had a previous SARS-CoV-2 infection). Samples were taken a median of 82 days (IQR 72-100) after the second vaccination. Antibody responses following dual vaccination were strong and equivalent between participants younger then 65 years and those aged 65 years and older in the infection-primed group (median 125 285 Au/mL [1128 BAU/mL] for <65 year olds vs 157 979 Au/mL [1423 BAU/mL] for ≥65 year olds; p=0·47). The antibody response was reduced by 2·4-times (467 BAU/mL; p≤0·0001) in infection-naive people younger than 65 years and 8·1-times (174 BAU/mL; p≤0·0001) in infection-naive residents compared with their infection-primed counterparts. Antibody response was 2·6-times lower in infection-naive residents than in infection-naive people younger than 65 years (p=0·0006). Impaired neutralisation of delta (1.617.2) variant spike binding was also apparent in infection-naive people younger than 65 years and in those aged 65 years and older. Spike-specific T-cell responses were also significantly enhanced in the infection-primed group. Infection-naive people aged 65 years and older (203 SFU per million [IQR 89-374]) had a 52% lower T-cell response compared with infection-naive people younger than 65 years (85 SFU per million [30-206]; p≤0·0001). Post-vaccine spike-specific CD4 T-cell responses displayed single or dual production of IFN-γ and IL-2 were similar across infection status groups, whereas the infection-primed group had an extended functional profile with TNFα and CXCL10 production. Interpretation: These data reveal suboptimal post-vaccine immune responses within infection-naive residents of LTCFs, and they suggest the need for optimisation of immune protection through the use of booster vaccination. Funding: UK Government Department of Health and Social Care.

Severe Acute Respiratory Syndrome Coronavirus 2 Anti-Spike Antibody Levels Following Second Dose of ChAdOx1 nCov-19 or BNT162b2 Vaccine in Residents of Long-term Care Facilities in England (VIVALDI).

General population studies have shown strong humoral response following severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) vaccination with subsequent waning of anti-spike antibody levels. Vaccine-induced immune responses are often attenuated in frail and older populations, but published data are scarce. We measured SARS-CoV-2 anti-spike antibody levels in long-term care facility residents and staff following a second vaccination dose with Oxford-AstraZeneca or Pfizer-BioNTech. Vaccination elicited robust antibody responses in older residents, suggesting comparable levels of vaccine-induced immunity to that in the general population. Antibody levels are higher after Pfizer-BioNTech vaccination but fall more rapidly compared to Oxford-AstraZeneca recipients and are enhanced by prior infection in both groups.

Prevalence and duration of detectable SARS-CoV-2 nucleocapsid antibodies in staff and residents of long-term care facilities over the first year of the pandemic (VIVALDI study): prospective cohort study in England.

BACKGROUND: Long-term care facilities (LTCFs) have reported high SARS-CoV-2 infection rates and related mortality, but the proportion of infected people among those who have survived, and duration of the antibody response to natural infection, is unknown. We determined the prevalence and stability of nucleocapsid antibodies (the standard assay for detection of previous infection) in staff and residents in LTCFs in England. METHODS: This was a prospective cohort study of residents 65 years or older and of staff 65 years or younger in 201 LTCFs in England between March 1, 2020, and May 7, 2021. Participants were linked to a unique pseudo-identifier based on their UK National Health Service identification number. Serial blood samples were tested for IgG antibodies against SARS-CoV-2 nucleocapsid protein using the Abbott ARCHITECT i-system (Abbott, Maidenhead, UK) immunoassay. Primary endpoints were prevalence and cumulative incidence of antibody positivity, which were weighted to the LTCF population. Incidence rate of loss of antibodies (seroreversion) was estimated from Kaplan-Meier curves. FINDINGS: 9488 samples were included, 8636 (91·0%) of which could be individually linked to 1434 residents and 3288 staff members. The cumulative incidence of nucleocapsid seropositivity was 34·6% (29·6-40·0) in residents and 26·1% (23·0-29·5) in staff over 11 months. 239 (38·6%) residents and 503 women (81·3%) were included in the antibody-waning analysis, and median follow-up was 149 days (IQR 107-169). The incidence rate of seroreversion was 2·1 per 1000 person-days at risk, and median time to reversion was 242·5 days. INTERPRETATION: At least a quarter of staff and a third of surviving residents were infected with SAR-CoV-2 during the first two waves of the pandemic in England. Nucleocapsid-specific antibodies often become undetectable within the first year following infection, which is likely to lead to marked underestimation of the true proportion of people with previous infection. Given that natural infection might act to boost vaccine responses, better assays to identify natural infection should be developed. FUNDING: UK Government Department of Health and Social Care.

Robust SARS-CoV-2-specific and heterologous immune responses in vaccine-naïve residents of long-term care facilities who survive natural infection.

We studied humoral and cellular immunity against severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) in 152 long-term care facility staff and 124 residents over a prospective 4-month period shortly after the first wave of infection in England. We show that residents of long-term care facilities developed high and stable levels of antibodies against spike protein and receptor-binding domain. Nucleocapsid-specific responses were also elevated but waned over time. Antibodies showed stable and equivalent levels of functional inhibition against spike-angiotensin-converting enzyme 2 binding in all age groups with comparable activity against viral variants of concern. SARS-CoV-2 seropositive donors showed high levels of antibodies to other beta-coronaviruses but serostatus did not impact humoral immunity to influenza or other respiratory syncytial viruses. SARS-CoV-2-specific cellular responses were similar across all ages but virus-specific populations showed elevated levels of activation in older donors. Thus, survivors of SARS-CoV-2 infection show a robust and stable immunity against the virus that does not negatively impact responses to other seasonal viruses.

Reinfection with new variants of SARS-CoV-2 after natural infection: a prospective observational cohort in 13 care homes in England.

BACKGROUND: Understanding the duration of protection and risk of reinfection after natural infection is crucial to planning COVID-19 vaccination for at-risk groups, including care home residents, particularly with the emergence of more transmissible variants. We report on the duration, neutralising activity, and protection against the alpha variant of previous SARS-CoV-2 infection in care home residents and staff infected more than 6 months previously. METHODS: We did this prospective observational cohort surveillance in 13 care homes in Greater London, England. All staff and residents were included. Staff and residents had regular nose and throat screening for SARS-CoV-2 by RT-PCR according to national guidelines, with ad hoc testing of symptomatic individuals. From January, 2021, antigen lateral flow devices were also used, but positive tests still required RT-PCR confirmation. Staff members took the swab samples for themselves and the residents. The primary outcome was SARS-CoV-2 RT-PCR positive primary infection or reinfection in previously infected individuals, as determined by previous serological testing and screening or diagnostic RT-PCR results. Poisson regression and Cox proportional hazards models were used to estimate protective effectiveness of previous exposure. SARS-CoV-2 spike, nucleoprotein, and neutralising antibodies were assessed at multiple timepoints as part of the longitudinal follow-up. FINDINGS: Between April 10 and Aug 3, 2020, we recruited and tested 1625 individuals (933 staff and 692 residents). 248 participants were lost to follow-up (123 staff and 125 residents) and 1377 participants were included in the follow-up period to Jan 31, 2021 (810 staff and 567 residents). There were 23 reinfections (ten confirmed, eight probable, five possible) in 656 previously infected individuals (366 staff and 290 residents), compared with 165 primary infections in 721 susceptible individuals (444 staff and 277 residents). Those with confirmed reinfections had no or low neutralising antibody concentration before reinfection, with boosting of titres after reinfection. Kinetics of binding and neutralising antibodies were similar in older residents and younger staff. INTERPRETATION: SARS-CoV-2 reinfections were rare in older residents and younger staff. Protection from SARS-CoV-2 was sustained for longer than 9 months, including against the alpha variant. Reinfection was associated with no or low neutralising antibody before reinfection, but significant boosting occurred on reinfection. FUNDING: Public Health England.

Changes in COVID-19 outbreak severity and duration in long-term care facilities following vaccine introduction, England, November 2020 to June 2021.

We describe the impact of changing epidemiology and vaccine introduction on characteristics of COVID-19 outbreaks in 330 long-term care facilities (LTCF) in England between November 2020 and June 2021. As vaccine coverage in LTCF increased and national incidence declined, the total number of outbreaks and outbreak severity decreased across the LTCF. The number of infected cases per outbreak decreased by 80.6%, while the proportion of outbreaks affecting staff only increased. Our study supports findings of vaccine effectiveness in LTCF.

Vaccine effectiveness of the first dose of ChAdOx1 nCoV-19 and BNT162b2 against SARS-CoV-2 infection in residents of long-term care facilities in England (VIVALDI): a prospective cohort study.

BACKGROUND: The effectiveness of SARS-CoV-2 vaccines in older adults living in long-term care facilities is uncertain. We investigated the protective effect of the first dose of the Oxford-AstraZeneca non-replicating viral-vectored vaccine (ChAdOx1 nCoV-19; AZD1222) and the Pfizer-BioNTech mRNA-based vaccine (BNT162b2) in residents of long-term care facilities in terms of PCR-confirmed SARS-CoV-2 infection over time since vaccination. METHODS: The VIVALDI study is a prospective cohort study that commenced recruitment on June 11, 2020, to investigate SARS-CoV-2 transmission, infection outcomes, and immunity in residents and staff in long-term care facilities in England that provide residential or nursing care for adults aged 65 years and older. In this cohort study, we included long-term care facility residents undergoing routine asymptomatic SARS-CoV-2 testing between Dec 8, 2020 (the date the vaccine was first deployed in a long-term care facility), and March 15, 2021, using national testing data linked within the COVID-19 Datastore. Using Cox proportional hazards regression, we estimated the relative hazard of PCR-positive infection at 0-6 days, 7-13 days, 14-20 days, 21-27 days, 28-34 days, 35-48 days, and 49 days and beyond after vaccination, comparing unvaccinated and vaccinated person-time from the same cohort of residents, adjusting for age, sex, previous infection, local SARS-CoV-2 incidence, long-term care facility bed capacity, and clustering by long-term care facility. We also compared mean PCR cycle threshold (Ct) values for positive swabs obtained before and after vaccination. The study is registered with ISRCTN, number 14447421. FINDINGS: 10 412 care home residents aged 65 years and older from 310 LTCFs were included in this analysis. The median participant age was 86 years (IQR 80-91), 7247 (69·6%) of 10 412 residents were female, and 1155 residents (11·1%) had evidence of previous SARS-CoV-2 infection. 9160 (88·0%) residents received at least one vaccine dose, of whom 6138 (67·0%) received ChAdOx1 and 3022 (33·0%) received BNT162b2. Between Dec 8, 2020, and March 15, 2021, there were 36 352 PCR results in 670 628 person-days, and 1335 PCR-positive infections (713 in unvaccinated residents and 612 in vaccinated residents) were included. Adjusted hazard ratios (HRs) for PCR-positive infection relative to unvaccinated residents declined from 28 days after the first vaccine dose to 0·44 (95% CI 0·24-0·81) at 28-34 days and 0·38 (0·19-0·77) at 35-48 days. Similar effect sizes were seen for ChAdOx1 (adjusted HR 0·32, 95% CI 0·15-0·66) and BNT162b2 (0·35, 0·17-0·71) vaccines at 35-48 days. Mean PCR Ct values were higher for infections that occurred at least 28 days after vaccination than for those occurring before vaccination (31·3 [SD 8·7] in 107 PCR-positive tests vs 26·6 [6·6] in 552 PCR-positive tests; p<0·0001). INTERPRETATION: Single-dose vaccination with BNT162b2 and ChAdOx1 vaccines provides substantial protection against infection in older adults from 4-7 weeks after vaccination and might reduce SARS-CoV-2 transmission. However, the risk of infection is not eliminated, highlighting the ongoing need for non-pharmaceutical interventions to prevent transmission in long-term care facilities. FUNDING: UK Government Department of Health and Social Care.