Using generative AI to investigate medical imagery models and datasets.

BACKGROUND: AI models have shown promise in performing many medical imaging tasks. However, our ability to explain what signals these models have learned is severely lacking. Explanations are needed in order to increase the trust of doctors in AI-based models, especially in domains where AI prediction capabilities surpass those of humans. Moreover, such explanations could enable novel scientific discovery by uncovering signals in the data that aren't yet known to experts. METHODS: In this paper, we present a workflow for generating hypotheses to understand which visual signals in images are correlated with a classification model's predictions for a given task. This approach leverages an automatic visual explanation algorithm followed by interdisciplinary expert review. We propose the following 4 steps: (i) Train a classifier to perform a given task to assess whether the imagery indeed contains signals relevant to the task; (ii) Train a StyleGAN-based image generator with an architecture that enables guidance by the classifier ("StylEx"); (iii) Automatically detect, extract, and visualize the top visual attributes that the classifier is sensitive towards. For visualization, we independently modify each of these attributes to generate counterfactual visualizations for a set of images (i.e., what the image would look like with the attribute increased or decreased); (iv) Formulate hypotheses for the underlying mechanisms, to stimulate future research. Specifically, present the discovered attributes and corresponding counterfactual visualizations to an interdisciplinary panel of experts so that hypotheses can account for social and structural determinants of health (e.g., whether the attributes correspond to known patho-physiological or socio-cultural phenomena, or could be novel discoveries). FINDINGS: To demonstrate the broad applicability of our approach, we present results on eight prediction tasks across three medical imaging modalities-retinal fundus photographs, external eye photographs, and chest radiographs. We showcase examples where many of the automatically-learned attributes clearly capture clinically known features (e.g., types of cataract, enlarged heart), and demonstrate automatically-learned confounders that arise from factors beyond physiological mechanisms (e.g., chest X-ray underexposure is correlated with the classifier predicting abnormality, and eye makeup is correlated with the classifier predicting low hemoglobin levels). We further show that our method reveals a number of physiologically plausible, previously-unknown attributes based on the literature (e.g., differences in the fundus associated with self-reported sex, which were previously unknown). INTERPRETATION: Our approach enables hypotheses generation via attribute visualizations and has the potential to enable researchers to better understand, improve their assessment, and extract new knowledge from AI-based models, as well as debug and design better datasets. Though not designed to infer causality, importantly, we highlight that attributes generated by our framework can capture phenomena beyond physiology or pathophysiology, reflecting the real world nature of healthcare delivery and socio-cultural factors, and hence interdisciplinary perspectives are critical in these investigations. Finally, we will release code to help researchers train their own StylEx models and analyze their predictive tasks of interest, and use the methodology presented in this paper for responsible interpretation of the revealed attributes. FUNDING: Google.

Health equity assessment of machine learning performance (HEAL): a framework and dermatology AI model case study.

Background: Artificial intelligence (AI) has repeatedly been shown to encode historical inequities in healthcare. We aimed to develop a framework to quantitatively assess the performance equity of health AI technologies and to illustrate its utility via a case study. Methods: Here, we propose a methodology to assess whether health AI technologies prioritise performance for patient populations experiencing worse outcomes, that is complementary to existing fairness metrics. We developed the Health Equity Assessment of machine Learning performance (HEAL) framework designed to quantitatively assess the performance equity of health AI technologies via a four-step interdisciplinary process to understand and quantify domain-specific criteria, and the resulting HEAL metric. As an illustrative case study (analysis conducted between October 2022 and January 2023), we applied the HEAL framework to a dermatology AI model. A set of 5420 teledermatology cases (store-and-forward cases from patients of 20 years or older, submitted from primary care providers in the USA and skin cancer clinics in Australia), enriched for diversity in age, sex and race/ethnicity, was used to retrospectively evaluate the AI model's HEAL metric, defined as the likelihood that the AI model performs better for subpopulations with worse average health outcomes as compared to others. The likelihood that AI performance was anticorrelated to pre-existing health outcomes was estimated using bootstrap methods as the probability that the negated Spearman's rank correlation coefficient (i.e., "R") was greater than zero. Positive values of R suggest that subpopulations with poorer health outcomes have better AI model performance. Thus, the HEAL metric, defined as p (R >0), measures how likely the AI technology is to prioritise performance for subpopulations with worse average health outcomes as compared to others (presented as a percentage below). Health outcomes were quantified as disability-adjusted life years (DALYs) when grouping by sex and age, and years of life lost (YLLs) when grouping by race/ethnicity. AI performance was measured as top-3 agreement with the reference diagnosis from a panel of 3 dermatologists per case. Findings: Across all dermatologic conditions, the HEAL metric was 80.5% for prioritizing AI performance of racial/ethnic subpopulations based on YLLs, and 92.1% and 0.0% respectively for prioritizing AI performance of sex and age subpopulations based on DALYs. Certain dermatologic conditions were significantly associated with greater AI model performance compared to a reference category of less common conditions. For skin cancer conditions, the HEAL metric was 73.8% for prioritizing AI performance of age subpopulations based on DALYs. Interpretation: Analysis using the proposed HEAL framework showed that the dermatology AI model prioritised performance for race/ethnicity, sex (all conditions) and age (cancer conditions) subpopulations with respect to pre-existing health disparities. More work is needed to investigate ways of promoting equitable AI performance across age for non-cancer conditions and to better understand how AI models can contribute towards improving equity in health outcomes. Funding: Google LLC.

An intentional approach to managing bias in general purpose embedding models.

Advances in machine learning for health care have brought concerns about bias from the research community; specifically, the introduction, perpetuation, or exacerbation of care disparities. Reinforcing these concerns is the finding that medical images often reveal signals about sensitive attributes in ways that are hard to pinpoint by both algorithms and people. This finding raises a question about how to best design general purpose pretrained embeddings (GPPEs, defined as embeddings meant to support a broad array of use cases) for building downstream models that are free from particular types of bias. The downstream model should be carefully evaluated for bias, and audited and improved as appropriate. However, in our view, well intentioned attempts to prevent the upstream components-GPPEs-from learning sensitive attributes can have unintended consequences on the downstream models. Despite producing a veneer of technical neutrality, the resultant end-to-end system might still be biased or poorly performing. We present reasons, by building on previously published data, to support the reasoning that GPPEs should ideally contain as much information as the original data contain, and highlight the perils of trying to remove sensitive attributes from a GPPE. We also emphasise that downstream prediction models trained for specific tasks and settings, whether developed using GPPEs or not, should be carefully designed and evaluated to avoid bias that makes models vulnerable to issues such as distributional shift. These evaluations should be done by a diverse team, including social scientists, on a diverse cohort representing the full breadth of the patient population for which the final model is intended.

Rapid antiretroviral therapy in primary HIV-1 infection enhances immune recovery.

OBJECTIVE: We present findings from a large cohort of individuals treated during primary HIV infection (PHI) and examine the impact of time from HIV-1 acquisition to antiretroviral therapy (ART) initiation on clinical outcomes. We also examine the temporal changes in the demographics of individuals presenting with PHI to inform HIV-1 prevention strategies. METHODS: Individuals who fulfilled the criteria of PHI and started ART within 3 months of confirmed HIV-1 diagnosis were enrolled between 2009 and 2020. Baseline demographics of those diagnosed between 2009 and 2015 (before preexposure prophylaxis (PrEP) and universal ART availability) and 2015-2020 (post-PrEP and universal ART availability) were compared. We examined the factors associated with immune recovery and time to viral suppression. RESULTS: Two hundred four individuals enrolled, 144 from 2009 to 2015 and 90 from 2015 to 2020; median follow-up was 33 months. At PHI, the median age was 33 years; 4% were women, 39% were UK-born, and 84% were MSM. The proportion of UK-born individuals was 47% in 2009-2015, compared with 29% in 2015-2020. There was an association between earlier ART initiation after PHI diagnosis and increased immune recovery; each day that ART was delayed was associated with a lower likelihood of achieving a CD4 + cell count more than 900 cells/µl [hazard ratio 0.99 (95% confidence interval, 95% CI 0.98-0.99), P  = 0.02) and CD4/CD8 more than 1.0 (hazard ratio 0.98 (95% CI 0.97-0.99). CONCLUSION: Early initiation of ART at PHI diagnosis is associated with enhanced immune recovery, providing further evidence to support immediate ART in the context of PHI. Non-UK-born MSM accounts for an increasing proportion of those with primary infection; UK HIV-1 prevention strategies should better target this group.

The value of standards for health datasets in artificial intelligence-based applications.

Artificial intelligence as a medical device is increasingly being applied to healthcare for diagnosis, risk stratification and resource allocation. However, a growing body of evidence has highlighted the risk of algorithmic bias, which may perpetuate existing health inequity. This problem arises in part because of systemic inequalities in dataset curation, unequal opportunity to participate in research and inequalities of access. This study aims to explore existing standards, frameworks and best practices for ensuring adequate data diversity in health datasets. Exploring the body of existing literature and expert views is an important step towards the development of consensus-based guidelines. The study comprises two parts: a systematic review of existing standards, frameworks and best practices for healthcare datasets; and a survey and thematic analysis of stakeholder views of bias, health equity and best practices for artificial intelligence as a medical device. We found that the need for dataset diversity was well described in literature, and experts generally favored the development of a robust set of guidelines, but there were mixed views about how these could be implemented practically. The outputs of this study will be used to inform the development of standards for transparency of data diversity in health datasets (the STANDING Together initiative).

Top ten priorities for global saltmarsh restoration, conservation and ecosystem service research.

Coastal saltmarshes provide globally important ecosystem services including 'blue carbon' sequestration, flood protection, pollutant remediation, habitat provision and cultural value. Large portions of marshes have been lost or fragmented as a result of land reclamation, embankment construction, and pollution. Sea level rise threatens marsh survival by blocking landward migration where coastlines have been developed. Research-informed saltmarsh conservation and restoration efforts are helping to prevent further loss, yet significant knowledge gaps remain. Using a mixed methods approach, this paper identifies ten research priorities through an online questionnaire and a residential workshop attended by an international, multi-disciplinary network of 35 saltmarsh experts spanning natural, physical and social sciences across research, policy, and practitioner sectors. Priorities have been grouped under four thematic areas of research: Saltmarsh Area Extent, Change and Restoration Potential (including past, present, global variation), Spatio-social contexts of Ecosystem Service delivery (e.g. influences of environmental context, climate change, and stakeholder groups on service provisioning), Patterns and Processes in saltmarsh functioning (global drivers of saltmarsh ecosystem structure/function) and Management and Policy Needs (how management varies contextually; challenges/opportunities for management). Although not intended to be exhaustive, the challenges, opportunities, and strategies for addressing each research priority examined here, providing a blueprint of the work that needs to be done to protect saltmarshes for future generations.

Large language models encode clinical knowledge.

Large language models (LLMs) have demonstrated impressive capabilities, but the bar for clinical applications is high. Attempts to assess the clinical knowledge of models typically rely on automated evaluations based on limited benchmarks. Here, to address these limitations, we present MultiMedQA, a benchmark combining six existing medical question answering datasets spanning professional medicine, research and consumer queries and a new dataset of medical questions searched online, HealthSearchQA. We propose a human evaluation framework for model answers along multiple axes including factuality, comprehension, reasoning, possible harm and bias. In addition, we evaluate Pathways Language Model1 (PaLM, a 540-billion parameter LLM) and its instruction-tuned variant, Flan-PaLM2 on MultiMedQA. Using a combination of prompting strategies, Flan-PaLM achieves state-of-the-art accuracy on every MultiMedQA multiple-choice dataset (MedQA3, MedMCQA4, PubMedQA5 and Measuring Massive Multitask Language Understanding (MMLU) clinical topics6), including 67.6% accuracy on MedQA (US Medical Licensing Exam-style questions), surpassing the prior state of the art by more than 17%. However, human evaluation reveals key gaps. To resolve this, we introduce instruction prompt tuning, a parameter-efficient approach for aligning LLMs to new domains using a few exemplars. The resulting model, Med-PaLM, performs encouragingly, but remains inferior to clinicians. We show that comprehension, knowledge recall and reasoning improve with model scale and instruction prompt tuning, suggesting the potential utility of LLMs in medicine. Our human evaluations reveal limitations of today's models, reinforcing the importance of both evaluation frameworks and method development in creating safe, helpful LLMs for clinical applications.

A large outbreak of COVID-19 in a UK prison, October 2020 to April 2021.

Prisons are susceptible to outbreaks. Control measures focusing on isolation and cohorting negatively affect wellbeing. We present an outbreak of coronavirus disease 2019 (COVID-19) in a large male prison in Wales, UK, October 2020 to April 2021, and discuss control measures.We gathered case-information, including demographics, staff-residence postcode, resident cell number, work areas/dates, test results, staff interview dates/notes and resident prison-transfer dates. Epidemiological curves were mapped by prison location. Control measures included isolation (exclusion from work or cell-isolation), cohorting (new admissions and work-area groups), asymptomatic testing (case-finding), removal of communal dining and movement restrictions. Facemask use and enhanced hygiene were already in place. Whole-genome sequencing (WGS) and interviews determined the genetic relationship between cases plausibility of transmission.Of 453 cases, 53% (n = 242) were staff, most aged 25-34 years (11.5% females, 27.15% males) and symptomatic (64%). Crude attack-rate was higher in staff (29%, 95% CI 26-64%) than in residents (12%, 95% CI 9-15%).Whole-genome sequencing can help differentiate multiple introductions from person-to-person transmission in prisons. It should be introduced alongside asymptomatic testing as soon as possible to control prison outbreaks. Timely epidemiological investigation, including data visualisation, allowed dynamic risk assessment and proportionate control measures, minimising the reduction in resident welfare.

Acamprosate may be safer than baclofen for the treatment of alcohol use disorder in patients with cirrhosis: a first description of use in real-world clinical practice.

OBJECTIVES: Patients with alcohol use disorder (AUD) and liver cirrhosis benefit from stopping alcohol intake. Baclofen has been trialled for AUD in cirrhosis and appears to be effective. However, in patients without cirrhosis acamprosate is safer and more efficacious. Acamprosate is rarely used in cirrhosis due to safety concerns: the only published report was for 24 h in a controlled setting. Our centre uses both medications off-label in cirrhotic patients. We performed an audit to pragmatically compare the safety of acamprosate to baclofen in these patients. METHODS: The electronic records of patients prescribed acamprosate or baclofen between 01/04/17 and 31/03/20 were retrospectively reviewed. Adverse events and abstinence at last follow-up were compared by Student's t-test, Mann-Whitney U or chi-square test. Confounding variables were evaluated by logistic regression. RESULTS: In total 48 cirrhotic patients taking acamprosate (median 84 days, range 2-524); 44 baclofen (247 days, 8-910) met inclusion criteria. At baseline, 41% had Childs-Pugh B or C cirrhosis. More patients taking baclofen had an unplanned hospital admission or attendance (23 vs 13; P = 0.013) and the mean number per patient was higher (1.6 vs 0.6; P = 0.032). Sub-group analysis revealed increased admissions in actively drinking patients prescribed baclofen to achieve abstinence (mean 2.4 vs 0.6; P = 0.020); acamprosate use was associated with a reduced chance of admission or attendance (OR, 0.284; 0.095-0.854; P = 0.025) independent of treatment length. No difference in efficacy was observed. CONCLUSIONS: In patients with cirrhosis, acamprosate was associated with fewer unplanned admissions than baclofen, hence may be safer despite historical concerns.

SARS-CoV-2 sero-prevalence in the workforces of three large workplaces in South Wales: a sero-epidemiological study.

BACKGROUND: Sero-prevalence studies quantify the proportion of a population that has antibodies against SARS-CoV-2, and can be used to identify the extent of the COVID-19 pandemic at a population level. The aim of the study was to assess the sero-prevalence of SARS-CoV-2 antibodies in the workforce at three workplaces: a food factory, non-food factory and call-centre. METHODS: Nine hundred ninety-three participants were recruited from three workplaces in South Wales. Participants completed a questionnaire and had a lateral flow point-of-care SARS-CoV-2 antibody test administered by a healthcare professional. The data were analysed using multivariable logistic regression, both using complete records only and following multiple imputation. RESULTS: The sero-prevalence of SARS-CoV-2 antibodies ranged from 4% (n = 17/402) in the non-food factory to 10% (n = 28/281) in the food factory (OR 2.93; 95% CI 1.26 to 6.81). After taking account of confounding factors evidence of a difference remained (cOR comparing food factory to call centre (2.93; 95% CI 1.26 to 6.81) and non-food factory (3.99; 95% CI 1.97 to 8.08) respectively). The SARS-CoV-2 antibody prevalence also varied between roles within workplaces. People working in office based roles had a 2.23 times greater conditional odds (95% CI 1.02 to 4.87) of being positive for SARS-CoV-2 antibodies than those working on the factory floor. CONCLUSION: The sero-prevalence of SARS-CoV-2 antibodies varied by workplace and work role. Whilst it is not possible to state whether these differences are due to COVID-19 transmission within the workplaces, it highlights the importance of considering COVID-19 transmission in a range of workplaces and work roles.

Conventional histological and cytological staining with simultaneous immunohistochemistry enabled by invisible chromogens.

Conventional histological stains, such as hematoxylin plus eosin (H&E), and immunohistochemistry (IHC) are mainstays of histology that provide complementary diagnostic information. H&E and IHC currently require separate slides, because the stains would otherwise obscure one another. This consumes small specimen, limiting the total amount of testing. Additionally, performing H&E and IHC on different slides does not permit comparison of staining at the single cell level, since the same cells are not present on each slide, and alignment of tissue features can be problematic due to changes in tissue landscape with sectioning. We have solved these problems by performing conventional staining and IHC on the same slide using invisible IHC chromogens, such that the chromogens are not visible when viewing the conventional stain and the conventional stain is excluded from images of the IHC. Covalently deposited chromogens provided a convenient route to invisible chromogen design and are stable to reagents used in conventional staining. A dual-camera brightfield microscope system was developed that permits simultaneous viewing of both visible conventional stains and invisible IHC chromogens. Simultaneous staining was demonstrated on several formalin-fixed paraffin-embedded tissue specimens using single and duplex IHC, with chromogens that absorb ultraviolet and near infrared light, followed by H&E staining. The concept was extended to other conventional stains, including mucicarmine special stain and Papanicoulou stain, and further extended to cytology specimens. In addition to interactive video review, images were recorded using multispectral imaging and image processing to provide flexible production of color composite images and enable quantitative analysis.

Behavior science in the evolving world of digital health: considerations on anticipated opportunities and challenges.

Digital health promises to increase intervention reach and effectiveness for a range of behavioral health outcomes. Behavioral scientists have a unique opportunity to infuse their expertise in all phases of a digital health intervention, from design to implementation. The aim of this study was to assess behavioral scientists' interests and needs with respect to digital health endeavors, as well as gather expert insight into the role of behavioral science in the evolution of digital health. The study used a two-phased approach: (a) a survey of behavioral scientists' current needs and interests with respect to digital health endeavors (n = 346); (b) a series of interviews with digital health stakeholders for their expert insight on the evolution of the health field (n = 15). In terms of current needs and interests, the large majority of surveyed behavioral scientists (77%) already participate in digital health projects, and from those who have not done so yet, the majority (65%) reported intending to do so in the future. In terms of the expected evolution of the digital health field, interviewed stakeholders anticipated a number of changes, from overall landscape changes through evolving models of reimbursement to more significant oversight and regulations. These findings provide a timely insight into behavioral scientists' current needs, barriers, and attitudes toward the use of technology in health care and public health. Results might also highlight the areas where behavioral scientists can leverage their expertise to both enhance digital health's potential to improve health, as well as to prevent the potential unintended consequences that can emerge from scaling the use of technology in health care.

An iterative, interdisciplinary, collaborative framework for developing and evaluating digital behavior change interventions.

The rapid expansion of technology promises to transform the behavior science field by revolutionizing the ways in which individuals can monitor and improve their health behaviors. To fully live into this promise, the behavior science field must address distinct challenges, including: building interventions that are not only scientifically sound but also engaging; using evaluation methods to precisely assess intervention components for intervention optimization; and building personalized interventions that acknowledge and adapt to the dynamic ecosystem of individual and contextual variables that impact behavior change. The purpose of this paper is to provide a framework to address these challenges by leveraging behavior science, human-centered design, and data science expertise throughout the cycle of developing and evaluating digital behavior change interventions (DBCIs). To define this framework, we reviewed current models and practices for intervention development and evaluation, as well as technology industry models for product development. The framework promotes an iterative process, aiming to maximize outcomes by incorporating faster and more frequent testing cycles into the lifecycle of a DBCI. Within the framework provided, we describe each phase, from development to evaluation, to discuss the optimal practices, necessary stakeholders, and proposed evaluation methods. The proposed framework may inform practices in both academia and industry, as well as highlight the need to offer collaborative platforms to ensure successful partnerships that can lead to more effective DBCIs that reach broad and diverse populations.

Understanding Health Behavior Technology Engagement: Pathway to Measuring Digital Behavior Change Interventions.

Researchers and practitioners of digital behavior change interventions (DBCI) use varying and, often, incongruent definitions of the term "engagement," thus leading to a lack of precision in DBCI measurement and evaluation. The objective of this paper is to propose discrete definitions for various types of user engagement and to explain why precision in the measurement of these engagement types is integral to ensuring the intervention is effective for health behavior modulation. Additionally, this paper presents a framework and practical steps for how engagement can be measured in practice and used to inform DBCI design and evaluation. The key purpose of a DBCI is to influence change in a target health behavior of a user, which may ultimately improve a health outcome. Using available literature and practice-based knowledge of DBCI, the framework conceptualizes two primary categories of engagement that must be measured in DBCI. The categories are health behavior engagement, referred to as "Big E," and DBCI engagement, referred to as "Little e." DBCI engagement is further bifurcated into two subclasses: (1) user interactions with features of the intervention designed to encourage frequency of use (ie, simple login, games, and social interactions) and make the user experience appealing, and (2) user interactions with behavior change intervention components (ie, behavior change techniques), which influence determinants of health behavior and subsequently influence health behavior. Achievement of Big E in an intervention delivered via digital means is contingent upon Little e. If users do not interact with DBCI features and enjoy the user experience, exposure to behavior change intervention components will be limited and less likely to influence the behavioral determinants that lead to health behavior engagement (Big E). Big E is also dependent upon the quality and relevance of the behavior change intervention components within the solution. Therefore, the combination of user interactions and behavior change intervention components creates Little e, which is, in turn, designed to improve Big E. The proposed framework includes a model to support measurement of DBCI that describes categories of engagement and details how features of Little e produce Big E. This framework can be applied to DBCI to support various health behaviors and outcomes and can be utilized to identify gaps in intervention efficacy and effectiveness.

How I treat alcohol-related liver disease.

Age-standardised mortality from liver disease in the United Kingdom has risen by 400% since 1970, with three-quarters of deaths from alcohol-related liver disease (ARLD). The 2013 National Confidential Enquiry into Patient Outcome and Death report found that only 47% of the patients dying in hospital from liver disease experienced 'good' care. We discuss common complications in the care of patients with ARLD and the evidence-based best practice that can improve patient outcomes, with a focus on the initial management of patients presenting acutely to the medical take.

Effectiveness of SMS Technology on Timely Community Health Worker Follow-Up for Childhood Malnutrition: A Retrospective Cohort Study in sub-Saharan Africa.

BACKGROUND: The Millennium Villages Project facilitated technology-based health interventions in rural under-resourced areas of sub-Saharan Africa. Our study examined whether data entry using SMS compared with paper forms by community health workers (CHWs) led to higher proportion of timely follow-up visits for malnutrition screening in under-5 children in Ghana, Rwanda, Senegal, and Uganda. METHODS: Children under 5 years were screened for malnutrition every 90 days by CHWs using mid-upper arm circumference (MUAC) readings. CHWs used either SMS texts or paper forms to enter MUAC data. Reminder texts were sent at 15 days before follow-up was needed. Chi-square tests assessed proportion of timely follow-up visits within 90 days between SMS and paper groups. Logistic regression analysis was conducted in a step-wise multivariate model. Post-hoc power calculations were conducted to verify strength of associations. RESULTS: SMS data entry was associated with a higher proportion of timely malnutrition follow-up visits compared with paper forms across all sites. The association was strongest with consistent SMS use over consecutive visits. SMS use at the first of 2 consecutive visits was most effective, highlighting the importance of SMS reminder alerts. CONCLUSIONS: SMS technology with reminders increased timely CHW malnutrition screening visits for under-5 children in Ghana, Rwanda, Senegal, and Uganda, highlighting the importance of such technology for improving health worker behavior in low-resource settings.