Illuminating protein space with a programmable generative model.

Three billion years of evolution has produced a tremendous diversity of protein molecules1, but the full potential of proteins is likely to be much greater. Accessing this potential has been challenging for both computation and experiments because the space of possible protein molecules is much larger than the space of those likely to have functions. Here we introduce Chroma, a generative model for proteins and protein complexes that can directly sample novel protein structures and sequences, and that can be conditioned to steer the generative process towards desired properties and functions. To enable this, we introduce a diffusion process that respects the conformational statistics of polymer ensembles, an efficient neural architecture for molecular systems that enables long-range reasoning with sub-quadratic scaling, layers for efficiently synthesizing three-dimensional structures of proteins from predicted inter-residue geometries and a general low-temperature sampling algorithm for diffusion models. Chroma achieves protein design as Bayesian inference under external constraints, which can involve symmetries, substructure, shape, semantics and even natural-language prompts. The experimental characterization of 310 proteins shows that sampling from Chroma results in proteins that are highly expressed, fold and have favourable biophysical properties. The crystal structures of two designed proteins exhibit atomistic agreement with Chroma samples (a backbone root-mean-square deviation of around 1.0 Å). With this unified approach to protein design, we hope to accelerate the programming of protein matter to benefit human health, materials science and synthetic biology.

National Needs Assessment of Utilization of Common Newborn Clinical Decision Support Tools.

OBJECTIVE: Clinical decision support tools (CDSTs) are common in neonatology, but utilization is rarely examined. We examined the utilization of four CDSTs in newborn care. STUDY DESIGN: A 72-field needs assessment was developed. It was distributed to listservs encompassing trainees, nurse practitioners, hospitalists, and attendings. At the conclusion of data collection, responses were downloaded and analyzed. RESULTS: We received 339 fully completed questionnaires. BiliTool and the Early-Onset Sepsis (EOS) tool were used by > 90% of respondents, the Bronchopulmonary Dysplasia tool by 39%, and the Extremely Preterm Birth tool by 72%. Common reasons CDSTs did not impact clinical care included lack of electronic health record integration, lack of confidence in prediction accuracy, and unhelpful predictions. CONCLUSION: From a national sample of neonatal care providers, there is frequent but variable use of four CDSTs. Understanding the factors that contribute to tool utility is vital prior to development and implementation. KEY POINTS: · Clinical decision support tools are common in medicine.. · There is a varied use of neonatal CDST.. · Understanding the use of CDST is vital for future development..

A structural characterization of shortcut features for prediction.

With the rising use of machine learning for healthcare applications, practitioners are increasingly confronted with the limitations of prediction models that are trained in one setting but meant to be deployed in several others. One recently identified limitation is so-called shortcut learning, whereby a model learns to associate features with the prediction target that do not maintain their relationship across settings. Famously, the watermark on chest x-rays has been demonstrated to be an instance of a shortcut feature. In this viewpoint, we attempt to give a structural characterization of shortcut features in terms of causal DAGs. This is the first attempt at defining shortcut features in terms of their causal relationship with a model's prediction target.

Regional differences in utilization of 17α-hydroxyprogesterone caproate (17-OHP).

OBJECTIVES: To describe regional differences in utilization of 17α-hydroxyprogesterone caproate (17-OHP). METHODS: Retrospective cohort study of a large, US commercial managed care plan claims database with pharmacy coverage from 2008 to 2018. Singleton pregnancies with at least one prior spontaneous preterm birth (sPTB) were included. Regional and state-based differences in 17-OHP use were compared. Data were analyzed using t-tests and Fisher's exact tests. RESULTS: Of the 4,514 individuals with an indication for 17-OHP, 580 (12.8%) were prescribed 17-OHP. Regional and state-based differences in 17-OHP utilization were identified; Northeast 15.7%, Midwest 13.7%, South 12.0%, and West 10.4% (p=0.003). CONCLUSIONS: While significant regional differences in 17-OHP utilization were demonstrated, 17-OHP utilization remained low despite this cohort having insurance through a US commercial managed care plan. Suboptimal utilization demonstrates a disconnect between research and uptake in clinical practice. This underscores a need for implementation science in obstetrics to translate updated recommendations more effectively and efficiently into clinical practice.

Cost Savings Without Increased Risk of Respiratory Hospitalization for Preterm Children after the 2014 Palivizumab Policy Update.

OBJECTIVE: Our objective was to compare rates of hospitalizations for respiratory illnesses in preterm and full-term (FT) children for 4 years before and after the 2014 update to the American Academy of Pediatrics (AAP) respiratory syncytial virus (RSV) immunoprophylaxis guidance, which restricted eligibility among infants born at 29 to 34 weeks in the first winter and all preterm infants in the second winter after neonatal discharge. STUDY DESIGN: We conducted pre-post and interrupted time series analyses on claims data from a commercial national managed care plan. We compared the number of RSV and all respiratory hospital admissions in the first and second RSV seasons after neonatal discharge among a cohort of preterm children, regardless of palivizumab status, in the 4 years before and after the implementation of the 2014 palivizumab eligibility change. A FT group was included for reference. RESULTS: The cohort included 821 early preterm (EP, <29 weeks), 4,790 moderate preterm (MP, 29-34 weeks), and 130,782 FT children. Palivizumab use after the policy update decreased among MP children in the first and second RSV seasons after neonatal discharge, without any change in the odds of hospitalization with RSV or respiratory illness. For the EP group, there was no change in the rate of palivizumab or the odds of hospitalization with RSV or respiratory illness after the policy update. For the FT group, there was a slight decrease in odds of hospitalization post-2014 after the policy update. The interrupted time series did not reveal any secular trends over time in hospitalization rates among preterm children. Following the policy change, there were cost savings for MP children in the first and second RSV seasons, when accounting for the cost of hospitalizations and the cost of palivizumab. CONCLUSION: Hospitalizations for RSV or respiratory illness did not increase, and cost savings were obtained after the implementation of the 2014 AAP palivizumab prophylaxis policy. KEY POINTS: · Palivizumab use decreased among children born moderate preterm (29 to34 weeks) after the 2014 palivizuamb policy update.. · There was no change in odds of hospitalization with respiratory syncitial virus or respiratory illness among preterm infants after the policy update when compared to before.. · There were cost savings, when accounting for the cost of hospitalizations and the cost of palivizumab, after the policy update among children born moderate preterm..

Effectiveness of 17-OHP for Prevention of Recurrent Preterm Birth: A Retrospective Cohort Study.

OBJECTIVE: 17-α-hydroxyprogesterone caproate (17-OHP) has been recommended by professional societies for the prevention of recurrent preterm birth, but subsequent clinical studies have reported conflicting efficacy results. This study aimed to contribute to the evidence base regarding the effectiveness of 17-OHP in clinical practice using real-world data. STUDY DESIGN: A total of 4,422 individuals meeting inclusion criteria representing recurrent spontaneous preterm birth (sPTB) were identified in a database of insurance claims, and 568 (12.8%) received 17-OHP. Crude and propensity score-matched recurrence rates and risk ratios (RRs) for the association of receiving 17-OHP on recurrent sPTB were calculated. RESULTS: Raw sPTB recurrence rates were higher among those treated versus not treated; after propensity score matching, no association was detected (26.3 vs. 23.8%, RR = 1.1, 95% CI: 0.9-1.4). CONCLUSION: We failed to identify a beneficial effect of 17-OHP for the prevention of spontaneous recurrent preterm birth in our observational, U.S. based cohort. KEY POINTS: · •We observed higher risk for sPTB in the group receiving 17-OHP in the unmatched analysis. · •After propensity-score matching, we still failed to identify a beneficial effect of 17-OHP on sPTB. · •Sensitivity analyses demonstrated robustness to the inclusion criteria and modeling assumptions..

Empirical Frequentist Coverage of Deep Learning Uncertainty Quantification Procedures.

Uncertainty quantification for complex deep learning models is increasingly important as these techniques see growing use in high-stakes, real-world settings. Currently, the quality of a model's uncertainty is evaluated using point-prediction metrics, such as the negative log-likelihood (NLL), expected calibration error (ECE) or the Brier score on held-out data. Marginal coverage of prediction intervals or sets, a well-known concept in the statistical literature, is an intuitive alternative to these metrics but has yet to be systematically studied for many popular uncertainty quantification techniques for deep learning models. With marginal coverage and the complementary notion of the width of a prediction interval, downstream users of deployed machine learning models can better understand uncertainty quantification both on a global dataset level and on a per-sample basis. In this study, we provide the first large-scale evaluation of the empirical frequentist coverage properties of well-known uncertainty quantification techniques on a suite of regression and classification tasks. We find that, in general, some methods do achieve desirable coverage properties on in distribution samples, but that coverage is not maintained on out-of-distribution data. Our results demonstrate the failings of current uncertainty quantification techniques as dataset shift increases and reinforce coverage as an important metric in developing models for real-world applications.

Automated grouping of medical codes via multiview banded spectral clustering.

OBJECTIVE: With its increasingly widespread adoption, electronic health records (EHR) have enabled phenotypic information extraction at an unprecedented granularity and scale. However, often a medical concept (e.g. diagnosis, prescription, symptom) is described in various synonyms across different EHR systems, hindering data integration for signal enhancement and complicating dimensionality reduction for knowledge discovery. Despite existing ontologies and hierarchies, tremendous human effort is needed for curation and maintenance - a process that is both unscalable and susceptible to subjective biases. This paper aims to develop a data-driven approach to automate grouping medical terms into clinically relevant concepts by combining multiple up-to-date data sources in an unbiased manner. METHODS: We present a novel data-driven grouping approach - multi-view banded spectral clustering (mvBSC) combining summary data from multiple healthcare systems. The proposed method consists of a banding step that leverages the prior knowledge from the existing coding hierarchy, and a combining step that performs spectral clustering on an optimally weighted matrix. RESULTS: We apply the proposed method to group ICD-9 and ICD-10-CM codes together by integrating data from two healthcare systems. We show grouping results and hierarchies for 13 representative disease categories. Individual grouping qualities were evaluated using normalized mutual information, adjusted Rand index, and F1-measure, and were found to consistently exhibit great similarity to the existing manual grouping counterpart. The resulting ICD groupings also enjoy comparable interpretability and are well aligned with the current ICD hierarchy. CONCLUSION: The proposed approach, by systematically leveraging multiple data sources, is able to overcome bias while maximizing consensus to achieve generalizability. It has the advantage of being efficient, scalable, and adaptive to the evolving human knowledge reflected in the data, showing a significant step toward automating medical knowledge integration.

Feature extraction for phenotyping from semantic and knowledge resources.

OBJECTIVE: Phenotyping algorithms can efficiently and accurately identify patients with a specific disease phenotype and construct electronic health records (EHR)-based cohorts for subsequent clinical or genomic studies. Previous studies have introduced unsupervised EHR-based feature selection methods that yielded algorithms with high accuracy. However, those selection methods still require expert intervention to tweak the parameter settings according to the EHR data distribution for each phenotype. To further accelerate the development of phenotyping algorithms, we propose a fully automated and robust unsupervised feature selection method that leverages only publicly available medical knowledge sources, instead of EHR data. METHODS: SEmantics-Driven Feature Extraction (SEDFE) collects medical concepts from online knowledge sources as candidate features and gives them vector-form distributional semantic representations derived with neural word embedding and the Unified Medical Language System Metathesaurus. A number of features that are semantically closest and that sufficiently characterize the target phenotype are determined by a linear decomposition criterion and are selected for the final classification algorithm. RESULTS: SEDFE was compared with the EHR-based SAFE algorithm and domain experts on feature selection for the classification of five phenotypes including coronary artery disease, rheumatoid arthritis, Crohn's disease, ulcerative colitis, and pediatric pulmonary arterial hypertension using both supervised and unsupervised approaches. Algorithms yielded by SEDFE achieved comparable accuracy to those yielded by SAFE and expert-curated features. SEDFE is also robust to the input semantic vectors. CONCLUSION: SEDFE attains satisfying performance in unsupervised feature selection for EHR phenotyping. Both fully automated and EHR-independent, this method promises efficiency and accuracy in developing algorithms for high-throughput phenotyping.

Bayesian neural networks for detecting epistasis in genetic association studies.

BACKGROUND: Discovering causal genetic variants from large genetic association studies poses many difficult challenges. Assessing which genetic markers are involved in determining trait status is a computationally demanding task, especially in the presence of gene-gene interactions. RESULTS: A non-parametric Bayesian approach in the form of a Bayesian neural network is proposed for use in analyzing genetic association studies. Demonstrations on synthetic and real data reveal they are able to efficiently and accurately determine which variants are involved in determining case-control status. By using graphics processing units (GPUs) the time needed to build these models is decreased by several orders of magnitude. In comparison with commonly used approaches for detecting interactions, Bayesian neural networks perform very well across a broad spectrum of possible genetic relationships. CONCLUSIONS: The proposed framework is shown to be a powerful method for detecting causal SNPs while being computationally efficient enough to handle large datasets.

Artificial intelligence in the neonatal intensive care unit: the time is now.

Artificial intelligence (AI) has the potential to revolutionize the neonatal intensive care unit (NICU) care by leveraging the large-scale, high-dimensional data that are generated by NICU patients. There is an emerging recognition that the confluence of technological progress, commercialization pathways, and rich data sets provides a unique opportunity for AI to make a lasting impact on the NICU. In this perspective article, we discuss four broad categories of AI applications in the NICU: imaging interpretation, prediction modeling of electronic health record data, integration of real-time monitoring data, and documentation and billing. By enhancing decision-making, streamlining processes, and improving patient outcomes, AI holds the potential to transform the quality of care for vulnerable newborns, making the excitement surrounding AI advancements well-founded and the potential for significant positive change stronger than ever before.

Statin Twitter: Human and Automated Bot Contributions, 2010 to 2022.

BACKGROUND: Many individuals eligible for statin therapy decline treatment, often due to fear of adverse effects. Misinformation about statins is common and drives statin reluctance, but its prevalence on social media platforms, such as Twitter (now X) remains unclear. Social media bots are known to proliferate medical misinformation, but their involvement in statin-related discourse is unknown. This study examined temporal trends in volume, author type (bot or human), and sentiment of statin-related Twitter posts (tweets). METHODS AND RESULTS: We analyzed original tweets with statin-related terms from 2010 to 2022 using a machine learning-derived classifier to determine the author's bot probability, natural language processing to assign each tweet a negative or positive sentiment, and manual qualitative analysis to identify statin skepticism in a random sample of all tweets and in highly influential tweets. We identified 1 155 735 original statin-related tweets. Bots produced 333 689 (28.9%), humans produced 699 876 (60.6%), and intermediate probability accounts produced 104 966 (9.1%). Over time, the proportion of bot tweets decreased from 47.8% to 11.3%, and human tweets increased from 43.6% to 79.8%. The proportion of negative-sentiment tweets increased from 27.8% to 43.4% for bots and 30.9% to 38.4% for humans. Manually coded statin skepticism increased from 8.0% to 19.0% for bots and from 26.0% to 40.0% for humans. CONCLUSIONS: Over the past decade, humans have overtaken bots as generators of statin-related content on Twitter. Negative sentiment and statin skepticism have increased across all user types. Twitter may be an important forum to combat statin-related misinformation.

Concordance of randomised controlled trials for artificial intelligence interventions with the CONSORT-AI reporting guidelines.

The Consolidated Standards of Reporting Trials extension for Artificial Intelligence interventions (CONSORT-AI) was published in September 2020. Since its publication, several randomised controlled trials (RCTs) of AI interventions have been published but their completeness and transparency of reporting is unknown. This systematic review assesses the completeness of reporting of AI RCTs following publication of CONSORT-AI and provides a comprehensive summary of RCTs published in recent years. 65 RCTs were identified, mostly conducted in China (37%) and USA (18%). Median concordance with CONSORT-AI reporting was 90% (IQR 77-94%), although only 10 RCTs explicitly reported its use. Several items were consistently under-reported, including algorithm version, accessibility of the AI intervention or code, and references to a study protocol. Only 3 of 52 included journals explicitly endorsed or mandated CONSORT-AI. Despite a generally high concordance amongst recent AI RCTs, some AI-specific considerations remain systematically poorly reported. Further encouragement of CONSORT-AI adoption by journals and funders may enable more complete adoption of the full CONSORT-AI guidelines.

Safe and reliable transport of prediction models to new healthcare settings without the need to collect new labeled data.

How can practitioners and clinicians know if a prediction model trained at a different institution can be safely used on their patient population? There is a large body of evidence showing that small changes in the distribution of the covariates used by prediction models may cause them to fail when deployed to new settings. This specific kind of dataset shift, known as covariate shift, is a central challenge to implementing existing prediction models in new healthcare environments. One solution is to collect additional labels in the target population and then fine tune the prediction model to adapt it to the characteristics of the new healthcare setting, which is often referred to as localization. However, collecting new labels can be expensive and time-consuming. To address these issues, we recast the core problem of model transportation in terms of uncertainty quantification, which allows one to know when a model trained in one setting may be safely used in a new healthcare environment of interest. Using methods from conformal prediction, we show how to transport models safely between different settings in the presence of covariate shift, even when all one has access to are covariates from the new setting of interest (e.g. no new labels). Using this approach, the model returns a prediction set that quantifies its uncertainty and is guaranteed to contain the correct label with a user-specified probability (e.g. 90%), a property that is also known as coverage. We show that a weighted conformal inference procedure based on density ratio estimation between the source and target populations can produce prediction sets with the correct level of coverage on real-world data. This allows users to know if a model's predictions can be trusted on their population without the need to collect new labeled data.

The Diagnostic and Triage Accuracy of the GPT-3 Artificial Intelligence Model.

Importance: Artificial intelligence (AI) applications in health care have been effective in many areas of medicine, but they are often trained for a single task using labeled data, making deployment and generalizability challenging. Whether a general-purpose AI language model can perform diagnosis and triage is unknown. Objective: Compare the general-purpose Generative Pre-trained Transformer 3 (GPT-3) AI model's diagnostic and triage performance to attending physicians and lay adults who use the Internet. Design: We compared the accuracy of GPT-3's diagnostic and triage ability for 48 validated case vignettes of both common (e.g., viral illness) and severe (e.g., heart attack) conditions to lay people and practicing physicians. Finally, we examined how well calibrated GPT-3's confidence was for diagnosis and triage. Setting and Participants: The GPT-3 model, a nationally representative sample of lay people, and practicing physicians. Exposure: Validated case vignettes (<60 words; <6th grade reading level). Main Outcomes and Measures: Correct diagnosis, correct triage. Results: Among all cases, GPT-3 replied with the correct diagnosis in its top 3 for 88% (95% CI, 75% to 94%) of cases, compared to 54% (95% CI, 53% to 55%) for lay individuals (p<0.001) and 96% (95% CI, 94% to 97%) for physicians (p=0.0354). GPT-3 triaged (71% correct; 95% CI, 57% to 82%) similarly to lay individuals (74%; 95% CI, 73% to 75%; p=0.73); both were significantly worse than physicians (91%; 95% CI, 89% to 93%; p<0.001). As measured by the Brier score, GPT-3 confidence in its top prediction was reasonably well-calibrated for diagnosis (Brier score = 0.18) and triage (Brier score = 0.22). Conclusions and Relevance: A general-purpose AI language model without any content-specific training could perform diagnosis at levels close to, but below physicians and better than lay individuals. The model was performed less well on triage, where its performance was closer to that of lay individuals.