Large language models encode clinical knowledge.

Large language models (LLMs) have demonstrated impressive capabilities, but the bar for clinical applications is high. Attempts to assess the clinical knowledge of models typically rely on automated evaluations based on limited benchmarks. Here, to address these limitations, we present MultiMedQA, a benchmark combining six existing medical question answering datasets spanning professional medicine, research and consumer queries and a new dataset of medical questions searched online, HealthSearchQA. We propose a human evaluation framework for model answers along multiple axes including factuality, comprehension, reasoning, possible harm and bias. In addition, we evaluate Pathways Language Model1 (PaLM, a 540-billion parameter LLM) and its instruction-tuned variant, Flan-PaLM2 on MultiMedQA. Using a combination of prompting strategies, Flan-PaLM achieves state-of-the-art accuracy on every MultiMedQA multiple-choice dataset (MedQA3, MedMCQA4, PubMedQA5 and Measuring Massive Multitask Language Understanding (MMLU) clinical topics6), including 67.6% accuracy on MedQA (US Medical Licensing Exam-style questions), surpassing the prior state of the art by more than 17%. However, human evaluation reveals key gaps. To resolve this, we introduce instruction prompt tuning, a parameter-efficient approach for aligning LLMs to new domains using a few exemplars. The resulting model, Med-PaLM, performs encouragingly, but remains inferior to clinicians. We show that comprehension, knowledge recall and reasoning improve with model scale and instruction prompt tuning, suggesting the potential utility of LLMs in medicine. Our human evaluations reveal limitations of today's models, reinforcing the importance of both evaluation frameworks and method development in creating safe, helpful LLMs for clinical applications.

Health system-scale language models are all-purpose prediction engines.

Physicians make critical time-constrained decisions every day. Clinical predictive models can help physicians and administrators make decisions by forecasting clinical and operational events. Existing structured data-based clinical predictive models have limited use in everyday practice owing to complexity in data processing, as well as model development and deployment1-3. Here we show that unstructured clinical notes from the electronic health record can enable the training of clinical language models, which can be used as all-purpose clinical predictive engines with low-resistance development and deployment. Our approach leverages recent advances in natural language processing4,5 to train a large language model for medical language (NYUTron) and subsequently fine-tune it across a wide range of clinical and operational predictive tasks. We evaluated our approach within our health system for five such tasks: 30-day all-cause readmission prediction, in-hospital mortality prediction, comorbidity index prediction, length of stay prediction, and insurance denial prediction. We show that NYUTron has an area under the curve (AUC) of 78.7-94.9%, with an improvement of 5.36-14.7% in the AUC compared with traditional models. We additionally demonstrate the benefits of pretraining with clinical text, the potential for increasing generalizability to different sites through fine-tuning and the full deployment of our system in a prospective, single-arm trial. These results show the potential for using clinical language models in medicine to read alongside physicians and provide guidance at the point of care.

Communicating doctors' consensus persistently increases COVID-19 vaccinations.

The reluctance of people to get vaccinated represents a fundamental challenge to containing the spread of deadly infectious diseases1,2, including COVID-19. Identifying misperceptions that can fuel vaccine hesitancy and creating effective communication strategies to overcome them are a global public health priority3-5. Medical doctors are a trusted source of advice about vaccinations6, but media reports may create an inaccurate impression that vaccine controversy is prevalent among doctors, even when a broad consensus exists7,8. Here we show that public misperceptions about the views of doctors on the COVID-19 vaccines are widespread, and correcting them increases vaccine uptake. We implement a survey among 9,650 doctors in the Czech Republic and find that 90% of doctors trust the vaccines. Next, we show that 90% of respondents in a nationally representative sample (n = 2,101) underestimate doctors' trust; the most common belief is that only 50% of doctors trust the vaccines. Finally, we integrate randomized provision of information about the true views held by doctors into a longitudinal data collection that regularly monitors vaccination status over 9 months. The treatment recalibrates beliefs and leads to a persistent increase in vaccine uptake. The approach demonstrated in this paper shows how the engagement of professional medical associations, with their unparalleled capacity to elicit individual views of doctors on a large scale, can help to create a cheap, scalable intervention that has lasting positive impacts on health behaviour.

Artificial Intelligence for Drug Discovery: Are We There Yet?

Drug discovery is adapting to novel technologies such as data science, informatics, and artificial intelligence (AI) to accelerate effective treatment development while reducing costs and animal experiments. AI is transforming drug discovery, as indicated by increasing interest from investors, industrial and academic scientists, and legislators. Successful drug discovery requires optimizing properties related to pharmacodynamics, pharmacokinetics, and clinical outcomes. This review discusses the use of AI in the three pillars of drug discovery: diseases, targets, and therapeutic modalities, with a focus on small-molecule drugs. AI technologies, such as generative chemistry, machine learning, and multiproperty optimization, have enabled several compounds to enter clinical trials. The scientific community must carefully vet known information to address the reproducibility crisis. The full potential of AI in drug discovery can only be realized with sufficient ground truth and appropriate human intervention at later pipeline stages.

Gene-Environment Interactions: My Unique Journey.

I am deeply honored to be invited to write this scientific autobiography. As a physician-scientist, pediatrician, molecular biologist, and geneticist, I have authored/coauthored more than 600 publications in the fields of clinical medicine, biochemistry, biophysics, pharmacology, drug metabolism, toxicology, molecular biology, cancer, standardized gene nomenclature, developmental toxicology and teratogenesis, mouse genetics, human genetics, and evolutionary genomics. Looking back, I think my career can be divided into four distinct research areas, which I summarize mostly chronologically in this article: (a) discovery and characterization of the AHR/CYP1 axis, (b) pharmacogenomics and genetic prediction of response to drugs and other environmental toxicants, (c) standardized drug-metabolizing gene nomenclature based on evolutionary divergence, and (d) discovery and characterization of the SLC39A8 gene encoding the ZIP8 metal cation influx transporter. Collectively, all four topics embrace gene-environment interactions, hence the title of my autobiography.

Hospital Prices for Physician-Administered Drugs for Patients with Private Insurance.

BACKGROUND: Hospitals can leverage their position between the ultimate buyers and sellers of drugs to retain a substantial share of insurer pharmaceutical expenditures. METHODS: In this study, we used 2020-2021 national Blue Cross Blue Shield claims data regarding patients in the United States who had drug-infusion visits for oncologic conditions, inflammatory conditions, or blood-cell deficiency disorders. Markups of the reimbursement prices were measured in terms of amounts paid by Blue Cross Blue Shield plans to hospitals and physician practices relative to the amounts paid by these providers to drug manufacturers. Acquisition-price reductions in hospital payments to drug manufacturers were measured in terms of discounts under the federal 340B Drug Pricing Program. We estimated the percentage of Blue Cross Blue Shield drug spending that was received by drug manufacturers and the percentage retained by provider organizations. RESULTS: The study included 404,443 patients in the United States who had 4,727,189 drug-infusion visits. The median price markup (defined as the ratio of the reimbursement price to the acquisition price) for hospitals eligible for 340B discounts was 3.08 (interquartile range, 1.87 to 6.38). After adjustment for drug, patient, and geographic factors, price markups at hospitals eligible for 340B discounts were 6.59 times (95% confidence interval [CI], 6.02 to 7.16) as high as those in independent physician practices, and price markups at noneligible hospitals were 4.34 times (95% CI, 3.77 to 4.90) as high as those in physician practices. Hospitals eligible for 340B discounts retained 64.3% of insurer drug expenditures, whereas hospitals not eligible for 340B discounts retained 44.8% and independent physician practices retained 19.1%. CONCLUSIONS: This study showed that hospitals imposed large price markups and retained a substantial share of total insurer spending on physician-administered drugs for patients with private insurance. The effects were especially large for hospitals eligible for discounts under the federal 340B Drug Pricing Program on acquisition costs paid to manufacturers. (Funded by Arnold Ventures and the National Institute for Health Care Management.).

Trends in U.S. public confidence in science and opportunities for progress.

In recent years, many questions have been raised about whether public confidence in science is changing. To clarify recent trends in the public's confidence and factors that are associated with these feelings, an effort initiated by the National Academies' Strategic Council for Research Excellence, Integrity, and Trust (the Strategic Council) analyzed findings from multiple survey research organizations. The Strategic Council's effort, which began in 2022, found that U.S. public confidence in science, the scientific community, and leaders of scientific communities is high relative to other civic, cultural, and governmental institutions for which researchers regularly collect such data. However, confidence in these institutions has fallen during the previous 5 years. Science's decline, while real, is similar to or less than that in the other groups. A recent study goes into greater detail by exploring public views of science. From these data, we observe that many of the surveyed U.S. public question the extent to which scientists share their values or overcome personal biases when presenting conclusions. At the same time, large majorities agree on certain types of actions that they want scientists to take. For example, 84% respond that it is "somewhat important" or "very important" for scientists to disclose their funders. Ninety-two percent (92%) offer the same responses to scientists "being open to changing their minds based on new evidence." Collectively, these data clarify how the U.S. public views science and scientists. They also suggest actions that can affect public confidence in science and scientists in the years to come.

Physician-patient racial concordance and newborn mortality.

The racial gap in infant mortality is a pressing public-health concern, and [B. N. Greenwood et al., Proc. Natl. Acad. Sci. U.S.A. 117, 21194-21200 (2020), 10.1073/pnas.1913405117] suggest that Black newborns are more likely to survive if cared for by Black physicians after birth, even in models that control for numerous variables, including hospital and physician fixed effects, and the 65 most common comorbidities affecting newborns (as described by International Classification of Disease codes). We acquired the data used in the study, covering Florida hospital discharges from 1992 through the third quarter of 2015, to replicate and extend the analysis. We find that the magnitude of the concordance effect is substantially reduced after controlling for diagnoses indicating very low birth weight (<1,500 g), which are a strong predictor of neonatal mortality but not among the 65 most common comorbidities. In fact, the estimated effect is near zero and statistically insignificant in the expanded specifications that control for very low birth weight and include hospital and physician fixed effects.

Sex bias in pain management decisions.

In the pursuit of mental and physical health, effective pain management stands as a cornerstone. Here, we examine a potential sex bias in pain management. Leveraging insights from psychological research showing that females' pain is stereotypically judged as less intense than males' pain, we hypothesize that there may be tangible differences in pain management decisions based on patients' sex. Our investigation spans emergency department (ED) datasets from two countries, including discharge notes of patients arriving with pain complaints (N = 21,851). Across these datasets, a consistent sex disparity emerges. Female patients are less likely to be prescribed pain-relief medications compared to males, and this disparity persists even after adjusting for patients' reported pain scores and numerous patient, physician, and ED variables. This disparity extends across medical practitioners, with both male and female physicians prescribing less pain-relief medications to females than to males. Additional analyses reveal that female patients' pain scores are 10% less likely to be recorded by nurses, and female patients spend an additional 30 min in the ED compared to male patients. A controlled experiment employing clinical vignettes reinforces our hypothesis, showing that nurses (N = 109) judge pain of female patients to be less intense than that of males. We argue that the findings reflect an undertreatment of female patients' pain. We discuss the troubling societal and medical implications of females' pain being overlooked and call for policy interventions to ensure equal pain treatment.

The physician scientist: balancing clinical and research duties.

Physician scientists bridge the gap between biomedical research and clinical practice. However, the continuing decrease in number of people who choose this career path poses a threat to the advancement of biomedical science and the translation of research findings to clinical practice.

Brachytherapy: An overview for clinicians.

Brachytherapy is a specific form of radiotherapy consisting of the precise placement of radioactive sources directly into or next to the tumor. This technique is indicated for patients affected by various types of cancers. It is an optimal tool for delivering very high doses to the tumor focally while minimizing the probability of normal tissue complications. Physicians from a wide range of specialties may be involved in either the referral to or the placement of brachytherapy. Many patients require brachytherapy as either primary treatment or as part of their oncologic care. On the basis of high-level evidence from randomized controlled trials, brachytherapy is mainly indicated: 1) as standard in combination with chemoradiation in patients with locally advanced cervical cancer; 2) in surgically treated patients with uterine endometrial cancer for decreasing the risk of vaginal vault recurrence; 3) in patients with high-risk prostate cancer to perform dose escalation and improve progression-free survival; and 4) in patients with breast cancer as adjuvant, accelerated partial breast irradiation or to boost the tumor bed. In this review, the authors discuss the clinical relevance of brachytherapy with a focus on indications, levels of evidence, and results in the overall context of radiation use for patients with cancer.

Factors Assessing Science's Self-Presentation model and their effect on conservatives' and liberals' support for funding science.

A confirmatory factor analysis (CFA) of responses to 13 questions from a 2022 national probability sample of 1,154 US adults supported the existence of five factors that we argue assess perceptions of Factors Assessing Science's Self-Presentation (FASS). These factors also predict support for increasing federal funding of science and, separately, supporting federal funding of basic research. Each of the factors reflects perceptions of a key facet of scientists' self-presentation, science/scientists' adherence to professed norms, or science's benefits: specifically, that scientists are Credible, Prudent, and Unbiased and that science is Self-Correcting and Beneficial. The FASS model explained 40.6% of the variance in support for increasing federal funding for science and 33.7% in support for basic research. For both dependent variables, conservatives were less likely to be supportive when they perceived that science/scientists fail to overcome biases. The interactions between political ideology and both Prudence and Beneficial, however, were significant only when predicting Basic Research support. In that case, there were no differences between conservatives and liberals when perceptions of benefit were low, but when high, liberals' perception of benefit had a stronger association with support for funding than conservatives'. Among those perceiving that scientists lack prudence, liberals were more likely to support funding basic research than conservatives, but the difference disappeared when perceptions of prudence were very high. The factors could serve as across-time indicators of the public's assessment of the state of science.

Human verifications: Computable with truth values outside logic.

Cognitive scientists treat verification as a computation in which descriptions that match the relevant situation are true, but otherwise false. The claim is controversial: The logician Gödel and the physicist Penrose have argued that human verifications are not computable. In contrast, the theory of mental models treats verification as computable, but the two truth values of standard logics, true and false, as insufficient. Three online experiments (n = 208) examined participants' verifications of disjunctive assertions about a location of an individual or a journey, such as: 'You arrived at Exeter or Perth'. The results showed that their verifications depended on observation of a match with one of the locations but also on the status of other locations (Experiment 1). Likewise, when they reached one destination and the alternative one was impossible, their use of the truth value: could be true and could be false increased (Experiment 2). And, when they reached one destination and the only alternative one was possible, they used the truth value, true and it couldn't have been false, and when the alternative one was impossible, they used the truth value: true but it could have been false (Experiment 3). These truth values and those for falsity embody counterfactuals. We implemented a computer program that constructs models of disjunctions, represents possible destinations, and verifies the disjunctions using the truth values in our experiments. Whether an awareness of a verification's outcome is computable remains an open question.

Widespread support for a global species list with a formal governance system.

Taxonomic data are a scientific common. Unlike nomenclature, which has strong governance institutions, there are currently no generally accepted governance institutions for the compilation of taxonomic data into an accepted global list. This gap results in challenges for conservation, ecological research, policymaking, international trade, and other areas of scientific and societal importance. Consensus on a global list and its management requires effective governance and standards, including agreed mechanisms for choosing among competing taxonomies and partial lists. However, governance frameworks are currently lacking, and a call for governance in 2017 generated critical responses. Any governance system to which compliance is voluntary requires a high level of legitimacy and credibility among those by and for whom it is created. Legitimacy and credibility, in turn, require adequate and credible consultation. Here, we report on the results of a global survey of taxonomists, scientists from other disciplines, and users of taxonomy designed to assess views and test ideas for a new system of taxonomic list governance. We found a surprisingly high degree of agreement on the need for a global list of accepted species and their names, and consistent views on what such a list should provide to users and how it should be governed. The survey suggests that consensus on a mechanism to create, manage, and govern a single widely accepted list of all the world's species is achievable. This finding was unexpected given past controversies about the merits of list governance.

A gender perspective on the global migration of scholars.

Although considerable progress toward gender equality in science has been made in recent decades, female researchers continue to face significant barriers in the academic labor market. International mobility has been increasingly recognized as a strategy for scientists to expand their professional networks, and that could help narrow the gender gap in academic careers. Using bibliometric data on over 33 million Scopus publications, we provide a global and dynamic view of gendered patterns of transnational scholarly mobility, as measured by volume, distance, diversity, and distribution, from 1998 to 2017. We find that, while female researchers continued to be underrepresented among internationally mobile researchers and migrate over shorter distances, this gender gap was narrowing at a faster rate than the gender gap in the population of general active researchers. Globally, the origin and destination countries of both female and male mobile researchers became increasingly diversified, which suggests that scholarly migration has become less skewed and more globalized. However, the range of origin and destination countries continued to be narrower for women than for men. While the United States remained the leading academic destination worldwide, the shares of both female and male scholarly inflows to that country declined from around 25% to 20% over the study period, partially due to the growing relevance of China. This study offers a cross-national measurement of gender inequality in global scholarly migration that is essential for promoting gender-equitable science policies and for monitoring the impact of such interventions.

Pushing Forward the Future Tense: Perspectives of a Scientist.

This review is a somewhat chronological tale of my scientific life, emphasizing the why of the questions we asked in the lab and lessons learned that may be of value to nascent scientists. The reader will come to realize that the flow of my life has been driven by a combined life of the mind and life of the soul, intertwining like the strands of DNA.

Medical School Regulatory Compliance Burdens for the Physician-Scientist.

Medical school research faculty is increasingly required to complete more comprehensive and time consuming compliance steps for regulatory oversight. These relate to animal studies, information technology, biosafety, and human resources. For physician-scientists, the additional role in clinical care adds to these research areas with regulatory compliance in patient care and ever-growing web trainings. The sum of all these compliance regimes is a considerable time and cost burden, diminished research performance, and disengagement of faculty from colleagues, collaborations, and institutions. Many research and clinical compliance processes were put in place, often using legacy systems, in well-meaning attempts to address straightforward regulations in humane animal care, safe use of biological agents, and medical care delivery. However, their accumulation and negative impact on faculty performance demand time, energy, and resources that impact academic productivity. There are solutions to a relentlessly increasing regulatory load for research faculty, which involve vertical integration, convergence, and performance assessment in medical school and health system compliance regimes. ANN NEUROL 2024;96:417-422.

A qualitative approach to assess the opinion of physicians about the challenges and prospects of pharmacogenomic testing implementation in clinical practice in Greece.

BACKGROUND: Pharmacogenomics (PGx) constitutes an important part of personalized medicine and has several clinical applications. PGx role in clinical practice is known, however, it has not been widely adopted yet. In this study, we aim to investigate the perspectives of Greek physicians regarding the implementation of PGx testing in clinical practice and the key issues associated with it. METHODS: Fourteen interviews were conducted with physicians of various specialties for which PGx applications are available. A semi-structured interview guide was utilized based on the Consolidated Framework for Implementation Research (CFIR) context and the Diffusion of Innovation model. Transcripts were coded independently and compared by two members of the research team. Descriptive statistics were generated using Microsoft Excel. RESULTS: Six main themes emerged: awareness and use of PGx testing; source of information; key stakeholders of the PGx supply chain, their interactions and change agents; clinical benefit and significance of PGx testing; barriers and lack of reimbursement; and recommendations to boost the PGx adoption rate. Most respondents were aware of PGx applications, but only three had already recommended PGx testing. Peer-reviewed journals along with clinical guidelines were regarded as the most used source of information while stakeholders of the PGx supply chain were discussed. PGx was considered that promote patient-centered care, enhance medication clinical effectiveness, decrease the risk of side effects, and reduce healthcare costs. Lack of reimbursement, scarcity of resources, and high PGx cost were the foremost barriers affecting PGx adoption. CONCLUSIONS: It was concluded that if case PGx testing is reimbursed and physicians' training is reinforced, PGx implementation will be boosted and improved shortly.