Large language models encode clinical knowledge.

Large language models (LLMs) have demonstrated impressive capabilities, but the bar for clinical applications is high. Attempts to assess the clinical knowledge of models typically rely on automated evaluations based on limited benchmarks. Here, to address these limitations, we present MultiMedQA, a benchmark combining six existing medical question answering datasets spanning professional medicine, research and consumer queries and a new dataset of medical questions searched online, HealthSearchQA. We propose a human evaluation framework for model answers along multiple axes including factuality, comprehension, reasoning, possible harm and bias. In addition, we evaluate Pathways Language Model1 (PaLM, a 540-billion parameter LLM) and its instruction-tuned variant, Flan-PaLM2 on MultiMedQA. Using a combination of prompting strategies, Flan-PaLM achieves state-of-the-art accuracy on every MultiMedQA multiple-choice dataset (MedQA3, MedMCQA4, PubMedQA5 and Measuring Massive Multitask Language Understanding (MMLU) clinical topics6), including 67.6% accuracy on MedQA (US Medical Licensing Exam-style questions), surpassing the prior state of the art by more than 17%. However, human evaluation reveals key gaps. To resolve this, we introduce instruction prompt tuning, a parameter-efficient approach for aligning LLMs to new domains using a few exemplars. The resulting model, Med-PaLM, performs encouragingly, but remains inferior to clinicians. We show that comprehension, knowledge recall and reasoning improve with model scale and instruction prompt tuning, suggesting the potential utility of LLMs in medicine. Our human evaluations reveal limitations of today's models, reinforcing the importance of both evaluation frameworks and method development in creating safe, helpful LLMs for clinical applications.

LifeTime and improving European healthcare through cell-based interceptive medicine.

Here we describe the LifeTime Initiative, which aims to track, understand and target human cells during the onset and progression of complex diseases, and to analyse their response to therapy at single-cell resolution. This mission will be implemented through the development, integration and application of single-cell multi-omics and imaging, artificial intelligence and patient-derived experimental disease models during the progression from health to disease. The analysis of large molecular and clinical datasets will identify molecular mechanisms, create predictive computational models of disease progression, and reveal new drug targets and therapies. The timely detection and interception of disease embedded in an ethical and patient-centred vision will be achieved through interactions across academia, hospitals, patient associations, health data management systems and industry. The application of this strategy to key medical challenges in cancer, neurological and neuropsychiatric disorders, and infectious, chronic inflammatory and cardiovascular diseases at the single-cell level will usher in cell-based interceptive medicine in Europe over the next decade.

Creation and Adoption of Large Language Models in Medicine.

Importance: There is increased interest in and potential benefits from using large language models (LLMs) in medicine. However, by simply wondering how the LLMs and the applications powered by them will reshape medicine instead of getting actively involved, the agency in shaping how these tools can be used in medicine is lost. Observations: Applications powered by LLMs are increasingly used to perform medical tasks without the underlying language model being trained on medical records and without verifying their purported benefit in performing those tasks. Conclusions and Relevance: The creation and use of LLMs in medicine need to be actively shaped by provisioning relevant training data, specifying the desired benefits, and evaluating the benefits via testing in real-world deployments.

ChatGPT Performs on the Chinese National Medical Licensing Examination.

ChatGPT, a language model developed by OpenAI, uses a 175 billion parameter Transformer architecture for natural language processing tasks. This study aimed to compare the knowledge and interpretation ability of ChatGPT with those of medical students in China by administering the Chinese National Medical Licensing Examination (NMLE) to both ChatGPT and medical students. We evaluated the performance of ChatGPT in three years' worth of the NMLE, which consists of four units. At the same time, the exam results were compared to those of medical students who had studied for five years at medical colleges. ChatGPT's performance was lower than that of the medical students, and ChatGPT's correct answer rate was related to the year in which the exam questions were released. ChatGPT's knowledge and interpretation ability for the NMLE were not yet comparable to those of medical students in China. It is probable that these abilities will improve through deep learning.

Virtual and Augmented Reality Applications in Medicine: Analysis of the Scientific Literature.

BACKGROUND: Virtual reality (VR) and augmented reality (AR) have recently become popular research themes. However, there are no published bibliometric reports that have analyzed the corresponding scientific literature in relation to the application of these technologies in medicine. OBJECTIVE: We used a bibliometric approach to identify and analyze the scientific literature on VR and AR research in medicine, revealing the popular research topics, key authors, scientific institutions, countries, and journals. We further aimed to capture and describe the themes and medical conditions most commonly investigated by VR and AR research. METHODS: The Web of Science electronic database was searched to identify relevant papers on VR research in medicine. Basic publication and citation data were acquired using the "Analyze" and "Create Citation Report" functions of the database. Complete bibliographic data were exported to VOSviewer and Bibliometrix, dedicated bibliometric software packages, for further analyses. Visualization maps were generated to illustrate the recurring keywords and words mentioned in the titles and abstracts. RESULTS: The analysis was based on data from 8399 papers. Major research themes were diagnostic and surgical procedures, as well as rehabilitation. Commonly studied medical conditions were pain, stroke, anxiety, depression, fear, cancer, and neurodegenerative disorders. Overall, contributions to the literature were globally distributed with heaviest contributions from the United States and United Kingdom. Studies from more clinically related research areas such as surgery, psychology, neurosciences, and rehabilitation had higher average numbers of citations than studies from computer sciences and engineering. CONCLUSIONS: The conducted bibliometric analysis unequivocally reveals the versatile emerging applications of VR and AR in medicine. With the further maturation of the technology and improved accessibility in countries where VR and AR research is strong, we expect it to have a marked impact on clinical practice and in the life of patients.

Drawing lessons from the COVID-19 pandemic: science and epistemic humility should go together.

During the COVID-19 pandemic, scientific experts advised governments for measures to be promptly taken; they also helped people to understand the situation. They carried out this role in the face of a worldwide emergency, when scientific understanding was still underway. Public scientific disputes also arose, creating confusion among people. This article highlights the importance of experts' epistemic stance under these circumstances. It suggests they should embrace the intellectual virtue of epistemic humility, regulating their epistemic behavior and communication accordingly. In so doing, they would also favour the functioning of the broad network of knowledge-based experts, which is required to properly address all the aspects of the global pandemic.

Preventive medicine's equivalence problem.

The structure of preventive medicine residency training in the U.S. warrants serious examination. U.S. public health and general preventive medicine residencies have suffered a 17% decline in the number of residency programs since 2000, and current residency programs are, on average, half-empty. The required clinical year is not unique to preventive medicine, a basic, undifferentiated MPH for preventive medicine doesn't distinguish the preventive medicine specialist, and practicum year requirements are overly broad and not necessarily specific to the specialty, leaving the specialty vulnerable to equivalence by most other specialties. Strategies including creation of an additional preventive medicine-specific clinical year, developing a new public health degree for the specialty, and more specific practicum rotations, as well as potentially changing the specialty's name and altering the annual structure of training, are proposed along with an equivalence test.