Evaluation and mitigation of the limitations of large language models in clinical decision-making.

Clinical decision-making is one of the most impactful parts of a physician's responsibilities and stands to benefit greatly from artificial intelligence solutions and large language models (LLMs) in particular. However, while LLMs have achieved excellent performance on medical licensing exams, these tests fail to assess many skills necessary for deployment in a realistic clinical decision-making environment, including gathering information, adhering to guidelines, and integrating into clinical workflows. Here we have created a curated dataset based on the Medical Information Mart for Intensive Care database spanning 2,400 real patient cases and four common abdominal pathologies as well as a framework to simulate a realistic clinical setting. We show that current state-of-the-art LLMs do not accurately diagnose patients across all pathologies (performing significantly worse than physicians), follow neither diagnostic nor treatment guidelines, and cannot interpret laboratory results, thus posing a serious risk to the health of patients. Furthermore, we move beyond diagnostic accuracy and demonstrate that they cannot be easily integrated into existing workflows because they often fail to follow instructions and are sensitive to both the quantity and order of information. Overall, our analysis reveals that LLMs are currently not ready for autonomous clinical decision-making while providing a dataset and framework to guide future studies.

A Numerical Investigation of Dimensionless Numbers Characterizing Meltpool Morphology of the Laser Powder Bed Fusion Process.

Microstructure evolution in metal additive manufacturing (AM) is a complex multi-physics and multi-scale problem. Understanding the impact of AM process conditions on the microstructure evolution and the resulting mechanical properties of the printed component remains an active area of research. At the meltpool scale, the thermo-fluidic governing equations have been extensively modeled in the literature to understand the meltpool conditions and the thermal gradients in its vicinity. In many phenomena governed by partial differential equations, dimensional analysis and identification of important dimensionless numbers can provide significant insights into the process dynamics. In this context, we present a novel strategy using dimensional analysis and the linear least-squares regression method to numerically investigate the thermo-fluidic governing equations of the Laser Powder Bed Fusion AM process. First, the governing equations are solved using the Finite Element Method, and the model predictions are validated by comparing with experimentally estimated cooling rates, and with numerical results from the literature. Then, through dimensional analysis, an important dimensionless quantity interpreted as a measure of heat absorbed by the powdered material and the meltpool, is identified. This dimensionless measure of absorbed heat, along with classical dimensionless quantities such as Péclet, Marangoni, and Stefan numbers, are employed to investigate advective transport in the meltpool for different alloys. Further, the framework is used to study variations in the thermal gradients and the solidification cooling rate. Important correlations linking meltpool morphology and microstructure-evolution-related variables with classical dimensionless numbers are the key contribution of this work.