Tumor microenvironment-modulated nanoparticles with cascade energy transfer as internal light sources for photodynamic therapy of deep-seated tumors.

Photodynamic therapy (PDT) is an appealing modality for cancer treatments. However, the limited tissue penetration depth of external-excitation light makes PDT impossible in treating deep-seated tumors. Meanwhile, tumor hypoxia and intracellular reductive microenvironment restrain the generation of reactive oxygen species (ROS). To overcome these limitations, a tumor-targeted self-illuminating supramolecular nanoparticle T-NPCe6-L-N is proposed by integrating photosensitizer Ce6 with luminol and nitric oxide (NO) for chemiluminescence resonance energy transfer (CRET)-activated PDT. The high H2O2 level in tumor can trigger chemiluminescence of luminol to realize CRET-activated PDT without exposure of external light. Meanwhile, the released NO significantly relieves tumor hypoxia via vascular normalization and reduces intracellular reductive GSH level, further enhancing ROS abundance. Importantly, due to the different ROS levels between cancer cells and normal cells, T-NPCe6-L-N can selectively trigger PDT in cancer cells while sparing normal cells, which ensured low side effect. The combination of CRET-based photosensitizer-activation and tumor microenvironment modulation overcomes the innate challenges of conventional PDT, demonstrating efficient inhibition of orthotopic and metastatic tumors on mice. It also provoked potent immunogenic cell death to ensure long-term suppression effects. The proof-of-concept research proved as a new strategy to solve the dilemma of PDT in treatment of deep-seated tumors.

Dynamics of epitranscriptomes uncover translational reprogramming directed by ac4C in rice during pathogen infection.

Messenger RNA modifications play pivotal roles in RNA biology, but comprehensive landscape changes of epitranscriptomes remain largely unknown in plant immune response. Here we report translational reprogramming directed by ac4C mRNA modification upon pathogen challenge. We first investigate the dynamics of translatomes and epitranscriptomes and uncover that the change in ac4C at single-base resolution promotes translational reprogramming upon Magnaporthe oryzae infection. Then by characterizing the specific distributions of m1A, 2'O-Nm, ac4C, m5C, m6A and m7G, we find that ac4Cs, unlike other modifications, are enriched at the 3rd position of codons, which stabilizes the Watson-Crick base pairing. Importantly, we demonstrate that upon pathogen infection, the increased expression of the ac4C writer OsNAT10/OsACYR (N-ACETYLTRANSFERASE FOR CYTIDINE IN RNA) promotes translation to facilitate rapid activation of immune responses, including the enhancement of jasmonic acid biosynthesis. Our study provides an atlas of mRNA modifications and insights into ac4C function in plant immunity.

Unmasking and quantifying racial bias of large language models in medical report generation.

BACKGROUND: Large language models like GPT-3.5-turbo and GPT-4 hold promise for healthcare professionals, but they may inadvertently inherit biases during their training, potentially affecting their utility in medical applications. Despite few attempts in the past, the precise impact and extent of these biases remain uncertain. METHODS: We use LLMs to generate responses that predict hospitalization, cost and mortality based on real patient cases. We manually examine the generated responses to identify biases. RESULTS: We find that these models tend to project higher costs and longer hospitalizations for white populations and exhibit optimistic views in challenging medical scenarios with much higher survival rates. These biases, which mirror real-world healthcare disparities, are evident in the generation of patient backgrounds, the association of specific diseases with certain racial and ethnic groups, and disparities in treatment recommendations, etc. CONCLUSIONS: Our findings underscore the critical need for future research to address and mitigate biases in language models, especially in critical healthcare applications, to ensure fair and accurate outcomes for all patients.

Large language models (LLMs) such as GPT-3.5-turbo and GPT-4 are advanced computer programs that can understand and generate text. They have the potential to help doctors and other healthcare professionals to improve patient care. We looked at how well these models predicted the cost of healthcare for patients, and the chances of them being hospitalized or dying. We found that these models often projected higher costs and longer hospital stays for white people than people from other racial or ethnicity groups. These biases mirror the disparities in real-world healthcare. Our findings show the need for more research to ensure that inappropriate biases are removed from LLMs to ensure fair and accurate healthcare predictions of possible outcomes for all patients. This will help ensure that these tools can be used effectively to improve healthcare for everyone.

Closing the gap between open source and commercial large language models for medical evidence summarization.

Large language models (LLMs) hold great promise in summarizing medical evidence. Most recent studies focus on the application of proprietary LLMs. Using proprietary LLMs introduces multiple risk factors, including a lack of transparency and vendor dependency. While open-source LLMs allow better transparency and customization, their performance falls short compared to the proprietary ones. In this study, we investigated to what extent fine-tuning open-source LLMs can further improve their performance. Utilizing a benchmark dataset, MedReview, consisting of 8161 pairs of systematic reviews and summaries, we fine-tuned three broadly-used, open-sourced LLMs, namely PRIMERA, LongT5, and Llama-2. Overall, the performance of open-source models was all improved after fine-tuning. The performance of fine-tuned LongT5 is close to GPT-3.5 with zero-shot settings. Furthermore, smaller fine-tuned models sometimes even demonstrated superior performance compared to larger zero-shot models. The above trends of improvement were manifested in both a human evaluation and a larger-scale GPT4-simulated evaluation.

Nanomaterial-based therapeutics for enhanced antifungal therapy.

The application of nanotechnology in antifungal therapy is gaining increasing attention. Current antifungal drugs have significant limitations, such as severe side effects, low bioavailability, and the rapid development of resistance. Nanotechnology offers an innovative solution to address these issues. This review discusses three key strategies of nanotechnology to enhance antifungal efficacy. Firstly, nanomaterials can enhance their interaction with fungal cells via ingenious surface tailoring of nanomaterials. Effective adhesion of nanoparticles to fungal cells can be achieved by electrostatic interaction or specific targeting to the fungal cell wall and cell membrane. Secondly, stimuli-responsive nanomaterials are developed to realize smart release of drugs in the specific microenvironment of pathological tissues, such as the fungal biofilm microenvironment and inflammatory microenvironment. Thirdly, nanomaterials can be designed to cross different physiological barriers, effectively addressing challenges posed by skin, corneal, and blood-brain barriers. Additionally, some new nanomaterial-based strategies in treating fungal infections are discussed, including the development of fungal vaccines, modulation of macrophage activity, phage therapy, the application of high-throughput screening in drug discovery, and so on. Despite the challenges faced in applying nanotechnology to antifungal therapy, its significant potential and innovation open new possibilities for future clinical antifungal applications.

Enhanced MALDI-2 Sensitivity with Reflecting Post-Ionization Laser for High-Resolution MS Imaging Combined with Real-Time Microscope Imaging.

Laser-induced matrix-assisted laser desorption/ionization post-ionization (MALDI-2) could improve the MALDI sensitivity of biological metabolites by over 1 order of magnitude. Herein, we demonstrate that MALDI-2 sensitivity can be further enhanced with reflecting post-ionization laser that multiplies the intersection times between laser and MALDI plume. This method, which we named MALDI-2+, typically brought over 2 times sensitivity improvement from conventional MALDI-2. Advancing in sensitivity thereby prompted us to pursue higher mass spectrometry imaging (MSI) spatial resolution. A dedicated T-shaped ion guide was designed to allow perpendicular incidence of ablation laser in reflection geometry MALDI. Although 8-10 µm pixel was used in MALDI imaging due to the limited precision of the motorized stage, the laser spot diameter could be down to 2.5 µm for potentially higher spatial resolution. In addition, this ion source enabled real-time and high-quality microscope imaging from backward of the sample plate. Beneficially, we were able to monitor the actual laser spot condition in real time as well as obtain high-resolution microscopic sample images that inherently register with MSI images. All of these benefits have been demonstrated by analyzing standard samples and imaging of cells. We believe that the enhancement in sensitivity, spatial resolution, and microscope capacity of our design could facilitate spatial omics studies.

Removable Photocatalysis Microneedle Reactor for Carbon Monoxide Delivery to Enhance Chemosensitization.

Carbon monoxide (CO) has emerged as a promising therapeutic agent, yet ensuring safe and precise CO delivery remains challenging. Here, we report a removable hydrogel-forming microneedle (MN) reactor for CO delivery via photocatalysis, with an emphasis on chemosensitization. Upon application, body fluids absorbed by the MNs dissolve the effervescent agents, leading to the generation of carbon dioxide (CO2) and triggering the release of the chemotherapeutics cisplatin. Meanwhile, the photocatalysts (PCs) trapped within MNs convert CO2 to CO under 660 nm light irradiation. These PCs can be removed by hydrogel-forming MNs, thereby mitigating potential biological risks associated with residual PCs. Both in vitro and in vivo experiments showed that MN-mediated CO delivery significantly improved tumor sensitivity to cisplatin by suppressing DNA repair, using an A375/CDDP melanoma model. This removable photocatalysis MN reactor offers safe and precise local delivery of CO, potentially creating new opportunities for CO or its combination therapies.

Hidden flaws behind expert-level accuracy of multimodal GPT-4 vision in medicine.

Recent studies indicate that Generative Pre-trained Transformer 4 with Vision (GPT-4V) outperforms human physicians in medical challenge tasks. However, these evaluations primarily focused on the accuracy of multi-choice questions alone. Our study extends the current scope by conducting a comprehensive analysis of GPT-4V's rationales of image comprehension, recall of medical knowledge, and step-by-step multimodal reasoning when solving New England Journal of Medicine (NEJM) Image Challenges-an imaging quiz designed to test the knowledge and diagnostic capabilities of medical professionals. Evaluation results confirmed that GPT-4V performs comparatively to human physicians regarding multi-choice accuracy (81.6% vs. 77.8%). GPT-4V also performs well in cases where physicians incorrectly answer, with over 78% accuracy. However, we discovered that GPT-4V frequently presents flawed rationales in cases where it makes the correct final choices (35.5%), most prominent in image comprehension (27.2%). Regardless of GPT-4V's high accuracy in multi-choice questions, our findings emphasize the necessity for further in-depth evaluations of its rationales before integrating such multimodal AI models into clinical workflows.

Syntropic spin alignment at the interface between ferromagnetic and superconducting nitrides.

The magnetic correlations at the superconductor/ferromagnet (S/F) interfaces play a crucial role in realizing dissipation-less spin-based logic and memory technologies, such as triplet-supercurrent spin-valves and 'π' Josephson junctions. Here we report the observation of an induced large magnetic moment at high-quality nitride S/F interfaces. Using polarized neutron reflectometry and DC SQUID measurements, we quantitatively determined the magnetization profile of the S/F bilayer and confirmed that the induced magnetic moment in the adjacent superconductor only exists below T C. Interestingly, the direction of the induced moment in the superconductors was unexpectedly parallel to that in the ferromagnet, which contrasts with earlier findings in S/F heterostructures based on metals or oxides. First-principles calculations verified that the unusual interfacial spin texture observed in our study was caused by the Heisenberg direct exchange coupling with constant J∼4.28 meV through d-orbital overlapping and severe charge transfer across the interfaces. Our work establishes an incisive experimental probe for understanding the magnetic proximity behavior at S/F interfaces and provides a prototype epitaxial 'building block' for superconducting spintronics.

Through virtual saturation mutagenesis and rational design for superior substrate conversion in engineered d-amino acid oxidase.

The d-amino acid oxidase (DAAO) is pivotal in obtaining optically pure l-glufosinate (l-PPT) by converting d-glufosinate (d-PPT) to its deamination product. We screened and designed a Rasamsonia emersonii DAAO (ReDAAO), making it more suitable for oxidizing d-PPT. Using Caver 3.0, we delineated three substrate binding pockets and, via alanine scanning, identified nearby key residues. Pinpointing key residues influencing activity, we applied virtual saturation mutagenesis (VSM), and experimentally validated mutants which reduced substrate binding energy. Analysis of positive mutants revealed elongated side-chain prevalence in substrate binding pocket periphery. Although computer-aided approaches can rapidly identify advantageous mutants and guide further design, the mutations obtained in the first round may not be suitable for combination with other advantageous mutations. Therefore, each round of combination requires reasonable iteration. Employing VSM-assisted screening multiple times and after four rounds of combining mutations, we ultimately obtained a mutant, N53V/F57Q/V94R/V242R, resulting in a mutant with a 5097% increase in enzyme activity compared to the wild type. It provides valuable insights into the structural determinants of enzyme activity and introduces a novel rational design procedure.

Physiological and molecular mechanisms of silicon and potassium on mitigating iron-toxicity stress in Panax ginseng.

Iron plays a crucial role in plant chlorophyll synthesis, respiration, and plant growth. However, excessive iron content can contribute to ginseng poisoning. We previously discovered that the application of silicon (Si) and potassium (K) can mitigate the iron toxicity on ginseng. To elucidate the molecular mechanism of how Si and K alleviate iron toxicity stress in ginseng. We investigated the physiological and transcriptional effects of exogenous Si and K on Panax ginseng. The results suggested that the leaves of ginseng with Si and K addition under iron stress increased antioxidant enzyme activity or secondary metabolite content, such as phenylalanine amino-lyase, polyphenol oxidase, ascorbate peroxidase, total phenols and lignin, by 6.21%-25.94%, 30.12%-309.19%, 32.26%-38.82%, 7.81%-23.66%, and 4.68%-48.42%, respectively. Moreover, Si and K increased the expression of differentially expressed genes (DEGs) associated with resistance to both biotic and abiotic stress, including WRKY (WRKY1, WRKY5, and WRKY65), bHLH (bHLH35, bHLH66, bHLH128, and bHLH149), EREBP, ERF10 and ZIP. Additionally, the amount of DEGs of ginseng by Si and K addition was enriched in metabolic processes, single-organism process pathways, signal transduction, metabolism, synthesis and disease resistance. In conclusion, the utilization of Si and K can potentially reduce the accumulation of iron in ginseng, regulate the expression of iron tolerance genes, and enhance the antioxidant enzyme activity and secondary metabolite production in both leaves and roots, thus alleviating the iron toxicity stress in ginseng.

The negative spillover impact of the COVID-19 pandemic on Hong Kong diabetes patients' mortality and cardiovascular disease.

BACKGROUND: The spillover impact from disrupted healthcare services for non-COVID-infected diabetes mellitus (DM) patients caused by the reshuffling of the manpower during the pandemic remains understudied, especially in Hong Kong where healthcare resources were already strained before the pandemic. AIM: To evaluate the spill-over effect of the Pandemic on Hong Kong diabetes patients, we examined the change in all-cause mortality and the incidence of cardiovascular disease (CVD) from 2012 to 2021. METHOD: This retrospective cohort study analyzed data from Hong Kong Hospital Authority healthcare records covering all publicly provided care. Adults diagnosed with DM on/before December 31, 2010, without CVD before January 2012 were included. The 2016-2019 average all-cause mortality and the incidence of CVD after age-standardization represented the pre-pandemic levels. Subjects would leave the cohort after being infected with COVID-19. RESULTS: A cohort of 159,693 patients with diabetes was identified and followed up for 10 years from January 2012 to December 2021. Compared to the pre-pandemic levels, 2020 saw a 12% increase in age-standardized mortality per 10,000 diabetic patients (incidence rate ratio [95% CI]: 1.12 [1.10 - 1.14]), but no significant change in age-standardized CVD incidence. However, in 2021, there were 11% (1.11[1.10 - 1.13]) and 13% (1.13 [1.11 - 1.15]) more new CVD cases and deaths, respectively, versus the pre-pandemic period. CONCLUSION: The COVID-19 outbreak in 2020 had negative spillover impacts on DM patients without COVID-19 in Hong Kong, with a higher mortality in 2020 and 2021 compared with the pre-pandemic level.

GeneAgent: Self-verification Language Agent for Gene Set Knowledge Discovery using Domain Databases.

Gene set knowledge discovery is essential for advancing human functional genomics. Recent studies have shown promising performance by harnessing the power of Large Language Models (LLMs) on this task. Nonetheless, their results are subject to several limitations common in LLMs such as hallucinations. In response, we present GeneAgent, a first-of-its-kind language agent featuring self-verification capability. It autonomously interacts with various biological databases and leverages relevant domain knowledge to improve accuracy and reduce hallucination occurrences. Benchmarking on 1,106 gene sets from different sources, GeneAgent consistently outperforms standard GPT-4 by a significant margin. Moreover, a detailed manual review confirms the effectiveness of the self-verification module in minimizing hallucinations and generating more reliable analytical narratives. To demonstrate its practical utility, we apply GeneAgent to seven novel gene sets derived from mouse B2905 melanoma cell lines, with expert evaluations showing that GeneAgent offers novel insights into gene functions and subsequently expedites knowledge discovery.

Harnessing PubMed User Query Logs for Post Hoc Explanations of Recommended Similar Articles.

Searching for a related article based on a reference article is an integral part of scientific research. PubMed, like many academic search engines, has a "similar articles" feature that recommends articles relevant to the current article viewed by a user. Explaining recommended items can be of great utility to users, particularly in the literature search process. With more than a million biomedical papers being published each year, explaining the recommended similar articles would facilitate researchers and clinicians in searching for related articles. Nonetheless, the majority of current literature recommendation systems lack explanations for their suggestions. We employ a post hoc approach to explaining recommendations by identifying relevant tokens in the titles of similar articles. Our major contribution is building PubCLogs by repurposing 5.6 million pairs of coclicked articles from PubMed's user query logs. Using our PubCLogs dataset, we train the Highlight Similar Article Title (HSAT), a transformer-based model designed to select the most relevant parts of the title of a similar article, based on the title and abstract of a seed article. HSAT demonstrates strong performance in our empirical evaluations, achieving an F1 score of 91.72 percent on the PubCLogs test set, considerably outperforming several baselines including BM25 (70.62), MPNet (67.11), MedCPT (62.22), GPT-3.5 (46.00), and GPT-4 (64.89). Additional evaluations on a separate, manually annotated test set further verifies HSAT's performance. Moreover, participants of our user study indicate a preference for HSAT, due to its superior balance between conciseness and comprehensiveness. Our study suggests that repurposing user query logs of academic search engines can be a promising way to train state-of-the-art models for explaining literature recommendation.

Leveraging Professional Radiologists' Expertise to Enhance LLMs' Evaluation for Radiology Reports.

In radiology, Artificial Intelligence (AI) has significantly advanced report generation, but automatic evaluation of these AI-produced reports remains challenging. Current metrics, such as Conventional Natural Language Generation (NLG) and Clinical Efficacy (CE), often fall short in capturing the semantic intricacies of clinical contexts or overemphasize clinical details, undermining report clarity. To overcome these issues, our proposed method synergizes the expertise of professional radiologists with Large Language Models (LLMs), like GPT-3.5 and GPT-4. Utilizing In-Context Instruction Learning (ICIL) and Chain of Thought (CoT) reasoning, our approach aligns LLM evaluations with radiologist standards, enabling detailed comparisons between human and AI-generated reports. This is further enhanced by a Regression model that aggregates sentence evaluation scores. Experimental results show that our "Detailed GPT-4 (5-shot)" model achieves a 0.48 score, outperforming the METEOR metric by 0.19, while our "Regressed GPT-4" model shows even greater alignment with expert evaluations, exceeding the best existing metric by a 0.35 margin. Moreover, the robustness of our explanations has been validated through a thorough iterative strategy. We plan to publicly release annotations from radiology experts, setting a new standard for accuracy in future assessments. This underscores the potential of our approach in enhancing the quality assessment of AI-driven medical reports.

Endowing Pt(IV) with Perfluorocarbon Chains and Human Serum Albumin Encapsulation for Highly Effective Antitumor Chemoimmunotherapy.

Tumor metastases and reoccurrence are considered the leading causes of cancer-associated deaths. As an emerging therapeutic method, increasing research efforts have been devoted to immunogenic cell death (ICD)-inducing compounds to solve the challenge. The clinically approved chemotherapeutic Pt complexes are not or are only poorly able to trigger ICD. Herein, the axial functionalization of the Pt(II) complex cisplatin with perfluorocarbon chains into ICD-inducing Pt(IV) prodrugs is reported. Strikingly, while the Pt(II) complex as well as the perfluorocarbon ligands did not induce ICD, the Pt(IV) prodrug demonstrated unexpectantly the induction of ICD through accumulation in the endoplasmic reticulum and generation of reactive oxygen species in this organelle. To enhance the pharmacological properties, the compound was encapsulated with human serum albumin into nanoparticles. While selectively accumulating in the tumorous tissue, the nanoparticles demonstrated a strong tumor growth inhibitory effect against osteosarcoma inside a mouse model. In vivo tumor vaccine analysis also demonstrated the ability of Pt(IV) to be an ideal ICD inducer. Overall, this study reports on axially perfluorocarbon chain-modified Pt(IV) complexes for ICD induction and chemoimmunotherapy in osteosarcoma.

An Organophosphorescence Probe with Ultralong Lifetime and Intrinsic Tissue Selectivity for Specific Tumor Imaging and Guided Tumor Surgery.

Organic phosphorescent materials are excellent candidates for use in tumor imaging. However, a systematic comparison of the effects of the intensity, lifetime, and wavelength of phosphorescent emissions on bioimaging performance has not yet been undertaken. In addition, there have been few reports on organic phosphorescent materials that specifically distinguish tumors from normal tissues. This study addresses these gaps and reveals that longer lifetimes effectively increase the signal intensity, whereas longer wavelengths enhance the penetration depth. Conversely, a strong emission intensity with a short lifetime does not necessarily yield robust imaging signals. Building upon these findings, an organo-phosphorescent material with a lifetime of 0.94 s was designed for tumor imaging. Remarkably, the phosphorescent signals of various organic nanoparticles are nearly extinguished in blood-rich organs because of the quenching effect of iron ions. Moreover, for the first time, we demonstrated that iron ions universally quench the phosphorescence of organic room-temperature phosphorescent materials, which is an inherent property of such substances. Leveraging this property, both the normal liver and hepatitis tissues exhibit negligible phosphorescent signals, whereas liver tumors display intense phosphorescence. Therefore, phosphorescent materials, unlike chemiluminescent or fluorescent materials, can exploit this unique inherent property to selectively distinguish liver tumor tissues from normal tissues without additional modifications or treatments.

Associations and attributable burden between risk factors and all-cause and cause-specific mortality at different ages in patients with hypertension.

It remains unclear the age-specific associations of risk factors with deaths and mortality burden attributable across age. In a territory-wide retrospective cohort, 1,012,228 adults with hypertension were identified. Comorbidities including diabetes, chronic kidney disease (CKD), cardiovascular disease (CVD), heart failure, and cancer, and risk factors including current smoking and suboptimal control of blood pressure (BP), glucose and low-density lipoprotein cholesterol were defined. Associations of comorbidities/risk factors with all-cause and cause-specific mortality across age groups (18-54, 55-64, 65-74, and ≥75 years) were assessed. Population attributable fractions were also quantified. During a median follow-up of 10.7 years, 244,268 (24.1%) patients died, with pneumonia (7.2%), cancer (5.1%), and CVD (4.2%) being the leading causes. Despite increasing deaths with age, relative risk of mortality related to comorbidities/risk factors decreased with age; similar patterns were found for cause-specific mortality. The assessed risk factors accounted for 24.0% (95% CI 22.5%, 25.4%) deaths, with highest proportion in the youngest group (33.5% [28.1%, 38.5%] in 18-54 years vs 19.4% [17.0%, 21.6%] in ≥75 years). For mortality burden, CKD was the overall leading risk factor (12.7% [12.4%, 12.9%]) with higher proportions in older patients (11.1-13.1% in ≥65 years), while diabetes was the leading risk factor in younger patients (15.9-13.5% in 18-54 years). The association of comorbidities or risk factors with mortality is stronger in younger patients with hypertension, despite lower absolute mortality in young patients than in the elderly. Leading risk factors differed across age, highlighting the importance of targeted and precise risk management.

Quality of Answers of Generative Large Language Models Versus Peer Users for Interpreting Laboratory Test Results for Lay Patients: Evaluation Study.

BACKGROUND: Although patients have easy access to their electronic health records and laboratory test result data through patient portals, laboratory test results are often confusing and hard to understand. Many patients turn to web-based forums or question-and-answer (Q&A) sites to seek advice from their peers. The quality of answers from social Q&A sites on health-related questions varies significantly, and not all responses are accurate or reliable. Large language models (LLMs) such as ChatGPT have opened a promising avenue for patients to have their questions answered. OBJECTIVE: We aimed to assess the feasibility of using LLMs to generate relevant, accurate, helpful, and unharmful responses to laboratory test-related questions asked by patients and identify potential issues that can be mitigated using augmentation approaches. METHODS: We collected laboratory test result-related Q&A data from Yahoo! Answers and selected 53 Q&A pairs for this study. Using the LangChain framework and ChatGPT web portal, we generated responses to the 53 questions from 5 LLMs: GPT-4, GPT-3.5, LLaMA 2, MedAlpaca, and ORCA_mini. We assessed the similarity of their answers using standard Q&A similarity-based evaluation metrics, including Recall-Oriented Understudy for Gisting Evaluation, Bilingual Evaluation Understudy, Metric for Evaluation of Translation With Explicit Ordering, and Bidirectional Encoder Representations from Transformers Score. We used an LLM-based evaluator to judge whether a target model had higher quality in terms of relevance, correctness, helpfulness, and safety than the baseline model. We performed a manual evaluation with medical experts for all the responses to 7 selected questions on the same 4 aspects. RESULTS: Regarding the similarity of the responses from 4 LLMs; the GPT-4 output was used as the reference answer, the responses from GPT-3.5 were the most similar, followed by those from LLaMA 2, ORCA_mini, and MedAlpaca. Human answers from Yahoo data were scored the lowest and, thus, as the least similar to GPT-4-generated answers. The results of the win rate and medical expert evaluation both showed that GPT-4's responses achieved better scores than all the other LLM responses and human responses on all 4 aspects (relevance, correctness, helpfulness, and safety). LLM responses occasionally also suffered from lack of interpretation in one's medical context, incorrect statements, and lack of references. CONCLUSIONS: By evaluating LLMs in generating responses to patients' laboratory test result-related questions, we found that, compared to other 4 LLMs and human answers from a Q&A website, GPT-4's responses were more accurate, helpful, relevant, and safer. There were cases in which GPT-4 responses were inaccurate and not individualized. We identified a number of ways to improve the quality of LLM responses, including prompt engineering, prompt augmentation, retrieval-augmented generation, and response evaluation.