Data Set and Benchmark (MedGPTEval) to Evaluate Responses From Large Language Models in Medicine: Evaluation Development and Validation.

Background: Large language models (LLMs) have achieved great progress in natural language processing tasks and demonstrated the potential for use in clinical applications. Despite their capabilities, LLMs in the medical domain are prone to generating hallucinations (not fully reliable responses). Hallucinations in LLMs' responses create substantial risks, potentially threatening patients' physical safety. Thus, to perceive and prevent this safety risk, it is essential to evaluate LLMs in the medical domain and build a systematic evaluation. Objective: We developed a comprehensive evaluation system, MedGPTEval, composed of criteria, medical data sets in Chinese, and publicly available benchmarks. Methods: First, a set of evaluation criteria was designed based on a comprehensive literature review. Second, existing candidate criteria were optimized by using a Delphi method with 5 experts in medicine and engineering. Third, 3 clinical experts designed medical data sets to interact with LLMs. Finally, benchmarking experiments were conducted on the data sets. The responses generated by chatbots based on LLMs were recorded for blind evaluations by 5 licensed medical experts. The evaluation criteria that were obtained covered medical professional capabilities, social comprehensive capabilities, contextual capabilities, and computational robustness, with 16 detailed indicators. The medical data sets include 27 medical dialogues and 7 case reports in Chinese. Three chatbots were evaluated: ChatGPT by OpenAI; ERNIE Bot by Baidu, Inc; and Doctor PuJiang (Dr PJ) by Shanghai Artificial Intelligence Laboratory. Results: Dr PJ outperformed ChatGPT and ERNIE Bot in the multiple-turn medical dialogues and case report scenarios. Dr PJ also outperformed ChatGPT in the semantic consistency rate and complete error rate category, indicating better robustness. However, Dr PJ had slightly lower scores in medical professional capabilities compared with ChatGPT in the multiple-turn dialogue scenario. Conclusions: MedGPTEval provides comprehensive criteria to evaluate chatbots by LLMs in the medical domain, open-source data sets, and benchmarks assessing 3 LLMs. Experimental results demonstrate that Dr PJ outperforms ChatGPT and ERNIE Bot in social and professional contexts. Therefore, such an assessment system can be easily adopted by researchers in this community to augment an open-source data set.

Enantiomeric separation of tryptophan via novel chiral polyamide composite membrane.

The separation of chiral drugs continues to pose a significant challenge. However, in recent years, the emergence of membrane-based chiral separation has shown promising effectiveness due to its environmentally friendly, energy-efficient, and cost-effective characteristics. In this study, we prepared chiral composite membrane via interfacial polymerization (IP), utilizing ß-cyclodextrin (ß-CD) and piperazine (PIP) as mixed monomers in the aqueous phase. The chiral separation process was facilitated by ß-CD, serving as a chiral selective agent. The resulting membrane were characterized using SEM, FT-IR, and XPS. Subsequently, the chiral separation performance of the membrane for DL-tryptophan (Trp) was investigated. Lastly, the water flux, dye rejection, and stability of the membrane were also examined. The results showed that the optimized chiral PIP0.5ß-CD0.5 membrane achieved an enantiomeric excess percentage (ee%) of 43.0% for D-Trp, with a solute flux of 66.18 nmol·cm-2·h-1, and maintained a good chiral separation stability. Additionally, the membrane demonstrated positive performance in the selective separation of mixed dyes, allowing for steady operation over a long period of time. This study offers fresh insights into membrane-based chiral separations.

Efficacy and Safety of Longyizhengqi Granule in Treatment of Mild COVID-19 Patients Caused by SARS-CoV-2 Omicron Variant: A Prospective Study.

Purpose: This study aimed to evaluate the clinical efficacy of Longyizhengqi granule, a traditional Chinese medicine, in patients with mild COVID-19. Patients and Methods: We conducted a prospective study including mild COVID-19 participants conducted at Mobile Cabin Hospital in Shanghai, China. Participants were assigned to receive Longyizhengqi granule or conventional treatment. The primary outcome was the time for nucleic acid to turn negative and the secondary outcomes are hospital stay and changes in cycle threshold (Ct) values for N gene and Orf gene. Multilevel random-intercept model was performed to analyze the effects of treatment. Results: A total of 3243 patients were included in this study (Longyizhengqi granule 667 patients; conventional treatment 2576 patients). Age (43.5 vs 42.1, p<0.01) and vaccination doses (not vaccinated: 15.8% vs 21.7%, 1 dose: 3.5% vs 2.9%, 2 doses: 27.9% vs 25.6%, 3 doses: 52.8% vs 49.8%. p<0.01) show statistical difference between Conventional treatment group and LYZQ granules group. The use of Longyizhengqi granule could significantly reduce the time for nucleic acid to turn negative (14.2 days vs 10.7 days, p<0.01), shorten hospital stay (12.5 days vs 9.9 days, p<0.01), and increase the changes in Ct value for N gene (8.44 vs 10.33, p<0.01) and Orf gene (7.31 vs 8.44, p<0.01) to approximately 1.5. Moreover, the difference in the changes of Ct values on the 4th, 6th, 8th, and 10th days seem to increase between two groups. No serious adverse events were reported. Conclusion: Longyizhengqi granule might be a promising drug for the treatment of mild COVID-19, and it might be beneficial to effectively shorten the negative transition time of nucleic acid, the total days of hospitalization, and increase the changes of Ct values. Long-term randomized controlled trials with follow-up evaluations are required to confirm its long-term efficacy.

Serum Nutritional Biomarkers and All-Cause and Cause-Specific Mortality in U.S. Adults with Metabolic Syndrome: The Results from National Health and Nutrition Examination Survey 2001-2006.

BACKGROUND: There is limited research on the associations between serum nutritional biomarkers and mortality risk in patients with metabolic syndrome (MetS). Existing studies merely investigated the single-biomarker effect. Thus, this study aimed to investigate the combined effect of nutritional biomarker mixtures and mortality risk using the Bayesian kernel machine regression (BKMR) model in patients with MetS. METHODS: We included the MetS patients, defined according to the 2018 Guideline on the Management of Blood Cholesterol from the National Health and Nutrition Examination Survey (NHANES) 2001-2006. A total of 20 serum nutritional biomarkers were measured and evaluated in this study. The Cox proportional hazard model and restricted cubic spline models were used to evaluate the individual linear and non-linear association of 20 nutritional biomarkers with mortality risk. Bayesian kernel machine regression (BKMR) was used to assess the associations between mixture of nutritional biomarkers and mortality risk. RESULTS: A total of 1455 MetS patients had a median age of 50 years (range: 20-85). During a median of 17.1-year follow-up, 453 (24.72%) died: 146 (7.20%) caused by CVD and 87 (5.26%) by cancer. Non-linear and linear analyses indicated that, in total, eight individual biomarkers (α-carotene, ß-carotene, bicarbonate, lutein/zeaxanthin, lycopene, potassium, protein, and vitamin A) were significantly associated with all-cause mortality (all p-values < 0.05). Results from BKMR showed an association between the low levels of the mixture of nutritional biomarkers and high risk of all-cause mortality with the estimated effects ranging from 0.04 to 0.14 (referent: medians). α-Carotene (PIP = 0.971) and potassium (PIP = 0.796) were the primary contributors to the combined effect of the biomarker mixture. The nutritional mixture levels were found to be negatively associated with the risk of cardiovascular disease (CVD) mortality and positively associated with the risk of cancer mortality. After it was stratified by nutrients, the mixture of vitamins showed a negative association with all-cause and CVD mortality, whereas the mixture of mineral-related biomarkers was positively associated with all-cause and cancer mortality. CONCLUSION: Our findings support the evidence that nutritional status was associated with long-term health outcomes in MetS patients. It is necessary for MetS patients to be concerned with certain nutritional status (i.e., vitamins and mineral elements).

Clinical characteristics of 1139 mild cases of the SARS-CoV-2 Omicron variant infected patients in Shanghai.

In March 2022, the Omicron variant of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) surged during the Coronavirus Disease 2019 (COVID-19) pandemic in Shanghai, but over 90% of patients were mild. This study included 1139 COVID-19 patients mildly infected with the Omicron variant of SARS-CoV-2 in Shanghai from May 1 to 10, 2022, aiming to clarify the demographic characteristics and clinical symptoms of patients with mild Omicron infection. The clinical phenotypes of Omicron infection were identified by model-based cluster analysis to explore the features of different clusters. The median age of the patients was 41.0 years [IQR: 31.0-52.0 years] and 73.0% were male. The top three clinical manifestations are cough (57.5%), expectoration (48.3%), and nasal congestion and runny nose (43.4%). The prevalence of nasal congestion and runny nose varied significantly across the doses of vaccinations, with 23.1% in the unvaccinated population and 30%, 45.9%, and 44.3% in the 1-dose, 2-dose and 3-dose vaccinated populations, respectively. In addition, there were significant differences for fever (23.1%, 26.0%, 28.6%, 18.4%), head and body heaviness (15.4%, 14.0%, 26.7%, 22.4%), and loss of appetite (25.6%, 30.0%, 33.6%, 27.7%). The unvaccinated population had a lower incidence of symptoms than the vaccinated population. Cluster analysis revealed that all four clusters had multisystemic symptoms and were dominated by both general and respiratory symptoms. The more severe the degree of the symptoms was, the higher the prevalence of multisystemic symptoms will be. The Omicron variant produced a lower incidence of symptoms in mildly infected patients than previous SARS-CoV-2 variants, but the clinical symptoms caused by the Omicron variant are more complex, so that it needs to be differentiated from influenza.

Theoretical Approach and Scale Construction of Patient Privacy Protection Behavior of Doctors in Public Medical Institutions in China: Pilot Development Study.

BACKGROUND: Considering the high incidence of medical privacy disclosure, it is of vital importance to study doctors' privacy protection behavior and its influencing factors. OBJECTIVE: We aim to develop a scale for doctors' protection of patients' privacy in Chinese public medical institutions, following construction of a theoretical model framework through grounded theory, and subsequently to validate the scale to measure this protection behavior. METHODS: Combined with the theoretical paradigm of protection motivation theory (PMT) and semistructured interview data, the grounded theory research method, followed by the Delphi expert and group discussion methods, a theoretical framework and initial scale for doctors in Chinese public medical institutions to protect patients' privacy was formed. The adjusted scale was collected online using a WeChat electronic survey measured using a 5-point Likert scale. Exploratory and confirmatory factor analysis (EFA and CFA) and tests to analyze reliability and validity were performed on the sample data. SPSS 19.0 and Amos 26.0 statistical analysis software were used for EFA and CFA of the sample data, respectively. RESULTS: According to the internal logic of PMT, we developed a novel theoretical framework of a "storyline," which was a process from being unaware of patients' privacy to having privacy protection behavior, that affected doctors' cognitive intermediary and changed the development of doctors' awareness, finally affecting actual privacy protection behavior in Chinese public medical institutions. Ultimately, we created a scale to measure 18 variables in the theoretical model, comprising 63 measurement items, with a total of 208 doctors participating in the scaling survey, who were predominantly educated to the master's degree level (n=151, 72.6%). The department distribution was relatively balanced. Prior to EFA, the Kaiser-Meyer-Olkin (KMO) value was 0.702, indicating that the study was suitable for factor analysis. The minimum value of Cronbach α for each study variable was .754, which met the internal consistency requirements of the scale. The standard factor loading value of each potential measurement item in CFA had scores greater than 0.5, which signified that all the items in the scale could effectively converge to the corresponding potential variables. CONCLUSIONS: The theoretical framework and scale to assess doctors' patient protection behavior in public medical institutions in China fills a significant gap in the literature and can be used to further the current knowledge of physicians' thought processes and adoption decisions.

Association of Stool Frequency and Consistency with the Risk of All-Cause and Cause-Specific Mortality among U.S. Adults: Results from NHANES 2005-2010.

BACKGROUND: Prior studies on the relationship between bowel health and mortality have generally focused on the individual association of stool frequency or consistency with mortality but did not present a joint association. Therefore, we aimed to systematically evaluate the individual and joint associations of stool frequency and consistency with all-cause and cause-specific mortality in this study. METHODS: A total of 14,574 participants from the National Health and Nutrition Examination Survey 2005-2010 were incorporated in this analysis. Survey sample-weighted Cox proportional hazards models adjusted for potential confounders were used to estimate hazard ratios (HRs) between bowel health measures and mortality risks. RESULTS: During a median of 7.6 years of follow-up, 1502 deaths occurred, including 357 cancer deaths and 284 cardiovascular disease (CVD) deaths. The bowel habit of the most participants was 7 times/week (50.7%), and the most common type was "Like a sausage or snake, smooth and soft" (51.8%). Stool frequency displayed a parabolic relationship with all-cause mortality, and less than 7 times/week is a significant risk factor for mortality (HR for 1 time/week: 1.43, p-values = 0.04. HR for 6 times/week: 1.05, p-value = 0.03). Analyzing the joint association of stool frequency and consistency on mortality clarified the limitations of only inspecting the effects of either individual factor. Compared with 7 times/week of normal stool, infrequent soft stools at 4 times/week were associated with 1.78-, 2.42-, and 2.27-times higher risks of all-cause, cancer, and CVD mortality, respectively. CONCLUSION: Analyses of bowel health should consider the joint effects of stool frequency and stool consistency. Self-appraisal of stool frequency and consistency may be a simple but useful tool for informing about major chronic illnesses.

Identification of IκBα in Japanese eel Anguilla japonica that impairs the IKKα-dependent activation of NF-κB, AP1, and type I IFN signaling pathways.

As a member of inhibitory κB family (IκB) family, IκBα is best-characterized and plays a central negative feedback regulator of NF-κB pathway in mammals, but the information about IκBα in the regulation of immune responses is still limited in teleost fishes. In the present study, the full-length cDNA of an IκBα homologue, AjIκBα, was cloned by 5' and 3' SMART RACE from Japanese eel, and its characteristics of expression in response to various PAMPs and A. hydrophila infection were investigated both in vivo and in vitro using quantitative real-time polymerase chain reaction (qRT-PCR). In addition, the subcellular localization of AjIκBα GFP fusion protein and the induction of AjIκBα alone or co-expression with Japanese eel IKKα (AjIKKα) in the activation of NF-κB, type I IFN and AP1 performed using Dual-Glo luciferase assay system were also detected. Sequence comparison analysis revealed that AjIκBα has typical conserved domains, including the N-terminal conserved degradation motif, the ankyrin repeats, and the C-terminal PEST domain. The predicted three-dimensional structure of AjIκBα is similar to that of human IκBα. qRT-PCR analysis revealed a broad expression for AjIκBα in a wide range of tissues, with the highest expression in the spleen, followed by intestine, liver, gills, skin, kidney, and with a lower expression in the heart and muscle. The AjIκBα expressions in the kidney, spleen, and especially in liver were significantly induced following injection with Gram-negative bacterial component LPS, the viral mimic poly I:C and Aeromonas hydrophila infection. In vitro, the AjIκBα transcripts of Japanese eel liver cells were significantly enhanced by the treatment of LPS, poly I:C, or the stimulation of different concentration of Aeromonas hydrophil. Luciferase assays demonstrated that not only could the AjIκBα expression significantly decrease the activation of NF-κB, AP1, and IFNß-responsive promoters in HEK293 cells and EPC cells, but also robustly inhibited the activity of these three promoters in HEK293 cells or NF-κB and AP1-responsive promoters in EPC cells induced by AjIKKα. Additionally, subcellular localization studies showed that AjIκBα was evenly distributed in the cytoplasm and nucleus both in HEK293 cells and EPC cells under natural state. AjIκBα was found to aggregate into spots in the cytoplasm and nucleus stimulated by LPS or mostly aggregate into nucleus with the treatment of poly I:C in HEK293 cells, whereas the elevated expression of AjIκBα was observed in the cytoplasm of EPC cells upon the stimulation of poly I:C. These results collectively indicated that AjIκBα function as an important negative regulation in innate immunity of host against antibacterial and antiviral infection likely via the inhibition of the activation of NF-κB, AP1, and type I IFN signaling pathways.

Fish IKKα from Japanese eel (Anguilla japonica) can activate NF-κB, AP1, and type I IFN signaling pathways.

Inhibitor of nuclear factor kappa-B kinase subunit alpha (IKKα) plays a pivotal role in the activation of nuclear factor kappa-B (NF-κB) pathway in response to pathogens infections in mammals, but the information about IKKα in the regulation of immune responses is still limited in teleost fishes. In the present study, the full-length cDNA of an IKKα homologue, AjIKKα, was cloned by 5' and 3' SMART RACE from Japanese eel, and its characteristics of expression in response to various PAMPs and A. hydrophila infection were investigated both in vivo and in vitro using quantitative real-time polymerase chain reaction (qRT-PCR). In addition, the subcellular localization of AjIKKα GFP fusion protein and the induction of AjIKKα in the activation of NF-κB, type I IFN and AP1 performed using Dual-Glo luciferase assay system were also detected. Sequence comparison analysis revealed that AjIKKα has typical conserved domains, including an N-terminal kinase domain, an ubiquitin-like domain, a scaffold dimerization domain, and a C-terminal NEMO-binding domain. The predicted three-dimensional structure of AjIKKα is similar to that of human IKKα. Quantitative real-time polymerase chain reaction (qRT-PCR) analysis revealed a broad expression for AjIKKα in a wide range of tissues, with the highest expression in the liver, followed by the intestine, gills, and spleen, and with a lower expression in the muscle and heart. The AjIKKα expressions in the liver and kidney were significantly induced following injection with the viral mimic poly I:C and Aeromonas hydrophila infection, whereas the bacterial mimic LPS down-regulated the expression of AjIKKα in the spleen. In vitro, the AjIKKα transcripts of Japanese eel liver cells were significantly enhanced by the treatment of LPS, poly I:C, CpG-DNA, and PGN or the stimulation of different concentration of Aeromonas hydrophila (1 × 106 cfu/mL, 1 × 107 cfu/mL, and 1 × 108 cfu/mL). Luciferase assays demonstrated that AjIKKα expression could significantly induce NF-κB, AP-1 and type I IFN promoter activation in a dose-dependent manner. Additionally, subcellular localization studies showed that AjIKKα was evenly distributed in the cytoplasm in the natural state, but AjIKKα was found to aggregate into spots in the cytoplasm after the stimulation of LPS and poly I:C. These results collectively indicated that AjIKKα plays an important role in innate immunity of host against antibacterial and antiviral infection likely via the activation of NF-κB, AP1and type I IFN signaling pathway.