Performance of ChatGPT on the Chinese Postgraduate Examination for Clinical Medicine: Survey Study.

BACKGROUND: ChatGPT, an artificial intelligence (AI) based on large-scale language models, has sparked interest in the field of health care. Nonetheless, the capabilities of AI in text comprehension and generation are constrained by the quality and volume of available training data for a specific language, and the performance of AI across different languages requires further investigation. While AI harbors substantial potential in medicine, it is imperative to tackle challenges such as the formulation of clinical care standards; facilitating cultural transitions in medical education and practice; and managing ethical issues including data privacy, consent, and bias. OBJECTIVE: The study aimed to evaluate ChatGPT's performance in processing Chinese Postgraduate Examination for Clinical Medicine questions, assess its clinical reasoning ability, investigate potential limitations with the Chinese language, and explore its potential as a valuable tool for medical professionals in the Chinese context. METHODS: A data set of Chinese Postgraduate Examination for Clinical Medicine questions was used to assess the effectiveness of ChatGPT's (version 3.5) medical knowledge in the Chinese language, which has a data set of 165 medical questions that were divided into three categories: (1) common questions (n=90) assessing basic medical knowledge, (2) case analysis questions (n=45) focusing on clinical decision-making through patient case evaluations, and (3) multichoice questions (n=30) requiring the selection of multiple correct answers. First of all, we assessed whether ChatGPT could meet the stringent cutoff score defined by the government agency, which requires a performance within the top 20% of candidates. Additionally, in our evaluation of ChatGPT's performance on both original and encoded medical questions, 3 primary indicators were used: accuracy, concordance (which validates the answer), and the frequency of insights. RESULTS: Our evaluation revealed that ChatGPT scored 153.5 out of 300 for original questions in Chinese, which signifies the minimum score set to ensure that at least 20% more candidates pass than the enrollment quota. However, ChatGPT had low accuracy in answering open-ended medical questions, with only 31.5% total accuracy. The accuracy for common questions, multichoice questions, and case analysis questions was 42%, 37%, and 17%, respectively. ChatGPT achieved a 90% concordance across all questions. Among correct responses, the concordance was 100%, significantly exceeding that of incorrect responses (n=57, 50%; P<.001). ChatGPT provided innovative insights for 80% (n=132) of all questions, with an average of 2.95 insights per accurate response. CONCLUSIONS: Although ChatGPT surpassed the passing threshold for the Chinese Postgraduate Examination for Clinical Medicine, its performance in answering open-ended medical questions was suboptimal. Nonetheless, ChatGPT exhibited high internal concordance and the ability to generate multiple insights in the Chinese language. Future research should investigate the language-based discrepancies in ChatGPT's performance within the health care context.

How does ChatGPT-4 preform on non-English national medical licensing examination? An evaluation in Chinese language.

ChatGPT, an artificial intelligence (AI) system powered by large-scale language models, has garnered significant interest in healthcare. Its performance dependent on the quality and quantity of training data available for a specific language, with the majority of it being in English. Therefore, its effectiveness in processing the Chinese language, which has fewer data available, warrants further investigation. This study aims to assess the of ChatGPT's ability in medical education and clinical decision-making within the Chinese context. We utilized a dataset from the Chinese National Medical Licensing Examination (NMLE) to assess ChatGPT-4's proficiency in medical knowledge in Chinese. Performance indicators, including score, accuracy, and concordance (confirmation of answers through explanation), were employed to evaluate ChatGPT's effectiveness in both original and encoded medical questions. Additionally, we translated the original Chinese questions into English to explore potential avenues for improvement. ChatGPT scored 442/600 for original questions in Chinese, surpassing the passing threshold of 360/600. However, ChatGPT demonstrated reduced accuracy in addressing open-ended questions, with an overall accuracy rate of 47.7%. Despite this, ChatGPT displayed commendable consistency, achieving a 75% concordance rate across all case analysis questions. Moreover, translating Chinese case analysis questions into English yielded only marginal improvements in ChatGPT's performance (p = 0.728). ChatGPT exhibits remarkable precision and reliability when handling the NMLE in Chinese. Translation of NMLE questions from Chinese to English does not yield an improvement in ChatGPT's performance.

Association of serum copper (Cu) with cardiovascular mortality and all-cause mortality in a general population: a prospective cohort study.

BACKGROUND: Copper (Cu) homeostasis and Cu-induced cell death are gaining recognition as crucial processes in the pathogenesis of cardiovascular disease (CVD). Circulating Cu associated with CVD and mortality is yet to be fully elucidated. OBJECTIVE: This national prospective cohort study is to estimate relationship between serum Cu and the risk of CVD and all-cause mortality. METHODS: This study included participants from the National Health and Nutrition Examination Survey 2011-2016. Weighted Cox proportional hazards regression analysis and exposure-response curves were applied. RESULTS: This included 5,412 adults, representing 76,479,702 individuals. During a mean of 5.85 years of follow-up (31,653 person-years), 96 CVD and 356 all-cause mortality events occurred. Age and sex-adjusted survival curves showed that individuals with higher levels of serum Cu experienced increased CVD and all-cause death rates (tertiles, p < 0.05). Compared with the participant in tertile 1 of serum Cu (< 16.31 mol/L), those in tertile 3 (≥ 19.84 mol/L) were significantly associated with CVD mortality (HR: 7.06, 95%CI: 1.85,26.96), and all-cause mortality (HR: 2.84, 95% CI: 1.66,4.87). The dose-response curve indicated a linear relationship between serum Cu and CVD mortality (p -nonlinear = 0.48) and all-cause (p -nonlinear = 0.62). A meta-analysis included additional three prospective cohorts with 13,189 patients confirmed the association between higher serum Cu and CVD (HR: 2.08, 95% CI: 1.63,2.65) and all-cause mortality (HR: 1.89, 95%CI: 1.58,2.25). CONCLUSION: The present study suggests excessive serum Cu concentrations are associated with the risk of CVD and all-cause mortality in American adults. Our findings and the causal relationships require further investigation.