ABSTRACT
BACKGROUND: ChatGPT is a large language model designed to generate responses based on a contextual understanding of user queries and requests. This study utilised the entrance examination for the Master of Clinical Medicine in Traditional Chinese Medicine to assesses the reliability and practicality of ChatGPT within the domain of medical education. METHODS: We selected 330 single and multiple-choice questions from the 2021 and 2022 Chinese Master of Clinical Medicine comprehensive examinations, which did not include any images or tables. To ensure the test's accuracy and authenticity, we preserved the original format of the query and alternative test texts, without any modifications or explanations. RESULTS: Both ChatGPT3.5 and GPT-4 attained average scores surpassing the admission threshold. Noteworthy is that ChatGPT achieved the highest score in the Medical Humanities section, boasting a correct rate of 93.75%. However, it is worth noting that ChatGPT3.5 exhibited the lowest accuracy percentage of 37.5% in the Pathology division, while GPT-4 also displayed a relatively lower correctness percentage of 60.23% in the Biochemistry section. An analysis of sub-questions revealed that ChatGPT demonstrates superior performance in handling single-choice questions but performs poorly in multiple-choice questions. CONCLUSION: ChatGPT exhibits a degree of medical knowledge and the capacity to aid in diagnosing and treating diseases. Nevertheless, enhancements are warranted to address its accuracy and reliability limitations. Imperatively, rigorous evaluation and oversight must accompany its utilization, accompanied by proactive measures to surmount prevailing constraints.
Subject(s)
Artificial Intelligence , Clinical Medicine , Educational Measurement , Language , Reproducibility of ResultsABSTRACT
Background: Randomized controlled trials (RCTs) have demonstrated that combining Chinese herbal injections (CHIs) with oxaliplatin plus tegafur (SOX) chemotherapy regimens improves clinical effectiveness and reduces adverse reactions in patients with advanced gastric cancer (AGC). These RCTs highlight the potential applications of CHIs and their impact on AGC patient prognosis. However, there is insufficient comparative evidence on the clinical effectiveness and safety of different CHIs when combined with SOX. Therefore, we performed a network meta-analysis to rank the clinical effectiveness and safety of different CHIs when combined with SOX chemotherapy regimens. This study aimed to provide evidence for selecting appropriate CHIs in the treatment of patients with AGC. Methods: We searched eight databases from their inception until March 2023. Surface Under the Cumulative Ranking Curve (SUCRA) probability values were used to rank the treatment measures, and the Confidence in Network Meta-Analysis (CINeMA) software assessed the grading of evidence. Results: A total of 51 RCTs involving 3,703 AGC patients were identified. Huachansu injections + SOX demonstrated the highest clinical effectiveness (SUCRA: 78.17%), significantly reducing the incidence of leukopenia (93.35%), thrombocytopenia (80.19%), and nausea and vomiting (95.15%). Shenfu injections + SOX improved Karnofsky's Performance Status (75.59%) and showed a significant reduction in peripheral neurotoxicity incidence (88.26%). Aidi injections + SOX were most effective in reducing the incidence of liver function damage (75.16%). According to CINeMA, most confidence rating results were classified as "low". Conclusion: The combination of CHIs and SOX shows promising effects in the treatment of AGC compared to SOX alone. Huachansu and Shenfu injections offer the greatest overall advantage among the CHIs, while Aidi injections are optimal for reducing the incidence of liver damage. However, further rigorous RCTs with larger sample sizes and additional pharmacological studies are necessary to reinforce these findings.