Accuracy of ChatGPT on Medical Questions in the National Medical Licensing Examination in Japan: Evaluation Study.

BACKGROUND: ChatGPT (OpenAI) has gained considerable attention because of its natural and intuitive responses. ChatGPT sometimes writes plausible-sounding but incorrect or nonsensical answers, as stated by OpenAI as a limitation. However, considering that ChatGPT is an interactive AI that has been trained to reduce the output of unethical sentences, the reliability of the training data is high and the usefulness of the output content is promising. Fortunately, in March 2023, a new version of ChatGPT, GPT-4, was released, which, according to internal evaluations, was expected to increase the likelihood of producing factual responses by 40% compared with its predecessor, GPT-3.5. The usefulness of this version of ChatGPT in English is widely appreciated. It is also increasingly being evaluated as a system for obtaining medical information in languages other than English. Although it does not reach a passing score on the national medical examination in Chinese, its accuracy is expected to gradually improve. Evaluation of ChatGPT with Japanese input is limited, although there have been reports on the accuracy of ChatGPT's answers to clinical questions regarding the Japanese Society of Hypertension guidelines and on the performance of the National Nursing Examination. OBJECTIVE: The objective of this study is to evaluate whether ChatGPT can provide accurate diagnoses and medical knowledge for Japanese input. METHODS: Questions from the National Medical Licensing Examination (NMLE) in Japan, administered by the Japanese Ministry of Health, Labour and Welfare in 2022, were used. All 400 questions were included. Exclusion criteria were figures and tables that ChatGPT could not recognize; only text questions were extracted. We instructed GPT-3.5 and GPT-4 to input the Japanese questions as they were and to output the correct answers for each question. The output of ChatGPT was verified by 2 general practice physicians. In case of discrepancies, they were checked by another physician to make a final decision. The overall performance was evaluated by calculating the percentage of correct answers output by GPT-3.5 and GPT-4. RESULTS: Of the 400 questions, 292 were analyzed. Questions containing charts, which are not supported by ChatGPT, were excluded. The correct response rate for GPT-4 was 81.5% (237/292), which was significantly higher than the rate for GPT-3.5, 42.8% (125/292). Moreover, GPT-4 surpassed the passing standard (>72%) for the NMLE, indicating its potential as a diagnostic and therapeutic decision aid for physicians. CONCLUSIONS: GPT-4 reached the passing standard for the NMLE in Japan, entered in Japanese, although it is limited to written questions. As the accelerated progress in the past few months has shown, the performance of the AI will improve as the large language model continues to learn more, and it may well become a decision support system for medical professionals by providing more accurate information.

Temporal Arteritis Caused by Tertiary Syphilis.

A 77-year-old man arrived at our hospital with bilateral shoulder pain persisting for several months and headache for 1 month. Giant cell arteritis with polymyalgia rheumatica was suspected. However, considering his medical history of testing positive for syphilis, we submitted a sample for a syphilis serology test, which yielded positive results. The Treponema pallidum hemagglutination assay of cerebrospinal fluid was positive, and a temporal artery biopsy revealed vasculitis, confirming the diagnosis of tertiary syphilis. He was successfully treated for two weeks with penicillin G infusions. Symptoms reminiscent of giant cell arteritis and polymyalgia rheumatica may reveal syphilis, which is called the "great imitator."

Drug Fever Due to Favipiravir Administration for the Treatment of a COVID-19 Patient.

A 55-year-old Japanese man was hospitalized with the novel coronavirus disease 2019 (COVID-19). On the 14th day after the start of favipiravir administration, the patient developed a fever with a temperature of 38.1°C. His pulse rate also became elevated to 128 bpm, so relative bradycardia was not suspected. Since he was in good overall health and no concomitant symptoms and signs were apparent, we considered it to be drug fever due to favipiravir. After the completion of favipiravir treatment, the patient's temperature normalized within 24 hours. We herein report this case of drug fever caused by favipiravir.

The importance of the frequency of suicide attempts as a risk factor of suicide.

The number of suicides in Japan has increased from approximately 22,000 per year in 1988-1997 to over 30,000 per year since then. The number has also increased in Mie Prefecture during this period. According to many reports, suicide attempts are one of the risk factors of suicide. In the present study, we investigated the incidences and circumstances of all suicide cases between 1996 and 2002, focusing in detail on the frequency of suicide attempts and general differences in the frequency of all suicide cases. During the 7 years, 1979 male and 969 female suicides were reported to Mie Prefectural Police Headquarters. During the test period, there were suicide attempts in 13.4% of all suicidal cases, 10.7% for men and 18.9% for women. The average age was 48.98 years (SD 17.68), 47.07 years (SD 16.32) for men, and 51.19 years (SD 18.89) for women. Among men, we discovered the risk factors of suicide attempts in completed suicides, which were "single time" of suicide attempts, "living with family" for the family constitution, and "visiting treatment" of the admission statuses. Among women, the risk factors were "living with family" and "visiting treatment". We must pay attention to those risk factors.