Performance evaluation of ChatGPT in detecting diagnostic errors and their contributing factors: an analysis of 545 case reports of diagnostic errors.

BACKGROUND: Manual chart review using validated assessment tools is a standardised methodology for detecting diagnostic errors. However, this requires considerable human resources and time. ChatGPT, a recently developed artificial intelligence chatbot based on a large language model, can effectively classify text based on suitable prompts. Therefore, ChatGPT can assist manual chart reviews in detecting diagnostic errors. OBJECTIVE: This study aimed to clarify whether ChatGPT could correctly detect diagnostic errors and possible factors contributing to them based on case presentations. METHODS: We analysed 545 published case reports that included diagnostic errors. We imputed the texts of case presentations and the final diagnoses with some original prompts into ChatGPT (GPT-4) to generate responses, including the judgement of diagnostic errors and contributing factors of diagnostic errors. Factors contributing to diagnostic errors were coded according to the following three taxonomies: Diagnosis Error Evaluation and Research (DEER), Reliable Diagnosis Challenges (RDC) and Generic Diagnostic Pitfalls (GDP). The responses on the contributing factors from ChatGPT were compared with those from physicians. RESULTS: ChatGPT correctly detected diagnostic errors in 519/545 cases (95%) and coded statistically larger numbers of factors contributing to diagnostic errors per case than physicians: DEER (median 5 vs 1, p<0.001), RDC (median 4 vs 2, p<0.001) and GDP (median 4 vs 1, p<0.001). The most important contributing factors of diagnostic errors coded by ChatGPT were 'failure/delay in considering the diagnosis' (315, 57.8%) in DEER, 'atypical presentation' (365, 67.0%) in RDC, and 'atypical presentation' (264, 48.4%) in GDP. CONCLUSION: ChatGPT accurately detects diagnostic errors from case presentations. ChatGPT may be more sensitive than manual reviewing in detecting factors contributing to diagnostic errors, especially for 'atypical presentation'.

The Big Three diagnostic errors through reflections of Japanese internists.

OBJECTIVES: To analyze the Big Three diagnostic errors (malignant neoplasms, cardiovascular diseases, and infectious diseases) through internists' self-reflection on their most memorable diagnostic errors. METHODS: This secondary analysis study, based on a web-based cross-sectional survey, recruited participants from January 21 to 31, 2019. The participants were asked to recall the most memorable diagnostic error cases in which they were primarily involved. We gathered data on internists' demographics, time to error recognition, and error location. Factors causing diagnostic errors included environmental conditions, information processing, and cognitive bias. Participants scored the significance of each contributing factor on a Likert scale (0, unimportant; 10, extremely important). RESULTS: The Big Three comprised 54.1 % (n=372) of the 687 cases reviewed. The median physician age was 51.5 years (interquartile range, 42-58 years); 65.6 % of physicians worked in hospital settings. Delayed diagnoses were the most common among malignancies (n=64, 46 %). Diagnostic errors related to malignancy were frequent in general outpatient settings on weekdays and in the mornings and were not identified for several months following the event. Environmental factors often contributed to cardiovascular disease-related errors, which were typically identified within days in emergency departments, during night shifts, and on holidays. Information gathering and interpretation significantly impacted infectious disease diagnoses. CONCLUSIONS: The Big Three accounted for the majority of cases recalled by Japanese internists. The most relevant contributing factors were different for each of the three categories. Addressing these errors may require a unique approach based on the disease associations.

Distribution of internal medicine rotations among resident physicians in Japan: a nationwide, multicenter, cross-sectional study.

BACKGROUND: In Japan, postgraduate clinical training encompasses a 2-year residency program, including at least 24 weeks of internal medicine (IM) rotations. However, the fragmented structure of these rotations can compromise the training's quality and depth. For example, a resident might spend only a few weeks in cardiology before moving to endocrinology, without sufficient time to deepen their understanding or have clinical experience. This study examined current patterns and lengths of IM rotations within the Japanese postgraduate medical system. It scrutinized the piecemeal approach-whereby residents may engage in multiple short-term stints across various subspecialties without an overarching, integrated experience-and explored potential consequences for their clinical education. METHODS: This nationwide, multicenter, cross-sectional study used data from self-reported questionnaires completed by participants in the 2022 General Medicine In-Training Examination (GM-ITE). Data of 1,393 postgraduate year (PGY) one and two resident physicians who participated in the GM-ITE were included. We examined the IM rotation duration and number of IM subspecialties chosen by resident physicians during a 2-year rotation. RESULTS: Approximately half of the participants chose IM rotation periods of 32-40 weeks. A significant proportion of participants rotated in 5-7 internal medicine departments throughout the observation period. Notable variations in the distribution of rotations were observed, characterized by a common pattern where resident physicians typically spend 4 weeks in each department before moving to the next. This 4-week rotation is incrementally repeated across different subspecialties without a longer, continuous period in any single area. Notably, 39.7% of participants did not undertake general internal medicine rotations. These results suggest a narrowed exposure to medical conditions and patient care practices. CONCLUSIONS: Our study highlights the need to address the fragmented structure of IM rotations in Japan. We suggest that short, specialized learning periods may limit the opportunity to gain broad in-depth knowledge and practical experience. To improve the efficacy of postgraduate clinical education, we recommend fostering more sustained and comprehensive learning experiences.

How Do We Establish the Utility and Evidence of General Medicine in Japan?

Hospital Medicine in the United States has achieved significant progress in the accumulation of evidence. This development has influenced the increasing societal demand for General Medicine in Japan. Generalists in Japan actively engage in a wide range of interdisciplinary clinical practices, education, and management. Furthermore, Generalists have also contributed to advances in research. However, there is limited evidence regarding the benefits of General Medicine in Japan in all these areas, with most of the evidence derived from single-center studies. In Japan, the roles of Generalists are diverse, and the comprehensive definition of General Medicine makes it difficult to clearly delineate its scope. This results in an inadequate accumulation of evidence regarding the benefits of General Medicine, potentially making it less attractive to the public and younger physicians. Therefore, it is necessary to categorize General Medicine and collect clear evidence regarding its benefits.

Characteristics of Medical School Deans and University Hospital Directors in Japan.

This cross-sectional study investigates characteristics, including sex, postgraduate experience, and specialty, of medical school deans and university hospital directors in Japan.

Association of sleep quality with duty hours, mental health, and medical errors among Japanese postgraduate residents: a cross-sectional study.

Long duty hours (DH) impair sleep and negatively affect residents' health and medical safety. This cross-sectional study investigated the association among residents' DH, sleep duration, insomnia, sleep impairment, depressive symptoms, and self-reported medical errors among 5579 residents in Japan who completed the General Medicine In-Training Examination (2021) and participated in the training-environment survey. Weekly DH was classified under seven categories. Sleep duration and insomnia symptoms, from the Athens Insomnia Scale, were analysed to determine sleep impairment; depressive symptoms and medical errors were self-reported. Among 5095 residents, 15.5% slept < 5 h/day, and 26.7% had insomnia. In multivariable analysis, compared with ≥ 60 and < 70, DH ≥ 90 h/week associated with shorter sleep duration and worsen insomnia symptoms. Shorter durations of sleep and more intense symptoms of insomnia were associated with increased depressive symptoms. Medical errors increased only among residents with insomnia, but were not associated with sleep duration. DH > 90 h/week could lead to shorter sleep duration, worsen insomnia symptoms, and negatively impact well-being and medical safety. There was no significant association between sleep duration and medical errors; however, insomnia conferred an increased risk of medical errors. Limiting DH for residents to avoid excessive workload can help improve resident sleep, enhance resident well-being, and potentially reduce insomnia-associated medical errors.

Educational strategies related to information science and technology for medical students: General medicine physicians' perspective.

The Model Core Curriculum for Medical Education in Japan was revised in 2022. It aimed to reflect changes in the nature of medical care, including the advancement of medical technology through the use of information science and technology and artificial intelligence in the Society 5.0 era. We summarize recommendations for good practice regarding learning strategies from the perspective of general medicine.

Association between mentorship and mental health among junior residents: A nationwide cross-sectional study in Japan.

Background: Mentorship is a dynamic, reciprocal relationship in which an advanced careerist (mentor) encourages the growth of a novice (mentee). Mentorship may protect the mental health of residents at risk for depression and burnout, yet despite its frequent use and known benefits, limited reports exist regarding the prevalence and mental effects of mentorship on residents in Japan. Methods: We conducted a cross-sectional study involving postgraduate year 1 and 2 (PGY-1 and PGY-2) residents in Japan who took the General Medicine In-Training Examination (GM-ITE) at the end of the 2021 academic year. Data on mentorship were collected using surveys administered immediately following GM-ITE completion. The primary outcome was the Patient Health Questionaire-2 (PHQ-2), which consisted depressed mood and loss of interest. A positive response for either item indicated PHQ-2 positive. We examined associations between self-reported mentorship and PHQ-2 by multi-level analysis. Results: Of 4929 residents, 3266 (66.3%) residents reported having at least one mentor. Compared to residents without any mentor, those with a mentor were associated with a lower likelihood of a positive PHQ-2 response (adjusted odds ratio [aOR] 0.75; 95% confidence interval [95% CI] 0.65-0.86). Mentor characteristic significantly associated with negative PHQ-2 response was a formal mentor (aOR; 0.68; 95% CI 0.55-0.84). Conclusions: A mentor-based support system was positively associated with residents' mental health. Further research is needed to determine the quality of mentorship during clinical residency in Japan.

Three Subtypes of Cutaneous Nerve Entrapment Syndrome: A Narrative Review.

Anterior, lateral, and posterior cutaneous nerve entrapment syndromes have been proposed as etiologies of trunk pain. However, while these syndromes are analogous, comprehensive reports contrasting the three subtypes are lacking. We therefore reviewed the literature on anterior, lateral, and posterior cutaneous nerve entrapment syndrome. We searched the PubMed and Cochrane Library databases twice for relevant articles published between March and September 2022. In addition to 16 letters, technical reports, and review articles, a further 62, 6, and 3 articles concerning anterior, lateral, and posterior cutaneous nerve entrapment syndromes, respectively, were included. These syndromes are usually diagnosed based solely on unique history and examination findings; however, the diagnostic process may be prolonged, and multiple re-evaluations are required. The most common first-line treatment is trigger point injection; however, the management of refractory cases remains unclear. Awareness of this disease should be expanded to medical departments other than general medicine.

Idiopathic segmental anhidrosis in an older patient presenting with recurrent dizziness.

Dizziness is one of the most common complaints encountered in the outpatient clinic, which is difficult to diagnose, especially in older patients because of the multifactorial nature of the disease. Although not commonly recognised, anhidrosis can also cause dizziness.We report a case of a woman in her 70s who presented with long-term recurrent dizziness. She had a history of frequent hospitalisations for heatstroke. Physical examination revealed markedly less sweating in the left axilla and soles than in the right. Minol test revealed that most of the left side of her body, including the face, was anhidrotic. She was diagnosed with idiopathic segmental anhidrosis. We administered steroid pulse therapy without observing any significant effects.Although anhidrosis is a rare disorder, a careful interview and physical examination should be conducted to confirm a history of heatstroke and the absence of sweating to avoid missing the disease.

Empathy Among Physicians and Nurses in Japan: A Nationwide Cross-sectional Study.

BACKGROUND: Empathy with patients improves clinical outcomes. Although previous studies have shown no significant differences in empathy levels between physicians and nurses, investigations have not considered differences in cultural backgrounds and related factors of healthcare providers at the individual level. OBJECTIVE: This study compares empathy between physicians and nurses in Japan and identifies relevant factors that contribute to these differences. DESIGN: A cross-sectional survey design was used in the study. The online survey was conducted using the Nikkei Medical Online website. PARTICIPANTS: A total of 5441 physicians and 965 nurses in Japan who were registered as members of Nikkei Medical Online were included. MAIN MEASURES: Empathy was measured by the Jefferson Scale of Empathy (JSE). KEY RESULTS: Cronbach's α was 0.89. The mean JSE score for Japanese physicians was significantly lower at 100.05 (SD = 15.75) than the mean score of 110.63 (SD = 12.25) for nurses (p<0.001). In related factors, higher age (increasing by one year) (+0.29; 95% CI 0.25 to 0.32; p<0.001), self-identified female gender (+5.45; 95% CI 4.40 to 6.49; p<0.001), having children (+1.20; 95% CI 0.30 to 2.10; p=0.009), and working at a hospital with 20-99 beds (+1.73; 95% CI 0.03 to 3.43; p=0.046) were significantly associated with higher scores, whereas those whose mother is a physician (-6.65; 95% CI -8.82 to -4.47; p<0.001) and father is a nurse (-9.53; 95% CI -16.54 to -2.52; p=0.008) or co-medical professional (-3.85; 95% CI -5.49 to -2.21; p<0.001) were significantly associated with lower scores. CONCLUSIONS: Physicians had significantly lower scores on the JSE than nurses in Japan. Higher age, self-identified female gender, having children, working at a small hospital, having a mother who is a physician, and having a father who is a nurse or co-medical professional were factors associated with the level of empathy.

Japanese Internists' Most Memorable Diagnostic Error Cases: A Self-reflection Survey.

Objective The etiologies of diagnostic errors among internal medicine physicians are unclear. To understand the causes and characteristics of diagnostic errors through reflection by those involved in them. Methods We conducted a cross-sectional study using a web-based questionnaire in Japan in January 2019. Over a 10-day period, a total of 2,220 participants agreed to participate in the study, of whom 687 internists were included in the final analysis. Participants were asked about their most memorable diagnostic error cases, in which the time course, situational factors, and psychosocial context could be most vividly recalled and where the participant provided care. We categorized diagnostic errors and identified contributing factors (i.e., situational factors, data collection/interpretation factors, and cognitive biases). Results Two-thirds of the identified diagnostic errors occurred in the clinic or emergency department. Errors were most frequently categorized as wrong diagnoses, followed by delayed and missed diagnoses. Errors most often involved diagnoses related to malignancy, circulatory system disorders, or infectious diseases. Situational factors were the most cited error cause, followed by data collection factors and cognitive bias. Common situational factors included limited consultation during office hours and weekends and barriers that prevented consultation with a supervisor or another department. Conclusion Internists reported situational factors as a significant cause of diagnostic errors. Other factors, such as cognitive biases, were also evident, although the difference in clinical settings may have influenced the proportions of the etiologies of the errors that were observed. Furthermore, wrong, delayed, and missed diagnoses may have distinctive associated cognitive biases.

The current state of antimicrobial and urine culture stewardship in Thailand: Results from a national survey.

BACKGROUND: Antimicrobial stewardship in Thailand has made major progress backed by a national strategic plan. The current study aimed to assess the antimicrobial stewardship program (ASP) composition, reach, and breadth, as well as urine culture stewardship in Thai hospitals. METHODS: We sent an electronic survey to 100 Thai hospitals between February 1, 2021 and August 31, 2021. This hospital sample represented 20 hospitals in each of Thailand's 5 geographical regions. RESULTS: The response rate was 100%. A total of 86 of 100 hospitals had an ASP. These were often multi-disciplinary in nature, with half including infectious disease-trained physicians and pharmacists, infection preventionists, and nursing staff. Urine culture stewardship protocols existed in 51% of hospitals. CONCLUSIONS: The national strategic plan in Thailand has allowed the country to stand up robust ASPs. Further research should examine the effectiveness of such programs and ways to expand them into other medical settings, like nursing homes, urgent care, and outpatient while continuing to grow telehealth and urine culture stewardship.

Differences in the resident encounter of disruptive behavior by gender.

This study aimed to investigate the prevalence of gender-based differences in disruptive behaviors (DBs) among trainee physicians to shed light on the extent and nature of the problem. Using a national cross-sectional design, data were collected through a web-based, self-administered questionnaire administered to post-graduate first-year (PGY1) and second-year (PGY2) residents participating in the General Medicine Intermittent Examination (GM-ITE). A total of 5,403 participants, representing a response rate of 71.9%, were included in the study. Of these, approximately 35% of residents reported encounters with DBs in the past year. A gender-based comparison revealed that 38.4% of male residents faced DBs from physicians, compared to 27.6% of their female counterparts (p < 0.001). Conversely, a higher proportion of male residents (35.8%) experienced DBs from nurses than did female residents (32.9%; p = 0.037). After adjusting for factors such as hospital size, hospital type, urban location, age, and PGY, male residents exhibited an increased likelihood of experiencing DB from both physicians (adjusted OR 1.59, 95% CI 1.40-1.81) and nurses (adjusted OR 1.17, 95% CI 1.03-1.32) relative to women. Moreover, the study provides valuable insight into the prevalence of various types of DBs experienced by trainee physicians, including disrespectful behavior, exclusion from patient discussions, and reprimands. Understanding and addressing the gender-based differences in DBs among trainee physicians is crucial for improving the educational environment and promoting respectful behavior in healthcare settings. These findings highlight the need for targeted interventions based on gender to mitigate the negative impact of DBs on patient care and the well-being of medical residents.

Gender Inequality Improvement in Medical School Admissions in Japan.

Purpose: To investigate the effects of gender discrimination in Japan's medical school admission process and to assess whether the situation has improved since the disclosure of such discrimination in 2018. Materials and Methods: A cross-sectional study was conducted using secondary data from the Ministry of Education, Culture, Sports, Science, and Technology. The proportions of male and female applicants vis-à-vis all successful candidates admitted from 2016 to 2021 were analyzed; four medical schools were found to be systematically guilty of discriminatory admission practices. Acceptance rate ratios (ARRs) were estimated, and difference-in-differences (DID) analysis was used to examine the differences in ARRs between the two groups-the 4 and 75 medical schools that were and were not reported, respectively-in the predisclosure (2016-2018) and postdisclosure (2019-2021) periods. Results: Female applicants were subjected to discriminatory admission practices at the four reported medical schools in the predisclosure period. However, postdisclosure, those four medical schools had higher female than male acceptance rates in all 3 years. DID analysis revealed a statistically significant estimated average treatment effect on the treated of 0.25148 (95% confidence interval [0.00455-0.49840]), indicating a 0.25-point increase in ARRs relative to the other 75 medical schools. Conclusions: Discriminatory practices against female applicants have decreased since the disclosure in 2018, with the acceptance rate of female students exceeding that of male students for the first time in 2021. In response to these findings, we propose recommendations to further promote gender equality in medicine.