Human versus Artificial Intelligence: ChatGPT-4 Outperforming Bing, Bard, ChatGPT-3.5 and Humans in Clinical Chemistry Multiple-Choice Questions.

Introduction: Artificial intelligence (AI) chatbots excel in language understanding and generation. These models can transform healthcare education and practice. However, it is important to assess the performance of such AI models in various topics to highlight its strengths and possible limitations. This study aimed to evaluate the performance of ChatGPT (GPT-3.5 and GPT-4), Bing, and Bard compared to human students at a postgraduate master's level in Medical Laboratory Sciences. Methods: The study design was based on the METRICS checklist for the design and reporting of AI-based studies in healthcare. The study utilized a dataset of 60 Clinical Chemistry multiple-choice questions (MCQs) initially conceived for assessing 20 MSc students. The revised Bloom's taxonomy was used as the framework for classifying the MCQs into four cognitive categories: Remember, Understand, Analyze, and Apply. A modified version of the CLEAR tool was used for the assessment of the quality of AI-generated content, with Cohen's κ for inter-rater agreement. Results: Compared to the mean students' score which was 0.68±0.23, GPT-4 scored 0.90 ± 0.30, followed by Bing (0.77 ± 0.43), GPT-3.5 (0.73 ± 0.45), and Bard (0.67 ± 0.48). Statistically significant better performance was noted in lower cognitive domains (Remember and Understand) in GPT-3.5 (P=0.041), GPT-4 (P=0.003), and Bard (P=0.017) compared to the higher cognitive domains (Apply and Analyze). The CLEAR scores indicated that ChatGPT-4 performance was "Excellent" compared to the "Above average" performance of ChatGPT-3.5, Bing, and Bard. Discussion: The findings indicated that ChatGPT-4 excelled in the Clinical Chemistry exam, while ChatGPT-3.5, Bing, and Bard were above average. Given that the MCQs were directed to postgraduate students with a high degree of specialization, the performance of these AI chatbots was remarkable. Due to the risk of academic dishonesty and possible dependence on these AI models, the appropriateness of MCQs as an assessment tool in higher education should be re-evaluated.

ChatGPT Performance in Diagnostic Clinical Microbiology Laboratory-Oriented Case Scenarios.

BACKGROUND: Artificial intelligence (AI)-based tools can reshape healthcare practice. This includes ChatGPT which is considered among the most popular AI-based conversational models. Nevertheless, the performance of different versions of ChatGPT needs further evaluation in different settings to assess its reliability and credibility in various healthcare-related tasks. Therefore, the current study aimed to assess the performance of the freely available ChatGPT-3.5 and the paid version ChatGPT-4 in 10 different diagnostic clinical microbiology case scenarios. METHODS: The current study followed the METRICS (Model, Evaluation, Timing/Transparency, Range/Randomization, Individual factors, Count, Specificity of the prompts/language) checklist for standardization of the design and reporting of AI-based studies in healthcare. The models tested on December 3, 2023 included ChatGPT-3.5 and ChatGPT-4 and the evaluation of the ChatGPT-generated content was based on the CLEAR tool (Completeness, Lack of false information, Evidence support, Appropriateness, and Relevance) assessed on a 5-point Likert scale with a range of the CLEAR scores of 1-5. ChatGPT output was evaluated by two raters independently and the inter-rater agreement was based on the Cohen's κ statistic. Ten diagnostic clinical microbiology laboratory case scenarios were created in the English language by three microbiologists at diverse levels of expertise following an internal discussion of common cases observed in Jordan. The range of topics included bacteriology, mycology, parasitology, and virology cases. Specific prompts were tailored based on the CLEAR tool and a new session was selected following prompting each case scenario. RESULTS: The Cohen's κ values for the five CLEAR items were 0.351-0.737 for ChatGPT-3.5 and 0.294-0.701 for ChatGPT-4 indicating fair to good agreement and suitability for analysis. Based on the average CLEAR scores, ChatGPT-4 outperformed ChatGPT-3.5 (mean: 2.64±1.06 vs. 3.21±1.05, P=.012, t-test). The performance of each model varied based on the CLEAR items, with the lowest performance for the "Relevance" item (2.15±0.71 for ChatGPT-3.5 and 2.65±1.16 for ChatGPT-4). A statistically significant difference upon assessing the performance per each CLEAR item was only seen in ChatGPT-4 with the best performance in "Completeness", "Lack of false information", and "Evidence support" (P=0.043). The lowest level of performance for both models was observed with antimicrobial susceptibility testing (AST) queries while the highest level of performance was seen in bacterial and mycologic identification. CONCLUSIONS: Assessment of ChatGPT performance across different diagnostic clinical microbiology case scenarios showed that ChatGPT-4 outperformed ChatGPT-3.5. The performance of ChatGPT demonstrated noticeable variability depending on the specific topic evaluated. A primary shortcoming of both ChatGPT models was the tendency to generate irrelevant content lacking the needed focus. Although the overall ChatGPT performance in these diagnostic microbiology case scenarios might be described as "above average" at best, there remains a significant potential for improvement, considering the identified limitations and unsatisfactory results in a few cases.

Assessing Healthcare Workers' Knowledge and Their Confidence in the Diagnosis and Management of Human Monkeypox: A Cross-Sectional Study in a Middle Eastern Country.

The ongoing multi-country human monkeypox (HMPX) outbreak was declared as a public health emergency of international concern. Considering the key role of healthcare workers (HCWs) in mitigating the HMPX outbreak, we aimed to assess their level of knowledge and their confidence in diagnosis and management of the disease, besides the assessment of their attitude towards emerging virus infections from a conspiracy point of view. An online survey was distributed among HCWs in Jordan, a Middle Eastern country, during May−July 2022 using a questionnaire published in a previous study among university students in health schools in Jordan. The study sample comprised 606 HCWs, with about two-thirds being either physicians (n = 204, 33.7%) or nurses (n = 190, 31.4%). Four out of the 11 HMPX knowledge items had <50% correct responses with only 33.3% of the study respondents having previous knowledge that vaccination is available to prevent HMPX. A majority of study respondents (n = 356, 58.7%) strongly agreed, agreed or somewhat agreed that the spread of HMPX is related to a role of male homosexuals. Confidence in the ability of diagnosis based on the available monkeypox virus diagnostic tests was reported by 50.2% of the respondents, while the confidence levels were lower for the ability to manage (38.9%) and to diagnose (38.0%) HMPX cases based on their current level of knowledge and skills. Higher confidence levels for HMPX diagnosis and management were found among physicians compared to nurses. The endorsement of conspiracy beliefs about virus emergence was associated with lower HMPX knowledge, the belief in the role of male homosexuals in HMPX spread, and with lower diagnosis and management confidence levels. The current study highlighted the gaps in knowledge regarding HMPX among HCWs in Jordan as well as the lack of confidence to diagnose and manage cases among physicians and nurses. Raising the awareness about the disease is needed urgently considering the rapid escalation in the number of cases worldwide with reported cases in the Middle East. The attitude towards male homosexuals' role in HMPX spread necessitates proper intervention measures to prevent stigma and discrimination among this risk group. The adoption of conspiratorial beliefs regarding virus emergence was widely prevalent and this issue needs to be addressed with proper and accurate knowledge considering its potential harmful impact.

The Role of Psychological Factors and Vaccine Conspiracy Beliefs in Influenza Vaccine Hesitancy and Uptake among Jordanian Healthcare Workers during the COVID-19 Pandemic.

Vaccination to prevent influenza virus infection and to lessen its severity is recommended among healthcare workers (HCWs). Health professionals have a higher risk of exposure to viruses and could transmit the influenza virus to vulnerable patients who are prone to severe disease and mortality. The aim of the current study was to evaluate the levels of influenza vaccine acceptance and uptake as well as its determinants, among Jordanian HCWs over the last influenza season of 2021/2022. This study was based on a self-administered electronic survey that was distributed in March 2022. Psychological determinants of influenza vaccine acceptance and vaccine conspiracy beliefs were assessed using the previously validated 5C scale questionnaire (confidence, complacency, constraints, calculation and collective responsibility) and the vaccine conspiracy beliefs scale. The study sample comprised a total of 1218 HCWs: nurses (n = 412, 33.8%), physicians (n = 367, 30.1%), medical technicians (n = 182, 14.9%), pharmacists (n = 161, 13.2%) and dentists (n = 87, 7.1%), among others. About two-thirds of the study sample expressed willingness to receive influenza vaccination if provided free of charge (n = 807, 66.3%), whereas less than one-third were willing to pay for the vaccine (n = 388, 31.9%). The self-reported uptake of the influenza vaccine in the last influenza season was 62.8%. The following factors were significantly associated with higher acceptance of influenza vaccination if provided freely, as opposed to vaccine hesitancy/rejection: male sex; physicians and dentists among HCW categories; higher confidence and collective responsibility; and lower complacency, constraints and calculation. Higher influenza vaccine uptake was significantly correlated with nurses and physicians among HCW categories, older age, a higher monthly income, higher confidence and collective responsibility, lower complacency and constraints and lower embrace of general vaccine conspiracy beliefs. The results of the current study can provide helpful clues to improve influenza vaccine coverage among HCWs in Jordan. Consequently, this can help to protect vulnerable patient groups and reserve valuable resources in healthcare settings. Psychological determinants appeared to be the most significant factors for vaccine acceptance and uptake, whereas the embrace of general vaccine conspiracy beliefs was associated with lower rates of influenza vaccine uptake, which should be considered in educational and interventional measures aiming to promote influenza vaccination.

Knowledge of Human Monkeypox and Its Relation to Conspiracy Beliefs among Students in Jordanian Health Schools: Filling the Knowledge Gap on Emerging Zoonotic Viruses.

Background and Objective: The recent multi-country outbreak of human monkeypox (HMPX) in non-endemic regions poses an emerging public health concern. University students in health schools/faculties represent a core knowledgeable group that can be helpful to study from a public health point of view. As future healthcare workers, assessment of their knowledge and attitude towards emerging zoonotic viral infections can be helpful to assess their taught material and courses with potential improvement if gaps in knowledge were identified. Therefore, we aimed to evaluate the level of HMPX knowledge, conspiracy beliefs regarding emerging virus infections, as well as their associated determinants among university students studying Medicine, Nursing, Dentistry, Pharmacy, Medical Laboratory Sciences, and Rehabilitation in Jordanian health schools/faculties. In addition, we sought to evaluate the correlation between HMPX knowledge and the extent of holding conspiracy beliefs regarding emerging viral infection. Materials and Methods: A convenient sample of university students was obtained through an electronic survey distributed in late May 2022 using the chain-referral approach. Assessment of HMPX knowledge and general attitude towards emerging virus infections was based on survey items adopted from previously published literature. Results: The study sample comprised 615 students with a mean age of 20 years and a majority of females (432, 70.2%) and medical students (n = 351, 57.1%). Out of eleven monkeypox knowledge items, three were identified correctly by >70% of the respondents. Only 26.2% of the respondents (n = 161) knew that vaccination to prevent monkeypox is available. Age was significantly associated with better HMPX knowledge for a majority of items. Older age, females, and affiliation to non-medical schools/faculties were associated with harboring higher levels of conspiracy beliefs regarding emerging virus infections. Our data also indicate that lower levels of HMPX knowledge were associated with higher levels of conspiracy beliefs. Conclusion: The current study pointed to generally unsatisfactory levels of knowledge regarding the emerging HMPX among university students in Jordanian health schools/faculties. Conspiracy beliefs regarding emerging virus infections were widely prevalent, and its potential detrimental impact on health behavior should be evaluated in future studies.

Lack of knowledge regarding HPV and its relation to oropharyngeal cancer among medical students.

BACKGROUND: Oropharyngeal cancer (OPC) is an important cause of cancer-related mortality. Early detection of OPC results in a favorable prognosis and higher survival rates. Infection by high-risk types of human papillomavirus (HPV) is a risk factor for OPC with an upward trend globally. Medical students' knowledge and awareness of HPV-related OPC can be crucial in the preventive efforts. AIM: To assess HPV knowledge among medical students at the University of Jordan, with particular focus on its relation to different cancers. METHODS: This paper-based survey study was conducted in November 2019. The survey items were based on previously validated surveys used to evaluate HPV-related OPC knowledge among dental students and professionals. To assess HPV knowledge and students' confidence in personal history taking and physical examination, we developed a knowledge and confidence scores that showed acceptable reliability. RESULTS: The total number of participants was 1198 students, with a median age of 21 and female predominance (n = 697, 58.2%). Among the participants, 93.3% heard of HPV prior to this survey (n = 1118). Higher levels of knowledge regarding cervical cancer, OPC and HPV vaccination was seen among clinical students compared to their preclinical counterparts, but their overall HPV knowledge was low. Only 18.4% and 21.0% of the clinical students correctly identified the association of HPV with penile and oropharyngeal cancers, respectively. Additionally, 34.5% of the clinical students were not aware of the availability of HPV vaccines. The majority of students (92.0%) reported that the university courses were their major source of knowledge about HPV. CONCLUSION: A profound lack of knowledge regarding HPV role in OPC was found among medical students. This insufficiency included several aspects of the virus and its associated diseases. Such gaps in knowledge could have negative consequences in early detection and prevention of OPC and should be addressed by evaluation of the current curriculum.

Tracking the Epidemiologic Shifts in Hepatitis A Sero-Prevalence Using Age Stratification: A Cross-Sectional Study at Jordan University Hospital.

The study of hepatitis A virus (HAV) epidemiology and sero-prevalence has important public health implications. Changes in the epidemiology of hepatitis A can result in a larger pool of susceptible persons in countries with improved sanitation and hygienic conditions if vaccination is not provided. The aim of this study was to investigate the prevalence of HAV immunoglobulin G (IgG) in Jordan. In addition, we aimed to identify the potential differences in HAV sero-prevalence based on age, among other variables. We recruited the study participants at Jordan University Hospital in Amman, Jordan, during October 2020-June 2021. Assessment of participants' socio-demographic variables was done using a paper-based questionnaire. Testing for HAV IgG was based on a competitive enzyme linked immunosorbent assay (ELISA). The study population comprised 360 individuals with a median age of 18 years. The overall sero-prevalence of HAV in our study sample was 38.3%. Divided by age, the sero-prevalence of HAV was 8.2%, 12.3%, and 20.8% among individuals aged 10 years or less, 15 years or less, and 30 years or less, respectively. The estimated age at mid-population immunity was between 21 and 30 years. Besides age, individuals residing outside the Central region of Jordan had a significantly higher HAV sero-prevalence. Additionally, the use of filtered municipal water was associated with a lower sero-prevalence of HAV compared with the use of unfiltered municipal water among individuals aged 15 years or less. The results of this study suggest an intermediate to low endemicity of HAV in Jordan. An epidemiologic shift of HAV sero-prevalence with a declining rate of positivity for HAV IgG was noticed in this study. This highlights the importance of the recently introduced HAV vaccination in Jordan. Future research to evaluate the public health benefits of HAV vaccination in Jordan is recommended.