RESUMO
Background Artificial intelligence (AI)-based conversational models, such as Chat Generative Pre-trained Transformer (ChatGPT), Microsoft Bing, and Google Bard, have emerged as valuable sources of health information for lay individuals. However, the accuracy of the information provided by these AI models remains a significant concern. This pilot study aimed to test a new tool with key themes for inclusion as follows: Completeness of content, Lack of false information in the content, Evidence supporting the content, Appropriateness of the content, and Relevance, referred to as "CLEAR", designed to assess the quality of health information delivered by AI-based models. Methods Tool development involved a literature review on health information quality, followed by the initial establishment of the CLEAR tool, which comprised five items that aimed to assess the following: completeness, lack of false information, evidence support, appropriateness, and relevance. Each item was scored on a five-point Likert scale from excellent to poor. Content validity was checked by expert review. Pilot testing involved 32 healthcare professionals using the CLEAR tool to assess content on eight different health topics deliberately designed with varying qualities. The internal consistency was checked with Cronbach's alpha (α). Feedback from the pilot test resulted in language modifications to improve the clarity of the items. The final CLEAR tool was used to assess the quality of health information generated by four distinct AI models on five health topics. The AI models were ChatGPT 3.5, ChatGPT 4, Microsoft Bing, and Google Bard, and the content generated was scored by two independent raters with Cohen's kappa (κ) for inter-rater agreement. Results The final five CLEAR items were: (1) Is the content sufficient?; (2) Is the content accurate?; (3) Is the content evidence-based?; (4) Is the content clear, concise, and easy to understand?; and (5) Is the content free from irrelevant information? Pilot testing on the eight health topics revealed acceptable internal consistency with a Cronbach's α range of 0.669-0.981. The use of the final CLEAR tool yielded the following average scores: Microsoft Bing (mean=24.4±0.42), ChatGPT-4 (mean=23.6±0.96), Google Bard (mean=21.2±1.79), and ChatGPT-3.5 (mean=20.6±5.20). The inter-rater agreement revealed the following Cohen κ values: for ChatGPT-3.5 (κ=0.875, P<.001), ChatGPT-4 (κ=0.780, P<.001), Microsoft Bing (κ=0.348, P=.037), and Google Bard (κ=.749, P<.001). Conclusions The CLEAR tool is a brief yet helpful tool that can aid in standardizing testing of the quality of health information generated by AI-based models. Future studies are recommended to validate the utility of the CLEAR tool in the quality assessment of AI-generated health-related content using a larger sample across various complex health topics.
RESUMO
Macrophages play an important role in attempt to eliminate mycobacteria, via production of cytokines, including interleukin-1, and interleukin-12. Bacillus Calmitte Guerin (BCG) vaccination, known to induce interleukin-1ß in tuberculosis, was originally aimed at tuberculosis control, but it showed efficacy against leprosy. Our aim was to estimate serum levels of interleukin-1ß and interleukin-12, in leprosy, and to assess the impact of previous BCG vaccination on their levels. Serum interleukin-1ß and interleukin-12 p70 were estimated in 43 leprotic patients and 43 controls by enzyme-linked immunosorbent assay. Patients were grouped according to presence or absence of reactions, as well as bacillary load. Serum interleukin-1ß was significantly higher in patients as compared to controls (p = 0.047), and was significantly different in patients' groups (p = 0.036); with significantly higher level in multibacillary patients, both non reactional and with erythema nodosum leprosum, compared with paucibacillary/non reactional patients (p = 0.012 and 0.049 respectively). A statistically significant higher interleukin-1ß was found in BCG vaccinated paucibacillary patients as compared to unvaccinated patients (p = 0.031). Significantly elevated interleukin-12 was present in patients as compared to controls (p < 0.001), with no statistically significant difference comparing patients' groups. BCG vaccination showed stimulatory effect on monocytes only in the immunocompetent paucibacillary leprosy patients, as evidenced by higher Interleukin-1ß in this group. Interleukin-1ß was shown to have a pro-inflammatory role in multibacillary patients with or without erythema nodosum leprosum. Targeting interleukin-1ß may be promising to control episodic refractory erythema nodosum leprosum. Interleukin-12 may be a general marker of active Mycobacterium leprae infection.