Your browser doesn't support javascript.
loading
Show: 20 | 50 | 100
Results 1 - 20 de 7.154
Filter
Add more filters

Publication year range
1.
Nat Immunol ; 23(3): 411-422, 2022 03.
Article in English | MEDLINE | ID: mdl-35165446

ABSTRACT

The increasing implication of lymphocytes in general physiology and immune surveillance outside of infection poses the question of how their antigen receptors might be involved. Here, we show that macromolecular aggregates of intraepidermal γδ T cell antigen receptors (TCRs) in the mouse skin aligned with and depended on Skint1, a butyrophilin-like (BTNL) protein expressed by differentiated keratinocytes (KCs) at steady state. Interruption of TCR-mediated 'normality sensing' had no impact on γδ T cell numbers but altered their signature phenotype, while the epidermal barrier function was compromised. In addition to the regulation of steady-state physiology, normality sensing licensed intraepidermal T cells to respond rapidly to subsequent tissue perturbation by using innate tumor necrosis factor (TNF) superfamily receptors. Thus, interfering with Skint1-dependent interactions between local γδ T cells and KCs at steady state increased the susceptibility to ultraviolet B radiation (UVR)-induced DNA damage and inflammation, two cancer-disposing factors.


Subject(s)
Intraepithelial Lymphocytes , Receptors, Antigen, T-Cell, gamma-delta , Animals , Butyrophilins , Epidermis , Intraepithelial Lymphocytes/metabolism , Licensure , Mice , Receptors, Antigen, T-Cell, gamma-delta/metabolism
2.
Nat Immunol ; 23(6): 892-903, 2022 06.
Article in English | MEDLINE | ID: mdl-35624206

ABSTRACT

Intracellular sensing of stress and danger signals initiates inflammatory innate immune responses by triggering inflammasome assembly, caspase-1 activation and pyroptotic cell death as well as the release of interleukin 1ß (IL-1ß), IL-18 and danger signals. NLRP3 broadly senses infectious patterns and sterile danger signals, resulting in the tightly coordinated and regulated assembly of the NLRP3 inflammasome, but the precise mechanisms are incompletely understood. Here, we identified NLRP11 as an essential component of the NLRP3 inflammasome in human macrophages. NLRP11 interacted with NLRP3 and ASC, and deletion of NLRP11 specifically prevented NLRP3 inflammasome activation by preventing inflammasome assembly, NLRP3 and ASC polymerization, caspase-1 activation, pyroptosis and cytokine release but did not affect other inflammasomes. Restored expression of NLRP11, but not NLRP11 lacking the PYRIN domain (PYD), restored inflammasome activation. NLRP11 was also necessary for inflammasome responses driven by NLRP3 mutations that cause cryopyrin-associated periodic syndrome (CAPS). Because NLRP11 is not expressed in mice, our observations emphasize the specific complexity of inflammasome regulation in humans.


Subject(s)
Inflammasomes , NLR Family, Pyrin Domain-Containing 3 Protein , Animals , Caspase 1/genetics , Caspases/metabolism , Humans , Inflammasomes/metabolism , Interleukin-1beta/metabolism , Licensure , Macrophages , Mice , NLR Family, Pyrin Domain-Containing 3 Protein/genetics , NLR Family, Pyrin Domain-Containing 3 Protein/metabolism
4.
Nature ; 620(7972): 172-180, 2023 Aug.
Article in English | MEDLINE | ID: mdl-37438534

ABSTRACT

Large language models (LLMs) have demonstrated impressive capabilities, but the bar for clinical applications is high. Attempts to assess the clinical knowledge of models typically rely on automated evaluations based on limited benchmarks. Here, to address these limitations, we present MultiMedQA, a benchmark combining six existing medical question answering datasets spanning professional medicine, research and consumer queries and a new dataset of medical questions searched online, HealthSearchQA. We propose a human evaluation framework for model answers along multiple axes including factuality, comprehension, reasoning, possible harm and bias. In addition, we evaluate Pathways Language Model1 (PaLM, a 540-billion parameter LLM) and its instruction-tuned variant, Flan-PaLM2 on MultiMedQA. Using a combination of prompting strategies, Flan-PaLM achieves state-of-the-art accuracy on every MultiMedQA multiple-choice dataset (MedQA3, MedMCQA4, PubMedQA5 and Measuring Massive Multitask Language Understanding (MMLU) clinical topics6), including 67.6% accuracy on MedQA (US Medical Licensing Exam-style questions), surpassing the prior state of the art by more than 17%. However, human evaluation reveals key gaps. To resolve this, we introduce instruction prompt tuning, a parameter-efficient approach for aligning LLMs to new domains using a few exemplars. The resulting model, Med-PaLM, performs encouragingly, but remains inferior to clinicians. We show that comprehension, knowledge recall and reasoning improve with model scale and instruction prompt tuning, suggesting the potential utility of LLMs in medicine. Our human evaluations reveal limitations of today's models, reinforcing the importance of both evaluation frameworks and method development in creating safe, helpful LLMs for clinical applications.


Subject(s)
Benchmarking , Computer Simulation , Knowledge , Medicine , Natural Language Processing , Bias , Clinical Competence , Comprehension , Datasets as Topic , Licensure , Medicine/methods , Medicine/standards , Patient Safety , Physicians
6.
Proc Natl Acad Sci U S A ; 120(30): e2304319120, 2023 07 25.
Article in English | MEDLINE | ID: mdl-37459511

ABSTRACT

Recurrence of advanced melanoma after therapy is a major risk factor for reduced survival, and treatment options are limited. Antitumor immune memory plays a critical role in preventing melanoma recurrence and memory T cells could be a potent cell-based therapy, but the identity, and functional properties of the required immune cells are incompletely understood. Here, we show that an IL-7Rhi tumor-specific CD8+ population is critical for antitumor memory and can be epigenetically augmented to drive powerful antitumor immune responses. Using a model of functional antimelanoma memory, we found that high IL-7R expression selectively marks a CD8+ population in lymphoid organs that plays critical roles in maintaining tumor remission after immunotherapy or surgical resection. This population has intrinsic cytotoxic activity, lacks markers of exhaustion and has superior antitumor efficacy. IL-7Rhi cells have a functionally poised epigenetic landscape regulated by DNA methylation, which can be augmented by hypomethylating agents to confer improved survival and complete melanoma clearance in naive mice. Importantly, greater than 95% of tumor-specific T cells in draining lymph nodes after therapy express high levels of IL-7R. This overlap between IL-7Rhi and antigen-specific T cells allows for enrichment of a potent functional CD8+ population without determining antigen-specificity, which we demonstrate in a melanoma model without a known antigen. We identify that IL-7R expression in human melanoma is an independent prognostic factor of improved survival. These findings advance our basic understanding of antitumor memory and suggest a cell-based therapy using high IL-7R expression to enrich for a lymph node population with superior antitumor activity that can be augmented by hypomethylating agents.


Subject(s)
CD8-Positive T-Lymphocytes , Melanoma , Mice , Humans , Animals , Memory T Cells , Melanoma/genetics , Melanoma/therapy , Signal Transduction , Antigens , Licensure , Immunologic Memory
7.
PLoS Pathog ; 19(6): e1010767, 2023 Jun.
Article in English | MEDLINE | ID: mdl-37279255

ABSTRACT

The inflammatory cytokine tumor necrosis factor (TNF) is necessary for host defense against many intracellular pathogens, including Legionella pneumophila. Legionella causes the severe pneumonia Legionnaires' disease and predominantly affects individuals with a suppressed immune system, including those receiving therapeutic TNF blockade to treat autoinflammatory disorders. TNF induces pro-inflammatory gene expression, cellular proliferation, and survival signals in certain contexts, but can also trigger programmed cell death in others. It remains unclear, however, which of the pleiotropic functions of TNF mediate control of intracellular bacterial pathogens like Legionella. In this study, we demonstrate that TNF signaling licenses macrophages to die rapidly in response to Legionella infection. We find that TNF-licensed cells undergo rapid gasdermin-dependent, pyroptotic death downstream of inflammasome activation. We also find that TNF signaling upregulates components of the inflammasome response, and that the caspase-11-mediated non-canonical inflammasome is the first inflammasome to be activated, with caspase-1 and caspase-8 mediating delayed pyroptotic death. We find that all three caspases are collectively required for optimal TNF-mediated restriction of bacterial replication in macrophages. Furthermore, caspase-8 is required for control of pulmonary Legionella infection. These findings reveal a TNF-dependent mechanism in macrophages for activating rapid cell death that is collectively mediated by caspases-1, -8, and -11 and subsequent restriction of Legionella infection.


Subject(s)
Legionnaires' Disease , Pneumonia , Mice , Animals , Humans , Caspase 1/metabolism , Caspase 8/metabolism , Inflammasomes , Mice, Knockout , Macrophages , Caspases/metabolism , Cell Death , Tumor Necrosis Factor-alpha/metabolism , Pneumonia/metabolism , Licensure
8.
Proc Natl Acad Sci U S A ; 119(1)2022 01 04.
Article in English | MEDLINE | ID: mdl-34949712

ABSTRACT

Appended to the 5' end of nascent RNA polymerase II transcripts is 7-methyl guanosine (m7G-cap) that engages nuclear cap-binding complex (CBC) to facilitate messenger RNA (mRNA) maturation. Mature mRNAs exchange CBC for eIF4E, the rate-limiting translation factor that is controlled through mTOR. Experiments in immune cells have now documented HIV-1 incompletely processed transcripts exhibited hypermethylated m7G-cap and that the down-regulation of the trimethylguanosine synthetase-1-reduced HIV-1 infectivity and virion protein synthesis by several orders of magnitude. HIV-1 cap hypermethylation required nuclear RNA helicase A (RHA)/DHX9 interaction with the shape of the 5' untranslated region (UTR) primer binding site (PBS) segment. Down-regulation of RHA or the anomalous shape of the PBS segment abrogated hypermethylated caps and derepressed eIF4E binding for virion protein translation during global down-regulation of host translation. mTOR inhibition was detrimental to HIV-1 proliferation and attenuated Tat, Rev, and Nef synthesis. This study identified mutually exclusive translation pathways and the calibration of virion structural/accessory protein synthesis with de novo synthesis of the viral regulatory proteins. The hypermethylation of select, viral mRNA resulted in CBC exchange to heterodimeric CBP80/NCBP3 that expanded the functional capacity of HIV-1 in immune cells.


Subject(s)
Guanosine/metabolism , HIV-1/metabolism , Host Microbial Interactions/physiology , TOR Serine-Threonine Kinases/metabolism , 5' Untranslated Regions , Binding Sites , DEAD-box RNA Helicases , Eukaryotic Initiation Factor-4E/metabolism , Guanosine/analogs & derivatives , Humans , Licensure , Methylation , Methyltransferases/metabolism , Neoplasm Proteins , RNA Caps , RNA, Messenger/metabolism , RNA, Viral/genetics , Virion/metabolism
9.
Am J Epidemiol ; 193(1): 1-5, 2024 Jan 08.
Article in English | MEDLINE | ID: mdl-37527824

ABSTRACT

The current US Food and Drug Administration (FDA) licensure process underestimates the potential benefits of vaccines at both the individual and population levels by considering only direct clinical outcomes of vaccination. While all approved vaccines do protect the person who takes them from poor clinical outcomes for a specific infectious disease, many vaccines also have the potential to offer measurable, direct nonclinical benefits. For example, coronavirus disease 2019 (COVID-19) vaccinations for school-aged children may prevent school absenteeism. Also, by preventing infection or reducing its length and severity, some vaccines also protect-to some extent-the patient's immediate contacts from contracting the same disease. These nonclinical and population-level benefits are not considered as part of the FDA's current vaccine approval process, but they could be. We argue that the FDA's structured benefit-risk assessment framework, used for vaccine approvals, can and should consider both clinical and nonclinical benefits of vaccination when sufficient evidence exists to make an informed assessment. Including them could incentivize vaccine developers to measure additional vaccination effects, inform population health, and address health inequalities-including inequalities in the social determinants of health.


Subject(s)
Drug Approval , Population Health , Social Determinants of Health , Vaccines , Humans , Risk Assessment , Vaccination , Licensure , United States Food and Drug Administration
10.
EMBO J ; 39(10): e105012, 2020 05 18.
Article in English | MEDLINE | ID: mdl-32338387

ABSTRACT

Rhomboid intramembrane serine proteases are present in all kingdoms of life, but as we do not know their substrates in many species, it remains puzzling why rhomboids are among the most-conserved integral membrane proteins. Two new studies in The EMBO Journal by Began et al and Liu et al now link bacterial rhomboid proteases to membrane protein degradation, showing striking similarities to what is known about eukaryotic rhomboid family proteins, thus pointing toward a conserved membrane surveillance mechanism.


Subject(s)
Bacterial Proteins , Membrane Proteins , ATPases Associated with Diverse Cellular Activities , Bacteria/metabolism , Bacterial Proteins/metabolism , Endoplasmic Reticulum-Associated Degradation , Licensure , Membrane Proteins/metabolism , Membranes/metabolism , Peptide Hydrolases/metabolism
11.
Health Econ ; 33(11): 2525-2557, 2024 Nov.
Article in English | MEDLINE | ID: mdl-39020467

ABSTRACT

We study the impact of vertical identification card laws, which changed the orientation of driver's licenses and state identification cards from horizontal to vertical for those under 21 years, on teenage tobacco and alcohol use. We study this question using four national datasets (pooled national and state Youth Risk Behavior Surveillance System, National Youth Tobacco Survey, Current Population Survey to Tobacco Use Supplements, and Behavioral Risk Factor Surveillance System). We improve previous databases of vertical ID law implementation by using original archival research to identify the exact date of the law change. We estimate models using standard two-way fixed effects and stacked difference-in-differences that avoid bias from dynamic and heterogeneous treatment effects. Using data through 2021, we do not find evidence of reductions in teenage tobacco and alcohol use. While these laws reduce retail-based purchasing, they also increase social sourcing, thus leading to no net impact on use.


Subject(s)
Tobacco Use , Humans , Adolescent , Male , Female , United States , Underage Drinking , Licensure , Behavioral Risk Factor Surveillance System , Automobile Driving/legislation & jurisprudence , Adolescent Behavior , Young Adult , Alcohol Drinking/epidemiology , Alcohol Drinking/legislation & jurisprudence
12.
Global Health ; 20(1): 52, 2024 Jul 02.
Article in English | MEDLINE | ID: mdl-38956614

ABSTRACT

During the COVID-19 pandemic, intellectual property licensing through bilateral agreements and the Medicines Patent Pool were used to facilitate access to new COVID-19 therapeutics in low- and middle-income countries (LMICs). The lessons learnt from the application of the model to COVID-19 could be relevant for preparedness and response to future pandemics and other health emergencies.The speed at which affordable versions of a new product are available in LMICs is key to the realization of the potential global impact of the product. When initiated early in the research and development life cycle, licensing could facilitate rapid development of generic versions of innovative products in LMICs during a pandemic. The pre-selection of qualified manufacturers, for instance building on the existing network of generic manufacturers engaged during the COVID-19 pandemic, the sharing of know-how and the quick provision of critical inputs such as reference listed drugs (RLDs) could also result in significant time saved. It is important to find a good balance between speed and quality. Necessary quality assurance terms need to be included in licensing agreements, and the potentials of the new World Health Organization Listed Authority mechanism could be explored to promote expedited regulatory reviews and timely access to safe and quality-assured products.The number, capacity, and geographical distribution of licensed companies and the transparency of licensing agreements have implications for the sufficiency of supply, affordability, and supply security. To foster competition and support supply security, licenses should be non-exclusive. There is also a need to put modalities in place to de-risk the development of critical pandemic therapeutics, particularly where generic product development is initiated before the innovator product is proven to be effective and approved. IP licensing and technology transfer can be effective tools to improve the diversification of manufacturing and need to be explored for regional manufacturing for accelerated access at scale in in LMICs and supply security in future pandemics.


Subject(s)
COVID-19 , Developing Countries , Intellectual Property , Licensure , Humans , COVID-19/epidemiology , COVID-19/prevention & control , Pandemics , COVID-19 Drug Treatment , Antiviral Agents/therapeutic use , Drug Industry/legislation & jurisprudence , Drug Industry/organization & administration , Pandemic Preparedness
13.
Med Educ ; 58(2): 247-257, 2024 02.
Article in English | MEDLINE | ID: mdl-37455132

ABSTRACT

BACKGROUND: Although the accreditation approach is widely used to ensure the quality of medical education in many countries, there is scant empirical evidence on whether and how it improves actual medical school performance. We focused on conditions in China, which introduced an accreditation system during the 2010s. Specifically, we examined the relationship between first-round accreditation and actual performance based on the results of medical licensing examinations. Referring to organisation theory, we hypothesised that the impacts of accreditation would depend on existing performance gaps. METHOD: In 2022, we analysed panel data from 105 Chinese medical schools during accreditation (2012 to 2021) and pass rates on medical licensing examinations (2011 to 2019), as matched into 834 school-year records in a window of years before and after accreditation. We employed fixed-effects regression models with a comparison group to exclude factors that may have confounded the impacts of accreditation time. We also demonstrated the heterogeneous effects of accreditation by tier and performance gap of medical schools. RESULTS: The conservative estimates showed a substantial cumulative improvement (over 15 percentage points) in pass rates during the years before accreditation, with no clear trend indicating performance drops in the years after accreditation. Lower-tiered medical schools gained greater benefits from accreditation. Medical schools with a larger prior performance gap achieved a greater percentage point increase in pass rates with the passage of time in pre-accreditation years. CONCLUSIONS: This is the first empirical study to investigate whether accreditation has bridged performance gaps among medical schools. The results support the value of accreditation in China, a country that recently established the system, and might work as a substitute for missing information on early accreditation history in countries with long-established accreditation systems. We encourage more studies in countries that have recently introduced accreditation systems.


Subject(s)
Education, Medical , Schools, Medical , Humans , Accreditation , Licensure , China
14.
BMC Public Health ; 24(1): 2520, 2024 Sep 16.
Article in English | MEDLINE | ID: mdl-39285397

ABSTRACT

A form of food retail regulation called the Stores Licensing Scheme was introduced by the Australian Government in 2007-2022 to ensure food security in remote Indigenous communities of the Northern Territory. We examined evaluations of this Scheme implemented under the Northern Territory National Emergency Response and Stronger Futures Northern Territory Acts. Grey literature search identified nine primary source evaluations. Reported outcomes were extracted and thematic analysis utilised to determine barriers and enablers. Outcomes included improved availability and quality of groceries, financial structures, and retail practices, albeit not consistently reported. Governance and food cost were perceived barriers. Future policy aimed to improve food security through community stores should consider food cost subsidy, measures to incentivise all stores to improve standards, and improved governance arrangements enabling self-determination for Aboriginal and Torres Strait Islander Store Directors.


Subject(s)
Food Supply , Humans , Commerce , Food Security , Licensure , Northern Territory , Rural Population , Australian Aboriginal and Torres Strait Islander Peoples
15.
BMC Public Health ; 24(1): 2586, 2024 Sep 27.
Article in English | MEDLINE | ID: mdl-39334076

ABSTRACT

BACKGROUND: National legislative guidelines for sleep and rest are lacking in the Canadian licensed childcare sector. No review of Canadian legislation for licensed childcare facilities has focused on sleep. This paper provides a review of the Canadian provincial and territorial legislative landscape, regarding sleep, rest, and naps in licensed childcare centers. METHODS: Childcare statutes and regulations for each province and territory were identified and downloaded on a particular date. Statutes and regulations were reviewed focusing on sections articulating licensed childcare facility mandates governing sleep, rest, naps, and sleep equipment. An excel file was used to facilitate systematic data retrieval and comparisons across provinces and territories. Two authors developed and discussed themes that summarized data from the documents. RESULTS: No statutes indicated recommendations for sleep, rest, or naps. Only one regulation defined rest (Alberta). Our analysis of regulations identified four themes representing sleep, rest, and naps: programming (general programming, daily programming); space (dedicated space, amount of space, age-specific space); equipment (developmental appropriateness, acceptable sleep equipment, age-specific equipment); and safety (staffing during sleep/rest, sleep position, sleep monitoring, sleep equipment safety, prohibited practices). In Canada, minimal regulatory consistency is evident in required sleep programming, space, acceptability of sleep equipment, and sleep safety considerations. Most jurisdictions' regulations indicated necessity for developmentally appropriate rest or sleep areas and equipment, in particular for infants, but there was minimal consistency in defining infant age groups. CONCLUSIONS: Although we identified themes related to sleep across regulations, childcare regulations differ in their definitions of infants and specifications for children's sleep and rest in licensed Canadian childcare facilities. Without adequate definitions in legislative components of appropriate sleep duration linked to children's developmental stages, childcare facilities lack guidance to support healthy sleep for children in their care. Future research can examine translation of healthy sleep guidelines into government legislation and mandates for sleep, rest, and naps among young children in licensed childcare.


Subject(s)
Child Day Care Centers , Sleep , Humans , Canada , Child Day Care Centers/legislation & jurisprudence , Child, Preschool , Infant , Child , Policy Making , Licensure/legislation & jurisprudence
16.
Med Teach ; 46(3): 366-372, 2024 03.
Article in English | MEDLINE | ID: mdl-37839017

ABSTRACT

PURPOSE: ChatGPT-4 is an upgraded version of an artificial intelligence chatbot. The performance of ChatGPT-4 on the United States Medical Licensing Examination (USMLE) has not been independently characterized. We aimed to assess the performance of ChatGPT-4 at responding to USMLE Step 1, Step 2CK, and Step 3 practice questions. METHOD: Practice multiple-choice questions for the USMLE Step 1, Step 2CK, and Step 3 were compiled. Of 376 available questions, 319 (85%) were analyzed by ChatGPT-4 on March 21st, 2023. Our primary outcome was the performance of ChatGPT-4 for the practice USMLE Step 1, Step 2CK, and Step 3 examinations, measured as the proportion of multiple-choice questions answered correctly. Our secondary outcomes were the mean length of questions and responses provided by ChatGPT-4. RESULTS: ChatGPT-4 responded to 319 text-based multiple-choice questions from USMLE practice test material. ChatGPT-4 answered 82 of 93 (88%) questions correctly on USMLE Step 1, 91 of 106 (86%) on Step 2CK, and 108 of 120 (90%) on Step 3. ChatGPT-4 provided explanations for all questions. ChatGPT-4 spent 30.8 ± 11.8 s on average responding to practice questions for USMLE Step 1, 23.0 ± 9.4 s per question for Step 2CK, and 23.1 ± 8.3 s per question for Step 3. The mean length of practice USMLE multiple-choice questions that were answered correctly and incorrectly by ChatGPT-4 was similar (difference = 17.48 characters, SE = 59.75, 95%CI = [-100.09,135.04], t = 0.29, p = 0.77). The mean length of ChatGPT-4's correct responses to practice questions was significantly shorter than the mean length of incorrect responses (difference = 79.58 characters, SE = 35.42, 95%CI = [9.89,149.28], t = 2.25, p = 0.03). CONCLUSIONS: ChatGPT-4 answered a remarkably high proportion of practice questions correctly for USMLE examinations. ChatGPT-4 performed substantially better at USMLE practice questions than previous models of the same AI chatbot.


Subject(s)
Artificial Intelligence , Software , Humans , Licensure , Physical Examination
17.
Occup Med (Lond) ; 74(6): 455-457, 2024 09 23.
Article in English | MEDLINE | ID: mdl-39031950

ABSTRACT

BACKGROUND: Medical incapacity at the wheel is a rare but high-profile factor in accident causation. The UK Driver and Vehicle Licensing Agency (DVLA) does not require a review of medical records on the application for a bus licence, but applicants sign a self-declaration of medical history. There is debate over what, if any, verification of medical information is required for doctors who complete the medical assessment. AIMS: To assess how self-declaration compares against General Practitioner (GP) patient records for a series of bus drivers undergoing driver licensing assessment in a 12-month period. METHODS: Review of GP reports received for bus driver medicals undertaken in a 12-month period between 1 July 2022 and 30 June 2023. RESULTS: Of the 145 medicals undertaken, 12 (8.3%) GP reports contained undeclared medical conditions that required further evaluation and may have impacted on fitness to drive. CONCLUSIONS: Studies into the contribution of medical incapacity at the wheel to vehicle accidents are sparse, as, thankfully, are the events themselves. Nevertheless, given the updated General Medical Council guidance to doctors on confidentiality and public protection responsibilities, and evidence to suggest that doctors' knowledge of the DVLA guidelines is poor, it might be pertinent for the DVLA to reconsider its approach to driver self-reporting in some circumstances, given the discrepancy highlighted here.


Subject(s)
Automobile Driving , Medical Records , Self Report , Humans , Male , Motor Vehicles , Accidents, Traffic/statistics & numerical data , Adult , United Kingdom , Female , Middle Aged , Licensure , Automobile Driver Examination
18.
BMC Med Educ ; 24(1): 143, 2024 Feb 14.
Article in English | MEDLINE | ID: mdl-38355517

ABSTRACT

BACKGROUND: Large language models like ChatGPT have revolutionized the field of natural language processing with their capability to comprehend and generate textual content, showing great potential to play a role in medical education. This study aimed to quantitatively evaluate and comprehensively analysis the performance of ChatGPT on three types of national medical examinations in China, including National Medical Licensing Examination (NMLE), National Pharmacist Licensing Examination (NPLE), and National Nurse Licensing Examination (NNLE). METHODS: We collected questions from Chinese NMLE, NPLE and NNLE from year 2017 to 2021. In NMLE and NPLE, each exam consists of 4 units, while in NNLE, each exam consists of 2 units. The questions with figures, tables or chemical structure were manually identified and excluded by clinician. We applied direct instruction strategy via multiple prompts to force ChatGPT to generate the clear answer with the capability to distinguish between single-choice and multiple-choice questions. RESULTS: ChatGPT failed to pass the accuracy threshold of 0.6 in any of the three types of examinations over the five years. Specifically, in the NMLE, the highest recorded accuracy was 0.5467, which was attained in both 2018 and 2021. In the NPLE, the highest accuracy was 0.5599 in 2017. In the NNLE, the most impressive result was shown in 2017, with an accuracy of 0.5897, which is also the highest accuracy in our entire evaluation. ChatGPT's performance showed no significant difference in different units, but significant difference in different question types. ChatGPT performed well in a range of subject areas, including clinical epidemiology, human parasitology, and dermatology, as well as in various medical topics such as molecules, health management and prevention, diagnosis and screening. CONCLUSIONS: These results indicate ChatGPT failed the NMLE, NPLE and NNLE in China, spanning from year 2017 to 2021. but show great potential of large language models in medical education. In the future high-quality medical data will be required to improve the performance.


Subject(s)
Artificial Intelligence , Educational Measurement , Licensure , China , Data Accuracy , Education, Nursing , Education, Pharmacy , Education, Medical
19.
BMC Med Educ ; 24(1): 1013, 2024 Sep 16.
Article in English | MEDLINE | ID: mdl-39285377

ABSTRACT

BACKGROUND: ChatGPT, a recently developed artificial intelligence (AI) chatbot, has demonstrated improved performance in examinations in the medical field. However, thus far, an overall evaluation of the potential of ChatGPT models (ChatGPT-3.5 and GPT-4) in a variety of national health licensing examinations is lacking. This study aimed to provide a comprehensive assessment of the ChatGPT models' performance in national licensing examinations for medical, pharmacy, dentistry, and nursing research through a meta-analysis. METHODS: Following the PRISMA protocol, full-text articles from MEDLINE/PubMed, EMBASE, ERIC, Cochrane Library, Web of Science, and key journals were reviewed from the time of ChatGPT's introduction to February 27, 2024. Studies were eligible if they evaluated the performance of a ChatGPT model (ChatGPT-3.5 or GPT-4); related to national licensing examinations in the fields of medicine, pharmacy, dentistry, or nursing; involved multiple-choice questions; and provided data that enabled the calculation of effect size. Two reviewers independently completed data extraction, coding, and quality assessment. The JBI Critical Appraisal Tools were used to assess the quality of the selected articles. Overall effect size and 95% confidence intervals [CIs] were calculated using a random-effects model. RESULTS: A total of 23 studies were considered for this review, which evaluated the accuracy of four types of national licensing examinations. The selected articles were in the fields of medicine (n = 17), pharmacy (n = 3), nursing (n = 2), and dentistry (n = 1). They reported varying accuracy levels, ranging from 36 to 77% for ChatGPT-3.5 and 64.4-100% for GPT-4. The overall effect size for the percentage of accuracy was 70.1% (95% CI, 65-74.8%), which was statistically significant (p < 0.001). Subgroup analyses revealed that GPT-4 demonstrated significantly higher accuracy in providing correct responses than its earlier version, ChatGPT-3.5. Additionally, in the context of health licensing examinations, the ChatGPT models exhibited greater proficiency in the following order: pharmacy, medicine, dentistry, and nursing. However, the lack of a broader set of questions, including open-ended and scenario-based questions, and significant heterogeneity were limitations of this meta-analysis. CONCLUSIONS: This study sheds light on the accuracy of ChatGPT models in four national health licensing examinations across various countries and provides a practical basis and theoretical support for future research. Further studies are needed to explore their utilization in medical and health education by including a broader and more diverse range of questions, along with more advanced versions of AI chatbots.


Subject(s)
Artificial Intelligence , Educational Measurement , Licensure , Humans , Education, Nursing/standards , Educational Measurement/methods , Educational Measurement/standards , Licensure/standards , Education, Pharmacy/standards , Education, Medical/standards , Education, Dental/standards
20.
J Am Pharm Assoc (2003) ; 64(3): 102031, 2024.
Article in English | MEDLINE | ID: mdl-38341088

ABSTRACT

BACKGROUND: Despite variation in licensure requirements and models for pharmacy practice nationwide, there is little published data within the United States regarding pharmacist perspectives. OBJECTIVES: The purpose of this study was to identify the demographics, awareness, and perceptions about current pharmacist licensure models. METHODS: A fifteen-question mixed-methods survey was created and distributed via Qualtrics-XM Survey to all Utah licensed pharmacists (n = 4154). Data collection was August 22-September 22, 2022. Before survey distribution, pilot feedback was sought from the Utah Board of Pharmacy and pharmacists at the 118th National Association of Boards of Pharmacy (NABP) national conference. Exempt status was granted by Roseman University Institutional Review Board. Quantitative and qualitative data analysis allowed for descriptive statistics and thematic content identification. RESULTS: The survey collected 972 responses for a response rate of 23% and a completion rate of 94%. Respondents self-identified 36 practice areas. Distribution of years in practice was well dispersed between the predefined ranges. Primary state of licensure was Utah (80%), with additional representation from all 50 states and Guam. The survey showed a variation in awareness regarding other healthcare professional licensing models with 40.83% "aware," 40.62% "unaware," and 18.55% "unsure". A majority showed awareness of the NABP Verify program (55.8%), but unawareness of the Electronic Licensure Transfer Program program (56.14%). Respondents agreed with increased license portability for medically underserved and rural areas (71.79%) and preference for having a law exam (56.72%). Pharmacists (n = 405) noted concerns regarding multistate renewal requirements, fees, and continuing education. CONCLUSION: This study provided baseline data on a topic that is missing in existing literature. Results illustrated a high completion rate, a diversity of demographics including well dispersed age ranges, years in practice, and qualitative responses. The quantitative data shed light on a variety of pharmacist perspectives and varied awareness about NABP licensure programs and compacts.


Subject(s)
Pharmacists , Humans , Pharmacists/statistics & numerical data , Surveys and Questionnaires , Female , Male , Adult , Utah , Middle Aged , Licensure, Pharmacy , United States , Attitude of Health Personnel , Pharmaceutical Services/organization & administration , Feedback , Licensure
SELECTION OF CITATIONS
SEARCH DETAIL