Assessing accuracy of ChatGPT in response to questions from day to day pharmaceutical care in hospitals.

Background: The advent of Large Language Models (LLMs) such as ChatGPT introduces opportunities within the medical field. Nonetheless, use of LLM poses a risk when healthcare practitioners and patients present clinical questions to these programs without a comprehensive understanding of its suitability for clinical contexts. Objective: The objective of this study was to assess ChatGPT's ability to generate appropriate responses to clinical questions that hospital pharmacists could encounter during routine patient care. Methods: Thirty questions from 10 different domains within clinical pharmacy were collected during routine care. Questions were presented to ChatGPT in a standardized format, including patients' age, sex, drug name, dose, and indication. Subsequently, relevant information regarding specific cases were provided, and the prompt was concluded with the query "what would a hospital pharmacist do?". The impact on accuracy was assessed for each domain by modifying personification to "what would you do?", presenting the question in Dutch, and regenerating the primary question. All responses were independently evaluated by two senior hospital pharmacists, focusing on the availability of an advice, accuracy and concordance. Results: In 77% of questions, ChatGPT provided an advice in response to the question. For these responses, accuracy and concordance were determined. Accuracy was correct and complete for 26% of responses, correct but incomplete for 22% of responses, partially correct and partially incorrect for 30% of responses and completely incorrect for 22% of responses. The reproducibility was poor, with merely 10% of responses remaining consistent upon regeneration of the primary question. Conclusions: While concordance of responses was excellent, the accuracy and reproducibility were poor. With the described method, ChatGPT should not be used to address questions encountered by hospital pharmacists during their shifts. However, it is important to acknowledge the limitations of our methodology, including potential biases, which may have influenced the findings.

Narrow-spectrum antibiotics for community-acquired pneumonia in Dutch adults (CAP-PACT): a cross-sectional, stepped-wedge, cluster-randomised, non-inferiority, antimicrobial stewardship intervention trial.

BACKGROUND: Adults hospitalised to a non-intensive care unit (ICU) ward with moderately severe community-acquired pneumonia are frequently treated with broad-spectrum antibiotics, despite Dutch guidelines recommending narrow-spectrum antibiotics. Therefore, we investigated whether an antibiotic stewardship intervention would reduce the use of broad-spectrum antibiotics in patients with moderately severe community-acquired pneumonia without compromising their safety. METHODS: In this cross-sectional, stepped-wedge, cluster-randomised, non-inferiority trial (CAP-PACT) done in 12 hospitals in the Netherlands, we enrolled immunocompetent adults (≥18 years) who were admitted to a non-ICU ward and had a working diagnosis of moderately severe community-acquired pneumonia. All participating hospitals started in a control period and every 3 months a block of two hospitals transitioned from the control to the intervention period, with all hospitals eventually ending in the intervention period. The unit of randomisation was the hospital (cluster), and electronic randomisation (by an independent data manager) decided the sequence (the time of intervention) by which hospitals would cross over from the control period to the intervention period. Blinding was not possible. The antimicrobial stewardship intervention was a bundle targeting health-care providers and comprised education, engaging opinion leaders, and prospective audit and feedback of antibiotic use. The co-primary outcomes were broad-spectrum days of therapy per patient, tested by superiority, and 90-day all-cause mortality, tested by non-inferiority with a non-inferiority margin of 3%, and were analysed in the intention-to-treat population, comprising all patients who were enrolled in the control and intervention periods. This trial was prospectively registered at ClinicalTrials.gov, NCT02604628. FINDINGS: Between Nov 1, 2015, and Nov 1, 2017, 5683 patients were assessed for eligibility, of whom 4084 (2235 in the control period and 1849 in the intervention period) were included in the intention-to-treat analysis. The adjusted mean broad-spectrum days of therapy per patient were reduced from 6·5 days in the control period to 4·8 days in the intervention period, yielding an absolute reduction of -1·7 days (95% CI -2·4 to -1·1) and a relative reduction of 26·6% (95% CI 18·0-35·3). Crude 90-day mortality was 10·9% (242 of 2228 died) in the control period and 10·8% (199 of 1841) in the intervention period, yielding an adjusted absolute risk difference of 0·4% (90% CI -2·7 to 2·4), indicating non-inferiority. INTERPRETATION: In patients hospitalised with moderately severe community-acquired pneumonia, a multifaceted antibiotic stewardship intervention might safely reduce broad-spectrum antibiotic use. FUNDING: None.

Impact of antimicrobial de-escalation on mortality: a literature review of study methodology and recommendations for observational studies.

Introduction: The safety of de-escalation of empirical antimicrobial therapy is largely based on observational data, with many reporting protective effects on mortality. As there is no plausible biological explanation for this phenomenon, it is most probably caused by confounding by indication.Areas covered: We evaluate the methodology used in observational studies on the effects of de-escalation of antimicrobial therapy on mortality. We extended the search for a recent systematic review and identified 52 observational studies. The heterogeneity in study populations was large. Only 19 (36.5%) studies adjusted for confounders and four (8%) adjusted for clinical stability during admission, all as a fixed variable. All studies had methodological limitations, most importantly the lack of adjustment for clinical stability, causing bias toward a protective effect.Expert opinion: The methodology used in studies evaluating the effects of de-escalation on mortality requires improvement. We depicted all potential confounders in a directed acyclic graph to illustrate all associations between exposure (de-escalation) and outcome (mortality). Clinical stability is an important confounder in this association and should be modeled as a time-varying variable. We recommend to include de-escalation as time-varying exposure and use inverse-probability-of-treatment weighted marginal structural models to properly adjust for time-varying confounders.

Confounding by indication of the safety of de-escalation in community-acquired pneumonia: A simulation study embedded in a prospective cohort.

Observational studies have demonstrated that de-escalation of antimicrobial therapy is independently associated with lower mortality. This most probably results from confounding by indication. Reaching clinical stability is associated with the decision to de-escalate and with survival. However, studies rarely adjust for this confounder. We quantified the potential confounding effect of clinical stability on the estimated impact of de-escalation on mortality in patients with community-acquired pneumonia. Data were used from the Community-Acquired Pneumonia immunization Trial in Adults (CAPiTA). The primary outcome was 30-day mortality. We performed Cox proportional-hazards regression with de-escalation as time-dependent variable and adjusted for baseline characteristics using propensity scores. The potential impact of unmeasured confounding was quantified through simulating a variable representing clinical stability on day three, using data on prevalence and associations with mortality from the literature. Of 1,536 included patients, 257 (16.7%) were de-escalated, 123 (8.0%) were escalated and in 1156 (75.3%) the antibiotic spectrum remained unchanged. Crude 30-day mortality was 3.5% (9/257) and 10.9% (107/986) in the de-escalation and continuation groups, respectively. The adjusted hazard ratio of de-escalation for 30-day mortality (compared to patients with unchanged coverage), without adjustment for clinical stability, was 0.39 (95%CI: 0.19-0.79). If 90% to 100% of de-escalated patients were clinically stable on day three, the fully adjusted hazard ratio would be 0.56 (95%CI: 0.27-1.12) to 1.04 (95%CI: 0.49-2.23), respectively. The simulated confounder was substantially stronger than any of the baseline confounders in our dataset. Quantification of effects of de-escalation on patient outcomes without proper adjustment for clinical stability results in strong negative bias. This study suggests the effect of de-escalation on mortality needs further well-designed prospective research to determine effect size more accurately.

Inappropriate Use of Antimicrobials for Lower Respiratory Tract Infections in Elderly Patients: Patient- and Community-Related Implications and Possible Interventions.

The elderly are more susceptible to infections, which is reflected in the incidence and mortality of lower respiratory tract infections (LRTIs) increasing with age. Several aspects of antimicrobial use for LRTIs in elderly patients should be considered to determine appropriateness. We discuss possible differences in microbial etiology between elderly and younger adults, definitions of inappropriate antimicrobial use for LRTIs currently found in the literature, along with their results, and the possible negative impact of antimicrobial therapy at both an individual and community level. Finally, we propose that both antimicrobial stewardship interventions and novel rapid diagnostic techniques may optimize antimicrobial use in elderly patients with LRTIs.

Monitoring, documenting and reporting the quality of antibiotic use in the Netherlands: a pilot study to establish a national antimicrobial stewardship registry.

BACKGROUND: The Dutch Working Party on Antibiotic Policy is developing a national antimicrobial stewardship registry. This registry will report both the quality of antibiotic use in hospitals in the Netherlands and the stewardship activities employed. It is currently unclear which aspects of the quality of antibiotic use are monitored by antimicrobial stewardship teams (A-teams) and can be used as indicators for the stewardship registry. In this pilot study we aimed to determine which stewardship objectives are eligible for the envisioned registry. METHODS: We performed an observational pilot study among five Dutch hospitals. We assessed which of the 14 validated stewardship objectives (11 process of care recommendations and 3 structure of care recommendations) the A-teams monitored and documented in individual patients. They provided, where possible, data to compute quality indicator (QI) performance scores in line with recently developed QIs to measure appropriate antibiotic use in hospitalized adults for the period of January 2015 through December 2015 RESULTS: All hospitals had a local antibiotic guideline describing recommended antimicrobial use. All A-teams monitored the performance of bedside consultations in Staphylococcus aureus bacteremia and the prescription of restricted antimicrobials. Documentation and reporting were the best for the use of restricted antimicrobials: 80% of the A-teams could report data. Lack of time and the absence of an electronic medical record system enabling documentation during the daily work flow were the main barriers hindering documentation and reporting. CONCLUSIONS: Five out of 11 stewardship objectives were actively monitored by A-teams. Without extra effort, 4 A-teams could report on the quality of use of restricted antibiotics. Therefore, this aspect of antibiotic use should be the starting point of the national antimicrobial stewardship registry. Our registry is expected to become a powerful tool to evaluate progress and impact of antimicrobial stewardship programs in hospitals.

Treatment of epilepsy in daily clinical practice: have outcomes improved over the past 10 years?

In the past decade, many new antiepileptic drugs have become available, but their influence on patient outcomes in daily practice is not well known. In a community-based study, we assessed changes in epilepsy treatment and outcomes over a 10-year period. We compared two cross-sectional community-based samples that were obtained from the same Dutch suburban region in 2000 and 2010 using pharmacy records for recruitment, including 344 and 248 epilepsy patients, respectively. The main outcome was self-reported quality of life (QoL, using the QOLIE-31). Potential predictors of QoL (adverse effects, seizure control, seizure acceptability, demographic, epilepsy- or treatment-related determinants) were assessed by multivariable linear regression. New antiepileptic drugs were used by 9 % of patients in 2000 and 34 % in 2010, P < 0.001. More than 80 % in both samples reported to be treated by a neurologist. We found no significant differences in QOLIE-31 scores (72.57 vs. 72.44), or in the proportion of patients with complete seizure control (51 vs. 54 %), between the two study samples. Seizure control and adverse effects were important independent predictors of quality of life, explaining 58 % of the variance. We found no evidence that in this community the QoL of epilepsy patients has improved in the last decade, despite unrestricted availability of healthcare resources, including accessibility to new AEDs. The relatively low proportion with complete seizure control and the high rate of adverse effects in these community-based samples suggest that the effectiveness of epilepsy treatment is still far from optimal.