Integrating human expertise & automated methods for a dynamic and multi-parametric evaluation of large language models' feasibility in clinical decision-making.

BACKGROUND: Recent enhancements in Large Language Models (LLMs) such as ChatGPT have exponentially increased user adoption. These models are accessible on mobile devices and support multimodal interactions, including conversations, code generation, and patient image uploads, broadening their utility in providing healthcare professionals with real-time support for clinical decision-making. Nevertheless, many authors have highlighted serious risks that may arise from the adoption of LLMs, principally related to safety and alignment with ethical guidelines. OBJECTIVE: To address these challenges, we introduce a novel methodological approach designed to assess the specific feasibility of adopting LLMs within a healthcare area, with a focus on clinical nursing, evaluating their performance and thereby directing their choice. Emphasizing LLMs' adherence to scientific advancements, this approach prioritizes safety and care personalization, according to the "Organization for Economic Co-operation and Development" frameworks for responsible AI. Moreover, its dynamic nature is designed to adapt to future evolutions of LLMs. METHOD: Through integrating advanced multidisciplinary knowledge, including Nursing Informatics, and aided by a prospective literature review, seven key domains and specific evaluation items were identified as follows:A Peer Review by experts in Nursing and AI was performed, ensuring scientific rigor and breadth of insights for an essential, reproducible, and coherent methodological approach. By means of a 7-point Likert scale, thresholds are defined in order to classify LLMs as "unusable", "usable with high caution", and "recommended" categories. Nine state of the art LLMs were evaluated using this methodology in clinical oncology nursing decision-making, producing preliminary results. Gemini Advanced, Anthropic Claude 3 and ChatGPT 4 achieved the minimum score of the State of the Art Alignment & Safety domain for classification as "recommended", being also endorsed across all domains. LLAMA 3 70B and ChatGPT 3.5 were classified as "usable with high caution." Others were classified as unusable in this domain. CONCLUSION: The identification of a recommended LLM for a specific healthcare area, combined with its critical, prudent, and integrative use, can support healthcare professionals in decision-making processes.

A methodological framework for rigorous systematic reviews: Tailoring comprehensive analyses to clinicians and healthcare professionals.

Systematic reviews represent a fundamental study design, providing the highest level of evidence across diverse research inquiries, encompassing both public health and clinical research and practice. However, for healthcare professionals, the process of selecting, synthesizing, and interpreting evidence can be challenging, and requires specialized skills. Therefore, it is imperative to explore innovative solutions aimed at simplifying and making the traditional systematic review process more accessible while ensuring the validity and reliability of results. In this perspective, our research objective is to develop a systematic review framework that, while maintaining a rigorous methodological approach, streamlines the process for healthcare professionals. This study describes such approach in every phase, from the collection of evidence to the writing of the text, creating a guide for the healthcare professional who approaches this type of research. The qualitative and organizational analysis tools are also described, providing useful information for the use of non-paid programs. This systematic review aims to develop a framework with a rigorous methodological approach that allows simplify the process for clinicians and healthcare professionals. The implementation of this methodology in clinical practice offers new perspectives to ensure a thoughtful consideration and application of scientific evidence and opens the way to innovative and easily accessible solutions to facilitate the conduct of systematic reviews in the clinical care setting.

Inpatient deaths by medical service: characteristics in a comprehensive cancer centre.

AIM: To examine the intensity of care in the last 3 days of life in different medical settings of a comprehensive cancer centre. METHODS: In this retrospective study, the charts of patients who died in hospital from July 2019 to June 2023 were reviewed. The units taken into consideration were acute palliative care unit (APCU), oncology (ONC) and haematology (HEM), which included also the transplantation unit. Age, gender, diagnosis, Karnofsky or Eastern Cooperative Oncology Group, drugs used in the last 3 days of life, respiratory support, transfusion, parenteral nutrition, imaging studies, blood gas analysis, biochemistry, consultations and endoscopy were retrieved. RESULTS: Data on 177 patients who died in the period taken into consideration were reviewed. APCU was characterised by more frequent use of opioids, midazolam and scopolamine butylbromide. On the other side, HEM was characterised by higher use of vasoactive drugs, antibiotics, transfusions, imaging studies, growth factors, consultations, biochemistry, blood gas analysis, parenteral nutrition and ventilatory support. ONC was characterised by the larger use of anticoagulants and ventilatory support. CONCLUSION: A more aggressive treatment was performed in HEM, as denoted by a large use of life-sustaining treatments and investigations. Taken together, these data show that the way to die is different depending on the setting of care. End-of-life funnel is similar for every dying patient and choices should be based on short prognosis and expectation rather than on disproportionate and afinalistic treatments.

Pain Catastrophizing in Cancer Patients.

BACKGROUND: Pain catastrophizing is a group of negative irrational cognitions in the context of anticipated or actual pain. The aim of this study was to decipher the possible role of catastrophism on pain expression and outcomes after a comprehensive palliative care treatment. METHODS: A consecutive sample of patients with uncontrolled pain was assessed. Demographic characteristics, symptom intensity included in the Edmonton symptom assessment system (ESAS), and opioid drugs used were recorded at admission (T0). The Pain Catastrophizing Scale (PCS) was measured for patients. Patients were also asked about their personalized symptom goal (PSG) for each symptom of ESAS. One week after a comprehensive palliative care treatment (T7), ESAS and opioid doses used were recorded again, and the number of patients who achieved their PSG (PSGR) were calculated. At the same interval (T7), Minimal Clinically Important Difference (MCID) was calculated using patient global impression (PGI). RESULTS: Ninety-five patients were eligible. A significant decrease in symptom intensity was reported for all ESAS items. PGI was positive for all symptoms, with higher values for pain, poor well-being, and poor sleep. Only the rumination subscale of catastrophism was significantly associated with pain at T0 (B = 0.540; p = 0.034). CONCLUSIONS: Catastrophism was not associated with the levels of pain intensity, PSG, PSGR, and PGI for pain, except the rumination subscale that was associated with pain intensity at T0. A comprehensive palliative care management provided the relevant changes in symptom burden, undoing the pain expression associated with rumination.

Mortality rate and palliative sedation in an acute palliative care unit.

AIM: To assess the mortality rate and the use of palliative sedation (PS) in an advanced long-standing acute palliative care unit (APCU) METHODS: The charts of patients who died and eventually received PS, consecutively admitted to the APCU for 4 years, were reviewed. Patients' characteristics and symptom intensity were recorded at admission, 3 days before death and the day before death (T0, T-3, T-end, respectively). For patients who were administered midazolam for PS, initial and final doses of drugs, as well as duration of PS until death, were recorded. RESULTS: One hundred and forty-eight patients died in APCU (8.9%), and 45 of them (30.4%) received PS. Younger patients and those reporting high levels of dyspnoea at T-3 and T-end were more likely to be sedated (p=0.002, p=0.013 and 0.002, respectively). The mean duration of PS was 27.47 hours. Mean initial and final doses of midazolam were 35.45 mg/day (SD 19.7) and 45.57 mg/day (SD 20.6), respectively (p=0.001). CONCLUSION: Mortality rate in APCU was very low. As a percentage of the number of deaths, PS rate was similar to that reported in other settings. PS does not seem to accelerate impending death.

Maddalena Opioid Switching Score in patients with cancer pain.

ABSTRACT: Evaluation of opioid switching (OS) for cancer pain has not been properly assessed. The aim of this study was to assess an integrated score (Maddalena Opioid Switching Score) as a simple and repeatable tool to evaluate the outcomes of OS, facilitating the interpretation and comparison of studies, and information exchange among researchers. The integrated score took into account pain intensity, intensity of opioid-related symptoms, and cognitive function by using an author's formula. Physical and psychological symptoms were evaluated by the Edmonton Symptom Assessment Scale and Patient Global Impression (PGI) by the minimal clinically important difference. One hundred six patients were analyzed. Ninety-five patients were switched successfully, and 11 patients underwent a further OS and/or an alternative procedure. The Maddalena Opioid Switching Score significantly decreased after OS and was highly correlated to PGI of improvement ( P < 0.0005). In patients with unsuccessful OS, no significant changes in the Maddalena Opioid Switching Score and PGI were observed. A significant reduction in Edmonton Symptom Assessment Scale items intensity was observed after OS. The Maddalena Opioid Switching Score resulted to be a sensitive instrument for measuring the clinical improvement produced by OS.

Opioids in advanced cancer: use, storage and disposal in the home.

OBJECTIVES: To determine the patterns of storing, using and disposing of opioids among patients with advanced cancer followed at home. METHODS: Patients who were prescribed opioids were selected. Prescribed opioids and their doses used for background pain and breakthrough pain were collected, as well as CAGE (cut down, annoyed, guilty and eye opener) for alcohol and drugs, smoking and history of illicit substance use. Questions regarding the opioid use, storage and disposal were posed. RESULTS: 100 patients were surveyed. Fifty-one patients had unused opioids at home, 25 patients did not throw away the drugs, 40 patients saved opioids for future use and 35 patients were unaware of proper opioid disposal methods. A total of 28 patients reported unsafe use by sharing or losing their opioids; 12 patients were unaware that their opioid could be fatal when taken by others. Most patients acknowledged that pain medications could be dangerous when taken by others. Patients with a partner and who were married were more likely to keep their opioids locked (p=0.028 and p=0.025, respectively). CONCLUSION: A large number of patients with advanced cancer followed at home do not store, use and dispose of opioids safely. Patient education programmes should be incorporated to decrease the availability of opioids at home for abuse, diversion, and accidental poisoning.