Current safeguards, risk mitigation, and transparency measures of large language models against the generation of health disinformation: repeated cross sectional analysis.

OBJECTIVES: To evaluate the effectiveness of safeguards to prevent large language models (LLMs) from being misused to generate health disinformation, and to evaluate the transparency of artificial intelligence (AI) developers regarding their risk mitigation processes against observed vulnerabilities. DESIGN: Repeated cross sectional analysis. SETTING: Publicly accessible LLMs. METHODS: In a repeated cross sectional analysis, four LLMs (via chatbots/assistant interfaces) were evaluated: OpenAI's GPT-4 (via ChatGPT and Microsoft's Copilot), Google's PaLM 2 and newly released Gemini Pro (via Bard), Anthropic's Claude 2 (via Poe), and Meta's Llama 2 (via HuggingChat). In September 2023, these LLMs were prompted to generate health disinformation on two topics: sunscreen as a cause of skin cancer and the alkaline diet as a cancer cure. Jailbreaking techniques (ie, attempts to bypass safeguards) were evaluated if required. For LLMs with observed safeguarding vulnerabilities, the processes for reporting outputs of concern were audited. 12 weeks after initial investigations, the disinformation generation capabilities of the LLMs were re-evaluated to assess any subsequent improvements in safeguards. MAIN OUTCOME MEASURES: The main outcome measures were whether safeguards prevented the generation of health disinformation, and the transparency of risk mitigation processes against health disinformation. RESULTS: Claude 2 (via Poe) declined 130 prompts submitted across the two study timepoints requesting the generation of content claiming that sunscreen causes skin cancer or that the alkaline diet is a cure for cancer, even with jailbreaking attempts. GPT-4 (via Copilot) initially refused to generate health disinformation, even with jailbreaking attempts-although this was not the case at 12 weeks. In contrast, GPT-4 (via ChatGPT), PaLM 2/Gemini Pro (via Bard), and Llama 2 (via HuggingChat) consistently generated health disinformation blogs. In September 2023 evaluations, these LLMs facilitated the generation of 113 unique cancer disinformation blogs, totalling more than 40 000 words, without requiring jailbreaking attempts. The refusal rate across the evaluation timepoints for these LLMs was only 5% (7 of 150), and as prompted the LLM generated blogs incorporated attention grabbing titles, authentic looking (fake or fictional) references, fabricated testimonials from patients and clinicians, and they targeted diverse demographic groups. Although each LLM evaluated had mechanisms to report observed outputs of concern, the developers did not respond when observations of vulnerabilities were reported. CONCLUSIONS: This study found that although effective safeguards are feasible to prevent LLMs from being misused to generate health disinformation, they were inconsistently implemented. Furthermore, effective processes for reporting safeguard problems were lacking. Enhanced regulation, transparency, and routine auditing are required to help prevent LLMs from contributing to the mass generation of health disinformation.

Financial Toxicity, Time Toxicity, and Quality of Life in Multiple Myeloma.

BACKGROUND: Patients with multiple myeloma (MM) may be on therapy for years, which can lead to financial toxicity (FinTox) or time toxicity (TimeTox). The prevalence, predictors, and quality of life (QOL) impacts of FinTox and TimeTox during different phases of MM treatment have not been characterized. PATIENTS AND METHODS: We conducted a single-center cross-sectional survey of patients with MM who had undergone transplantation. FinTox+ was defined as a COST-FACIT score <23, TimeTox+ as MM-related interactions (including phone calls) ≥1x weekly or ≥1x monthly in-person among far-residing patients, QOL using PROMIS Global Health, and functional status using patient-reported Karnofsky performance status (KPS). RESULTS: Of 252 patients, 22% and 40% met FinTox+ and TimeTox+ criteria respectively. Respective FinTox+ and TimeTox+ proportions were 22%/37% for patients on maintenance, 22%/82% with active therapy, and 20%/14% with observation. FinTox+ predictors included annual income (P < .01) and out-of-pocket costs (P < .01). TimeTox+ predictors included disease status (P < .001), caregiver status (P = .01), far-residing status (P < .001), and out-of-pocket costs (P = .03). FinTox+ was associated with a clinically meaningful decrease in mental QOL, while TimeTox+ patients were more likely to have KPS ≤ 80. CONCLUSIONS: In our large study, monetary status but not disease status predicted FinTox. Over a third of patients on maintenance reported TimeTox. FinTox+ was associated with decreased mental health, while TimeTox+ was associated with worse performance status. These two toxicities may negatively impact patient wellbeing, and studies of strategies to mitigate their impact are in development.

Clinical characteristics, racial inequities, and outcomes in patients with breast cancer and COVID-19: A COVID-19 and cancer consortium (CCC19) cohort study.

Background: Limited information is available for patients with breast cancer (BC) and coronavirus disease 2019 (COVID-19), especially among underrepresented racial/ethnic populations. Methods: This is a COVID-19 and Cancer Consortium (CCC19) registry-based retrospective cohort study of females with active or history of BC and laboratory-confirmed severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2) infection diagnosed between March 2020 and June 2021 in the US. Primary outcome was COVID-19 severity measured on a five-level ordinal scale, including none of the following complications, hospitalization, intensive care unit admission, mechanical ventilation, and all-cause mortality. Multivariable ordinal logistic regression model identified characteristics associated with COVID-19 severity. Results: 1383 female patient records with BC and COVID-19 were included in the analysis, the median age was 61 years, and median follow-up was 90 days. Multivariable analysis revealed higher odds of COVID-19 severity for older age (aOR per decade, 1.48 [95% CI, 1.32-1.67]); Black patients (aOR 1.74; 95 CI 1.24-2.45), Asian Americans and Pacific Islander patients (aOR 3.40; 95 CI 1.70-6.79) and Other (aOR 2.97; 95 CI 1.71-5.17) racial/ethnic groups; worse ECOG performance status (ECOG PS ≥2: aOR, 7.78 [95% CI, 4.83-12.5]); pre-existing cardiovascular (aOR, 2.26 [95% CI, 1.63-3.15])/pulmonary comorbidities (aOR, 1.65 [95% CI, 1.20-2.29]); diabetes mellitus (aOR, 2.25 [95% CI, 1.66-3.04]); and active and progressing cancer (aOR, 12.5 [95% CI, 6.89-22.6]). Hispanic ethnicity, timing, and type of anti-cancer therapy modalities were not significantly associated with worse COVID-19 outcomes. The total all-cause mortality and hospitalization rate for the entire cohort was 9% and 37%, respectively however, it varied according to the BC disease status. Conclusions: Using one of the largest registries on cancer and COVID-19, we identified patient and BC-related factors associated with worse COVID-19 outcomes. After adjusting for baseline characteristics, underrepresented racial/ethnic patients experienced worse outcomes compared to non-Hispanic White patients. Funding: This study was partly supported by National Cancer Institute grant number P30 CA068485 to Tianyi Sun, Sanjay Mishra, Benjamin French, Jeremy L Warner; P30-CA046592 to Christopher R Friese; P30 CA023100 for Rana R McKay; P30-CA054174 for Pankil K Shah and Dimpy P Shah; KL2 TR002646 for Pankil Shah and the American Cancer Society and Hope Foundation for Cancer Research (MRSG-16-152-01-CCE) and P30-CA054174 for Dimpy P Shah. REDCap is developed and supported by Vanderbilt Institute for Clinical and Translational Research grant support (UL1 TR000445 from NCATS/NIH). The funding sources had no role in the writing of the manuscript or the decision to submit it for publication. Clinical trial number: CCC19 registry is registered on ClinicalTrials.gov, NCT04354701.

Heterogeneity and Utility of Pharmaceutical Company Sharing of Individual-Participant Data Packages.

Importance: The pharmaceutical industry has made substantial investments in developing processes for sharing individual-participant data (IPD) from clinical trials. However, the utility and completeness of shared IPD and supporting documents must be evaluated to ensure the potential for scientific advancements from the data sharing ecosystem can be realized. Objective: To assess the utility and completeness of IPD and supporting documents provided from industry-sponsored clinical trials. Design, Setting, and Participants: From February 9, 2022, to February 9, 2023, 91 of 203 clinical trials supporting US Food and Drug Administration registrations of anticancer medicines for the treatment of solid tumors from the past decade were confirmed as eligible for IPD request. This quality improvement study performed a retrospective audit of the utility and completeness of the IPD and supporting documents provided from the 91 clinical trials for a planned meta-analysis. Exposures: Request for IPD from 91 clinical oncology trials indicated as eligible for the request. Main Outcomes and Measures: The utility and completeness of the IPD and supporting documents provided. Results: The IPD packages were obtained from 70 of 91 requested clinical trials (77%). The median time to data provision was 123 (range, 117-352) days. Redactions were observed in 18 of the acquired IPD packages (26%) for outcome data, 11 (16%) for assessment variables, and 19 (27%) for adjustment data. Additionally, 20 IPD packages (29%) lacked a clinical study report, 4 (6%) had incomplete or missing data dictionaries, and 20 (29%) were missing anonymization or redaction description files. Access to IPD from 21 eligible trials (23%) was not granted. Conclusions and Relevance: In this quality improvement study, there was substantial variability within the provided IPD packages regarding the completeness of key data variables and supporting documents. To improve the data sharing ecosystem, key areas for enhancement include (1) ensuring that clinical trials are eligible for IPD sharing, (2) making eligible IPD transparently accessible, and (3) ensuring that IPD packages meet a standard of utility and completeness.

Real-World Impact of Prophylactic Growth Factor Use on Timing of Febrile Neutropenia and Infection After High-Risk Chemotherapy.

BACKGROUND: Prophylactic growth-factor therapy with granulocyte colony-stimulating factor (G-CSF) reduces the risk of febrile neutropenia (FN) in patients with breast cancer initiating myelosuppressive chemotherapy. However, little is known about the protective benefit early in the chemotherapy cycle. METHODS: To assess the relationship between G-CSF prophylaxis and incidence of FN/infection in week 1 versus beyond week 1 of the first chemotherapy cycle, a retrospective study was conducted using Medicare claims from 2005 through 2020 among patients with breast cancer initiating high-risk chemotherapy. Two cohorts were compared based on G-CSF prophylaxis within 3 days following chemotherapy initiation. The primary outcome was FN or infection, defined as hospitalization with neutropenia, fever, or infection diagnosis. Secondary outcomes were a stricter definition of FN and infection-related hospitalization. Unadjusted and regression-adjusted proportions of patients experiencing each outcome during week 1 versus beyond week 1 of the first chemotherapy cycle were compared. RESULTS: Of 78,810 patients meeting all inclusion criteria (>98% female; mean age, 69 years), 79% initiated TC (docetaxel/cyclophosphamide), 14% TCH (docetaxel/carboplatin/trastuzumab), and 7% TAC (docetaxel/doxorubicin/cyclophosphamide). Among patients receiving G-CSF (74%), incidence of first-cycle FN/infection was lower compared with patients not receiving G-CSF (overall, 6% vs 13%; TAC, 12% vs 19%; TC, 6% vs 12%; TCH, 5% vs 15%). However, patients who received G-CSF were generally more likely to experience FN/infection in week 1 (adjusted odds ratio [aOR], 1.24 for all; 1.73 for TAC; 1.35 for TC; and 0.76 for TCH). Results were similar for strictly defined FN (overall aOR, 1.29 for week 1 and 0.12 for beyond week 1) and infection-related hospitalization (overall aOR, 1.33 for week 1 and 0.27 for beyond week 1). CONCLUSIONS: Overall, the rates of chemotherapy-related FN and infection in week 1 of the first chemotherapy cycle are similar for patients receiving and not receiving G-CSF, suggesting continued risk in week 1 despite prophylactic G-CSF.

Mitigating long-term and delayed adverse events associated with cancer treatment: implications for survivorship.

Despite the importance of chemotherapy-associated adverse events in oncology practice and the broad range of interventions available to mitigate them, limited systematic efforts have been made to identify, critically appraise and summarize the totality of evidence on the effectiveness of these interventions. Herein, we review the most common long-term (continued beyond treatment) and late or delayed (following treatment) adverse events associated with chemotherapy and other anticancer treatments that pose major threats in terms of survival, quality of life and continuation of optimal therapy. These adverse effects often emerge during and continue beyond the course of therapy or arise among survivors in the months and years following treatment. For each of these adverse effects, we discuss and critically evaluate their underlying biological mechanisms, the most commonly used pharmacological and non-pharmacological treatment strategies, and evidence-based clinical practice guidelines for their appropriate management. Furthermore, we discuss risk factors and validated risk-assessment tools for identifying patients most likely to be harmed by chemotherapy and potentially benefit from effective interventions. Finally, we highlight promising emerging supportive-care opportunities for the ever-increasing number of cancer survivors at continuing risk of adverse treatment effects.

Clinical Characteristics, Racial Inequities, and Outcomes in Patients with Breast Cancer and COVID-19: A COVID-19 and Cancer Consortium (CCC19) Cohort Study.

Background: Limited information is available for patients with breast cancer (BC) and coronavirus disease 2019 (COVID-19), especially among underrepresented racial/ethnic populations. Methods: This is a COVID-19 and Cancer Consortium (CCC19) registry-based retrospective cohort study of females with active or history of BC and laboratory-confirmed severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2) infection diagnosed between March 2020 and June 2021 in the US. Primary outcome was COVID-19 severity measured on a five-level ordinal scale, including none of the following complications, hospitalization, intensive care unit admission, mechanical ventilation, and all-cause mortality. Multivariable ordinal logistic regression model identified characteristics associated with COVID-19 severity. Results: 1,383 female patient records with BC and COVID-19 were included in the analysis, the median age was 61 years, and median follow-up was 90 days. Multivariable analysis revealed higher odds of COVID-19 severity for older age (aOR per decade, 1.48 [95% CI, 1.32 - 1.67]); Black patients (aOR 1.74; 95 CI 1.24-2.45), Asian Americans and Pacific Islander patients (aOR 3.40; 95 CI 1.70 - 6.79) and Other (aOR 2.97; 95 CI 1.71-5.17) racial/ethnic groups; worse ECOG performance status (ECOG PS ≥2: aOR, 7.78 [95% CI, 4.83 - 12.5]); pre-existing cardiovascular (aOR, 2.26 [95% CI, 1.63 - 3.15])/pulmonary comorbidities (aOR, 1.65 [95% CI, 1.20 - 2.29]); diabetes mellitus (aOR, 2.25 [95% CI, 1.66 - 3.04]); and active and progressing cancer (aOR, 12.5 [95% CI, 6.89 - 22.6]). Hispanic ethnicity, timing and type of anti-cancer therapy modalities were not significantly associated with worse COVID-19 outcomes. The total all-cause mortality and hospitalization rate for the entire cohort was 9% and 37%, respectively however, it varied according to the BC disease status. Conclusions: Using one of the largest registries on cancer and COVID-19, we identified patient and BC related factors associated with worse COVID-19 outcomes. After adjusting for baseline characteristics, underrepresented racial/ethnic patients experienced worse outcomes compared to Non-Hispanic White patients.

Venous Thromboembolism Prophylaxis and Treatment in Patients With Cancer: ASCO Guideline Update.

PURPOSE: To conduct an update of the ASCO venous thromboembolism (VTE) guideline. METHODS: After publication of potentially practice-changing clinical trials, identified through ASCO's signals approach to updating, an updated systematic review was performed for two guideline questions: perioperative thromboprophylaxis and treatment of VTE. PubMed and the Cochrane Library were searched for randomized controlled trials (RCTs) published between November 1, 2018, and June 6, 2022. RESULTS: Five RCTs provided information that contributed to changes to the 2019 recommendations. Two RCTs addressed direct factor Xa inhibitors (either rivaroxaban or apixaban) for extended thromboprophylaxis after surgery. Each of these postoperative trials had important limitations but suggested that these two oral anticoagulants are safe and effective in the settings studied. An additional three RCTs addressed apixaban in the setting of VTE treatment. Apixaban was effective in reducing the risk of recurrent VTE, with a low risk of major bleeding. RECOMMENDATIONS: Apixaban and rivaroxaban were added as options for extended pharmacologic thromboprophylaxis after cancer surgery, with a weak strength of recommendation. Apixaban was also added as an option for the treatment of VTE, with high quality of evidence and a strong recommendation.Additional information is available at www.asco.org/supportive-care-guidelines.

Risk Model Development and Validation in Clinical Oncology: Lessons Learned.

Reliable risk models can greatly facilitate patient-centered inferences and decisions. Herein we summarize key considerations related to risk modeling in clinical oncology. Often overlooked challenges include data quality, missing data, effective sample size estimation, and selecting the variables to be included in the risk model. The stability and quality of the model should be carefully interrogated with particular emphasis on rigorous internal validation.

Low-Intensity Adjuvant Chemotherapy for Breast Cancer in Older Women: Results From the Prospective Multicenter HOPE Trial.

PURPOSE: Older women with high-risk early breast cancer (EBC) benefit from adjuvant chemotherapy, but their treatment is frequently complicated by toxic side effects, resulting in dose reductions and delays. This makes it challenging for oncologists to maintain a relative dose intensity (RDI) ≥ 85%, as recommended for optimal curative-intent treatment. Understanding which women are at risk of receiving suboptimal RDI may inform treatment discussions and guide early, targeted supportive care or geriatric comanagement interventions. METHODS: This was a prespecified secondary analysis of the HOPE trial, which enrolled women age ≥ 65 years with EBC initiating neoadjuvant or adjuvant chemotherapy. RDI was calculated as the ratio of delivered to planned chemotherapy dose intensity. The primary outcome was low RDI, defined as RDI < 85%. Multivariable logistic regression with stepwise selection was used to evaluate the association between baseline variables (demographic, clinical, and geriatric assessment) and low RDI. Survival probability was estimated using the Kaplan-Meier method, and the log-rank test was used to compare overall survival. RESULTS: Three hundred twenty-two patients (median age at diagnosis, 70 years; range, 65-86 years) were included. The median follow-up was 4 years. Sixty-six patients (21%) had a low RDI. Age ≥ 76 years (odds ratio [OR], 2.57; 95% CI, 1.12 to 5.91; P = .03), lower performance status (OR, 4.32; 95% CI, 1.98 to 9.42; P < .001), and use of anthracycline-based or cyclophosphamide, methotrexate, and fluorouracil regimens (OR, 3.47; 95% CI, 1.71 to 7.05; P < .001) were associated with low RDI. The 5-year overall survival probability was 0.80 versus 0.91 in patients with RDI < 85 versus ≥ 85%, respectively (log-rank P = .02). CONCLUSION: One in five older patients with EBC treated with standard chemotherapy received low RDI and had inferior survival outcomes. Older patients at risk for low RDI should be identified and targeted upfront before initiating chemotherapy.