Measuring the effectiveness of online search ads on blood donor recruitment and donation.

BACKGROUND: Acutely highlighted during the COVID-19 pandemic, the tenuousness of the blood supply continues to be a lynchpin of the most important medical procedures. Online advertisements have become a mainstay in donor recruitment. We set out to determine the effectiveness of online search ads and variations thereof on blood donations with an emphasis on first-time donors. STUDY DESIGN AND METHODS: From September 01, 2022 through March 31, 2023, we performed a campaign comparison experiment through a major search-ads platform with two distinct messages: one altruistic ("Altruistic") and one with a prospect of rewards ("Promotion"). We developed a method to track donation outcomes and associated them with impressions, click-throughs, and conversions. We compared the performance of the Altruistic and Promotion arms to a control group that was not associated with any search-ads ("Baseline"). RESULTS: Analyzing 34,157 donations during the study period, the Promotion group, and not Altruistic, had a significant difference of first-time donors over Baseline (24% vs. 12%, p = 7e-6). We analyzed 49,305 appointments and discovered that appointments made from the Altruistic arm resulted in a significantly higher percentage of donations when compared to Baseline (57% vs. 53%, p = .009); however, the Promotion group had a higher percentage of donations from first-time donors when compared to Baseline (12% vs. 8%, p = .006). CONCLUSION: We developed a method for determining the effectiveness of online search ads on donation outcomes. Rewards/promotions messaging was most effective at recruiting first-time donors. Our methodology is generalizable to different blood centers to explore messaging effectiveness among their unique communities.

Evaluating accuracy and fairness of clinical decision support algorithms when health care resources are limited.

OBJECTIVE: Guidance on how to evaluate accuracy and algorithmic fairness across subgroups is missing for clinical models that flag patients for an intervention but when health care resources to administer that intervention are limited. We aimed to propose a framework of metrics that would fit this specific use case. METHODS: We evaluated the following metrics and applied them to a Veterans Health Administration clinical model that flags patients for intervention who are at risk of overdose or a suicidal event among outpatients who were prescribed opioids (N = 405,817): Receiver - Operating Characteristic and area under the curve, precision - recall curve, calibration - reliability curve, false positive rate, false negative rate, and false omission rate. In addition, we developed a new approach to visualize false positives and false negatives that we named 'per true positive bars.' We demonstrate the utility of these metrics to our use case for three cohorts of patients at the highest risk (top 0.5 %, 1.0 %, and 5.0 %) by evaluating algorithmic fairness across the following age groups: <=30, 31-50, 51-65, and >65 years old. RESULTS: Metrics that allowed us to assess group differences more clearly were the false positive rate, false negative rate, false omission rate, and the new 'per true positive bars'. Metrics with limited utility to our use case were the Receiver - Operating Characteristic and area under the curve, the calibration - reliability curve, and the precision - recall curve. CONCLUSION: There is no "one size fits all" approach to model performance monitoring and bias analysis. Our work informs future researchers and clinicians who seek to evaluate accuracy and fairness of predictive models that identify patients to intervene on in the context of limited health care resources. In terms of ease of interpretation and utility for our use case, the new 'per true positive bars' may be the most intuitive to a range of stakeholders and facilitates choosing a threshold that allows weighing false positives against false negatives, which is especially important when predicting severe adverse events.

A call for better validation of opioid overdose risk algorithms.

Clinical decision support (CDS) systems powered by predictive models have the potential to improve the accuracy and efficiency of clinical decision-making. However, without sufficient validation, these systems have the potential to mislead clinicians and harm patients. This is especially true for CDS systems used by opioid prescribers and dispensers, where a flawed prediction can directly harm patients. To prevent these harms, regulators and researchers have proposed guidance for validating predictive models and CDS systems. However, this guidance is not universally followed and is not required by law. We call on CDS developers, deployers, and users to hold these systems to higher standards of clinical and technical validation. We provide a case study on two CDS systems deployed on a national scale in the United States for predicting a patient's risk of adverse opioid-related events: the Stratification Tool for Opioid Risk Mitigation (STORM), used by the Veterans Health Administration, and NarxCare, a commercial system.

Blood Demand Forecasting and Supply Management: An Analytical Assessment of Key Studies Utilizing Novel Computational Techniques.

Use of data-driven methodologies in enhancing blood transfusion practices is rising, leveraging big data, machine learning, and optimization techniques to improve demand forecasting and supply chain management. This review used a narrative approach to identify, evaluate, and synthesize key studies that considered novel computational techniques for blood demand forecasting and inventory management through a search of PubMed and Web of Sciences databases for studies published from January 01, 2016, to March 30, 2023. The studies were analyzed for their utilization of various techniques, and their strengths, limitations, and areas for improvement. Seven key studies were identified. The studies focused on different blood components using various computational methods, such as regression, machine learning, hybrid models, and time series models, across different locations and time periods. Key variables used for demand forecasting were largely derived from electronic health record data, including clinical related predictors such as laboratory test results and hospital census by location. Each study offered unique strengths and valuable insights into the use of data-driven methods in blood bank management. Common limitations were unknown generalizability to other healthcare settings or blood components, need for field-specific performance measures, lack of ABO compatibility consideration, and ethical challenges in resource allocation. While data-driven research in blood demand forecasting and management has progressed, limitations persist and further exploration is needed. Understanding these innovative, interdisciplinary methods and their complexities can help refine inventory strategies and address healthcare challenges more effectively, leading to more robust, accurate models to enhance blood management across diverse healthcare scenarios.