Can I trust my fake data - A comprehensive quality assessment framework for synthetic tabular data in healthcare.

BACKGROUND: Ensuring safe adoption of AI tools in healthcare hinges on access to sufficient data for training, testing and validation. Synthetic data has been suggested in response to privacy concerns and regulatory requirements and can be created by training a generator on real data to produce a dataset with similar statistical properties. Competing metrics with differing taxonomies for quality evaluation have been proposed, resulting in a complex landscape. Optimising quality entails balancing considerations that make the data fit for use, yet relevant dimensions are left out of existing frameworks. METHOD: We performed a comprehensive literature review on the use of quality evaluation metrics on synthetic data within the scope of synthetic tabular healthcare data using deep generative methods. Based on this and the collective team experiences, we developed a conceptual framework for quality assurance. The applicability was benchmarked against a practical case from the Dutch National Cancer Registry. CONCLUSION: We present a conceptual framework for quality assuranceof synthetic data for AI applications in healthcare that aligns diverging taxonomies, expands on common quality dimensions to include the dimensions of Fairness and Carbon footprint, and proposes stages necessary to support real-life applications. Building trust in synthetic data by increasing transparency and reducing the safety risk will accelerate the development and uptake of trustworthy AI tools for the benefit of patients. DISCUSSION: Despite the growing emphasis on algorithmic fairness and carbon footprint, these metrics were scarce in the literature review. The overwhelming focus was on statistical similarity using distance metrics while sequential logic detection was scarce. A consensus-backed framework that includes all relevant quality dimensions can provide assurance for safe and responsible real-life applications of synthetic data. As the choice of appropriate metrics are highly context dependent, further research is needed on validation studies to guide metric choices and support the development of technical standards.

Measuring cell-to-cell expression variability in single-cell RNA-sequencing data: a comparative analysis and applications to B cell aging.

BACKGROUND: Single-cell RNA-sequencing (scRNA-seq) technologies enable the capture of gene expression heterogeneity and consequently facilitate the study of cell-to-cell variability at the cell type level. Although different methods have been proposed to quantify cell-to-cell variability, it is unclear what the optimal statistical approach is, especially in light of challenging data structures that are unique to scRNA-seq data like zero inflation. RESULTS: We systematically evaluate the performance of 14 different variability metrics that are commonly applied to transcriptomic data for measuring cell-to-cell variability. Leveraging simulations and real datasets, we benchmark the metric performance based on data-specific features, sparsity and sequencing platform, biological properties, and the ability to recapitulate true levels of biological variability based on known gene sets. Next, we use scran, the metric with the strongest all-round performance, to investigate changes in cell-to-cell variability that occur during B cell differentiation and the aging processes. The analysis of primary cell types from hematopoietic stem cells (HSCs) and B lymphopoiesis reveals unique gene signatures with consistent patterns of variable and stable expression profiles during B cell differentiation which highlights the significance of these methods. Identifying differentially variable genes between young and old cells elucidates the regulatory changes that may be overlooked by solely focusing on mean expression changes and we investigate this in the context of regulatory networks. CONCLUSIONS: We highlight the importance of capturing cell-to-cell gene expression variability in a complex biological process like differentiation and aging and emphasize the value of these findings at the level of individual cell types.

Advanced breast cancer education for cancer nurses: A systematic review.

BACKGROUND: Access to comprehensive, integrated, multidisciplinary care is one of the most urgent and actionable recommendations of the Advanced Breast Cancer Global Alliance. However, access to specialist breast care units, and specialist breast cancer nurses is variable, influenced by access to specialist education and role recognition. To date, there has not been a synthesis of evidence regarding educational programmes related to advanced breast cancer education for nurses. OBJECTIVES: The aim of this review was to determine the content, mode of delivery, assessment and outcomes of education programmes related to advanced breast cancer for nurses. REVIEW METHODS: A systematic review was undertaken, according to the Joanna Briggs Institute's mixed methods review methodology. DATA SOURCES: MEDLINE, PUBMED, CINAHL, Scopus, PsycInfo, Joanna Briggs Institute, Web of Science and grey literature sources were systematically searched. Eleven publications met the inclusion criteria. Data relating to programme content, mode of delivery, assessment and outcomes were extracted and analysed. RESULTS: This review identifies a limited number of educational programmes within this specialist area of nursing practice. Shortcomings in the development, implementation and evaluation of advanced breast cancer education programmes included limited use of educational standards, theoretical frameworks and patient and public involvement to inform programme development. Evaluation of education programmes related to advanced breast cancer relied predominantly on self-reported learning, with limited consideration of the impacts of education on service delivery, patient experience or quality of care. CONCLUSIONS: Future development of advanced breast cancer education programmes must consider the alignment of programme content and learning outcomes with existing educational and competency standards. Evaluation of educational programmes in this field must endeavour to enhance rigour of methods, incorporating standardised questionnaires, and multiple methods and sources of data to evaluate the broader impacts of advanced breast cancer education for nurses.

Development of an Advance Care Planning Policy within an Evidenced-Based Evaluation Framework.

Background: As the population is aging and medical advancements enable people to live longer, advance care planning (ACP) becomes increasingly important in guiding future care decisions; however, they are often incomplete or absent from the patient chart. This study describes the development and implementation of an ACP policy in a post-acute care and long-term care setting using a systematic implementation framework. Methods: A process evaluation that parallels the Replicating Effective Programs (REP) framework was used to understand stakeholder experiences with ACP and identify gaps in practice. Physicians, multidisciplinary staff, patients, and substitute decision makers engaged in focus groups and interviews, and completed surveys. A retrospective chart review determined Plan for Life Sustaining Treatment (PLST) form completion rates. Results: Stakeholder feedback identified barriers and facilitators to ACP including a need for staff training, user-friendly resources, and standardization of ACP practice. The PLST form was developed and embedded in the electronic medical record, and had a 92% and an 87% PLST completion rate on 2 pilot units. Conclusion: The study showed the usefulness of the REP model in guiding the evaluation as an effective tool to enhance implementation practices and inform ACP policy development that can be replicated in other organizations.

A Protocol on Using the RE-AIM Framework in the Process Evaluation of the Primary Health Integrated Care Project for Four Chronic Conditions in Kenya.

Background: There has been a rapid increase in morbidity and mortality arising from non-communicable diseases (NCDs). The Academic Model Providing Access to Healthcare (AMPATH) program has established a chronic disease management program in collaboration with the Ministry of Health (MoH) in Kenya at over 150 health facilities in western Kenya. The primary health integrated care for chronic (PIC4C) disease project seeks to deliver preventive, promotive, and curative care for diabetes, hypertension, cervical and breast cancers at the primary health care level. We apply the RE-AIM framework to conduct a process evaluation of the integrated PIC4C model. This paper describes the protocol we are using in the PIC4C process evaluation planning and activities. Methods and Analysis: This evaluation utilizes clinic reports as well as primary data collected in two waves. Using mixed methods (secondary data, observation, semi-structured interviews, and focus group discussions), the process evaluation assesses the reach, effectiveness, adoption, implementation and maintenance of the PIC4C model in Busia and Trans Nzoia Kenya. The evaluation captures the PIC4C process, experiences of implementers and users, and the wishes of those using the PIC4C services. We will analyse our data across the RE-AIM dimensions using descriptive statistics and two-sample t-test to compare the mean scores for baseline and end line. Qualitative data will be analyzed thematically. Discussion: The process evaluation of the PIC4C model in Kenya allows implementers and users to reflect and question its implementation, uptake and maintenance. Our experiences thus far suggest practicable strategies to facilitate primary health care can benefit extensively from deliberate process evaluation of the programs undertaken. Furthermore, integrating the RE-AIM framework in the process evaluation of health programs is valuable due to its pragmatic and reporting usefulness.

A systematic review of the use and reporting of evaluation frameworks within evaluations of physical activity interventions.

BACKGROUND: Evaluation of physical activity interventions is vital to inform, and justify, evidence-based policy and practice to support population-wide changes in physical activity. Several evaluation frameworks and guidance documents have been developed to facilitate the evaluation and reporting of evaluation studies in public health. However, there is a lack of evidence about whether frameworks are being used to guide evaluation. There continues to be claims of poor and inconsistent reporting in evaluation studies. The aim of this review was to assess the use of evaluation frameworks and the quality of reporting of how they were applied within evaluation studies of physical activity interventions. OBJECTIVES: 1. To identify whether evaluation frameworks are reported to have been used within evaluation studies of physical activity interventions, and which frameworks have been used. 2. To appraise the quality of reporting with regards to how evaluation frameworks have been used. METHOD: We developed a checklist of indicators to enable a critical appraisal of the use and reporting of different evaluation frameworks in evaluation studies. We conducted a systematic search and review of evaluation studies published between 2015 and the date of the search to appraise the use and reporting of evaluation frameworks. A narrative synthesis is provided. RESULTS: The review identified 292 evaluation studies of physical activity interventions, only 69 (23%) of these mentioned using an evaluation framework, and only 16 different frameworks were referred to. There was variation in the quality of reporting of framework use. 51 (74%) studies were identified as being explicitly based on the stated framework, however only 26 (38%) provided detailed descriptions consistently across all the checklist indicators. Details of adaptations and limitations in how frameworks were applied were less frequently reported. The review also highlighted variability in the reporting of intervention components. More consistent and precise reporting of framework and intervention components is needed. CONCLUSION: Evaluation frameworks can facilitate a more systematic evaluation report and we argue their limited use suggests missed opportunities to apply frameworks to guide evaluation and reporting in evaluation studies. Variability in the quality of reporting of framework use limits the comparability and transferability of evidence. Where a framework has been used, the checklist of indicators can be employed to facilitate the reporting of an evaluation study and to review the quality of an evaluation report.

A conceptual framework for an oral cancer awareness campaign in Northern Germany - Challenges in campaign development and assessment.

Initial impetus for action: Oral cancer is still an underestimated disease in terms of incidence as well as mortality rates; it requires urgent prevention and early detection. At present, there is no best-practice systematic approach to raising awareness and informing the public about about this type of cancer in Germany. This article describes a framework that covers the significant stages of conceptual development and campaign design to promote oral cancer awareness in Germany. Solution: The challenges of the development, as well as evaluation of an oral cancer awareness campaign are shared in this article. Four key stages of the campaign are defined: (1) mass media, (2) target groups, (3) health care professionals, and (4) epidemiology. For each section, the following levels of assessment are proposed: (a) campaign development (formative assessment), (b) controlling and optimising campaign implementation (process assessment) and (c) measuring outcomes (summative assessment). Outcome: A process-oriented assessment concept for each of the four campaign sections was developed and merged to form a matrix, which includes each of the above sections regarding the prevention and early detection of oral cancer, as well as the three stages of campaign assessment. Future implications and learning points: The conceptual framework demonstrated that systematic planning and evaluation of different components helped to describe and evaluate an oral cancer campaign: For future campaigns, the use of a matrix covering different campaign targets as well as the entire campaign process, is recommended as a basis for campaign design and evaluation.

Evaluation of a comprehensive school physical activity program: Be a Champion!

BACKGROUND: Comprehensive School Physical Activity Programs (CSPAPs) are widely supported to increase physical activity (PA) in schools, but little has been reported to inform a comprehensive evaluation effort. The purpose of this report is to describe a comprehensive evaluation and monitoring strategy for CSPAP implementation. METHODS: The System for Observing Staff Promotion of Activity and Nutrition (SOSPAN), System for Observing Student Movement in Academic Routines and Transitions (SOSMART), and System for Observing Play and Leisure Activity (SOPLAY) were employed to record student and staff behaviors (2642 total scans). To assess policies, practices, and environments, a CSPAP Policies and Practices questionnaire was used, which included components of the School Health Index, the School PA Policy Assessment, the School Environment and Policies Survey, and the School Health Policies and Practices Study. Youth PA (minutes/day) was assessed using accelerometers. RESULTS: Working with school sites to ascertain the number of opportunities that are provided for PA and the amount of PA students engage in is challenging but feasible. The use of systematic observation, accelerometers, and questionnaires can provide a comprehensive representation of policies, practices, and behaviors to aid in establishing targeted PA action items. CONCLUSIONS: Developing an evaluation framework that was rigorous enough to assess the physical, social, and policy environment of a school in order to answer the overarching research questions of the study but flexible enough to be feasibly streamlined for utilization by a school implementation team proved to be challenging but possible.

Beyond cost-effectiveness, morbidity and mortality: a comprehensive evaluation of priority setting for HIV programming in Uganda.

BACKGROUND: While there has been progress in controlling the HIV epidemic, HIV still remains a disease of global concern. Some of the progress has been attributed to increased public awareness and uptake of public health interventions, as well as increased access to anti- retroviral treatment and the prevention of vertical HIV transmission. These interventions would not have been possible without substantial investments in HIV programs. However, donor fatigue introduces the need for low income countries to maximize the benefits of the available resources. This necessitates identification of priorities that should be funded. Evaluating prioritization processes would enable decision makers to assess the effectiveness of their processes, thereby designing intervention strategies. To date most evaluations have focused on cost-benefit analyses, which overlooks additional critical impacts of priority setting decisions. Kapiriri & Martin (2010) developed and validated a comprehensive framework for evaluating PS in low income countries. The objective of this paper report findings from a comprehensive evaluation of priority setting for HIV in Uganda, using the framework; and to identify lessons of good practice and areas for improvement. METHODS: This was a qualitative study based on forty interviews with decision makers and policy document review. Data were analysed using INVIVO 10, and based on the parameters in Kapiriri et al's evaluation framework. RESULTS: We found that HIV enjoys political support, which contributes to the availability of resources, strong planning institutions, and participatory prioritization process based on some criteria. Some of the identified limitations included; undue donor and political influence, priorities not being publicized, and lack of mechanisms for appealing the decisions. HIV prioritization had both positive and negative impacts on the health system. CONCLUSIONS: The framework facilitated a more comprehensive evaluation of HIV priority setting. While there were successful areas, the process could be strengthened by minimizing undue influence of external actors, and support the legitimate institutions to set priorities and implement them. These should also institute mechanisms for publicizing the decisions, appeals and increased accountability. While this paper looked at HIV, the framework is flexible enough to be used in evaluating priority setting for other health programs within similar context.

Implementation of an evidence-based intervention to promote colorectal cancer screening in community organizations: a cluster randomized trial.

UNLABELLED: The implementation of evidence-based strategies to promote colorectal cancer (CRC) screening remains challenging. The aim of this study is to evaluate two strategies to implement an evidence-based intervention to promote CRC screening in Filipino American community organizations. Twenty-two community organizations were randomized to either a basic or enhanced implementation strategy. In both arms, community health advisors recruited participants non-adherent to CRC screening guidelines, conducted educational sessions, distributed print materials and free fecal occult blood test kits, reminded participants to get screened, and mailed letters to participants' providers. In the enhanced arm, leaders of the organizations participated in implementation efforts. While the effectiveness was similar in both arms of the study (screening rate at 6-month follow-up was 53 % in the enhanced arm, 49 % in the basic arm), 223 participants were screened in the enhanced arm versus 122 in the basic arm. The enhanced implementation strategy reached 83 % more participants and achieved a higher public health impact. TRIAL REGISTRATION: NCT01351220 (ClinicalTrials.gov).

Public Health Impact After the Introduction of PsA-TT: The First 4 Years.

BACKGROUND: During the first introduction of a group A meningococcal vaccine (PsA-TT) in 2010-2011 and its rollout from 2011 to 2013, >150 million eligible people, representing 12 hyperendemic meningitis countries, have been vaccinated. METHODS: The new vaccine effectiveness evaluation framework was established by the World Health Organization and partners. Meningitis case-based surveillance was strengthened in PsA-TT first-introducer countries, and several evaluation studies were conducted to estimate the vaccination coverage and to measure the impact of vaccine introduction on meningococcal carriage and disease incidence. RESULTS: PsA-TT implementation achieved high vaccination coverage, and results from studies conducted showed significant decrease of disease incidence as well as significant reduction of oropharyngeal carriage of group A meningococci in vaccinated and unvaccinated individuals, demonstrating the vaccine's ability to generate herd protection and prevent group A epidemics. CONCLUSIONS: Lessons learned from this experience provide useful insights in how to guide and better prepare for future new vaccine introductions in resource-limited settings.