Cross-population evaluation of cervical cancer risk prediction algorithms.

BACKGROUND: Cervical cancer is a preventable disease, despite being one of the most common types of female cancers worldwide. Integrating existing programs for cervical cancer screening with personalized risk prediction algorithms can improve population-level cancer prevention by enabling more targeted screening and contrive preventive healthcare innovations. While algorithms developed for cervical cancer risk prediction have shown promising performance in internal validation on more homogeneous populations, their ability to generalize to external populations remains to be assessed. METHODS: To address this gap, we perform a cross-population comparative study of personalized prediction algorithms for more personalized cervical cancer screening. Using data from the Norwegian and Estonian populations, the algorithms are validated on internal and external datasets to study their potential biases and limitations when applied to different populations. We evaluate the algorithms in predicting progression from low-grade precancerous cervical lesions, simulating a clinically relevant application of more personalized risk stratification. RESULTS: As expected, our numerical experiments show that algorithm performance varies depending on the population. However, some algorithms show strong generalization capacity across different data sources. Using Kaplan-Meier estimates, we demonstrate the strengths and limitations of the algorithms in detecting cancer progression over time by comparing to the trends observed from data. We assess their overall discrimination performance in personalized risk predictions by analyzing the accuracy and confidence in individual risk estimates. DISCUSSION AND CONCLUSION: This study examines the effectiveness of personalized prediction algorithms across different populations. Our results demonstrate the potential for generalizing risk prediction algorithms to external populations. These findings highlight the importance of considering population diversity when developing risk prediction algorithms.

Can I trust my fake data - A comprehensive quality assessment framework for synthetic tabular data in healthcare.

BACKGROUND: Ensuring safe adoption of AI tools in healthcare hinges on access to sufficient data for training, testing and validation. Synthetic data has been suggested in response to privacy concerns and regulatory requirements and can be created by training a generator on real data to produce a dataset with similar statistical properties. Competing metrics with differing taxonomies for quality evaluation have been proposed, resulting in a complex landscape. Optimising quality entails balancing considerations that make the data fit for use, yet relevant dimensions are left out of existing frameworks. METHOD: We performed a comprehensive literature review on the use of quality evaluation metrics on synthetic data within the scope of synthetic tabular healthcare data using deep generative methods. Based on this and the collective team experiences, we developed a conceptual framework for quality assurance. The applicability was benchmarked against a practical case from the Dutch National Cancer Registry. CONCLUSION: We present a conceptual framework for quality assuranceof synthetic data for AI applications in healthcare that aligns diverging taxonomies, expands on common quality dimensions to include the dimensions of Fairness and Carbon footprint, and proposes stages necessary to support real-life applications. Building trust in synthetic data by increasing transparency and reducing the safety risk will accelerate the development and uptake of trustworthy AI tools for the benefit of patients. DISCUSSION: Despite the growing emphasis on algorithmic fairness and carbon footprint, these metrics were scarce in the literature review. The overwhelming focus was on statistical similarity using distance metrics while sequential logic detection was scarce. A consensus-backed framework that includes all relevant quality dimensions can provide assurance for safe and responsible real-life applications of synthetic data. As the choice of appropriate metrics are highly context dependent, further research is needed on validation studies to guide metric choices and support the development of technical standards.