Deep reinforcement learning for multi-class imbalanced training: applications in healthcare.

With the rapid growth of memory and computing power, datasets are becoming increasingly complex and imbalanced. This is especially severe in the context of clinical data, where there may be one rare event for many cases in the majority class. We introduce an imbalanced classification framework, based on reinforcement learning, for training extremely imbalanced data sets, and extend it for use in multi-class settings. We combine dueling and double deep Q-learning architectures, and formulate a custom reward function and episode-training procedure, specifically with the capability of handling multi-class imbalanced training. Using real-world clinical case studies, we demonstrate that our proposed framework outperforms current state-of-the-art imbalanced learning methods, achieving more fair and balanced classification, while also significantly improving the prediction of minority classes. Supplementary Information: The online version contains supplementary material available at 10.1007/s10994-023-06481-z.

A polygenic risk score added to a QRISK®2 cardiovascular disease risk calculator demonstrated robust clinical acceptance and clinical utility in the primary care setting.

AIMS: The aim of the study was to assess the real-world feasibility, acceptability, and impact of an integrated risk tool for cardiovascular disease (CVD IRT, combining the standard QRISK®2 risk algorithm with a polygenic risk score), implemented within routine primary practice in the UK National Health Service. METHODS AND RESULTS: The Healthcare Evaluation of Absolute Risk Testing Study (NCT05294419) evaluated participants undergoing primary care health checks. Both QRISK2 and CVD IRT scores were returned to the healthcare providers (HCPs), who then communicated the results to participants. The primary outcome of the study was feasibility of CVD IRT implementation. Secondary outcomes included changes in CVD risk (QRISK2 vs. CVD IRT) and impact of the CVD IRT on clinical decision-making. A total of 832 eligible participants (median age 55 years, 62% females, 97.5% White ethnicity) were enrolled across 12 UK primary care practices. Cardiovascular disease IRT scores were obtained on 100% of the blood samples. Healthcare providers stated that the CVD IRT could be incorporated into routine primary care in a straightforward manner in 90.7% of reports. Participants stated they were 'likely' or 'very likely' to recommend the use of this test to their family or friends in 86.9% of reports. Participants stated that the test was personally useful (98.8%) and that the results were easy to understand (94.6%). When CVD IRT exceeded QRISK2, HCPs planned changes in management for 108/388 (27.8%) of participants and 47% (62/132) of participants with absolute risk score changes of >2%. CONCLUSION: Amongst HCPs and participants who agreed to the trial of genetic data for refinement of clinical risk prediction in primary care, we observed that CVD IRT implementation was feasible and well accepted. The CVD IRT results were associated with planned changes in prevention strategies.

When a standard cardiovascular risk tool, as currently used in National Health Service Health Checks, was expanded to include genetic risk information, it was well accepted by both participants and healthcare providers and generated impactful changes in planned clinical decision-making.Most participants found the test useful and easy to understand, and healthcare providers found it straightforward to use in most cases.When risk was increased by the addition of genetic information, this influenced planned management decisions.

Validation of an Integrated Risk Tool, Including Polygenic Risk Score, for Atherosclerotic Cardiovascular Disease in Multiple Ethnicities and Ancestries.

The American College of Cardiology / American Heart Association pooled cohort equations tool (ASCVD-PCE) is currently recommended to assess 10-year risk for atherosclerotic cardiovascular disease (ASCVD). ASCVD-PCE does not currently include genetic risk factors. Polygenic risk scores (PRSs) have been shown to offer a powerful new approach to measuring genetic risk for common diseases, including ASCVD, and to enhance risk prediction when combined with ASCVD-PCE. Most work to date, including the assessment of tools, has focused on performance in individuals of European ancestries. Here we present evidence for the clinical validation of a new integrated risk tool (IRT), ASCVD-IRT, which combines ASCVD-PCE with PRS to predict 10-year risk of ASCVD across diverse ethnicity and ancestry groups. We demonstrate improved predictive performance of ASCVD-IRT over ASCVD-PCE, not only in individuals of self-reported White ethnicities (net reclassification improvement [NRI]; with 95% confidence interval = 2.7% [1.1 to 4.2]) but also Black / African American / Black Caribbean / Black African (NRI = 2.5% [0.6-4.3]) and South Asian (Indian, Bangladeshi or Pakistani) ethnicities (NRI = 8.7% [3.1 to 14.4]). NRI confidence intervals were wider and included zero for ethnicities with smaller sample sizes, including Hispanic (NRI = 7.5% [-1.4 to 16.5]), but PRS effect sizes in these ethnicities were significant and of comparable size to those seen in individuals of White ethnicities. Comparable results were obtained when individuals were analyzed by genetically inferred ancestry. Together, these results validate the performance of ASCVD-IRT in multiple ethnicities and ancestries, and favor their generalization to all ethnicities and ancestries.

Integrated Polygenic Tool Substantially Enhances Coronary Artery Disease Prediction.

BACKGROUND: There is considerable interest in whether genetic data can be used to improve standard cardiovascular disease risk calculators, as the latter are routinely used in clinical practice to manage preventative treatment. METHODS: Using the UK Biobank resource, we developed our own polygenic risk score for coronary artery disease (CAD). We used an additional 60 000 UK Biobank individuals to develop an integrated risk tool (IRT) that combined our polygenic risk score with established risk tools (either the American Heart Association/American College of Cardiology pooled cohort equations [PCE] or UK QRISK3), and we tested our IRT in an additional, independent set of 186 451 UK Biobank individuals. RESULTS: The novel CAD polygenic risk score shows superior predictive power for CAD events, compared with other published polygenic risk scores, and is largely uncorrelated with PCE and QRISK3. When combined with PCE into an IRT, it has superior predictive accuracy. Overall, 10.4% of incident CAD cases were misclassified as low risk by PCE and correctly classified as high risk by the IRT, compared with 4.4% misclassified by the IRT and correctly classified by PCE. The overall net reclassification improvement for the IRT was 5.9% (95% CI, 4.7-7.0). When individuals were stratified into age-by-sex subgroups, the improvement was larger for all subgroups (range, 8.3%-15.4%), with the best performance in 40- to 54-year-old men (15.4% [95% CI, 11.6-19.3]). Comparable results were found using a different risk tool (QRISK3) and also a broader definition of cardiovascular disease. Use of the IRT is estimated to avoid up to 12 000 deaths in the United States over a 5-year period. CONCLUSIONS: An IRT that includes polygenic risk outperforms current risk stratification tools and offers greater opportunity for early interventions. Given the plummeting costs of genetic tests, future iterations of CAD risk tools would be enhanced with the addition of a person's polygenic risk.

Multi-Label Random Forest Model for Tuberculosis Drug Resistance Classification and Mutation Ranking.

Resistance prediction and mutation ranking are important tasks in the analysis of Tuberculosis sequence data. Due to standard regimens for the use of first-line antibiotics, resistance co-occurrence, in which samples are resistant to multiple drugs, is common. Analysing all drugs simultaneously should therefore enable patterns reflecting resistance co-occurrence to be exploited for resistance prediction. Here, multi-label random forest (MLRF) models are compared with single-label random forest (SLRF) for both predicting phenotypic resistance from whole genome sequences and identifying important mutations for better prediction of four first-line drugs in a dataset of 13402 Mycobacterium tuberculosis isolates. Results confirmed that MLRFs can improve performance compared to conventional clinical methods (by 18.10%) and SLRFs (by 0.91%). In addition, we identified a list of candidate mutations that are important for resistance prediction or that are related to resistance co-occurrence. Moreover, we found that retraining our analysis to a subset of top-ranked mutations was sufficient to achieve satisfactory performance. The source code can be found at http://www.robots.ox.ac.uk/~davidc/code.php.

Pre-exchange training - Developing ethical and cultural competencies in medical students.

Background: The International Federation of Medical Students' Associations (IFMSA) organizes over 15,000 international medical exchanges per year in over 100 countries. In the past, there was no standardized Pre-Departure Training (PDT) for participants. A PDT is important to protect patient safety and prepare students for their exchange.Objective: To determine whether a two-hour case-based Pre-Departure Training can increase self-reported level of comfort on competencies in basic medical ethics, cultural competence, research ethics, and recognizing the limits of one's level of skill in medical students.Methods: In 2017, the PDT was implemented in nine countries for medical students prior to their IFMSA exchange. Participants self-evaluated their competencies in an online questionnaire before and after the PDT.Results: 234 students from 32 countries completed the pre-PDT evaluation and 104 completed both evaluations. Participants demonstrated statistically significant improvements in self-reported competencies in 16 out of 18 items including voicing lack of skill to a supervisor (p < 0.001) and recognizing personal cultural biases (p < 0.001).Conclusions: A case-based PDT can improve participants' self-reported comfort in treating patients from different cultural backgrounds and help maintain high ethical standards abroad. The PDT was implemented at large within IFMSA in 2018.

Branches From the "SEAD": Sustained and Feasible Expansion of the Surgical Exploration and Discovery Program.

PROBLEM: A predicted shortage of surgeons and attrition among surgical residents has highlighted the need to attract well-suited medical students to surgical specialties. Literature suggests that early exposure may increase interest by addressing misconceptions and allowing students more time to make an informed career decision. APPROACH: The Surgical Exploration and Discovery (SEAD) program was created in 2012 with the goal of providing medical students with comprehensive and multifaceted exposure to surgical specialties to develop their knowledge and skills, and in turn positively influence their interest in pursuing a surgical career. The purpose of this innovation report is to describe the challenges, successes, and evolution of the SEAD program. OUTCOMES: Since its inception, SEAD has expanded to include 5 North American institutions and has educated nearly 400 participants in 5 y. Through a replication strategy, SEAD has maintained its basic curriculum, while accommodating the constraints and innovative approaches unique to each institution. Short-term results have demonstrated improved knowledge of curricular objectives, student perception of significant value of the program, and the generation of interest in a career in surgery. CONCLUSIONS: Future directions include the evaluation of long-term impact on pursuing a career in surgery and continuing further expansion using the current replication model, while maintaining a high-quality surgical education program.

Deep neural network improves fracture detection by clinicians.

Suspected fractures are among the most common reasons for patients to visit emergency departments (EDs), and X-ray imaging is the primary diagnostic tool used by clinicians to assess patients for fractures. Missing a fracture in a radiograph often has severe consequences for patients, resulting in delayed treatment and poor recovery of function. Nevertheless, radiographs in emergency settings are often read out of necessity by emergency medicine clinicians who lack subspecialized expertise in orthopedics, and misdiagnosed fractures account for upward of four of every five reported diagnostic errors in certain EDs. In this work, we developed a deep neural network to detect and localize fractures in radiographs. We trained it to accurately emulate the expertise of 18 senior subspecialized orthopedic surgeons by having them annotate 135,409 radiographs. We then ran a controlled experiment with emergency medicine clinicians to evaluate their ability to detect fractures in wrist radiographs with and without the assistance of the deep learning model. The average clinician's sensitivity was 80.8% (95% CI, 76.7-84.1%) unaided and 91.5% (95% CI, 89.3-92.9%) aided, and specificity was 87.5% (95 CI, 85.3-89.5%) unaided and 93.9% (95% CI, 92.9-94.9%) aided. The average clinician experienced a relative reduction in misinterpretation rate of 47.0% (95% CI, 37.4-53.9%). The significant improvements in diagnostic accuracy that we observed in this study show that deep learning methods are a mechanism by which senior medical specialists can deliver their expertise to generalists on the front lines of medicine, thereby providing substantial improvements to patient care.

Surgical skills curricula in American College of Surgeons Accredited Education Institutes: An international survey.

BACKGROUND: A clear understanding of simulation-based curricula in use at American College of Surgeons Accredited Education Institutes (ACS-AEIs) is lacking. METHODS: A 25-question online survey was sent to ACS-AEIs. RESULTS: The response rate approached 60%. The most frequent specialties to use the ACS-AEIs are general surgery and obstetrics/gynecology (94%). Residents are the main target population for programming/training (96%). Elements of the ACS/Association of Program Directors in Surgery Surgical Skills Curriculum are used by 77% of responding ACS-AEIs. Only 49% of ACS-AEIs implement the entire curriculum and 96% have independently developed their own surgical skills curricula. "Home-grown" simulators have been designed at 71% of ACS-AEIs. Feasibility (80%), evidence of effectiveness (67%), and cost (60%) were reasons for curriculum adoption. All programs use operative assessment tools for resident performance, and 53% use Messick's unitary framework of validity. Most programs (88%) have financial support from their academic institute. Majority of ACS-AEIs had trainees evaluate their faculty instructors (90%), and the main form of such faculty evaluation was postcourse surveys (97%). CONCLUSION: This study provides specific information regarding simulation-based curricula at ACS-AEIs.