Validation of a Machine Learning Model to Predict Childhood Lead Poisoning.

Importance: Childhood lead poisoning causes irreversible neurobehavioral deficits, but current practice is secondary prevention. Objective: To validate a machine learning (random forest) prediction model of elevated blood lead levels (EBLLs) by comparison with a parsimonious logistic regression. Design, Setting, and Participants: This prognostic study for temporal validation of multivariable prediction models used data from the Women, Infants, and Children (WIC) program of the Chicago Department of Public Health. Participants included a development cohort of children born from January 1, 2007, to December 31, 2012, and a validation WIC cohort born from January 1 to December 31, 2013. Blood lead levels were measured until December 31, 2018. Data were analyzed from January 1 to October 31, 2019. Exposures: Blood lead level test results; lead investigation findings; housing characteristics, permits, and violations; and demographic variables. Main Outcomes and Measures: Incident EBLL (≥6 µg/dL). Models were assessed using the area under the receiver operating characteristic curve (AUC) and confusion matrix metrics (positive predictive value, sensitivity, and specificity) at various thresholds. Results: Among 6812 children in the WIC validation cohort, 3451 (50.7%) were female, 3057 (44.9%) were Hispanic, 2804 (41.2%) were non-Hispanic Black, 458 (6.7%) were non-Hispanic White, and 442 (6.5%) were Asian (mean [SD] age, 5.5 [0.3] years). The median year of housing construction was 1919 (interquartile range, 1903-1948). Random forest AUC was 0.69 compared with 0.64 for logistic regression (difference, 0.05; 95% CI, 0.02-0.08). When predicting the 5% of children at highest risk to have EBLLs, random forest and logistic regression models had positive predictive values of 15.5% and 7.8%, respectively (difference, 7.7%; 95% CI, 3.7%-11.3%), sensitivity of 16.2% and 8.1%, respectively (difference, 8.1%; 95% CI, 3.9%-11.7%), and specificity of 95.5% and 95.1% (difference, 0.4%; 95% CI, 0.0%-0.7%). Conclusions and Relevance: The machine learning model outperformed regression in predicting childhood lead poisoning, especially in identifying children at highest risk. Such a model could be used to target the allocation of lead poisoning prevention resources to these children.

Predictive Analytics for Retention in Care in an Urban HIV Clinic.

Consistent medical care among people living with HIV is essential for both individual and public health. HIV-positive individuals who are 'retained in care' are more likely to be prescribed antiretroviral medication and achieve HIV viral suppression, effectively eliminating the risk of transmitting HIV to others. However, in the United States, less than half of HIV-positive individuals are retained in care. Interventions to improve retention in care are resource intensive, and there is currently no systematic way to identify patients at risk for falling out of care who would benefit from these interventions. We developed a machine learning model to identify patients at risk for dropping out of care in an urban HIV care clinic using electronic medical records and geospatial data. The machine learning model has a mean positive predictive value of 34.6% [SD: 0.15] for flagging the top 10% highest risk patients as needing interventions, performing better than the previous state-of-the-art logistic regression model (PPV of 17% [SD: 0.06]) and the baseline rate of 11.1% [SD: 0.02]. Machine learning methods can improve the prediction ability in HIV care clinics to proactively identify patients at risk for not returning to medical care.

An Experience-Centered Approach to Training Effective Data Scientists.

Like medicine, psychology, or education, data science is fundamentally an applied discipline, with most students who receive advanced degrees in the field going on to work on practical problems. Unlike these disciplines, however, data science education remains heavily focused on theory and methods, and practical coursework typically revolves around cleaned or simplified data sets that have little analog in professional applications. We believe that the environment in which new data scientists are trained should more accurately reflect that in which they will eventually practice, and we propose here a data science master's degree program that takes inspiration from the residency model used in medicine. Students in the suggested program would spend their time working on a practical problem with an industry, government, or nonprofit partner, supplemented with coursework in data science methods and theory. We also discuss how this program can also be implemented in shorter formats to augment existing professional master's programs in different disciplines. This approach to learning by doing is designed to fill gaps in our current approach to data science education and ensure that students develop the skills they need to practice data science in a professional context and under the many constraints imposed by that context.

Combining Structured and Unstructured Data for Predicting Risk of Readmission for Heart Failure Patients.

Researchers have studied many models for predicting the risk of readmission for heart failure over the last decade. Most models have used a parametric statistical approach while a few have ventured into using machine learning methods such as statistical natural language processing. We created three predictive models by combining these two techniques for the cohort of 1,629 patients from six hosptials using structured data along with their 136,963 clinical notes till their index admission, stored in the EMR system over five years. The AUCs for structured and combined models were very close (0.6494 and 0.6447) and that for the unstructured model was 0.5219. The clinical impact of the models using decision curve analysis showed that, at a threshold predicted probability of 0.20, the combined model offered 15%, 30%, and 70% net benefit over its individual counterparts, treat-all, and treat-none strategy respectively.

Using Ensemble Machine Learning Methods for Predicting Risk of Readmission for Heart Failure.

Recently, researchers have been applying many new machine learning techniques for predicting the risk of readmission for heart failure. Combining such techniques through ensemble schemes holds a promise to further harness predictive performance of the resulting models. To that end, we examined two ensemble schemes and applied them to a real world dataset obtained from the EMR systems for 36,245 patients from 117 hospitals across the United States over five years. Both the ensemble schemes provided similar discriminative ability (AUC: 0.70, F1-score: 0.58) that was at least equal to or better than the base models that used a single machine learning method. The clinical impact of the models using decision curve analysis showed that at a threshold predicted probability of 0.40, the ensemble models offered 20% net benefit over the treat-all and treat-none strategies.

Machine Learning for Social Services: A Study of Prenatal Case Management in Illinois.

OBJECTIVES: To evaluate the positive predictive value of machine learning algorithms for early assessment of adverse birth risk among pregnant women as a means of improving the allocation of social services. METHODS: We used administrative data for 6457 women collected by the Illinois Department of Human Services from July 2014 to May 2015 to develop a machine learning model for adverse birth prediction and improve upon the existing paper-based risk assessment. We compared different models and determined the strongest predictors of adverse birth outcomes using positive predictive value as the metric for selection. RESULTS: Machine learning algorithms performed similarly, outperforming the current paper-based risk assessment by up to 36%; a refined paper-based assessment outperformed the current assessment by up to 22%. We estimate that these improvements will allow 100 to 170 additional high-risk pregnant women screened for program eligibility each year to receive services that would have otherwise been unobtainable. CONCLUSIONS: Our analysis exhibits the potential for machine learning to move government agencies toward a more data-informed approach to evaluating risk and providing social services. Overall, such efforts will improve the efficiency of allocating resource-intensive interventions.