Pretrained transformer framework on pediatric claims data for population specific tasks.

The adoption of electronic health records (EHR) has become universal during the past decade, which has afforded in-depth data-based research. By learning from the large amount of healthcare data, various data-driven models have been built to predict future events for different medical tasks, such as auto diagnosis and heart-attack prediction. Although EHR is abundant, the population that satisfies specific criteria for learning population-specific tasks is scarce, making it challenging to train data-hungry deep learning models. This study presents the Claim Pre-Training (Claim-PT) framework, a generic pre-training model that first trains on the entire pediatric claims dataset, followed by a discriminative fine-tuning on each population-specific task. The semantic meaning of medical events can be captured in the pre-training stage, and the effective knowledge transfer is completed through the task-aware fine-tuning stage. The fine-tuning process requires minimal parameter modification without changing the model architecture, which mitigates the data scarcity issue and helps train the deep learning model adequately on small patient cohorts. We conducted experiments on a real-world pediatric dataset with more than one million patient records. Experimental results on two downstream tasks demonstrated the effectiveness of our method: our general task-agnostic pre-training framework outperformed tailored task-specific models, achieving more than 10% higher in model performance as compared to baselines. In addition, our framework showed a potential to transfer learned knowledge from one institution to another, which may pave the way for future healthcare model pre-training across institutions.

A Machine Learning Approach to Uncovering Hidden Utilization Patterns of Early Childhood Dental Care Among Medicaid-Insured Children.

Background: Early childhood dental care (ECDC) is a significant public health opportunity since dental caries is largely preventable and a prime target for reducing healthcare expenditures. This study aims to discover underlying patterns in ECDC utilization among Ohio Medicaid-insured children, which have significant implications for public health prevention, innovative service delivery models, and targeted cost-saving interventions. Methods: Using 9 years of longitudinal Medicaid data of 24,223 publicly insured child members of an accountable care organization (ACO), Partners for Kids in Ohio, we applied unsupervised machine learning to cluster patients based on their cumulative dental cost curves in early childhood (24-60 months). Clinical validity, analytical validity, and reproducibility were assessed. Results: The clustering revealed five novel subpopulations: (1) early-onset of decay by age (0.5% of the population, as early as 28 months), (2) middle-onset of decay (3.0%, as early as 35 months), (3) late-onset of decay (5.8%, as early as 44 months), (4) regular preventive care (67.7%), and (5) zero utilization (23.0%). Patients with early-onset of decay incurred the highest dental cost [median annual cost (MAC) = $9,499, InterQuartile Range (IQR): $7,052-$11,216], while patients with regular preventive care incurred the lowest dental cost (MAC = $191, IQR: $99-$336). We also found a plausible correlation of early-onset of decay with complex medical conditions diagnosed at 0-24 months. Almost one-third of patients with early-onset of decay had complex medical conditions diagnosed at 0-24 months. Patients with early-onset of decay also incurred the highest medical cost (MAC = $7,513, IQR: $4,527-$12,546) at 0-24 months. Conclusion: Among Ohio Medicaid-insured children, five subpopulations with distinctive clinical, cost, and utilization patterns were discovered and validated through a data-driven approach. This novel discovery promotes innovative prevention strategies that differentiate Medicaid subpopulations, and allows for the development of cost-effective interventions that target high-risk patients. Furthermore, an integrated medical-dental care delivery model promises to reduce costs further while improving patient outcomes.

Multilevel Self-Attention Model and its Use on Medical Risk Prediction.

Various deep learning models have been developed for different healthcare predictive tasks using Electronic Health Records and have shown promising performance. In these models, medical codes are often aggregated into visit representation without considering their heterogeneity, e.g., the same diagnosis might imply different healthcare concerns with different procedures or medications. Then the visits are often fed into deep learning models, such as recurrent neural networks, sequentially without considering the irregular temporal information and dependencies among visits. To address these limitations, we developed a Multilevel Self-Attention Model (MSAM) that can capture the underlying relationships between medical codes and between medical visits. We compared MSAM with various baseline models on two predictive tasks, i.e., future disease prediction and future medical cost prediction, with two large datasets, i.e., MIMIC-3 and PFK. In the experiments, MSAM consistently outperformed baseline models. Additionally, for future medical cost prediction, we used disease prediction as an auxiliary task, which not only guides the model to achieve a stronger and more stable financial prediction, but also allows managed care organizations to provide a better care coordination.

A deep learning model for pediatric patient risk stratification.

OBJECTIVES: Current models for patient risk prediction rely on practitioner expertise and domain knowledge. This study presents a deep learning model-a type of machine learning that does not require human inputs-to analyze complex clinical and financial data for population risk stratification. STUDY DESIGN: A comparative predictive analysis of deep learning versus other popular risk prediction modeling strategies using medical claims data from a cohort of 112,641 pediatric accountable care organization members. METHODS: "Skip-Gram," an unsupervised deep learning approach that uses neural networks for prediction modeling, used data from 2014 and 2015 to predict the risk of hospitalization in 2016. The area under the curve (AUC) of the deep learning model was compared with that of both the Clinical Classifications Software and the commercial DxCG Intelligence predictive risk models, each with and without demographic and utilization features. We then calculated costs for patients in the top 1% and 5% of hospitalization risk identified by each model. RESULTS: The deep learning model performed the best across 6 predictive models, with an AUC of 75.1%. The top 1% of members selected by the deep learning model had a combined healthcare cost $5 million higher than that of the group identified by the DxCG Intelligence model. CONCLUSIONS: The deep learning model outperforms the traditional risk models in prospective hospitalization prediction. Thus, deep learning may improve the ability of managed care organizations to perform predictive modeling of financial risk, in addition to improving the accuracy of risk stratification for population health management activities.