Predicting COVID-19-Related Health Care Resource Utilization Across a Statewide Patient Population: Model Development Study.

BACKGROUND: The COVID-19 pandemic has highlighted the inability of health systems to leverage existing system infrastructure in order to rapidly develop and apply broad analytical tools that could inform state- and national-level policymaking, as well as patient care delivery in hospital settings. The COVID-19 pandemic has also led to highlighted systemic disparities in health outcomes and access to care based on race or ethnicity, gender, income-level, and urban-rural divide. Although the United States seems to be recovering from the COVID-19 pandemic owing to widespread vaccination efforts and increased public awareness, there is an urgent need to address the aforementioned challenges. OBJECTIVE: This study aims to inform the feasibility of leveraging broad, statewide datasets for population health-driven decision-making by developing robust analytical models that predict COVID-19-related health care resource utilization across patients served by Indiana's statewide Health Information Exchange. METHODS: We leveraged comprehensive datasets obtained from the Indiana Network for Patient Care to train decision forest-based models that can predict patient-level need of health care resource utilization. To assess these models for potential biases, we tested model performance against subpopulations stratified by age, race or ethnicity, gender, and residence (urban vs rural). RESULTS: For model development, we identified a cohort of 96,026 patients from across 957 zip codes in Indiana, United States. We trained the decision models that predicted health care resource utilization by using approximately 100 of the most impactful features from a total of 1172 features created. Each model and stratified subpopulation under test reported precision scores >70%, accuracy and area under the receiver operating curve scores >80%, and sensitivity scores approximately >90%. We noted statistically significant variations in model performance across stratified subpopulations identified by age, race or ethnicity, gender, and residence (urban vs rural). CONCLUSIONS: This study presents the possibility of developing decision models capable of predicting patient-level health care resource utilization across a broad, statewide region with considerable predictive performance. However, our models present statistically significant variations in performance across stratified subpopulations of interest. Further efforts are necessary to identify root causes of these biases and to rectify them.

Use of smartphone sensor data in detecting and predicting depression and anxiety in young people (12-25 years): A scoping review.

Digital phenotyping is a promising method for advancing scalable detection and prediction methods in mental health research and practice. However, little is known about how digital phenotyping data are used to make inferences about youth mental health. We conducted a scoping review of 35 studies to better understand how passive sensing (e.g., Global Positioning System, microphone etc) and electronic usage data (e.g., social media use, device activity etc) collected via smartphones are used in detecting and predicting depression and/or anxiety in young people between 12 and 25 years-of-age. GPS and/or Wifi association logs and accelerometers were the most used sensors, although a wide variety of low-level features were extracted and computed (e.g., transition frequency, time spent in specific locations, uniformity of movement). Mobility and sociability patterns were explored in more studies compared to other behaviours such as sleep, phone use, and circadian movement. Studies used machine learning, statistical regression, and correlation analyses to examine relationships between variables. Results were mixed, but machine learning indicated that models using feature combinations (e.g., mobility, sociability, and sleep features) were better able to predict and detect symptoms of youth anxiety and/or depression when compared to models using single features (e.g., transition frequency). There was inconsistent reporting of age, gender, attrition, and phone characteristics (e.g., operating system, models), and all studies were assessed to have moderate to high risk of bias. To increase translation potential for clinical practice, we recommend the development of a standardised reporting framework to improve transparency and replicability of methodology.

Natural language processing-driven state machines to extract social factors from unstructured clinical documentation.

Objective: This study sought to create natural language processing algorithms to extract the presence of social factors from clinical text in 3 areas: (1) housing, (2) financial, and (3) unemployment. For generalizability, finalized models were validated on data from a separate health system for generalizability. Materials and Methods: Notes from 2 healthcare systems, representing a variety of note types, were utilized. To train models, the study utilized n-grams to identify keywords and implemented natural language processing (NLP) state machines across all note types. Manual review was conducted to determine performance. Sampling was based on a set percentage of notes, based on the prevalence of social need. Models were optimized over multiple training and evaluation cycles. Performance metrics were calculated using positive predictive value (PPV), negative predictive value, sensitivity, and specificity. Results: PPV for housing rose from 0.71 to 0.95 over 3 training runs. PPV for financial rose from 0.83 to 0.89 over 2 training iterations, while PPV for unemployment rose from 0.78 to 0.88 over 3 iterations. The test data resulted in PPVs of 0.94, 0.97, and 0.95 for housing, financial, and unemployment, respectively. Final specificity scores were 0.95, 0.97, and 0.95 for housing, financial, and unemployment, respectively. Discussion: We developed 3 rule-based NLP algorithms, trained across health systems. While this is a less sophisticated approach, the algorithms demonstrated a high degree of generalizability, maintaining >0.85 across all predictive performance metrics. Conclusion: The rule-based NLP algorithms demonstrated consistent performance in identifying 3 social factors within clinical text. These methods may be a part of a strategy to measure social factors within an institution.

Machine learning identifies a COVID-19-specific phenotype in university students using a mental health app.

Background: Advances in smartphone technology have allowed people to access mental healthcare via digital apps from wherever and whenever they choose. University students experience a high burden of mental health concerns. Although these apps improve mental health symptoms, user engagement has remained low. Studies have shown that users can be subgrouped based on unique characteristics that just-in-time adaptive interventions (JITAIs) can use to improve engagement. To date, however, no studies have examined the effect of the COVID-19 pandemic on these subgroups. Objective: Here, we sought to examine user subgroup characteristics across three COVID-19-specific timepoints: during lockdown, immediately following lockdown, and three months after lockdown ended. Methods: To do this, we used a two-step machine learning approach combining unsupervised and supervised machine learning. Results: We demonstrate that there are three unique subgroups of university students who access mental health apps. Two of these, with either higher or lower mental well-being, were defined by characteristics that were stable across COVID-19 timepoints. The third, situational well-being, had characteristics that were timepoint-dependent, suggesting that they are highly influenced by traumatic stressors and stressful situations. This subgroup also showed feelings and behaviours consistent with burnout. Conclusions: Overall, our findings clearly suggest that user subgroups are unique: they have different characteristics and therefore likely have different mental healthcare goals. Our findings also highlight the importance of including questions and additional interventions targeting traumatic stress(ors), reason(s) for use, and burnout in JITAI-style mental health apps to improve engagement.