Understanding Time Series Patterns of Weight and Meal History Reports in Mobile Weight Loss Intervention Programs: Data-Driven Analysis.

BACKGROUND: Mobile apps for weight loss provide users with convenient features for recording lifestyle and health indicators; they have been widely used for weight loss recently. Previous studies in this field generally focused on the relationship between the cumulative nature of self-reported data and the results in weight loss at the end of the diet period. Therefore, we conducted an in-depth study to explore the relationships between adherence to self-reporting and weight loss outcomes during the weight reduction process. OBJECTIVE: We explored the relationship between adherence to self-reporting and weight loss outcomes during the time series weight reduction process with the following 3 research questions: "How does adherence to self-reporting of body weight and meal history change over time?", "How do weight loss outcomes depend on weight changes over time?", and "How does adherence to the weight loss intervention change over time by gender?" METHODS: We analyzed self-reported data collected weekly for 16 weeks (January 2017 to March 2018) from 684 Korean men and women who participated in a mobile weight loss intervention program provided by a mobile diet app called Noom. Analysis of variance (ANOVA) and chi-squared tests were employed to determine whether the baseline characteristics among the groups of weight loss results were different. Based on the ANOVA results and slope analysis of the trend indicating participant behavior along the time axis, we explored the relationship between adherence to self-reporting and weight loss results. RESULTS: Adherence to self-reporting levels decreased over time, as previous studies have found. BMI change patterns (ie, absolute BMI values and change in BMI values within a week) changed over time and were characterized in 3 time series periods. The relationships between the weight loss outcome and both meal history and self-reporting patterns were gender-dependent. There was no statistical association between adherence to self-reporting and weight loss outcomes in the male participants. CONCLUSIONS: Although mobile technology has increased the convenience of self-reporting when dieting, it should be noted that technology itself is not the essence of weight loss. The in-depth understanding of the relationship between adherence to self-reporting and weight loss outcome found in this study may contribute to the development of better weight loss interventions in mobile environments.

Limitations of Deep Learning Attention Mechanisms in Clinical Research: Empirical Case Study Based on the Korean Diabetic Disease Setting.

BACKGROUND: Despite excellent prediction performance, noninterpretability has undermined the value of applying deep-learning algorithms in clinical practice. To overcome this limitation, attention mechanism has been introduced to clinical research as an explanatory modeling method. However, potential limitations of using this attractive method have not been clarified to clinical researchers. Furthermore, there has been a lack of introductory information explaining attention mechanisms to clinical researchers. OBJECTIVE: The aim of this study was to introduce the basic concepts and design approaches of attention mechanisms. In addition, we aimed to empirically assess the potential limitations of current attention mechanisms in terms of prediction and interpretability performance. METHODS: First, the basic concepts and several key considerations regarding attention mechanisms were identified. Second, four approaches to attention mechanisms were suggested according to a two-dimensional framework based on the degrees of freedom and uncertainty awareness. Third, the prediction performance, probability reliability, concentration of variable importance, consistency of attention results, and generalizability of attention results to conventional statistics were assessed in the diabetic classification modeling setting. Fourth, the potential limitations of attention mechanisms were considered. RESULTS: Prediction performance was very high for all models. Probability reliability was high in models with uncertainty awareness. Variable importance was concentrated in several variables when uncertainty awareness was not considered. The consistency of attention results was high when uncertainty awareness was considered. The generalizability of attention results to conventional statistics was poor regardless of the modeling approach. CONCLUSIONS: The attention mechanism is an attractive technique with potential to be very promising in the future. However, it may not yet be desirable to rely on this method to assess variable importance in clinical settings. Therefore, along with theoretical studies enhancing attention mechanisms, more empirical studies investigating potential limitations should be encouraged.

What Clinical Information Is Valuable to Doctors Using Mobile Electronic Medical Records and When?

BACKGROUND: There has been a lack of understanding on what types of specific clinical information are most valuable for doctors to access through mobile-based electronic medical records (m-EMRs) and when they access such information. Furthermore, it has not been clearly discussed why the value of such information is high. OBJECTIVE: The goal of this study was to investigate the types of clinical information that are most valuable to doctors to access through an m-EMR and when such information is accessed. METHODS: Since 2010, an m-EMR has been used in a tertiary hospital in Seoul, South Korea. The usage logs of the m-EMR by doctors were gathered from March to December 2015. Descriptive analyses were conducted to explore the overall usage patterns of the m-EMR. To assess the value of the clinical information provided, the usage patterns of both the m-EMR and a hospital information system (HIS) were compared on an hourly basis. The peak usage times of the m-EMR were defined as continuous intervals having normalized usage values that are greater than 0.5. The usage logs were processed as an indicator representing specific clinical information using factor analysis. Random intercept logistic regression was used to explore the type of clinical information that is frequently accessed during the peak usage times. RESULTS: A total of 524,929 usage logs from 653 doctors (229 professors, 161 fellows, and 263 residents; mean age: 37.55 years; males: 415 [63.6%]) were analyzed. The highest average number of m-EMR usage logs (897) was by medical residents, whereas the lowest (292) was by surgical residents. The usage amount for three menus, namely inpatient list (47,096), lab results (38,508), and investigation list (25,336), accounted for 60.1% of the peak time usage. The HIS was used most frequently during regular hours (9:00 AM to 5:00 PM). The peak usage time of the m-EMR was early in the morning (6:00 AM to 10:00 AM), and the use of the m-EMR from early evening (5:00 PM) to midnight was higher than during regular business hours. Four factors representing the types of clinical information were extracted through factor analysis. Factors related to patient investigation status and patient conditions were associated with the peak usage times of the m-EMR (P<.01). CONCLUSIONS: Access to information regarding patient investigation status and patient conditions is crucial for decision making during morning activities, including ward rounds. The m-EMRs allow doctors to maintain the continuity of their clinical information regardless of the time and location constraints. Thus, m-EMRs will best evolve in a manner that enhances the accessibility of clinical information helpful to the decision-making process under such constraints.

Depression Screening Using Daily Mental-Health Ratings from a Smartphone Application for Breast Cancer Patients.

BACKGROUND: Mobile mental-health trackers are mobile phone apps that gather self-reported mental-health ratings from users. They have received great attention from clinicians as tools to screen for depression in individual patients. While several apps that ask simple questions using face emoticons have been developed, there has been no study examining the validity of their screening performance. OBJECTIVE: In this study, we (1) evaluate the potential of a mobile mental-health tracker that uses three daily mental-health ratings (sleep satisfaction, mood, and anxiety) as indicators for depression, (2) discuss three approaches to data processing (ratio, average, and frequency) for generating indicator variables, and (3) examine the impact of adherence on reporting using a mobile mental-health tracker and accuracy in depression screening. METHODS: We analyzed 5792 sets of daily mental-health ratings collected from 78 breast cancer patients over a 48-week period. Using the Patient Health Questionnaire-9 (PHQ-9) as the measure of true depression status, we conducted a random-effect logistic panel regression and receiver operating characteristic (ROC) analysis to evaluate the screening performance of the mobile mental-health tracker. In addition, we classified patients into two subgroups based on their adherence level (higher adherence and lower adherence) using a k-means clustering algorithm and compared the screening accuracy between the two groups. RESULTS: With the ratio approach, the area under the ROC curve (AUC) is 0.8012, indicating that the performance of depression screening using daily mental-health ratings gathered via mobile mental-health trackers is comparable to the results of PHQ-9 tests. Also, the AUC is significantly higher (P=.002) for the higher adherence group (AUC=0.8524) than for the lower adherence group (AUC=0.7234). This result shows that adherence to self-reporting is associated with a higher accuracy of depression screening. CONCLUSIONS: Our results support the potential of a mobile mental-health tracker as a tool for screening for depression in practice. Also, this study provides clinicians with a guideline for generating indicator variables from daily mental-health ratings. Furthermore, our results provide empirical evidence for the critical role of adherence to self-reporting, which represents crucial information for both doctors and patients.

Intraoperative Hypotension Prediction Based on Features Automatically Generated Within an Interpretable Deep Learning Model.

The monitoring of arterial blood pressure (ABP) in anesthetized patients is crucial for preventing hypotension, which can lead to adverse clinical outcomes. Several efforts have been devoted to develop artificial intelligence-based hypotension prediction indices. However, the use of such indices is limited because they may not provide a compelling interpretation of the association between the predictors and hypotension. Herein, an interpretable deep learning model is developed that forecasts hypotension occurrence 10 min before a given 90-s ABP record. Internal and external validations of the model performance show the area under the receiver operating characteristic curves of 0.9145 and 0.9035, respectively. Furthermore, the hypotension prediction mechanism can be physiologically interpreted using the predictors automatically generated from the proposed model for representing ABP trends. Finally, the applicability of a deep learning model with high accuracy is demonstrated, thus providing an interpretation of the association between ABP trends and hypotension in clinical practice.

A comparison of machine learning models and Cox proportional hazards models regarding their ability to predict the risk of gastrointestinal cancer based on metabolic syndrome and its components.

Background: Little is known about applying machine learning (ML) techniques to identify the important variables contributing to the occurrence of gastrointestinal (GI) cancer in epidemiological studies. We aimed to compare different ML models to a Cox proportional hazards (CPH) model regarding their ability to predict the risk of GI cancer based on metabolic syndrome (MetS) and its components. Methods: A total of 41,837 participants were included in a prospective cohort study. Incident cancer cases were identified by following up with participants until December 2019. We used CPH, random survival forest (RSF), survival trees (ST), gradient boosting (GB), survival support vector machine (SSVM), and extra survival trees (EST) models to explore the impact of MetS on GI cancer prediction. We used the C-index and integrated Brier score (IBS) to compare the models. Results: In all, 540 incident GI cancer cases were identified. The GB and SSVM models exhibited comparable performance to the CPH model concerning the C-index (0.725). We also recorded a similar IBS for all models (0.017). Fasting glucose and waist circumference were considered important predictors. Conclusions: Our study found comparably good performance concerning the C-index for the ML models and CPH model. This finding suggests that ML models may be considered another method for survival analysis when the CPH model's conditions are not satisfied.

Development of a Bispectral index score prediction model based on an interpretable deep learning algorithm.

BACKGROUND: Proper maintenance of hypnosis is crucial for ensuring the safety of patients undergoing surgery. Accordingly, indicators, such as the Bispectral index (BIS), have been developed to monitor hypnotic levels. However, the black-box nature of the algorithm coupled with the hardware makes it challenging to understand the underlying mechanisms of the algorithms and integrate them with other monitoring systems, thereby limiting their use. OBJECTIVE: We propose an interpretable deep learning model that forecasts BIS values 25 s in advance using 30 s electroencephalogram (EEG) data. MATERIAL AND METHODS: The proposed model utilized EEG data as a predictor, which is then decomposed into amplitude and phase components using fast Fourier Transform. An attention mechanism was applied to interpret the importance of these components in predicting BIS. The predictability of the model was evaluated on both regression and binary classification tasks, where the former involved predicting a continuous BIS value, and the latter involved classifying a dichotomous status at a BIS value of 60. To evaluate the interpretability of the model, we analyzed the attention values expressed in the amplitude and phase components according to five ranges of BIS values. The proposed model was trained and evaluated using datasets collected from two separate medical institutions. RESULTS AND CONCLUSION: The proposed model achieved excellent performance on both the internal and external validation datasets. The model achieved a root-mean-square error of 6.614 for the regression task, and an area under the receiver operating characteristic curve of 0.937 for the binary classification task. Interpretability analysis provided insight into the relationship between EEG frequency components and BIS values. Specifically, the attention mechanism revealed that higher BIS values were associated with increased amplitude attention values in high-frequency bands and increased phase attention values in various frequency bands. This finding is expected to facilitate a more profound understanding of the BIS prediction mechanism, thereby contributing to the advancement of anesthesia technologies.

A nomogram for predicting probability of low risk of MammaPrint results in women with clinically high-risk breast cancer.

We aimed to develop a prediction MammaPrint (MMP) genomic risk assessment nomogram model for hormone-receptor positive (HR+) and human epidermal growth factor receptor-2 negative (HER2-) breast cancer and minimal axillary burden (N0-1) tumors using clinicopathological factors of patients who underwent an MMP test for decision making regarding adjuvant chemotherapy. A total of 409 T1-3 N0-1 M0 HR + and HER2- breast cancer patients whose MMP genomic risk results and clinicopathological factors were available from 2017 to 2020 were analyzed. With randomly selected 306 patients, we developed a nomogram for predicting a low-risk subgroup of MMP results and externally validated with remaining patients (n = 103). Multivariate analysis revealed that the age at diagnosis, progesterone receptor (PR) score, nuclear grade, and Ki-67 were significantly associated with MMP risk results. We developed an MMP low-risk predictive nomogram. With a cut off value at 5% and 95% probability of low-risk MMP, the nomogram accurately predicted the results with 100% positive predictive value (PPV) and negative predictive value respectively. When applied to cut-off value at 35%, the specificity and PPV was 95% and 86% respectively. The area under the receiver operating characteristic curve was 0.82 (95% confidence interval [CI] 0.77 to 0.87). When applied to the validation group, the nomogram was accurate with an area under the curve of 0.77 (95% CI 0.68 to 0.86). Our nomogram, which incorporates four traditional prognostic factors, i.e., age, PR, nuclear grade, and Ki-67, could predict the probability of obtaining a low MMP risk in a cohort of high clinical risk patients. This nomogram can aid the prompt selection of patients who does not need additional MMP testing.

Development of a Real-Time Risk Prediction Model for In-Hospital Cardiac Arrest in Critically Ill Patients Using Deep Learning: Retrospective Study.

BACKGROUND: Cardiac arrest is the most serious death-related event in intensive care units (ICUs), but it is not easily predicted because of the complex and time-dependent data characteristics of intensive care patients. Given the complexity and time dependence of ICU data, deep learning-based methods are expected to provide a good foundation for developing risk prediction models based on large clinical records. OBJECTIVE: This study aimed to implement a deep learning model that estimates the distribution of cardiac arrest risk probability over time based on clinical data and assesses its potential. METHODS: A retrospective study of 759 ICU patients was conducted between January 2013 and July 2015. A character-level gated recurrent unit with a Weibull distribution algorithm was used to develop a real-time prediction model. Fivefold cross-validation testing (training set: 80% and validation set: 20%) determined the consistency of model accuracy. The time-dependent area under the curve (TAUC) was analyzed based on the aggregation of 5 validation sets. RESULTS: The TAUCs of the implemented model were 0.963, 0.942, 0.917, 0.875, 0.850, 0.842, and 0.761 before cardiac arrest at 1, 8, 16, 24, 32, 40, and 48 hours, respectively. The sensitivity was between 0.846 and 0.909, and specificity was between 0.923 and 0.946. The distribution of risk between the cardiac arrest group and the non-cardiac arrest group was generally different, and the difference rapidly increased as the time left until cardiac arrest reduced. CONCLUSIONS: A deep learning model for forecasting cardiac arrest was implemented and tested by considering the cumulative and fluctuating effects of time-dependent clinical data gathered from a large medical center. This real-time prediction model is expected to improve patient's care by allowing early intervention in patients at high risk of unexpected cardiac arrests.

A Nomogram for Predicting the Oncotype DX Recurrence Score in Women with T1-3N0-1miM0 Hormone Receptor‒Positive, Human Epidermal Growth Factor 2 (HER2)‒Negative Breast Cancer.

PURPOSE: This preliminary study was conducted to evaluate the association between Oncotype DX (ODX) recurrence score and traditional prognostic factors. We also developed a nomogram to predict subgroups with low ODX recurrence scores (less than 25) and to avoid additional chemotherapy treatments for those patients. MATERIALS AND METHODS: Clinicopathological and immunohistochemical variables were retrospectively retrieved and analyzed from a series of 485 T1-3N0-1miM0 hormone receptor-positive, human epidermal growth factor 2‒negative breast cancer patients with available ODX test results at Asan Medical Center from 2010 to 2016. One hundred twenty-seven patients (26%) had positive axillary lymph node micrometastases, and 408 (84%) had ODX recurrence scores of ≤25. Logistic regression was performed to build a nomogram for predicting a low-risk subgroup of the ODX assay. RESULTS: Multivariate analysis revealed that estrogen receptor (ER) score, progesterone receptor (PR) score, histologic grade, lymphovascular invasion (LVI), and Ki-67 had a statistically significant association with the low-risk subgroup. With these variables, we developed a nomogram to predict the low-risk subgroup with ODX recurrence scores of ≤25. The area under the receiver operating characteristic curve was 0.90 (95% confidence interval [CI], 0.85 to 0.96). When applied to the validation group the nomogram was accurate with an area under the curve = 0.88 (95% CI, 0.83 to 0.95). CONCLUSION: The low ODX recurrence score subgroup can be predicted by a nomogram incorporating five traditional prognostic factors: ER, PR, histologic grade, LVI, and Ki-67. Our nomogram, which predicts a low-risk ODX recurrence score, will be a useful tool to help select patients who may or may not need additional ODX testing.

Usage Pattern Differences and Similarities of Mobile Electronic Medical Records Among Health Care Providers.

BACKGROUND: Recently, many hospitals have introduced mobile electronic medical records (mEMRs). Although numerous studies have been published on the usability or usage patterns of mEMRs through user surveys, investigations based on the real data usage are lacking. OBJECTIVE: Asan Medical Center, a tertiary hospital in Seoul, Korea, implemented an mEMR program in 2010. On the basis of the mEMR usage log data collected over a period of 4.5 years, we aimed to identify a usage pattern and trends in accordance with user occupation and to disseminate the factors that make the mEMR more effective and efficient. METHODS: The mEMR log data were collected from March 2012 to August 2016. Descriptive analyses were completed according to user occupation, access time, services, and wireless network type. Specifically, analyses targeted were as follows: (1) the status of the mEMR usage and distribution of users, (2) trends in the number of users and usage amount, (3) 24-hour usage patterns, and (4) trends in service usage based on user occupations. Linear regressions were performed to model the relationship between the time, access frequency, and the number of users. The differences between the user occupations were examined using Student t tests for categorical variables. RESULTS: Approximately two-thirds of the doctors and nurses used the mEMR. The number of logs studied was 7,144,459. Among 3859 users, 2333 (60.46%) users were nurses and 1102 (28.56%) users were doctors. On average, the mEMR was used 1044 times by 438 users per day. The number of users and amount of access logs have significantly increased since 2012 (P<.001). Nurses used the mEMR 3 times more often than doctors. The use of mEMR by nurses increased by an annual average of 51.5%, but use by doctors decreased by an annual average of 7.7%. For doctors, the peak usage periods were observed during 08:00 to 09:00 and 17:00 to 18:00, which were coincident with the beginning of ward rounds. Conversely, the peak usage periods for the nurses were observed during 05:00 to 06:00, 12:00 to 13:00, and 20:00 to 21:00, which effectively occurred 1 or 2 hours before handover. In more than 80% of all cases, the mEMR was accessed via a nonhospital wireless network. CONCLUSIONS: The usage patterns of the mEMR differed between doctors and nurses according to their different workflows. In both occupations, mEMR was highly used when personal computer access was limited and the need for patient information was high, such as during ward rounds or handover periods.