Shortcomings in the Evaluation of Blood Glucose Forecasting.

OBJECTIVE: Recent years have seen an increase in machine learning (ML)-based blood glucose (BG) forecasting models, with a growing emphasis on potential application to hybrid or closed-loop predictive glucose controllers. However, current approaches focus on evaluating the accuracy of these models using benchmark data generated under the behavior policy, which may differ significantly from the data the model may encounter in a control setting. This study challenges the efficacy of such evaluation approaches, demonstrating that they can fail to accurately capture an ML-based model's true performance in closed-loop control settings. METHODS: Forecast error measured using current evaluation approaches was compared to the control performance of two forecasters-a machine learning-based model (LSTM) and a rule-based model (Loop)-in silico when the forecasters were utilized with a model-based controller in a hybrid closed-loop setting. RESULTS: Under current evaluation standards, LSTM achieves a significantly lower (better) forecast error than Loop with a root mean squared error (RMSE) of 11.57 ±0.05 mg/dL vs. 18.46 ±0.07 mg/dL at the 30-minute prediction horizon. Yet in a control setting, LSTM led to significantly worse control performance with only 77.14% (IQR 66.57-84.03) time-in-range compared to 86.20% (IQR 78.28-91.21) for Loop. CONCLUSION: Prevailing evaluation methods can fail to accurately capture the forecaster's performance when utilized in closed-loop settings. SIGNIFICANCE: Our findings underscore the limitations of current evaluation standards and the need for alternative evaluation metrics and training strategies when developing BG forecasters for closed-loop control systems.

From Stability to Variability: Classification of Healthy Individuals, Prediabetes, and Type 2 Diabetes using Glycemic Variability Indices from Continuous Glucose Monitoring Data.

OBJECTIVE: This study aims to investigate the continuum of glucose control from normoglycemia to dysglycemia (HbA1c ≥ 5.7% / 39 mmol/mol) using metrics derived from Continuous Glucose Monitoring (CGM). Additionally, we aim to develop a machine learning-based classification model to classify dysglycemia based on observed patterns. METHODS: Data from five distinct studies, each featuring at least two days of CGM, were pooled. Participants included individuals classified as healthy, with prediabetes, or with type 2 diabetes mellitus (T2DM). Various CGM indices were extracted and compared across groups. The dataset was split 70/30 for training and testing two classification models (XGBoost / Logistic Regression) to differentiate between prediabetes or dysglycemia and the healthy group. RESULTS: The analysis included 836 participants (healthy: n=282; prediabetes: n=133; T2DM: n=432). Across all CGM indices, a progressive shift was observed from the healthy group to those with diabetes (p<0.001). Statistically significant differences (p<0.01) were noted in mean glucose, Time Below Range, Time Above 140 mg/dl, Mmobility, Multiscale Complexity Index and Glycemic Risk Index when transitioning from health to prediabetes. The XGBoost models achieved the highest Receiver Operating Characteristic Area Under the Curve (ROC-AUC) values on the test dataset ranging from 0.91 [CI: 0.87-0.95] (prediabetes identification) to 0.97 [CI: 0.95-0.98] (Dysglycemia identification). CONCLUSION: Our findings demonstrate a gradual deterioration of glucose homeostasis and increased glycemic variability across the spectrum from normo- to dysglycemia, as evidenced by CGM metrics. The performance of CGM-based indices in classifying healthy individuals and those with prediabetes and diabetes is promising.

Cardiovascular autonomic reflex tests using a handheld device in the diagnosis of cardiovascular autonomic neuropathy in patients with schizophrenia.

Study objective: This study investigated whether schizophrenia and the duration of schizophrenia were associated with cardiovascular autonomic neuropathy (CAN) by using heart rate variability (HRV) as a marker. Design: Cross-sectional study. Setting: The examinations were conducted at the Centre for Psychosis Research and at the Department of Cardiology, Aalborg University Hospital, Aalborg, Denmark. Participants: 240 patients with first-episode and chronic schizophrenia and 180 controls. Interventions: CAN was assessed by the cardiovascular reflex tests (CARTs): HR, RS ratio, E:I ratio, and VM using a handheld device. Main outcome measures: One abnormal CART was interpreted as borderline CAN and ≥2 abnormal CARTs established definitive CAN. Borderline CAN and definitive CAN together was categorized as overall CAN. Analyses were adjusted for age, sex, smoking, overweight, and hypercholesterolemia. Results: A total of 240 patients with schizophrenia (median age 42.5 [28.8, 52.3], 42.9 % women) and 180 controls (median age 45.8 [24.0, 60.1], 47.8 % women) were included, with 50.8 % of patients with schizophrenia having overall CAN compared to 27.2 % among controls. Dividing patients into patients with first-episode and chronic schizophrenia, 32.9 % vs 10 % (p < 0.001) and 59.1 % vs 41 % (p < 0.001) had overall CAN compared with controls, respectively. Schizophrenia was significantly associated with overall CAN (OR, 2.80; 95%CI, 1.75-4.50), with an OR of 2.31 (95%CI, 1.14-4.68) for first-episode schizophrenia and an OR of 2.97 (95%CI, 1.81-4.87) for chronic schizophrenia. Conclusion: It was demonstrated that a diagnosis of schizophrenia was associated with CAN. Patients with chronic schizophrenia had a significantly higher prevalence of CAN compared to patients with first-episode schizophrenia, suggesting an association between the duration of schizophrenia and CAN.