Developing Methods for Assessing Mental Activity Using Human-Smartphone Interactions: Comparative Analysis of Activity Levels and Phase Patterns in General Mental Activities, Working Mental Activities, and Physical Activities.

BACKGROUND: Human biological rhythms are commonly assessed through physical activity (PA) measurement, but mental activity may offer a more substantial reflection of human biological rhythms. OBJECTIVE: This study proposes a novel approach based on human-smartphone interaction to compute mental activity, encompassing general mental activity (GMA) and working mental activity (WMA). METHODS: A total of 24 health care professionals participated, wearing wrist actigraphy devices and using the "Staff Hours" app for more than 457 person-days, including 332 workdays and 125 nonworkdays. PA was measured using actigraphy, while GMA and WMA were assessed based on patterns of smartphone interactions. To model WMA, machine learning techniques such as extreme gradient boosting and convolutional neural networks were applied, using human-smartphone interaction patterns and GPS-defined work hours. The data were organized by date and divided into person-days, with an 80:20 split for training and testing data sets to minimize overfitting and maximize model robustness. The study also adopted the M10 metric to quantify daily activity levels by calculating the average acceleration during the 10-hour period of highest activity each day, which facilitated the assessment of the interrelations between PA, GMA, and WMA and sleep indicators. Phase differences, such as those between PA and GMA, were defined using a second-order Butterworth filter and Hilbert transform to extract and calculate circadian rhythms and instantaneous phases. This calculation involved subtracting the phase of the reference signal from that of the target signal and averaging these differences to provide a stable and clear measure of the phase relationship between the signals. Additionally, multilevel modeling explored associations between sleep indicators (total sleep time, midpoint of sleep) and next-day activity levels, accounting for the data's nested structure. RESULTS: Significant differences in activity levels were noted between workdays and nonworkdays, with WMA occurring approximately 1.08 hours earlier than PA during workdays (P<.001). Conversely, GMA was observed to commence about 1.22 hours later than PA (P<.001). Furthermore, a significant negative correlation was identified between the activity level of WMA and the previous night's midpoint of sleep (ß=-0.263, P<.001), indicating that later bedtimes and wake times were linked to reduced activity levels in WMA the following day. However, there was no significant correlation between WMA's activity levels and total sleep time. Similarly, no significant correlations were found between the activity levels of PA and GMA and sleep indicators from the previous night. CONCLUSIONS: This study significantly advances the understanding of human biological rhythms by developing and highlighting GMA and WMA as key indicators, derived from human-smartphone interactions. These findings offer novel insights into how mental activities, alongside PA, are intricately linked to sleep patterns, emphasizing the potential of GMA and WMA in behavioral and health studies.

Developing a Machine Learning Algorithm to Predict the Probability of Medical Staff Work Mode Using Human-Smartphone Interaction Patterns: Algorithm Development and Validation Study.

BACKGROUND: Traditional methods for investigating work hours rely on an employee's physical presence at the worksite. However, accurately identifying break times at the worksite and distinguishing remote work outside the worksite poses challenges in work hour estimations. Machine learning has the potential to differentiate between human-smartphone interactions at work and off work. OBJECTIVE: In this study, we aimed to develop a novel approach called "probability in work mode," which leverages human-smartphone interaction patterns and corresponding GPS location data to estimate work hours. METHODS: To capture human-smartphone interactions and GPS locations, we used the "Staff Hours" app, developed by our team, to passively and continuously record participants' screen events, including timestamps of notifications, screen on or off occurrences, and app usage patterns. Extreme gradient boosted trees were used to transform these interaction patterns into a probability, while 1-dimensional convolutional neural networks generated successive probabilities based on previous sequence probabilities. The resulting probability in work mode allowed us to discern periods of office work, off-work, breaks at the worksite, and remote work. RESULTS: Our study included 121 participants, contributing to a total of 5503 person-days (person-days represent the cumulative number of days across all participants on which data were collected and analyzed). The developed machine learning model exhibited an average prediction performance, measured by the area under the receiver operating characteristic curve, of 0.915 (SD 0.064). Work hours estimated using the probability in work mode (higher than 0.5) were significantly longer (mean 11.2, SD 2.8 hours per day) than the GPS-defined counterparts (mean 10.2, SD 2.3 hours per day; P<.001). This discrepancy was attributed to the higher remote work time of 111.6 (SD 106.4) minutes compared to the break time of 54.7 (SD 74.5) minutes. CONCLUSIONS: Our novel approach, the probability in work mode, harnessed human-smartphone interaction patterns and machine learning models to enhance the precision and accuracy of work hour investigation. By integrating human-smartphone interactions and GPS data, our method provides valuable insights into work patterns, including remote work and breaks, offering potential applications in optimizing work productivity and well-being.

Widen the Applicability of a Convolutional Neural-Network-Assisted Glaucoma Detection Algorithm of Limited Training Images across Different Datasets.

Automated glaucoma detection using deep learning may increase the diagnostic rate of glaucoma to prevent blindness, but generalizable models are currently unavailable despite the use of huge training datasets. This study aims to evaluate the performance of a convolutional neural network (CNN) classifier trained with a limited number of high-quality fundus images in detecting glaucoma and methods to improve its performance across different datasets. A CNN classifier was constructed using EfficientNet B3 and 944 images collected from one medical center (core model) and externally validated using three datasets. The performance of the core model was compared with (1) the integrated model constructed by using all training images from the four datasets and (2) the dataset-specific model built by fine-tuning the core model with training images from the external datasets. The diagnostic accuracy of the core model was 95.62% but dropped to ranges of 52.5-80.0% on the external datasets. Dataset-specific models exhibited superior diagnostic performance on the external datasets compared to other models, with a diagnostic accuracy of 87.50-92.5%. The findings suggest that dataset-specific tuning of the core CNN classifier effectively improves its applicability across different datasets when increasing training images fails to achieve generalization.

Ground truth generalizability affects performance of the artificial intelligence model in automated vertebral fracture detection on plain lateral radiographs of the spine.

BACKGROUND CONTEXT: Computer-aided diagnosis with artificial intelligence (AI) has been used clinically, and ground truth generalizability is important for AI performance in medical image analyses. The AI model was trained on one specific group of older adults (aged≧60) has not yet been shown to work equally well in a younger adult group (aged 18-59). PURPOSE: To compare the performance of the developed AI model with ensemble method trained with the ground truth for those aged 60 years or older in identifying vertebral fractures (VFs) on plain lateral radiographs of spine (PLRS) between younger and older adult populations. STUDY DESIGN/SETTING: Retrospective analysis of PLRS in a single medical institution. OUTCOME MEASURES: Accuracy, sensitivity, specificity, and interobserver reliability (kappa value) were used to compare diagnostic performance of the AI model and subspecialists' consensus between the two groups. METHODS: Between January 2016 and December 2018, the ground truth of 941 patients (one PLRS per person) aged 60 years and older with 1101 VFs and 6358 normal vertebrae was used to set up the AI model. The framework of the developed AI model includes: object detection with You Only Look Once Version 3 (YOLOv3) at T0-L5 levels in the PLRS, data pre-preprocessing with image-size and quality processing, and AI ensemble model (ResNet34, DenseNet121, and DenseNet201) for identifying or grading VFs. The reported overall accuracy, sensitivity and specificity were 92%, 91% and 93%, respectively, and external validation was also performed. Thereafter, patients diagnosed as VFs and treated in our institution during October 2019 to August 2020 were the study group regardless of age. In total, 258 patients (339 VFs and 1725 normal vertebrae) in the older adult population (mean age 78±10.4; range, 60-106) were enrolled. In the younger adult population (mean age 36±9.43; range, 20-49), 106 patients (120 VFs and 728 normal vertebrae) were enrolled. After identification and grading of VFs based on the Genant method with consensus between two subspecialists', VFs in each PLRS with human labels were defined as the testing dataset. The corresponding CT or MRI scan was used for labeling in the PLRS. The bootstrap method was applied to the testing dataset. RESULTS: The model for clinical application, Digital Imaging and Communications in Medicine (DICOM) format, is uploaded directly (available at: http://140.113.114.104/vght_demo/svf-model (grading) and http://140.113.114.104/vght demo/svf-model2 (labeling). Overall accuracy, sensitivity and specificity in the older adult population were 93.36% (95% CI 93.34%-93.38%), 88.97% (95% CI 88.59%-88.99%) and 94.26% (95% CI 94.23%-94.29%), respectively. Overall accuracy, sensitivity and specificity in the younger adult population were 93.75% (95% CI 93.7%-93.8%), 65.00% (95% CI 64.33%-65.67%) and 98.49% (95% CI 98.45%-98.52%), respectively. Accuracy reached 100% in VFs grading once the VFs were labeled accurately. The unique pattern of limbus-like VFs, 43 (35.8%) were investigated only in the younger adult population. If limbus-like VFs from the dataset were not included, the accuracy increased from 93.75% (95% CI 93.70%-93.80%) to 95.78% (95% CI 95.73%-95.82%), sensitivity increased from 65.00% (95% CI 64.33%-65.67%) to 70.13% (95% CI 68.98%-71.27%) and specificity remained unchanged at 98.49% (95% CI 98.45%-98.52%), respectively. The main causes of false negative results in older adults were patients' lung markings, diaphragm or bowel airs (37%, n=14) followed by type I fracture (29%, n=11). The main causes of false negatives in younger adults were limbus-like VFs (45%, n=19), followed by type I fracture (26%, n=11). The overall kappa between AI discrimination and subspecialists' consensus in the older and younger adult populations were 0.77 (95% CI, 0.733-0.805) and 0.72 (95% CI, 0.6524-0.80), respectively. CONCLUSIONS: The developed VF-identifying AI ensemble model based on ground truth of older adults achieved better performance in identifying VFs in older adults and non-fractured thoracic and lumbar vertebrae in the younger adults. Different age distribution may have potential disease diversity and implicate the effect of ground truth generalizability on the AI model performance.

Ensemble classification and segmentation for intracranial metastatic tumors on MRI images based on 2D U-nets.

The extraction of brain tumor tissues in 3D Brain Magnetic Resonance Imaging (MRI) plays an important role in diagnosis before the gamma knife radiosurgery (GKRS). In this article, the post-contrast T1 whole-brain MRI images had been collected by Taipei Veterans General Hospital (TVGH) and stored in DICOM format (dated from 1999 to 2018). The proposed method starts with the active contour model to get the region of interest (ROI) automatically and enhance the image contrast. The segmentation models are trained by MRI images with tumors to avoid imbalanced data problem under model construction. In order to achieve this objective, a two-step ensemble approach is used to establish such diagnosis, first, classify whether there is any tumor in the image, and second, segment the intracranial metastatic tumors by ensemble neural networks based on 2D U-Net architecture. The ensemble for classification and segmentation simultaneously also improves segmentation accuracy. The result of classification achieves a F1-measure of [Formula: see text], while the result of segmentation achieves an IoU of [Formula: see text] and a DICE score of [Formula: see text]. Significantly reduce the time for manual labeling from 30 min to 18 s per patient.

Can 3D artificial intelligence models outshine 2D ones in the detection of intracranial metastatic tumors on magnetic resonance images?

BACKGROUND: This study aimed to compare the prediction performance of two-dimensional (2D) and three-dimensional (3D) semantic segmentation models for intracranial metastatic tumors with a volume ≥ 0.3 mL. METHODS: We used postcontrast T1 whole-brain magnetic resonance (MR), which was collected from Taipei Veterans General Hospital (TVGH). Also, the study was approved by the institutional review board (IRB) of TVGH. The 2D image segmentation model does not fully use the spatial information between neighboring slices, whereas the 3D segmentation model does. We treated the U-Net as the basic model for 2D and 3D architectures. RESULTS: For the prediction of intracranial metastatic tumors, the area under the curve (AUC) of the 3D model was 87.6% and that of the 2D model was 81.5%. CONCLUSION: Building a semantic segmentation model based on 3D deep convolutional neural networks might be crucial to achieve a high detection rate in clinical applications for intracranial metastatic tumors.

Can a Deep-learning Model for the Automated Detection of Vertebral Fractures Approach the Performance Level of Human Subspecialists?

BACKGROUND: Vertebral fractures are the most common osteoporotic fractures in older individuals. Recent studies suggest that the performance of artificial intelligence is equal to humans in detecting osteoporotic fractures, such as fractures of the hip, distal radius, and proximal humerus. However, whether artificial intelligence performs as well in the detection of vertebral fractures on plain lateral spine radiographs has not yet been reported. QUESTIONS/PURPOSES: (1) What is the accuracy, sensitivity, specificity, and interobserver reliability (kappa value) of an artificial intelligence model in detecting vertebral fractures, based on Genant fracture grades, using plain lateral spine radiographs compared with values obtained by human observers? (2) Do patients' clinical data, including the anatomic location of the fracture (thoracic or lumbar spine), T-score on dual-energy x-ray absorptiometry, or fracture grade severity, affect the performance of an artificial intelligence model? (3) How does the artificial intelligence model perform on external validation? METHODS: Between 2016 and 2018, 1019 patients older than 60 years were treated for vertebral fractures in our institution. Seventy-eight patients were excluded because of missing CT or MRI scans (24% [19]), poor image quality in plain lateral radiographs of spines (54% [42]), multiple myeloma (5% [4]), and prior spine instrumentation (17% [13]). The plain lateral radiographs of 941 patients (one radiograph per person), with a mean age of 76 ± 12 years, and 1101 vertebral fractures between T7 and L5 were retrospectively evaluated for training (n = 565), validating (n = 188), and testing (n = 188) of an artificial intelligence deep-learning model. The gold standard for diagnosis (ground truth) of a vertebral fracture is the interpretation of the CT or MRI reports by a spine surgeon and a radiologist independently. If there were any disagreements between human observers, the corresponding CT or MRI images would be rechecked by them together to reach a consensus. For the Genant classification, the injured vertebral body height was measured in the anterior, middle, and posterior third. Fractures were classified as Grade 1 (< 25%), Grade 2 (26% to 40%), or Grade 3 (> 40%). The framework of the artificial intelligence deep-learning model included object detection, data preprocessing of radiographs, and classification to detect vertebral fractures. Approximately 90 seconds was needed to complete the procedure and obtain the artificial intelligence model results when applied clinically. The accuracy, sensitivity, specificity, interobserver reliability (kappa value), receiver operating characteristic curve, and area under the curve (AUC) were analyzed. The bootstrapping method was applied to our testing dataset and external validation dataset. The accuracy, sensitivity, and specificity were used to investigate whether fracture anatomic location or T-score in dual-energy x-ray absorptiometry report affected the performance of the artificial intelligence model. The receiver operating characteristic curve and AUC were used to investigate the relationship between the performance of the artificial intelligence model and fracture grade. External validation with a similar age population and plain lateral radiographs from another medical institute was also performed to investigate the performance of the artificial intelligence model. RESULTS: The artificial intelligence model with ensemble method demonstrated excellent accuracy (93% [773 of 830] of vertebrae), sensitivity (91% [129 of 141]), and specificity (93% [644 of 689]) for detecting vertebral fractures of the lumbar spine. The interobserver reliability (kappa value) of the artificial intelligence performance and human observers for thoracic and lumbar vertebrae were 0.72 (95% CI 0.65 to 0.80; p < 0.001) and 0.77 (95% CI 0.72 to 0.83; p < 0.001), respectively. The AUCs for Grades 1, 2, and 3 vertebral fractures were 0.919, 0.989, and 0.990, respectively. The artificial intelligence model with ensemble method demonstrated poorer performance for discriminating normal osteoporotic lumbar vertebrae, with a specificity of 91% (260 of 285) compared with nonosteoporotic lumbar vertebrae, with a specificity of 95% (222 of 234). There was a higher sensitivity 97% (60 of 62) for detecting osteoporotic (dual-energy x-ray absorptiometry T-score ≤ -2.5) lumbar vertebral fractures, implying easier detection, than for nonosteoporotic vertebral fractures (83% [39 of 47]). The artificial intelligence model also demonstrated better detection of lumbar vertebral fractures compared with detection of thoracic vertebral fractures based on the external dataset using various radiographic techniques. Based on the dataset for external validation, the overall accuracy, sensitivity, and specificity on bootstrapping method were 89%, 83%, and 95%, respectively. CONCLUSION: The artificial intelligence model detected vertebral fractures on plain lateral radiographs with high accuracy, sensitivity, and specificity, especially for osteoporotic lumbar vertebral fractures (Genant Grades 2 and 3). The rapid reporting of results using this artificial intelligence model may improve the efficiency of diagnosing vertebral fractures. The testing model is available at http://140.113.114.104/vght_demo/corr/. One or multiple plain lateral radiographs of the spine in the Digital Imaging and Communications in Medicine format can be uploaded to see the performance of the artificial intelligence model. LEVEL OF EVIDENCE: Level II, diagnostic study.

The Clinical Application of the Deep Learning Technique for Predicting Trigger Origins in Patients With Paroxysmal Atrial Fibrillation With Catheter Ablation.

BACKGROUND: Non-pulmonary vein (NPV) trigger has been reported as an important predictor of recurrence post-atrial fibrillation ablation. Elimination of NPV triggers can reduce the recurrence of postablation atrial fibrillation. Deep learning was applied to preablation pulmonary vein computed tomography geometric slices to create a prediction model for NPV triggers in patients with paroxysmal atrial fibrillation. METHODS: We retrospectively analyzed 521 patients with paroxysmal atrial fibrillation who underwent catheter ablation of paroxysmal atrial fibrillation. Among them, pulmonary vein computed tomography geometric slices from 358 patients with nonrecurrent atrial fibrillation (1-3 mm interspace per slice, 20-200 slices for each patient, ranging from the upper border of the left atrium to the bottom of the heart, for a total of 23 683 images of slices) were used in the deep learning process, the ResNet34 of the neural network, to create the prediction model of the NPV trigger. There were 298 (83.2%) patients with only pulmonary vein triggers and 60 (16.8%) patients with NPV triggers±pulmonary vein triggers. The patients were randomly assigned to either training, validation, or test groups, and their data were allocated according to those sets. The image datasets were split into training (n=17 340), validation (n=3491), and testing (n=2852) groups, which had completely independent sets of patients. RESULTS: The accuracy of prediction in each pulmonary vein computed tomography image for NPV trigger was up to 82.4±2.0%. The sensitivity and specificity were 64.3±5.4% and 88.4±1.9%, respectively. For each patient, the accuracy of prediction for a NPV trigger was 88.6±2.3%. The sensitivity and specificity were 75.0±5.8% and 95.7±1.8%, respectively. The area under the curve for each image and patient were 0.82±0.01 and 0.88±0.07, respectively. CONCLUSIONS: The deep learning model using preablation pulmonary vein computed tomography can be applied to predict the trigger origins in patients with paroxysmal atrial fibrillation receiving catheter ablation. The application of this model may identify patients with a high risk of NPV trigger before ablation.

Diagnosis of common pulmonary diseases in children by X-ray images and deep learning.

Acute lower respiratory infection is the leading cause of child death in developing countries. Current strategies to reduce this problem include early detection and appropriate treatment. Better diagnostic and therapeutic strategies are still needed in poor countries. Artificial-intelligence chest X-ray scheme has the potential to become a screening tool for lower respiratory infection in child. Artificial-intelligence chest X-ray schemes for children are rare and limited to a single lung disease. We need a powerful system as a diagnostic tool for most common lung diseases in children. To address this, we present a computer-aided diagnostic scheme for the chest X-ray images of several common pulmonary diseases of children, including bronchiolitis/bronchitis, bronchopneumonia/interstitial pneumonitis, lobar pneumonia, and pneumothorax. The study consists of two main approaches: first, we trained a model based on YOLOv3 architecture for cropping the appropriate location of the lung field automatically. Second, we compared three different methods for multi-classification, included the one-versus-one scheme, the one-versus-all scheme and training a classifier model based on convolutional neural network. Our model demonstrated a good distinguishing ability for these common lung problems in children. Among the three methods, the one-versus-one scheme has the best performance. We could detect whether a chest X-ray image is abnormal with 92.47% accuracy and bronchiolitis/bronchitis, bronchopneumonia, lobar pneumonia, pneumothorax, or normal with 71.94%, 72.19%, 85.42%, 85.71%, and 80.00% accuracy, respectively. In conclusion, we provide a computer-aided diagnostic scheme by deep learning for common pulmonary diseases in children. This scheme is mostly useful as a screening for normal versus most of lower respiratory problems in children. It can also help review the chest X-ray images interpreted by clinicians and may remind possible negligence. This system can be a good diagnostic assistance under limited medical resources.

Automated extraction of left atrial volumes from two-dimensional computer tomography images using a deep learning technique.

BACKGROUND: Precise segmentation of the left atrium (LA) in computed tomography (CT) images constitutes a crucial preparatory step for catheter ablation in atrial fibrillation (AF). We aim to apply deep convolutional neural networks (DCNNs) to automate the LA detection/segmentation procedure and create three-dimensional (3D) geometries. METHODS: Five hundred eighteen patients who underwent procedures for circumferential isolation of four pulmonary veins were enrolled. Cardiac CT images (from 97 patients) were used to construct the LA detection and segmentation models. These images were reviewed by the cardiologists such that images containing the LA were identified/segmented as the ground truth for model training. Two DCNNs which incorporated transfer learning with the architectures of ResNet50/U-Net were trained for image-based LA classification/segmentation. The LA geometry created by the deep learning model was correlated to the outcomes of AF ablation. RESULTS: The LA detection model achieved an overall 99.0% prediction accuracy, as well as a sensitivity of 99.3% and a specificity of 98.7%. Moreover, the LA segmentation model achieved an intersection over union of 91.42%. The estimated mean LA volume of all the 518 patients studied herein with the deep learning model was 123.3 ± 40.4 ml. The greatest area under the curve with a LA volume of 139 ml yielded a positive predictive value of 85.5% without detectable AF episodes over a period of one year following ablation. CONCLUSIONS: The deep learning provides an efficient and accurate way for automatic contouring and LA volume calculation based on the construction of the 3D LA geometry.